JP2009301266A - User interface device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a user interface device wherein a user can easily find a template matched with his or her own equipment operation request from a list and equipment operability is good. <P>SOLUTION: This user interface device includes: a keyword DB102; a category DB104; a template DB106; a voice recognition means 103 for performing voice recognition by using a voice signal from a microphone 101 and a keyword DB102; a category acquisition means 105 for acquiring the category of a keyword recognized by using the category DB104, and for outputting the category list; a template retrieval control means 108 for creating a category attribute list, and for acquiring and sequencing a template from the template DB106, and for outputting a template list; and a template edition means 109 for creating a keyword list sequenced based on the template list and the category list, and for displaying the keyword and the template on an image display means 110, and for accepting an operation by a user. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a user interface device that performs device operation by searching and correcting a device operation template by voice recognition.

  In a translation apparatus using a conventional user interface, a device operation template is searched using a keyword input by speech recognition, and a template that matches a user's device operation request is obtained from a list of templates obtained as a result of the search. A desired device operation is performed by selecting and correcting the selected template using a keyword input by voice recognition (see, for example, Patent Document 1).

Japanese Patent Laid-Open No. 2002-14957 (page 4, FIGS. 9 to 13)

  In a translation apparatus using such a user interface, the list of templates obtained as a result of the search is arranged in the order in which the templates are stored in the database, or in alphabetical order or alphabetical order. For this reason, when there are a large number of templates, there is a problem that it takes time for the user to search the list for a template that meets his / her device operation request, and the device operability is poor.

  The present invention has been made to solve the above-described problems. A user interface that allows a user to easily find a template that meets his / her device operation request from a list and has a good device operability. The object is to obtain a device.

  The user interface device according to the present invention includes a keyword database in which a keyword for speech recognition is recorded, a category database in which a keyword category is recorded, and a blank in a part of an operation sentence of a device to be operated. Recognize a keyword by recognizing a template database in which a template and a category attribute determined in advance by combining one or more categories and a voice signal from a microphone and a keyword database are recorded. Voice recognition means for outputting a list of scores, a category for obtaining a category corresponding to the keyword recognized using the list from the voice recognition means and the category database, and outputting a category list including the recognized keyword and the recognition score Acquisition means and categories A category attribute list is created by obtaining points based on category attributes and recognition scores from one or more category lists obtained by the obtaining means. Based on this, templates are obtained from the template database and templates are ordered. Create a keyword list that is ordered based on the template search control means that outputs the list, the template list from the template search control means, and the category list from the category acquisition means, and displays the keywords in the keyword list and the templates in the template list as images A template editing means for displaying on the means and receiving an operation by the user.

  According to the present invention, since the templates are ordered and displayed, a template that meets the user's device operation request can be displayed, and a user interface device with good device operability can be obtained.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing a user interface device according to Embodiment 1 for carrying out the present invention. Although not shown in the figure, this interface device is connected to a device to be operated, such as a car navigation device or a television, or incorporated in a part of these devices, so that the user can operate the device. The execution request is received, and the execution of the operation is requested to the device to be operated. Now, in this figure, the microphone 101 converts the user's speech into an audio signal and outputs it. As shown in FIG. 2, the keyword DB (database) 102 stores a plurality of sets of keywords and keyword pronunciations as one set, as an example of the data structure and data. The voice recognition means 103 performs voice recognition using the voice signal from the microphone 101 and the keyword DB 102, and, as shown in FIG. 3, for example, a set of recognized keywords and corresponding recognition scores are arranged in the order of recognition scores. The keyword score list 601 is output. The recognition score, which is also referred to as recognition reliability, is obtained from the degree of coincidence between the user's utterance and the keyword in the speech recognition processing result.

  In the category DB 104, as shown in FIG. 4 as an example of the data structure and data, a plurality of sets of keywords and keyword categories are recorded as one set. Here, the category is a classification of “target” that is connected to the interface device and executes the function of the device to be operated. For example, in a “destination setting” function in a car navigation device, the “target” is a destination. Examples of categories that can be destinations include “specific facility name” and “address”. Examples of keywords belonging to the “specific facility name” category include “Oji Zoo” and “Meriken Park”. The category acquisition unit 105 receives the keyword score list 601 composed of a combination of a keyword and a recognition score from the speech recognition unit 103, acquires the category of each keyword from the category DB 104, and, as shown in FIGS. The category lists 701 and 801 in which pairs of keywords and recognition scores are arranged in the order of recognition scores are output.

  As shown in FIG. 7, the template DB 106 is connected to the interface device and includes a template having a blank in a part of an operation sentence of a device to be operated, and corresponds to the template in advance. Plural sets of recorded category attributes are recorded as one set. The category attribute is composed of a set of one or more categories, and as many categories as one category are associated with each blank as a category attribute. Here, the “target” for executing the function is an unspecified state (blank), and the “function execution request” of the device is expressed in text. The user of the device designates an arbitrary “keyword” as the “target”, thereby completing the “function execution request”.

  The template search unit 107 receives a category attribute list 901 (see FIG. 8) as a search key from the template search control unit 108, searches the template DB 106 for a template corresponding to the category attribute, and sets a template / category attribute pair. The list is output to the template search control means 108.

  The template search control means 108 receives one or more category lists comprising categories, keywords, and recognition scores from the category acquisition means 105, determines the category attributes and points for template search from the categories and recognition scores, and determines their ranks. Then, a category attribute list 901 as shown in FIG. 8 is created. Thereafter, the template search unit 107 is used to search the template using each category attribute as a key, and the result is obtained. As shown in FIG. 9, the combination of the obtained template and the category attribute of the template is assigned to the rank of the category attribute. A template list 1002 arranged according to the above is output.

  The template editing unit 109 receives the category list 701 and 801 including a category / keyword / recognition score combination from the category acquisition unit 105 and the template list 1002 including a template / category attribute combination from the template search control unit 108. The keyword to be displayed is determined from the category attribute. Thereafter, as shown in FIG. 10, a list of keywords and templates is output to the image display means 110, and a user's operation of selecting a template and assigning the keyword to a blank column of the template is accepted. The user interface screen 1501 includes an area 1502 for displaying a list of templates and areas 1503 and 1504 for displaying a list of keywords. The number of areas for displaying the keyword list is one or more, and it increases or decreases depending on the number of keywords input by voice. Each area includes a scroll bar 1505 for scrolling and displaying items of a list that cannot be displayed in the area. The list of each area is arranged in order with the recognition score or category attribute having the highest score, and the scroll position so that the top item of the list can be seen when the screen is first displayed. Is set. Thereafter, a template execution request from the user is received, and a device (not shown) is requested to execute the device operation described in the template.

  The screen display unit 110 converts the keyword and template list output from the template editing unit 109 into a video signal and outputs the video signal to the display 111. The display 111 visualizes the video signal.

  When the user interface device configured as described above is used in a car navigation device, the operation will be described with reference to the flowchart of FIG. When the user interface device is activated, the process proceeds to keyword input step S200. In keyword input step S200, a keyword spoken by the user is recognized by speech. Next, in a template search step S300, a template is searched from the keyword that has been recognized by speech. Thereafter, in the template editing step S400, the speech-recognized keyword and the searched template are displayed on the display 111, and the user edits the template and executes the template. Details of processing in each step of FIG. 11 will be described below.

  FIG. 12 is a detailed flowchart of the process in the keyword input step S200 of FIG. When the keyword input step S200 is started, the process proceeds to the voice input step S210. In the voice input step S210, the microphone 101 converts the keyword spoken by the user into a voice signal and outputs it. In the keyword recognition step S220, the voice recognition means 103 performs voice recognition using the voice signal from the microphone 101 and the keyword DB 102, and the recognized keyword and recognition score pairs are arranged in the order of the recognition score. As shown in FIG. A list 601 is output. Here, a case where the user speaks “Oji Zoo” is shown. The number of items in the list is not more than a predetermined fixed value. The example in the figure has four items, and the top four keywords of the recognition score are items in the list.

  In the category acquisition step S230, the category acquisition unit 105 receives the keyword score list 601 from the speech recognition unit 103, acquires the category of each keyword from the category DB 104, arranges the category, the keyword, and the recognition score in the order of the recognition score. Create 5 and 6 are examples of this. A category list 701 in FIG. 5 is an example when the category acquisition unit 105 receives the keyword score list 601 in FIG. 3 from the speech recognition unit 103. FIG. 6 is an example of the category list 801 that is output when the user further speaks “Family” and executes the keyword recognition step S220 and the category acquisition step S230.

  In the voice input end determination step S240, if there is a voice input end instruction from the user, the voice input ends, and the keyword input step S200 ends. If there is no instruction, the process returns to the voice input step S210. Note that the voice input end instruction by the user may be performed by operating a device button or a touch panel, or may be determined by not speaking the next keyword within a predetermined time.

  FIG. 13A is a detailed flowchart of the process in the template search step S300 of FIG. When template search step S300 starts, the process proceeds to category attribute list creation step S310. In category attribute list creation step S310, the template search control means 108 receives category lists 701 and 801 composed of pairs of categories, keywords, and recognition scores from the category acquisition means 105, and creates a category attribute list from them.

  Here, a method for creating a category attribute list will be described with reference to FIG. This figure is a detailed flowchart of the processing in the category attribute list creation step S310 of FIG.

  When the category attribute list creation step S310 is started, the process proceeds to a list receiving step S311 of a category / keyword / recognition score pair. Here, the template search control means 108 receives a category list composed of a combination of a category, a keyword, and a recognition score for the number of times voice is input in the keyword input step S200. For example, category lists 701 and 801 shown in FIGS.

  In category attribute / score acquisition step S312, a category attribute used for template search and its score are obtained from a combination of a category and a recognition score between category lists. Here, the score is a numerical value that is regarded as an “index of high fitness” of “user device operation request” and “category attribute” in this apparatus.

  Specifically, first, in the lists shown in FIGS. 5 and 6, if there are overlapping category items in the lists, items other than the item with the highest recognition score are deleted. The result of this processing is shown in FIG. The processing result of the category list 701 in FIG. 5 is a category intermediate list 1601, and the processing result of the category list 801 in FIG. 6 is a category intermediate list 1602.

  Next, all combinations of categories between the lists and their scores are obtained. The score is the sum of the recognition scores of each category. However, other calculation formulas may be used when calculating the score. The result of this processing is shown in the category combination table 1701 in FIG. In the category combination table 1701, category 1 corresponds to a category in the category intermediate list 1601, and category 2 corresponds to a category in the category intermediate list 1602.

  Next, a category attribute and its score are obtained from the category combination table 1701. For example, a combination of “specific facility” of category 1 and “general facility” of category 2 is set as a category attribute “specific facility / general facility”. At this time, the order of categories is not considered. In other words, “special facility / general facility” and “general facility / special facility” are regarded as the same. The result of this processing is shown in the category attribute intermediate list 1702. Next, when there are overlapping category attributes, items other than the item with the highest score are deleted. In the category attribute intermediate list 1702, items to be deleted are shown with a gray background.

  In category attribute sorting step S313, the obtained category attributes are arranged in descending order to create a category attribute list. Specifically, the category attributes and scores remaining in the category attribute intermediate list 1702 in FIG. 15 are arranged in descending order to create the category attribute list 901 in FIG. 8, and the category attribute list creation step S310 is completed.

  In the template DB search step S320, the template search control means 108 searches for a template using each category attribute as a key using the template search means 107 in the order of the category attribute list 901. FIG. 9 shows an example when the template search control means 108 searches for a template and obtains a template search result 1001. The template search unit 107 receives a category attribute as a search key, searches the template DB 106 for a template corresponding to the category attribute, and returns a list of template / category attribute pairs to the template search control unit 108.

  In the template list creation step S330, the template search control means 108 connects the template and category attribute pairs searched in the template DB search step S320 in the order of the category attribute list (see FIG. 9), and a list of template and category attribute pairs. Create As a result, a list of template / category attribute pairs arranged in the order of category attribute scores is created. Finally, a template list 1002 including the created template and category attribute pair is output to the template editing means 109, and the template search step S300 is terminated.

  FIG. 16 is a detailed flowchart of the process in template editing step S400 of FIG. When template editing step S400 is started, the process proceeds to keyword list creation step S410.

  In the keyword list creation step S410, the template editing unit 109 receives the category lists 701 and 801 (see FIGS. 5 and 6) including the category, the keyword, and the recognition score from the category acquisition unit 105, and the template search control unit 108 receives the template. A template list 1002 (see FIG. 9) consisting of a set of category attributes is received. Next, in order to select and order keywords, keywords in categories not included in the template are excluded and ordered in the order of recognition scores to create a keyword list. For example, in the case of the category lists 701 and 801 in FIGS. 5 and 6, the template list 1002 in FIG. 9 does not have the “song name” category. Therefore, if the keyword belonging to the “song name” is removed from the category lists 701 and 801, FIG. The keyword lists 702 and 802 shown in FIG.

  In the keyword / template display step S420, the template editing means 109 displays on the display 111 using the screen display means 110 as shown in FIG. 10 in the order of the keyword lists 702 and 802 and the template list 1002. In the editing operation accepting step S430, the template editing means 109 accepts an operation for assigning a keyword to a blank field of the selected template and an operation for executing the template after the user selects a template list. In template execution operation determination step S440, if the user operation is a template execution operation, the process proceeds to keyword substitution completion determination step S460. If it is not a template execution operation, the process proceeds to keyword substitution processing step S450.

  In keyword substitution processing step S450, the template editing means 109 writes the substituted keyword in the blank field of the template, and returns to the keyword / template display step S420 to redisplay the template.

  In the keyword substitution completion determination step S460, the template editing unit 109 proceeds to the template execution request step S470 if the keyword writing to all blanks of the selected template is completed. If there is a blank column in which no keyword is written in the template, the process returns to the keyword / template display step S420.

  In template execution request step S470, template editing means 109 requests the device to execute the device operation described in the template, and ends template editing step S400.

  The template DB may have a data structure shown in FIG. 18 in addition to the data structure shown in FIG. In the example of FIG. 7, the data structure has a category attribute for each template, but in this example, the data structure has a template belonging to a group of category attributes.

  According to the user interface device configured as described above, a template is searched from the keyword category and the voice recognition score input by voice recognition, the order of the templates obtained by the search, the selection and ordering of the keywords are performed, Since the list of keywords and templates is displayed in that order, keywords and templates that match the user's device operation request can be preferentially displayed at the top of the list, and the user can easily find keywords and templates from the list. Thus, a user interface device that has good device operability and can reduce the burden on the user can be obtained.

  Further, if the template DB has a data structure in which the template belongs to the group having the category attribute, the search for the same category becomes efficient, so that the operation speed is improved and the device operability is improved.

Embodiment 2. FIG.
In the first embodiment, when a plurality of templates are searched for and acquired by the same category attribute, the template is not superior or inferior, but here, a method for ordering templates even in such a case will be described. FIG. 19 shows an example of the data structure and data of the template DB in the user interface device according to the second embodiment of the present invention. Each template records the priority based on the usage frequency in addition to the category attribute. This priority is set in advance, but the priority may be changed according to the usage status of the user.

  In the user interface device configured as described above, the template search control means 108 has a higher template priority value between templates searched with the same category attribute in addition to the processing of the first embodiment. Arrange them so that they are at the top of the list.

  In this way, according to the configured user interface device, by setting a priority for each template, even when there are a large number of templates searched in the same category, a template that meets the user's device operation request is given priority. Since it can be displayed at the top of the list, the user can easily find the template from the list, the device operation is improved, and the burden on the user can be reduced.

Embodiment 3 FIG.
In the first and second embodiments, the templates are not superior or inferior when there is no template priority or in the same case, but here, a method for ordering templates will be described even in such a case. FIG. 20 shows an example of the data structure and data of the template DB 106 in the user interface device according to the third embodiment of the present invention. Each template has a category attribute recorded for each blank of the template. Each template can have a plurality of category attributes.

  In the user interface device configured as described above, the template search control unit 108 searches in the same category in addition to the processing in the first or second embodiment, and the template has no priority or the same priority. The template blanks are arranged so that the one with more category attributes is higher in the list. This is because a template with many blank category attributes is versatile, and is likely to be a template that meets a user's device operation request.

  In this way, according to the configured user interface device, since the number of category attributes of each template is used for ranking the templates, a template that matches the user's device operation request can be obtained even if the templates have no priority or the same priority. It can be preferentially displayed at the top of the list, and the user can easily find a template from the list.

Embodiment 4 FIG.
In the first embodiment, an example in which the user interface device is applied to a car navigation device has been described. Here, an example of a use for television or television recording will be described. The user interface device according to the fourth embodiment of the present invention has the same structure as the user interface according to the first embodiment shown in FIG. However, since the function of a device (not shown) that requires the user interface device to execute an operation is different, the data recorded in the keyword DB 102, the category DB 104, and the template DB 106 is changed according to the function of the device.

  FIG. 21 shows an example of data in the keyword DB 102 according to the fourth embodiment of the present invention. The data structure is the same as that of the keyword DB 102 in the first embodiment shown in FIG. FIG. 22 shows an example of data in the category DB 104. The data structure is the same as that of the category DB 104 in the first embodiment shown in FIG. Categories are arbitrarily defined according to the function of the device. FIG. 23 shows an example of data in the template DB 106. The data structure is the same as that of the template DB 106 in the first embodiment shown in FIG.

  In the user interface device configured as described above, from the keyword category and the speech recognition score input in the speech recognition, the template search and the order of the templates obtained by the search are performed, and the template according to the order is searched. Operate the device by displaying the list.

  The first embodiment is an example of a car navigation device, and the fourth embodiment is an example of a television. However, in the case of a device in which the car navigation device and the television are integrated, a category DB and a template corresponding to each of them. You may have DB. In this case, when each operation target is used, switching is performed to use a database corresponding to each operation target. For example, when using a television, a television database is used. In this case, “ABC” acquires the category of “Broadcasting station name” from the TV category DB, and acquires the template from the TV template DB corresponding thereto. On the other hand, when the car navigation device is used, the database is switched to the car navigation database. In this case, “ABC” acquires the category of “specific facility” from the category DB for car navigation, and acquires the template from the template DB for car navigation corresponding to this.

  Thus, according to the configured user interface device, since a database corresponding to the device to be operated is used, a template that meets the user's device operation request can be preferentially displayed at the top of the list. Therefore, the user can easily find a template from the list, and a user interface device with good device operation can be obtained.

It is a block diagram of a user interface device showing Embodiment 1 of the present invention. It is a data structure and data example of keyword DB which shows Embodiment 1 of this invention. It is an example of the keyword score list | wrist which shows Embodiment 1 of this invention. It is an example of the data structure and data of category DB which shows Embodiment 1 of this invention. It is an example of the category list | wrist which shows Embodiment 1 of this invention. It is another example of the category list | wrist which shows Embodiment 1 of this invention. It is an example of the data structure and data of template DB which shows Embodiment 1 of this invention. It is an example of the category attribute list | wrist which shows Embodiment 1 of this invention. It is an example of the search result of the template which shows Embodiment 1 of this invention. It is an example of the screen of the user interface apparatus which shows Embodiment 1 of this invention. It is a flowchart which shows Embodiment 1 of this invention. It is a flowchart in keyword input step S200 which shows Embodiment 1 of this invention. It is a flowchart in template search step S300 which shows Embodiment 1 of this invention. It is an example of a category intermediate list showing Embodiment 1 of the present invention. It is an example of the category combination table | surface which shows Embodiment 1 of this invention, and an example of a category attribute intermediate | middle list. It is a flowchart in template edit step S400 which shows Embodiment 1 of this invention. It is an example of the keyword list | wrist which shows Embodiment 1 of this invention. It is another example of the data structure and data of template DB which shows Embodiment 1 of this invention. It is a data structure and data example of template DB which shows Embodiment 2 of this invention. It is a data structure and data example of template DB which shows Embodiment 3 of this invention. It is a data structure and data example of keyword DB which shows Embodiment 4 of this invention. It is an example of the data structure and data of category DB which shows Embodiment 4 of this invention. It is a data structure and data example of template DB which shows Embodiment 4 of this invention.

Explanation of symbols

101 Mike 102 Keyword DB
103 Voice recognition means 104 Category DB
105 Category acquisition means 106 Template DB
107 Template search means 108 Template search control means 109 Template editing means 110 Screen display means 111 Display 601 Keyword score list 701 Category list 702 Keyword list 801 Category list 802 Keyword list 901 Category attribute list 1001 Template search result 1002 Template list 1101 Keyword DB storage Data 1201 Category DB storage data 1301 Template DB storage data 1401 Template DB storage data 1501 User interface screen 1502 Template list display area 1503 Keyword list display area (1)
1504 Keyword list display area (2)
1505 Scroll bar 1601 Category intermediate list (1)
1602 Category intermediate list (2)
1701 Category combination table 1702 Category attribute intermediate list 1801 Template DB storage data 1901 Template DB storage data 2001 Keyword DB storage data 2101 Category DB storage data 2201 Template DB storage data

Claims (5)

A keyword database in which keywords for speech recognition are recorded;
A category database in which the keyword categories are recorded;
A template database in which a template having a blank in a part of a statement for operation of a device to be operated and a category attribute determined in advance by combining one or more of the categories corresponding thereto are recorded;
Voice recognition means for performing voice recognition using a voice signal from a microphone and the keyword database, and outputting a list of recognized keywords and recognition scores;
Category acquisition means for acquiring a category corresponding to the recognized keyword using the list from the speech recognition means and the category database, and outputting a category list including the category, the recognized keyword, and the recognition score;
A category attribute list is created by obtaining a score based on the category attribute and the recognition score from one or more of the category lists obtained by the category acquisition means, and templates are obtained from the template database based on the score, and the templates are ordered. And a template search control means for outputting a template list,
Create a keyword list ordered based on the template list from the template search control means and the category list from the category acquisition means, and display the keywords in the keyword list and the templates in the template list on the image display means. A user interface device comprising template editing means for accepting an operation by a user. In the template database, priority information for each template is recorded,
2. The user interface device according to claim 1, wherein when a plurality of templates are acquired for the same category attribute, the template search control unit orders the templates based on the priority information for each template. In the template database, category attribute information for each blank is recorded,
The user interface device according to claim 1, wherein the template search control unit orders the templates based on the number of category attributes of the template when a plurality of templates are acquired for the same category attribute.   The user interface device according to claim 1, wherein the template database groups templates for each category attribute. The category database and the template database further include a database corresponding to each device to be operated,
The user interface device according to claim 1, wherein the category acquisition unit and the template search control unit acquire a category and a template by switching a database corresponding to each device to be operated.

Images