JPH1091309A - Device and method for inputting and outputting information - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable a user to correct an interpretation result in voice and also to select a candidate for the interpretation result in voice. SOLUTION: Of the information input/output device 101, an input interpretation candidate tentative output part 11 generates tentative display information for making the user to select a candidate out of respective interpretation candidates obtained from an input device L 102. A timer part 12 starts clocking the time with the output of the tentative display information and generates a timer signal a specific time later. When speech information for candidate selection is inputted from a speech recognition device 105 or when the timer part 12 generates the timer signal, an interpretation information selection and determination part 13 selects one of the respective interpretation candidates and supplies it as interpretation information to an information processor 103. An output integration part 15 integrates the output information from the information processor 103 and the tentative display information from the input interpretation candidate set tentative output part 11 together and supplies the result to an output device 104 as output information for the user.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a format required by a user by performing conversion processing or recognition processing on information input by a user, such as a kana-kanji conversion system and a pen input handwritten figure recognition system. The present invention relates to an information input / output device and an information input / output method applied to a system or a device that generates the above information.

[0002]

2. Description of the Related Art In recent years, with the development of computer technology, it has become possible to input various information to a computer in a manner more natural for a user. For example, in the case of a kana-kanji conversion system, a kana key is entered from a keyboard, or an alphabetic key is entered in romaji, and a kana character string is input. I can do it. With the advent of the Kana-Kanji conversion system, it was necessary to enter kanji codes or select characters from the kanji-type keyboard on which many kanji characters were lined up. Thus, input can be performed in a simple and more natural way for the user.

Further, for example, in a pen-input handwritten figure recognition system, a stroke sequence input by a user using a pen device or the like in a freehand manner is subjected to recognition processing and converted into a standardized unit figure such as a circle or a rectangle. Then, for example, the recognized figure is displayed on a display. With the advent of such a pen-input handwritten graphic recognition system, with respect to the previous graphic input device which had to select a graphic to be drawn from a menu or the like and input parameters such as a radius and a length of a side from a keyboard or the like, Simple and more natural input for users has become possible.

As described above, in an information input / output device that accepts a more natural input, information in a format desired by the user is generated by performing a conversion process or a recognition process on the information input by the user. I have.

However, for example, when using a kana-kanji conversion device, it is necessary to select a correct candidate from among homonymous word candidates. It is necessary to indicate a graphic intended by the user as an input from among graphic candidates, or to correct a recognition error.

That is, in an apparatus or a system that performs input processing involving such conversion processing and recognition processing, the ambiguity included in user input and the performance of the recognition apparatus are essentially insufficient. Therefore, a mechanism is required to deal with recognition errors caused by the above.

Conventionally, such processing that requires elimination of ambiguity or correction of recognition errors includes, for example, processing of homonymous words in kana-kanji conversion and correction of a segment break position, using a cursor movement key from the keyboard. Click the mouse to select the correct candidate to operate, etc.
In the sense of using a hand, it had to be performed in a manner that involved the same physical operation as input. That is, for example, when inputting Japanese sentences using the keyboard, it is necessary to correct the result of the Kana-Kanji conversion process using the mouse or the keyboard. There is a problem that hinders the mental concentration of the elderly.

Further, in an apparatus or a system in which the user needs to give some kind of feedback to resolve the ambiguity and correct the recognition error as described above, conventionally, the user selects the correct candidate from the candidates presented by the system. It was common to adopt a method of selecting and confirming.

However, since the conversion and recognition performance of systems such as kana-kanji conversion and online graphic recognition have been improved due to recent technological advances, it is highly likely that the first candidate is the correct answer, and the candidate is always the correct answer. There has been a problem that the operation form that requires confirmation may hinder usability. Further, there are problems that the user cannot set the candidate determination requirement and the system cannot automatically change the setting.

[0010]

As described above, conventionally, input processing involving ambiguity resolution and correction of recognition errors has been conventionally performed by selection processing for ambiguity resolution and correction processing of recognition errors. However, there is a problem that the user needs to perform the operation by a method involving the same physical operation as that at the time of inputting, and the operation is confused and the concentration of the user is prevented.

[0011] Further, even when the first candidate is a correct answer and thus no confirmation operation is necessary, the candidate is determined for the selection processing for resolving the ambiguity and the correction processing of the recognition error. There was also a problem that it required.

There are also problems such as that the user cannot set the requirements necessary for candidate determination or that the system cannot automatically change the settings. The present invention has been made in view of such circumstances, and regarding an interpretation result of an input device that accepts and interprets an ambiguous input,
It is an object of the present invention to provide an information input / output device and an information input / output method in which a user can make a correction of an interpretation result by voice, and a user can make a selection of a candidate of the interpretation result by voice. .

Further, according to the present invention, it is possible to automatically determine the result of the selection of the candidate made by the user and the correction of the interpretation result, and the user can select the candidate even after the set time has passed. An object of the present invention is to provide an information input / output device and an information input / output method that can automatically determine a certain candidate state unless selection or correction of an interpretation result is performed.

Further, according to the present invention, a user can set a time required for automatically determining a candidate state, and the time required for determining a candidate state is automatically set by adapting to a specific user. It is an object of the present invention to provide an information input / output device and an information input / output method that can perform the above.

[0015]

According to the present invention, there is provided an input means for processing an input signal from a user, ranking the plurality of interpretation candidates as a result of the processing, and outputting the result. Information processing means for processing a candidate selected from each interpretation candidate as interpretation information and outputting the processing result as output information; and output means for presenting the output information obtained by the information processing means to a user And an information input / output device provided with voice recognition means for recognizing voice information.

In such an information input / output device, the present invention provides an input interpretation candidate set provisional for generating provisional presentation information for allowing a user to select a candidate from among the interpretation candidates obtained by the input means. An output unit, a timer signal generation unit that starts a timer operation when the provisional presentation information is output from the input interpretation candidate provisional output unit, and generates a timer signal after a lapse of a predetermined time; When the audio information for selecting the interpretation candidate is input, or at the timing when the timer signal is output by the timer signal generation means, the timing is specified by the audio information from among the interpretation candidates. A candidate or a predetermined candidate is selected according to the generation of the timer signal, and an interpretation information selection / determination method for giving the selected candidate to the information processing means as the interpretation information. Output control means for providing the output information output as a result of processing of the interpretation information provided by the interpretation information selection / determination means by the information processing means to the output means as new output information to a user And characterized in that:

Further, the present invention further comprises timer setting input means for arbitrarily setting a clock time, and timer setting storage means for storing the clock time set by the timer setting input means. The timer signal generation means generates the timer signal based on the clock time stored in the timer setting storage means when the provisional presentation information is output.

Further, according to the present invention, after the provisional presentation information is output, a time required until the interpretation information is determined by the audio information or the timer signal is measured each time.
A fixed time sequence storing means for storing the sequence, a fixed time calculating means for calculating a proper fixed time for a user of the present apparatus from a set of times required for the fixing stored in the fixed time sequence storing means, A timer setting storage unit for storing the determined time calculated by the determined time calculation unit as a measured time, wherein the timer signal generation unit is configured to output the timer setting storage unit when the provisional presentation information is output. The timer signal is generated based on the clock time stored in the timer.

According to the above configuration, the interpretation result of the user's input (for example, the result of conversion processing or recognition processing)
, The user can make the correction by voice. Further, in addition to the above, the interpretation result candidate of the user's input allows the user to select the interpretation result candidate by voice.

For example, when operating a system that requires ambiguity resolution or correction of recognition errors, such as processing of homophonic meaning words in kana-kanji conversion and correction of segment break positions, cursor movement keys are operated from the keyboard. In the sense of using a hand, for example, selecting a correct candidate by clicking with a mouse on the display of a correct candidate, it was necessary to perform the method with a physical operation similar to that for input. That is, for example, when inputting Japanese sentences using the keyboard, it is necessary to correct the result of the kana-kanji conversion process using, for example, a mouse or a keyboard, and a finger may be moved away from the home position on the keyboard, There was a problem that hindered the user's mental concentration.

According to the present invention, the interpretation result can be corrected by the user by voice, and the interpretation result candidate can be selected by the user by voice. While inputting a character string and converting it using the Kana-Kanji conversion system, Japanese sentences such as selecting a conversion candidate or changing a segment break by uttering a word, for example, saying "third" to select a homonymous word. By assigning the different tasks of inputting characters and correcting the processing results of the Kana-Kanji conversion system to different devices such as a keyboard and a microphone, it reduces the mental burden on users and concentrates on the task of inputting Japanese kana character strings. You can work naturally.

Further, according to the present invention, it is possible to automatically determine the result of the selection of the candidate made by the user by voice and the correction of the interpretation result. In addition, if the user does not select a candidate or correct the interpretation result even after the set time has elapsed, a certain candidate state can be automatically set.

For example, in an on-line graphic recognition system, the system performs a graphic recognition process on a pen input stroke sequence performed by a user with the intention of inputting a rectangle, and as a result, the likelihood in the order of a rectangle, a square, and an ellipse is obtained. In the case where the recognition result is obtained, the conventional system presents these candidates to the user, and the user cannot select a correct answer from the candidates to continue the graphic input, or The system either forcibly selects candidates with high likelihood as recognition results. In the former case, even though the correct answer intended by the user is the candidate with the highest likelihood, the graphic input cannot be continued unless the user selects the first candidate from among the candidates, and There was a problem that hindered the mental concentration of users.
In the latter case, even if the user intends to input a rectangle, if the system obtains the recognition result of the likelihood in the order of square, rectangle, and ellipse, the user does not receive feedback. However, there is a problem in that the result is incorrect recognition.

According to the present invention, it is possible to automatically determine the result of the selection of the candidate made by the user and the correction of the interpretation result, and the user can select the candidate even after the set time has passed. Alternatively, if the interpretation result is not corrected, a candidate state can be automatically determined. For example, in an online graphic recognition system, when the user performs a pen input stroke sequence with the intention of inputting a rectangle, the system performs the graphic processing. As a result of performing the recognition processing, a recognition result is obtained in the order of likelihood of a rectangle, a square, and an ellipse, and when the system presents these candidates to the user with a ranking, within a set time of the user If the user does not perform a candidate selection operation, the first candidate rectangle is automatically determined as a recognition candidate. As a result of the system performing the figure recognition processing on the row sequence, if a recognition result is obtained in the order of likelihood of a square, a rectangle, and an ellipse, a rectangle that is a correct candidate within a set time by the user is, for example, " Recognition results can be corrected by saying "rectangle" and selecting, so that the user can concentrate on the original work, and the user can give feedback on the interpretation result of the system I have.

Further, according to the present invention, a user can set a time required for automatically determining a candidate state. In the configuration in which the candidate state is automatically determined as described above, the time from when the candidate is presented to when it is automatically determined is too short for the user, and when the candidate is selected, the time is automatically determined. Also, for another user, the time from when the candidate is presented to when it is automatically determined is too long, and a problem such as an erroneous operation cannot be made without waiting.

According to the present invention, the user can set the time required for automatically determining the candidate state. Therefore, by setting the automatic determination time that each user feels comfortable, the user can set the time. Can concentrate on the original work.

Further, according to the present invention, the time required to determine a candidate state can be automatically set by adapting it to a specific user. When the time required to automatically determine the candidate status is set by the user as described above, each user can set the time required to automatically determine the candidate status by himself / herself. The problem arises that the user has to re-set it many times by trial and error until finding a suitable time for himself.

According to the present invention, the time required to determine the candidate state can be automatically set by adapting it to a specific user. Therefore, the time from when a specific user presents a candidate to when the candidate is selected is selected. Is measured and stored, and an average value of the time series is calculated and set as a time required for a new determination, thereby making it possible to set an automatic determination time according to each user.

[0029]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The information input / output device according to the present invention has a function of allowing a user to make a voice correction of an interpretation result of a user input (for example, a result of a conversion process or a recognition process). , Has a function that allows the user to select a candidate for the interpretation result by voice with respect to the interpretation result of the user's input, and selects the candidate that the user has performed by voice after a set time, Also, it has a function that can automatically determine the result of interpretation result correction, and automatically determines a candidate state if the user does not select a candidate or correct the interpretation result even after the set time has passed It has a function that allows the user to set the time required to automatically determine the candidate state, and adapts the time required to determine the candidate state to a specific user automatically. It has a function that can be set to.

An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a schematic configuration diagram of an information input / output device according to an embodiment of the present invention. As shown in FIG. 1, an information input / output device 101 of the present invention is connected to an input device 102, an information processing device 103, and an output device 104.

The input device 102 performs an input process on an input from a user, and outputs a plurality of input interpretation candidates for each unit of input signal in order. In the example of FIG. 1, the input devices 102a, 102b, and 102c are connected to the input devices 102a, 102b, and 102c.
A multimodal input device for processing a user input signal provided asynchronously from 2c is shown.

The information processing apparatus 103 receives one unit of interpretation information as input information corresponding to one unit of input by the user, performs a predetermined process, and outputs the processing result as output information.

The output device 104 presents output information requested by another device to the user. In the example of FIG. 1, the plurality of output devices 104a, 104b, 104c are connected to the plurality of output devices 104a, 104b, 104c.
3C shows a multi-modal output device that sorts output information and presents it to the user.

In such a configuration, the input device 10
2, the input interpretation candidate set with ranking output from step 2 is received as an input, and a candidate having the highest likelihood as the input content intended by the user is selected from the candidate set and given as an input to the information processing apparatus 103; , Information processing device 103
Is output to the output device 104 and presented to the user.

Here, the information input / output device 101 of the present invention
Are input interpretation candidate provisional output section 11, timer section 12, interpretation information selection / determination section 13, timer set time storage section 14, output integration section 15, timer setting input section 16, fixed time series storage section 17, fixed time calculation -Consists of a setting unit 18.

When the input interpretation candidate provisional output unit 11 receives as input the ordered input interpretation candidate set generated by the input device 102 corresponding to the input from the user, the input interpretation candidate provisional output unit 11 Is output to the output integration unit 15 for the user to select. At this time,
The input interpretation candidate provisional output unit 11 outputs a reset signal to the timer unit 12 and then outputs an input interpretation candidate set to the interpretation information selection / determination unit 13 to transfer control.

When input interpretation candidate provisional output section 11 receives candidate confirmation information indicating that the candidate is confirmed from interpretation information selection / determination section 13, it instructs output integration section 15 to cancel provisional output. I do. However, when the number of elements of the input interpretation candidate set received by the input interpretation candidate provisional output unit 11 is one, that is, when there is no ambiguity in the interpretation result of the input, the input interpretation candidate provisional output unit 11 Does nothing,
Interpretation information selection / determination unit 1 for the received input interpretation candidate set
3 and transfer control.

When the reset signal is output from the input interpretation candidate provisional output unit 11, the timer unit 12 starts the time counting operation from that point. Then, when the time stored in the timer set time storage unit 14 has elapsed since the start of the timing operation, the timer unit 12 outputs a timer signal to the interpretation information selection / determination unit 13 and stops the timing operation. However, before the time stored in the timer set time storage unit 14 has elapsed since the start of the timekeeping operation, that is, before the timer signal is output, the timekeeping stop signal is transmitted from the interpretation information selection / determination unit 13. If it is done, stop the timing operation immediately.

When the input interpretation candidate set is output from the input interpretation candidate provisional output unit 11, the interpretation information selection / determination unit 13 generates a vocabulary corresponding to each input interpretation candidate, and converts the set to a speech recognition target vocabulary set. To the voice recognition device 105 to instruct the start of voice recognition and enter an input waiting state.

The speech recognition result is transmitted to the speech recognition device 105.
If the timer signal is output from the timer section 12 before the output is performed, the interpretation information selection / determination section 13 determines the first input interpretation candidate in the force interpretation candidate set as the input interpretation result. Conversely, before the timer signal is output from the timer unit 12, the speech recognition result is
05, the interpretation information selection / confirmation unit 1
Reference numeral 3 determines an input interpretation candidate (that is, a candidate specified by the user by voice) corresponding to the speech recognition result in the input interpretation candidate set as an input interpretation result, and outputs a timer stop signal to the timer unit 12.

When the input interpretation result is determined, the interpretation information selection / determination section 13 outputs candidate determination information to the input interpretation candidate provisional output section 11, and after the provisional interpretation candidate output section 11 receives the input interpretation candidate set. The time required until the interpretation result is determined, that is, the timer unit 12 when the interpretation result is determined.
Is output to the fixed time series storage unit 17.

The interpretation information selection / determination section 13 instructs the speech recognition device 105 to stop speech recognition, and gives the determined input interpretation result as an input to the information processing device 103. However, if the number of elements of the input interpretation candidate set received by the interpretation information selection / determination unit 13 is one, that is, if there is no ambiguity in the interpretation result of the input, the interpretation information selection / determination unit 13 Does nothing, determines only one element of the received input interpretation candidate set as the input interpretation result,
It is given as an input of the information processing device 103.

The timer set time storage section 14 stores the timer section 1
2 stores in advance the time to be measured,
And outputs the information in response to the request. When a new time to be stored is given from the timer setting input unit 16 or the fixed time calculation / setting unit 18, the timer setting time storage unit 14 cancels the previous time and stores the new time. Remember.

The output integrating section 15 outputs the output information given as a processing result from the information processing device 103 to the output device 104.
Output to Then, when the provisional output instruction is output together with the provisional output information from the input interpretation candidate provisional output section 11, the output integration section 15 continuously outputs the provisional output information until the provisional output cancellation instruction is output next. The output information from the information processing device 103 is integrated and output to the output device 104. For example, if the output device 104 is connected to a display device as the output device 104a, the information processing device 103
The output device 104 integrates the provisional output information in such a manner as to be superimposed on the output information from the device, and instructs the output device 104 to display it on the display device.

When an instruction to cancel provisional output is output from the input interpretation candidate provisional output section 11, the output integration section 15
Canceling the integration of the provisional output information and the output information from the information processing apparatus 103, until the next provisional output instruction is output,
Only the output information from the information processing device 103 is output to the output device 10
4 is output.

The timer setting input section 16 receives a change input of the time required for confirming the input result from the user, and outputs the information (timed time arbitrarily set by the user) to the timer setting time storage section 14. .

Each time there is an ambiguous input from the user, the fixed time series storage unit 17 stores the input interpretation candidate provisional output unit 1
1 is the time required from receiving the input interpretation candidate set to determining the interpretation result, that is, the value of the timer of the timer unit 12 when the interpretation result is determined is output from the interpretation information selection / determination unit 13 and the information is output. In addition to the addition to the fixed time sequence storage unit in the fixed time sequence storage unit 17, the arrival of new fixed time information is output to the fixed time calculation / setting unit 18.

When notified of the arrival of new fixed time information, the fixed time calculation / setting unit 18 sets a set of fixed time information stored in the fixed time series storage unit 17, that is, a certain user For a time from when an input interpretation candidate is presented to when one of the input interpretation candidates is determined as an interpretation result, predetermined processing such as taking an average value is performed based on information obtained up to that point. As a result, a fixed time to be newly set as the timer setting time is generated,
The information is output to the timer set time storage unit 14.

Next, the overall processing flow of the present apparatus will be described with reference to FIG. FIG. 2 is a flowchart showing the overall processing flow. The present invention is a so-called event-driven process, and various process requests are sent to an event queue. Then, the events are taken out one by one from the event queue, the process corresponding to the taken out event is performed, and when the process ends, the process is performed by repeating the process of the main loop of performing the process of the event queue. FIG. 2 shows an outline of the processing performed in the main loop of this processing.

That is, when the process is started (step S0), it is first checked whether there is an unprocessed event in the event queue (step S10). If there is no unprocessed event, it is confirmed whether there is an end request from the user (step S12), and if the user has issued the end request, the processing here ends (step S13). If the user has not issued the end request, the process returns to step S10. Step S10
In step S11, if there is one or more unprocessed events, the first event among them is extracted, and the process proceeds to that event (step S11).

In the process for the extracted event, first, it is verified whether the extracted event is a user event triggered by an input from the user (step S14). If the event is a user event, the process proceeds to a process corresponding to the content.

If it is determined in step S14 that the extracted event is not a user event, it is next verified whether or not the event is a system event (step S15). If the event is a system event, the process proceeds to a process corresponding to the content. Otherwise, the process for the event is not performed, and the process returns to step S10 to process the next event.

Here, in step S14, the events determined to be user events include (a) an arrival event of a set of input interpretation candidates, (b) arrival of a request to change the input confirmation time, and (c) arrival of a speech recognition result. There are three types.

Arrival event of input interpretation candidate set (a)
Is an event generated by the input device 102 in response to an input from the user. If the user event is obtained, the process proceeds to the input interpretation candidate set output process together with the ranked input interpretation candidate set information (step S16).

The arrival event (b) of the input confirmation time change request is an event that occurs in response to the user inputting the time required to determine the input interpretation candidate. If this event is obtained, the process proceeds to the timer setting time setting process together with the time setting information input by the user (step S22).

In the arrival event (c) of the speech recognition result, the vocabulary corresponding to the candidate intended by the user is uttered from the set of interpretation candidates presented to the user, and the speech recognition result for the utterance is obtained. An event that occurs in response to the event. When the user event is obtained, the process proceeds to the interpretation information selection / confirmation process together with the speech recognition result (step S1).
7).

On the other hand, events judged to be system events in step S15 include (d) arrival of candidate confirmation information, (e) arrival of a timer signal, (f) notification of input interpretation candidate provisional output, and (g) Notification of arrival of input interpretation candidates, (h) arrival of reset signal, (i) arrival of clock stop signal, (j)
There are nine types of arrival of output information, (k) arrival of provisional output information, and (l) arrival of provisional output cancellation instruction.

The arrival event (d) of the candidate determination information is an event that occurs when the input interpretation result is determined in the interpretation information selection / determination process (S17). If this event is obtained, the process proceeds to input interpretation candidate provisional output processing (step S16).

The arrival event (e) of the timer signal is an event that occurs when a set time has elapsed since the timer was started. If this event has been obtained, the process proceeds to interpretation information selection / confirmation processing (step S17).

The notification event (f) of the input interpretation candidate provisional output corresponds to the provisional output of the input interpretation candidate set performed in the input interpretation candidate provisional output processing (S16) with the arrival of the input interpretation candidate set. The event that occurs. If this event has been obtained, the process proceeds to interpretation information selection / confirmation processing (step S17).

Notification event of arrival of input interpretation candidate (g)
Indicates that the number of elements of the input interpretation candidate set is counted in the input interpretation candidate provisional output processing (S16) upon arrival of the input interpretation candidate set, and the number of elements is 1, and it is necessary for the user to select and confirm the candidate. This is an event that occurs in response to the absence. If this event is obtained, select interpretation information
The process proceeds to the confirmation process (step S17).

The reset signal arrival event (h) is an event that occurs in response to the arrival of the input interpretation candidate set to instruct the timer to be reset and the start of timing in the input interpretation candidate provisional output process (S16). When this event is obtained, the process proceeds to a timer process (step S18).

The arrival event (i) of the clock stop signal is an event that occurs when the input interpretation result is determined in the interpretation information selection / determination process (S17). When this event is obtained, the process proceeds to the timer process (step S1).
8).

The output information arrival event (j) is an event corresponding to the arrival of the output information to be presented to the user from the information processing device 103. If this event is obtained, the process proceeds to the output integration process ( Step S19).

The arrival event (k) of the provisional output information occurs in response to the provision of the provisional output of the input interpretation candidate set in the provisional output of the input interpretation candidate set (S16) following the arrival of the input interpretation candidate set. Event. When this event is obtained, the process proceeds to the output integration process (step S19).

Arrival event of provisional output cancellation instruction (l)
Is an event that is generated when the arrival event of the candidate confirmation information arrives and is processed in the candidate provisional output process (in step S16) to instruct cancellation of the provisional output of the input interpretation candidate previously performed. When this event is obtained, the process proceeds to the output integration process (step S19).

Next, each of the events (a) to
Various processes activated by (l) will be described. The input interpretation candidate provisional output processing in step S16 is started in response to the arrival event (a) of the input interpretation candidate set or the arrival event (d) of the candidate confirmation information.

When the input interpretation candidate provisional output process (S16) is activated in response to the input interpretation candidate set arrival event (a), first, the number of elements of the input interpretation candidate set is counted. If the number of elements is 0, the input interpretation candidate provisional output process (S
16) ends without doing anything else. When the number of elements is 1, since there is no need to ask the user to select / confirm the candidate, a notification event (g) of the input interpretation candidate arrival is generated, and the input interpretation candidate provisional output processing (S16) ends. When the number of elements is two or more, it is necessary to present the input interpretation candidate set to the user to determine the candidate selection / confirmation. Therefore, the provisional output information arrival event (k) is generated using the input interpretation candidate set as an argument. Then, the input interpretation candidate provisional output process (S16) ends.

The arrival event (d) of the candidate confirmation information
When the input interpretation candidate provisional output processing (S16) is started in response to the above, it is necessary to cancel the provisional output performed previously because the input interpretation result is determined. Therefore, the input interpretation candidate provisional output process (S16) ends after the provisional output cancellation instruction arrival event (l) occurs. When the input interpretation candidate provisional output process (S16) ends, the process proceeds to step S10 to perform the main loop process.

The interpretation information selection / confirmation processing in step S17 includes the notification event (g) of the arrival of the input interpretation candidate, the notification event (f) of the provisional output of the input interpretation candidate, the arrival event (e) of the timer signal, or voice recognition. Fired in response to the resulting arrival event (c).

When the interpretation information selection / confirmation process (S17) is started in response to the notification event (g) of the arrival of the input interpretation candidate, since the number of elements of the input interpretation candidate set is 1, the input interpretation candidate set becomes Has no ambiguity, and there is no need to present candidates to the user and perform selection / confirmation processing. In this case, in the interpretation information selection / determination processing (S17), only one element of the given input interpretation candidate set is determined as the input interpretation result, and the input interpretation result is provided to the information processing apparatus 103 as an input. Then, an arrival event (i) of the clock stop signal is generated, and the interpretation information selection / determination process (S17) ends.

Further, in response to the notification event (f) of the provisional output of the input interpretation candidate, the interpretation information selection / confirmation processing (S1)
When 7) is activated, there are two or more input interpretation candidates in the input interpretation candidate set, and it is necessary to determine the input interpretation result intended by the user from those candidates. For this purpose, the input interpretation candidate set has been provisionally presented to the user by this point. At this time, in the interpretation information selection / confirmation process (S17), an arrival event (h) of the reset signal is generated to start the timer, and the input interpretation candidate is selected so that the user can select the input interpretation candidate by voice. By generating vocabulary information corresponding to each input interpretation candidate in the set, a vocabulary set for speech recognition is generated, notified to the speech recognition apparatus 105, and instructed to start speech recognition processing. Thereafter, the interpretation information selection / confirmation process (S17) ends.

When the interpretation information selection / confirmation process (S17) is activated in response to the timer signal arrival event (e), the timer is activated before the user selects an input interpretation candidate by voice. This means that the elapse of a predetermined time has been detected. In this case, interpretation information selection / confirmation processing (S17)
Then, a predetermined input interpretation candidate among elements of the input interpretation candidate set, that is, an input interpretation candidate with the highest likelihood is determined as an input interpretation result, and the input interpretation result is provided to the information processing apparatus 103 as an input. At the same time, the interpretation information selection / confirmation processing (S1
7) generates an arrival event (i) of a clock stop signal and an arrival event (d) of candidate confirmation information, and instructs the speech recognition device 105 to stop speech recognition processing. afterwards,
The interpretation information selection / confirmation process (S17) ends.

The arrival event (c) of the speech recognition result
When the interpretation information selection / confirmation process (S17) is started in response to the above, before the timer detects the elapse of a predetermined time,
This means that the user has selected an input interpretation candidate by voice. In this case, in the interpretation information selection / confirmation processing (S17), the input interpretation candidate corresponding to the vocabulary obtained as the speech recognition result is determined as the input interpretation result, and the input interpretation result is provided to the information processing apparatus 103 as an input. At the same time, the interpretation information selection / confirmation process (S17) generates an arrival event (i) of the clock stop signal and an arrival event (d) of the candidate confirmation information, and instructs the speech recognition device 105 to stop the speech recognition process. . Thereafter, the interpretation information selection / confirmation process (S17) ends. Interpretation information selection / confirmation processing (S1
When step 7) is completed, it is determined whether or not the input interpretation result has been determined, and if it has been determined, whether or not it has been performed on an input interpretation candidate set having two or more elements (step S23).

Here, when the input interpretation result is determined from a plurality of elements, that is, the arrival event (e) of the timer signal or the arrival event (c) of the speech recognition result
When the interpretation information selection / confirmation process (S17) is started and terminated in response to the above, the process proceeds to a confirmation time series storage process (step S20). Otherwise, the process proceeds to step S10 to perform the main loop process.

In the fixed time series storing process in step S20, the time required to determine the input interpretation result, that is, the value of the timer at that time, is obtained and added to a list-like storage area called fixed time series storage. Then, the process proceeds to the fixed time calculation process (step S21).

The determination time calculation processing in step S21 is to record the time until the user or the system selects an input interpretation candidate and determines the input interpretation candidate in each input having an ambiguity stored in the determination time series memory. On the basis of the above, a predetermined calculation process, for example, a process of taking an average value of all the recorded times, is performed, a new determination time is set, and the process proceeds to a timer setting time setting process (step S22).

The timer setting time setting process of step S22 is started in response to the input fixed time change request arrival event (b), or a new setting of the input fixed time in the fixed time calculation process is determined. After that, the program is started (step S22). In any case, in the timer setting time setting process (S22), a new input interpretation candidate determination time value is given, and the new input interpretation candidate value is written to a storage area called input determination time storage. Thereafter, the timer set time setting process ends, and step S1 is performed to perform the main loop process.
It proceeds to the process of 0.

The timer processing in step S18 is started in response to either the arrival event (h) of the reset signal or the arrival event (i) of the clock stop signal. When the timer process (S18) is started in response to the reset signal arrival event (h), first, the current time of the timer is reset to 0, and the value stored in the fixed time storage is obtained. When the current time of the timer reaches that value, an arrival event of the timer signal is set to occur, and then the timer unit 12 starts measuring time. Thereafter, the timer process (S18) ends, and the process proceeds to step S10 in order to perform a main loop process.

When the timer processing (S18) is started in response to the arrival event (i) of the clock stop signal, the timer processing (S18) stops the operation of the timer section 12. Thereafter, the timer process (S18) ends, and the process proceeds to step S10 in order to perform a main loop process.

The output integration process of step S19 is started in response to the arrival event (j) of the output information, the arrival event (k) of the provisional output information, or the arrival event (l) of the provisional output cancel instruction.

When the output integration process (S19) is activated in response to the provisional output information arrival event (k), it is necessary to tentatively present the input interpretation candidate set to the user and perform the selection / confirmation process. Means that it happened. In this case, in the output integration process (S19), provisional output information to be presented to the user is generated from the input interpretation candidate set, stored in the provisional output storage unit, and output information in which the output information and the provisional output information are integrated is generated. And outputs it to the output device 104.

When the output integration processing (S19) is started in response to the output information arrival event (j), it means that the output information has arrived from the connected information processing apparatus 103. In this case, in the output integration processing (S19), it is necessary to generate new output information. For this reason, the output integration process (S19) stores the new output information in the output information storage unit, and if the information to be provisionally output is stored in the provisional output storage unit, the newly obtained output information and the provisional output information are temporarily stored. The output information is integrated to generate new output information, and the output device 10
4 is output. If the provisional output storage unit is empty, new output information is output to the output device 104 as output information as it is.

When the output integration processing (S19) is activated in response to the provisional output cancellation instruction arrival event (l), it means that provisional output information need not be presented to the user. In this case, in the output integration process (S19), the provisional output storage unit is emptied, new output information is generated from the information stored in the output information storage unit, and the output device 104
Output to

The above is the configuration and function of the information input / output device according to the present invention, and its processing method. Here, the online handwritten figure recognition system as shown in FIG. 3 will be described in more detail by way of example. In this example, as the input device 102a and the output device 104a in FIG. 1, a user inputs a handwritten stroke on a screen using a stylus pen or the like, for example, a liquid crystal pen tablet, and the input stroke is output as it is. Device is assumed. The information processing apparatus 103 is assumed to be a drawing creation apparatus in which a user inputs a graphic in order to create a drawing such as a design drawing.

When the user inputs a handwritten figure on the screen using a pen or the like as shown in FIG. 4, the input device 102 performs a recognition process on the stroke, and the following recognition result candidate figure and its A list in which the triples of the feature point sequence and the score of the recognition result are arranged in descending order of the score is generated as an output, and is provided as an input to the information input / output device 101 according to the present invention.

[Square, (100, 100, 200,
200), 0.94] • [rectangle, (97, 105, 204, 197), 0.
88] ・ [Parallelogram, (97, 105, 196, 105, 2)
04, 197, 89, 197), 0.82] The recognition candidate sequence (input interpretation candidate) from the input device 102 supplied to the information input / output device 101 is first supplied to the input interpretation candidate provisional output unit 11. Here, since there are a plurality of input interpretation candidates, the input interpretation candidate provisional output unit 11 superimposes each candidate in the original stroke format with a different color, for example, so that the user can select the candidate. A provisional output such as a format to be displayed is generated, and this is output to the output integration unit 15.

At this time, the input interpretation candidate provisional output unit 1
1 outputs a reset signal to the timer unit 12, and outputs, for example, [square, rectangle, parallelogram, upper, middle, lower, first, second based on given recognition interpretation candidates and generated temporary output information. , Third, first, last], a vocabulary expression set for speech recognition for limiting the vocabulary for recognizing speech input from the user is generated and delivered to the speech recognition device 105, and the interpretation information selection / determination unit Pass control to 13.

The timer section 12 has a timer set time storage section 1
The timer setting time information (for example, information of 5 seconds) stored in No. 4 is read out to set a timer, and the timer starts counting time. The output integration unit 15 integrates the output of the information processing device 103 as shown in FIG. 3 and the provisional output sent thereto, and outputs the integrated output to the output device 104. As a result, for example, as shown in FIG. 5, a menu listing the interpretation candidates in the order of their scores is superimposed on the output of the information processing apparatus 103 and displayed.

Here, before the set timer time is counted by the timer section 12, the user can select each interpretation candidate such as "up", "middle", or "parallelogram". When a word is uttered, the utterance of the user is subjected to recognition processing by the speech recognition device 105, and the result is given to the interpretation information selection / determination unit 13.

For example, when the word uttered by the user is recognized as “middle”, the interpretation information selection / determination unit 13 outputs the speech recognition information and the input interpretation candidate information passed from the input interpretation candidate provisional output unit 11. To identify that the user has selected “Rectangle” which is a candidate in the center of the menu. As a result, the interpretation information selection / determination unit 13 determines the “rectangle” as the figure recognition result, the input interpretation candidate provisional output unit 11 has the candidate determination information, the timer unit 12 has the clock stop signal, and the information processing device 103 has determined it. Information related to input [rectangle, (97, 105, 209, 197)]
Are output.

The input interpretation candidate provisional output unit 11 that has received the candidate determination information instructs the output integration unit 15 to cancel the provisional output, and in response, the output integration unit 15 cancels the candidate display by the menu. Further, the information processing apparatus 103 that has received [rectangle, (97, 105, 204, 197)] as the determined input interpretation result changes the output. As a result, as shown in FIG. 7, a rectangle intended by the user is input to the drawing creating apparatus.

As a method of tentatively outputting the input interpretation candidate, the interpretation candidate is superimposed on the original input stroke screen.
For example, "square" candidates are blue, "rectangles" are red,
FIG. 6 shows an example in which “parallelograms” are displayed in yellow with different colors. In such a case, the user may select a candidate by uttering “blue”, “red”, “yellow” in addition to uttering words such as “square”, “rectangle”, and “parallelogram”. it can.

As described above, the set of vocabulary expressions to be recognized, which is a set of vocabulary expressions to be subjected to the speech recognition, is generated by the input interpretation candidate provisional output unit 11.
The information is transmitted to the voice recognition device 105.

If the user selects a candidate before the timer measures the set time, a timer stop signal is output to the timer section 12 to stop the timer. Then, select interpretation information
The determination unit 13 receives the value of the timer from the timer unit 12,
The time required for input determination after the user is presented with the candidate is read out and given to the determined time series storage unit 17. The given latest determination time is added to the determination time series storage unit 17, the determination time calculation / setting unit 18 calculates a new determination time, and causes the timer setting time storage unit 14 to store the time. As a result, the time required for automatic determination from the next time changes according to the user.

Next, let us consider a case where when the menu shown in FIG. 5 is presented to the user, the square that is the first candidate is a figure intended by the user. In such a case, the user can select a candidate for “square” by saying “square” or “up”, but it can also be left without performing such a determination operation .

When the user leaves the apparatus without performing the determination operation and the timer measures the set time, the timer section 12 outputs a timer signal to the interpretation information selection / determination section 13. The interpretation information selection / determination unit 13 receives the information, determines the first candidate, inputs candidate determination information to the input interpretation candidate provisional output unit 11, and information related to the input determined to the information processing apparatus 103 [square, (100, 100, 200, 200)].

Upon receiving the candidate confirmation information, the input interpretation candidate provisional output unit 11 instructs the output integration unit 15 to cancel the provisional output, and in response, the output integration unit 15 cancels the candidate display using the menu. Further, [square, (100, 100, 200, 200)]
The information processing device 103 that has received the command changes the output. As a result, the square intended by the user is input to the drawing creation device.

In the drawing creating apparatus configured as described above,
Since the automatic determination time of the interpretation candidate can be automatically changed according to the user, for example, a long automatic determination time is maintained for a beginner, and errors in input confirmation can be reduced. Conversely, when the operation is performed by a skilled person, the automatic determination time is automatically shortened, and efficient input is possible.

Thus, according to the information input / output device and method of the present invention, it is possible for the user to make a voice correction of the interpretation result of the user's input. Furthermore, according to the present invention, it is possible for a user to select a candidate for an interpretation result by voice with respect to the interpretation result of a user input.

Further, according to the present invention, it is possible to automatically determine the result of the selection of the candidate made by the user by voice and the correction of the interpretation result. Furthermore, according to the present invention,
If the user does not select a candidate or correct the interpretation result even after the set time has elapsed, a certain candidate state can be automatically determined.

Further, according to the present invention, it is possible for the user to set the time required for automatically determining the candidate state. Further, according to the present invention, it is possible to automatically set the time required to determine the candidate state by adapting it to a specific user.

The present invention is not limited to the above embodiment. That is, in the above-described embodiment, as a method of feeding back the input of the user to the subsequent input, only the method of automatically adjusting the time required for automatically determining the interpretation candidate is described. When the user specifies an interpretation candidate selection result, that is, an interpretation other than the first candidate as the input candidate, as a correct interpretation result, the information is returned to the input device 102 and the input device 102 performs a learning process. This makes it possible to bias an input device 102 candidate method and to configure an input / output system more suitable for the user.

In the following, a device configuration in the case where the processing in the information input / output device is realized using software will be described with reference to FIG. In this case, the hardware portion of the information input / output device includes a CPU 21, a RAM 22 for storing programs and necessary data, a disk drive device 24, a storage device 25, and an input / output device 2.
6.

Also, the input interpretation candidate provisional output unit 1 shown in FIG.
1, a timer unit 12, an interpretation information selection / determination unit 13, a timer setting time storage unit 14, an output integration unit 15, a timer setting input unit 16, a determination time series storage unit 17, and a determination time calculation / setting unit 18 It is composed of a program describing the processing procedure, and the processing procedure combining them is as shown in FIG.

A program describing this processing procedure is stored in the RAM 22 as a program for controlling the computer system shown in FIG. The CPU 21 performs calculations and controls the storage device 25 or the input / output device 26 according to the procedure of the program stored in the RAM 22 to realize desired functions.

Various methods can be used to install the program in the RAM 22. For example, the program (input interpretation candidate provisional output unit 11, timer unit 1
2. A program describing the processing procedure of the interpretation information selection / determination unit 13, timer setting time storage unit 14, output integration unit 15, timer setting input unit 16, determination time series storage unit 17, and determination time calculation / setting unit 18. Then, a program for controlling the computer system is stored in a computer-readable storage medium (for example, a removable storage medium such as a floppy disk or a CD-ROM). Then, as shown in FIG. 8, the program is read using the disk drive device 24 corresponding to the storage medium and stored in the RAM 22. Alternatively, once installed in the disk drive device 24 or the like,
At the time of execution, it is stored in the RAM 22 from the same device.

Further, the storage medium storing the program is I
In the case of a C card, the program can be read using an IC card reader. Further, a program can be received from a predetermined interface device via a network. The present invention is not limited to the above-described embodiment, and can be implemented with various modifications within the technical scope thereof.

[0109]

As described above, according to the present invention, the interpretation result of the input device that accepts and interprets an ambiguous input can be corrected by the user using the speech. The user can select a result candidate by voice.

Further, according to the present invention, it is possible to automatically determine the result of the selection of the candidate made by the user and the correction of the interpretation result, and the user is able to determine the result even after the set time has elapsed. If the user does not select a candidate or correct the interpretation result, a certain candidate state can be automatically determined.

According to the present invention, the user can set the time required for automatically determining the candidate state.
The time required to determine the candidate state can be automatically set in accordance with the specific user.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a schematic configuration of an information input / output device according to an embodiment of the present invention.

FIG. 2 is a flowchart showing the overall processing flow.

FIG. 3 is a diagram showing an external configuration of an online graphic recognition system.

FIG. 4 is a diagram showing an input handwritten stroke.

FIG. 5 is a view showing a menu selection screen for interpretation candidates.

FIG. 6 is a view showing an interpretation candidate superimposed display selection screen.

FIG. 7 is a diagram showing a screen when an input is determined.

FIG. 8 is a block diagram showing a hardware configuration of the information input / output device of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 101 ... Information input / output device 102a-102c ... Input device 103 ... Information processing device 104a-104c ... Output device 105 ... Speech recognition device 11 ... Input interpretation candidate provisional output unit 12 ... Timer unit 13 ... Interpretation information selection / determination unit 14 ... Timer setting time storage unit 15 ... Output integration unit 16 ... Timer setting input unit 17 ... Definite time series storage unit 18 ... Definite time calculation / setting unit

Claims (9)

[Claims] 1. An input means for processing an input signal from a user, assigning a plurality of interpretation candidates as a result of the processing and outputting the result, and selecting one of the interpretation candidates obtained by the input means. Information processing means for processing the candidates as interpretation information and outputting the processing result as output information; output means for presenting the output information obtained by the information processing means to a user; and for recognizing voice information. An input / interpretation candidate set provisional output means for generating provisional presentation information for allowing a user to select a candidate from among the respective interpretation candidates obtained by the input means. Timer signal generating means for starting a timing operation when the provisional presentation information is output from the input interpretation candidate provisional output means, and for generating a timer signal after a lapse of a predetermined time. At the timing when the speech information for selecting each of the interpretation candidates is input to the speech recognition means, or at the time when the timer signal is output by the timer signal generation means, Means for selecting a candidate specified by the voice information or a predetermined candidate by the generation of the timer signal, and providing the selected candidate as the interpretation information to the information processing means; and an interpretation information selection / determination means. And output control means for providing the output information, which is output as a result of processing the interpretation information given by the information processing means, to the output means as new output information to a user. Information input / output device. A timer setting input means for arbitrarily setting a clock time; and a timer setting storage means for storing the clock time set by the timer setting input means. 2. The information input / output device according to claim 1, wherein when the provisional presentation information is output, the timer signal is generated based on the measured time stored in the timer setting storage means. 3. A fixed time sequence storage means for measuring a time required until the interpretation information is determined by the audio information or the timer signal after outputting the provisional presentation information, and storing the sequence. A determination time calculating means for calculating an appropriate determination time for a user of the apparatus from a set of times required for the determination stored in the determination time series storage means; and a determination time calculated by the determination time calculation means. Timer setting storage means for storing the time as a clock time, wherein the timer signal generation means, based on the clock time stored in the timer setting storage means when the provisional presentation information is output, The information input / output device according to claim 1, wherein the information input / output device generates a signal. 4. The multi-modal input device according to claim 1, wherein said input means is connected to a plurality of input devices and processes a user input signal asynchronously given from said input devices. Information input and output device. 5. The information according to claim 1, wherein said output means is a multi-modal output device connected to a plurality of output devices, for allocating output information to the output devices and presenting the output information to a user. I / O device. 6. The input interpretation candidate set provisional output means,
A speech recognition target vocabulary set for generating a set of vocabulary expressions acceptable as speech information for selecting an interpretation candidate from a user from each input interpretation candidate obtained from the input means and a provisional output method of the input interpretation candidate. A speech recognition target vocabulary that can be passed to the speech recognition unit as a speech recognition target vocabulary expression information that can limit a vocabulary expression for the generation function and the speech recognition unit to perform a speech input recognition process from a user 2. The information input / output device according to claim 1, having an expression transmitting function. 7. An input means for processing an input signal from a user, assigning a plurality of interpretation candidates as a result of the processing and outputting the result, and selecting one of the interpretation candidates obtained by the input means. Information processing means for processing the candidates as interpretation information and outputting the processing result as output information; output means for presenting the output information obtained by the information processing means to a user; and for recognizing voice information. In the information input / output method of the information input / output device provided with the voice recognition means, the provisional presentation information for causing the user to select a candidate from among the interpretation candidates obtained by the input means is generated. When the provisional presentation information is output, a timer operation is started, a timer signal is generated after a lapse of a predetermined time, and the voice information for selecting each of the interpretation candidates is input to the voice recognition means. When, or at the timing when the timer signal is output, a candidate specified by the audio information or a predetermined candidate is selected by the generation of the timer signal from among the interpretation candidates, and The output information is provided to the information processing means as interpretation information, and the output information output as a result of processing the interpretation information by the information processing means is provided to the output means as new output information to a user. Information input / output method. 8. The timekeeping time can be arbitrarily set, and the timekeeping time is stored in a memory. When the provisional presentation information is output, the timekeeping time is set based on the timekeeping time stored in the memory. 8. The information input / output method according to claim 7, wherein said timer signal is generated. 9. After the provisional presentation information is output, the time required until the interpretation information is determined by the audio information or the timer signal is measured each time, and the series is determined as a first series.
The determination time appropriate for the user of the apparatus is calculated from the set of the time required for determination stored in the first memory, and the determined time is used as the clocking time in the second memory. 8. The timer signal according to claim 7, wherein when the provisional presentation information is output, the timer signal is generated based on the clocking time stored in the second memory. Information input / output method.