JPH09114579A - Pointing calling confirmation system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To lighten the burden on an operator who answers whether or not screen operation is proper or improper by automatically judging the confirmation of the screen operation by pointing calling as to a monitor and control system to which a voice input device and a pointing device are connected. SOLUTION: A terminal controller 12 has a speech recognition part 21, a touch sensor detection part 24, and a process decision part 27 and the speech recognition part 21 has a voice input buffer 22 and a voice input information management table 23; when a figure on a screen is selected with a pointing device such as a touch screen, the words voiced by the operator are compared with words registered previously in the voice input information management table 23 to answer whether the operator's operation is proper or improper.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a pointing and pointing confirmation system, and in particular, it is suitable for a supervisory control system having a voice input device having a voice recognition and a pointing device and operating a screen while pointing and calling. The present invention relates to a pointing and calling confirmation system.

[0002]

2. Description of the Related Art In a recent plant supervisory control system, the start / stop of the plant and the setting of various control parameters are controlled by a display device and an input device.
Performed by RT device. By using the CRT device as described above, the plant operation can be performed by a simple operation, and it largely contributes to the labor saving of the plant monitoring control system. However, the simple operation has a drawback that an erroneous operation is likely to occur due to an operator's misunderstanding and erroneous recognition. Therefore, in order to prevent the above-mentioned erroneous operation, usually, the pointing operation, that is, the operator himself or another operation in which he / she is working jointly by uttering the operation content while pointing at the figure displayed on the screen. I am trying to get confirmation from the staff.

If the above-mentioned confirmation of pointing and pointing can be automated by a computer, it is possible to contribute to improvement of operation reliability and labor saving.

As a publicly known example relating to these, Japanese Patent Laid-Open No.
-175477 publication.

[0005]

Problems to be solved when constructing a plant monitoring control system by using the voice recognition technology as described above include the following two points.

[0006] First, in the current voice recognition technology, there is a problem that erroneous recognition occurs at a certain probability, which is a factor that hinders its application in a plant monitoring control system where erroneous operation has a great influence.

Another problem is that a plant monitoring and control system that requires a large amount of processing for voice recognition and requires real-time operation requires higher-speed and higher-performance hardware.

An object of the present invention is to provide a pointing and calling confirmation system with high reliability and small processing amount.

[0009]

A feature of the present invention for achieving the above object is that it has voice input means, pointing input means, and graphic display means, and stores voice input data in time series by encoding. The graphic information table for storing the graphic information in the voice buffer and the graphic information table has an associated voice data table, and the graphic information table stores the processing performed when the graphic is selected by pointing. Which has a next operation list, which indexes the audio data table associated with the pointed graphic, and outputs the result of the judgment of the coincidence with the audio data in the audio buffer as a voice or a display graphic according to the contents of the next operation list. To do.

Further, the content of the voice data table relating to the pointed figure and the content of the voice buffer within a certain time before and after the pointed time may be determined to be coincident with each other.

That is, in the plant monitoring and control system, the figure on the screen selected by the operator and the words to be called when the operator selects the figure on the screen are inevitably determined. Therefore, the number of vocabularies can be reduced by registering only the words that the operator should call for voice recognition in the system.

Further, by registering in advance a list in which the graphic on the screen and the voice corresponding to the graphic are to be associated with the system, it is registered in advance when the graphic on the screen is selected by the operator. The operator's operation can be confirmed by comparing whether the said word was uttered by the operator and responding the result.

According to this method, since only the words to be called when the operator selects a figure on the screen are registered in the system, it is possible to reduce the capacity less than that required for normal voice recognition, and to use the finger. It is only necessary to determine whether or not the operator has issued a pre-registered word at the time of the differential call, and the processing amount for voice recognition can be reduced.

Further, when the operator selects a figure on the screen, it is performed by a plurality of operators in order to compare whether or not a pre-registered word is uttered by the operator and respond the result. You can check the operation by yourself.

[0015]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 shows the system configuration of an embodiment of the present invention. The terminal controller 12 that controls input / output of the input device and the screen display inputs the touch screen 11 for the operator to directly touch and input the screen, the display device 15 for displaying the screen to which the touch screen is added, and the voice information of the operator. A voice input device 13 for outputting and voice output device 14 for outputting voice information and a warning sound are connected.

When the figure selected by touch input on the touch screen 11 and the predetermined voice information from the voice input device 13 are input, the processing when the figure is first selected is executed, for example, a new screen is displayed. If there is a mismatch, the audio output device 14 outputs a warning sound.

FIG. 2 shows the internal structure of the terminal controller 12. A voice recognition unit 21 including a voice input buffer 22 that stores voice data obtained by A / D converting voice data from the voice input device 13 and a voice input information management table 23 that stores registered voice data in advance. , The touch position management table 25 in which the screen display graphic and the graphic area are stored and the graphic selected by the operator are determined based on the input coordinates (x, y) sent from the touch screen 11 when a touch input is made. Touch sensor detector 2
4, a timer 26, a process determination unit 27 that determines the process to be executed next based on the information sent from the voice recognition unit 21 and the touch sensor detection unit 24, a voice output unit that repeats voice input information and generates a warning sound 28, and an image display controller 29 for displaying a figure.

FIG. 3 shows a touch position management table of the touch sensor detector 24. Touch position management table 2
5 has table information for each screen, and is composed of a screen No., a graphic No. of a graphic drawn on the screen to be selected by the operator, and a display area of the graphic. The display area of the graphic registered here consists of (x, y), width, and height of the graphic, so that the graphic can be recognized even if the touch position is slightly deviated from the actual graphic due to parallax at the time of touch input. The value of the effective area which is slightly larger than the actual figure display position and size is stored.

While the screen 31 on which a plurality of figures are drawn is displayed, when the figure 32 drawn on the screen 31 is selected by the touch input on the touch screen 11 by the operator, the touch screen is displayed. The input coordinates (x, y) are sent from 11 to the touch sensor detection unit 24.

The touch sensor detection unit 24 to which the input coordinate information has been sent searches for the selected figure No. from the touch position management table 33 from the sent information and the currently displayed screen No. The detected figure No. is sent to the voice recognition unit 27.

FIG. 4 shows a voice input information management table of the voice recognition unit 21. The voice input information management table 23 is
Voice registration table 41 for each screen and voice data table 42
Consists of

The screen-by-screen voice registration table 41 has a table for each screen, and is used as a screen No., the number of registered voice input information, and a figure used as a voice input registration information to determine the conformity between the figure and the information of voice data. The voice data table 42 stores voice data in the form of a word or a simple sentence together with the voice No ..

When the figure No. touch-inputted by the touch sensor detection section 24 is sent to the voice recognition section 21, it is selected from the voice registration table for each voice of the voice input information management table 23 based on the figure No. A voice No. that should correspond to the figure is searched, voice data is acquired from the voice data table 42 based on the detected voice No., and the acquired voice data is A / D converted from the voice input from the voice input device 13. The post-speech analysis is performed, and it is determined whether or not there is a portion that matches the voice data stored in the voice input buffer 22, and the result is sent to the processing determination unit 27.

FIG. 5 shows a voice-graphics next operation determination table 51 used for determining the next processing when the voice input information and the graphic selected by touch input match in the processing determination unit 27. Voice-figure next motion determination table 5
1 is a screen No., a figure No. that the operator should select on the screen No., a voice No. of a voice that the operator should say when selecting the figure, and a figure selected by the operator. It stores the next action to be taken when the word spoken at that time matches.

When the processing judgment unit 27 receives the notification of the conformity between the selected figure and the uttered word from the voice recognition unit 21, the screen N is displayed.
Based on o., voice No., and graphic No., the next process is searched by the voice-graphic next operation determination table 51, and the detected process is executed.

FIG. 6 shows a processing flow of this embodiment. When the operator selects a figure on the screen, the touch screen 11
More input coordinate information (x, y) is touch sensor detection unit 2
Sent to 4. In the touch sensor detection unit 24, the touch screen 11 is selected from the touch position management table 25.
Figure N containing the input coordinates (x, y) sent from
When o is detected (61), the figure No. is sent to the voice recognition unit 21.

Upon receipt of the figure No., the voice recognition section 21 confirms whether voice data is stored in the voice input buffer 22 (62). When there is no voice data in the voice input buffer 22, the determination processing unit 27 is immediately notified that there is no voice input. When the voice data is stored in the voice input buffer 22, the voice No. stored in the screen voice registration table 41 is searched from the screen No. and the figure No., and the voice data is based on the detected voice No. Table 42
Get audio data from. It is determined whether the acquired voice data is included in the voice data stored in the voice input buffer (63), and the determination result and the screen No., figure No., and voice No. used for the determination are processed and determined. Send to department 27.

When the processing judgment section 27 receives the report from the voice recognition section 21, when it receives the message that the selected figure and the utterance at that time match, the recognized voice data is sent to the voice output device 14. The voice input information is reproduced by using the voice input information, and the next motion processing is searched by the voice-graphic next motion determination table 51 based on the screen No., the voice No., and the graphic No., and the process is performed (64). Further, when a report indicating no voice input or nonconformity is received, an error is reported to the user by a warning sound or the like (65).

FIG. 7 shows a determination processing flow based on input information within a fixed time. That is, the touch input and the utterance of the operator are slightly different in timing such that the utterance is performed first or the touch input is performed first. Therefore, it is necessary to perform a process of determining whether or not the touch input information and the voice information detected during a certain time period before and after the touch input detection are compatible, and the flow thereof is shown in FIG. 7.

When the operator selects a figure on the screen, the input coordinate information (x, y) is sent from the touch screen 11 to the touch sensor detecting section 24. Touch sensor detector 2
4, the figure No. including the input coordinates (x, y) sent from the touch screen 11 is detected from the touch position management table 25 (61), and the voice recognition unit 21 is detected.
Send figure No. to.

Further, the voice recognition unit 21 holds the time when the voice data obtained by A / D converting the voice data from the voice input device 13 and storing the voice data is stored, and the touch sensor detection unit 24 displays the figure No. When received, it is confirmed whether voice data is stored in the voice input buffer 22 (6
2). If the time stored in the voice input buffer was stored before a certain time (for example, 3 seconds) before the current time, and if there is no voice data in the voice input buffer 22, a certain time (for example, 3 seconds). (Second) Wait for voice input (71).

When the voice data is stored in the voice input buffer 22, the voice No. stored in the voice registration table 41 for each screen is searched from the screen No. and the figure No., and the detected voice No. is used as the basis. The voice data is acquired from the voice data table 42. It is determined whether the acquired voice data is included in the voice data stored in the voice input buffer (63), and the determination result and the screen No., figure No., and voice No. used for the determination are processed and determined. Send to department 27.

When the process judging section 27 which has received the report from the voice recognizing section 21 receives the message that the selected figure and the utterance at that time match, the recognized voice data is sent to the voice output device 14. The voice input information is reproduced by using the voice input information, and the next motion processing is searched by the voice-graphic next motion determination table 51 based on the screen No., the voice No., and the graphic No., and the process is performed (64). Also, when a report indicating no voice input or nonconformity is received within a certain period of time, an error is reported to the user by a warning sound or the like (65).

According to this embodiment, since the operator registers only the words to be called, the number of registered vocabularies can be suppressed, and the voice information to be issued by the operator is retrieved from the figure selected by the operator, Since it is determined whether the same information as the voice information is included in the voice input buffer 22, the processing time by voice recognition is shortened, and the operator can confirm the operation alone to notify the determination result.

As described above, by connecting the figure to be selected by the operator and the word to be spoken at that time by pointing and calling, the number of registered vocabulary in the system for voice recognition can be suppressed to a small number. This has the effect of preventing an increase in the processing amount for voice recognition.

Next, since the operator confirms the operation of the operator on the system side, one operator can confirm the operation performed among a plurality of operators, which has the effect of reducing the burden on the operator. In addition, since the result judged by the machine side can be confirmed by hearing again, "visual inspection" and "operation" are more reliable because the interface with the operator is performed rather than "auditory" of another system. Improvement can be achieved.

Further, there is a slight time lag between the voice input and the input by the pointing on the screen, such that the voice is input before the pointing and the voice is input after the pointing, and the input is not always performed at the same time. Since there is no limitation, it is possible to eliminate the time lag in such a case.

[0039]

According to the present invention, it is possible to provide a highly reliable pointing and pointing confirmation system with a small processing amount.

[Brief description of the drawings]

FIG. 1 is a diagram showing a system configuration of an embodiment of the present invention.

FIG. 2 is a diagram showing an internal configuration of a terminal controller according to the embodiment of the present invention.

FIG. 3 is a diagram showing an example of a touch position management table according to the embodiment of the present invention.

FIG. 4 is a diagram showing an example of a voice input information management table according to the embodiment of the present invention.

FIG. 5 is a diagram showing an example of a voice-graphics next operation determination table for matching search of voice information and selected graphic information according to the embodiment of the present invention.

FIG. 6 is a diagram showing a basic processing flow according to the embodiment of the present invention.

FIG. 7 is a diagram showing a determination processing flow according to input information within a fixed time according to the embodiment of this invention.

[Explanation of symbols]

11 ... Touch screen, 13 ... Voice input device, 21 ...
Voice recognition unit, 22 ... Voice input buffer, 23 ... Voice input information management table, 24 ... Touch sensor detection unit, 25
... touch position management table, 51 ... voice-figure next operation determination table.

Claims (2)

[Claims] 1. A voice buffer having voice input means, pointing input means, and graphic display means, a voice buffer for encoding and storing voice input data in time series, a graphic information table for storing graphic information being displayed, and the graphic. The information table has a related voice data table, and the figure information table has a next operation list in which the processing to be performed when a figure is selected by pointing is stored, and the voice related to the pointed figure is stored. A pointing and calling confirmation system characterized in that a data table is indexed, and the result of matching judgment with the voice data in the voice buffer is output as voice or a display graphic according to the contents of the next operation list. 2. The finger according to claim 1, wherein the content of the voice data table related to the pointed figure and the content of the voice buffer within a fixed time before and after the pointed time are determined to be coincident with each other. Difference name confirmation system.