JPH09265378A - Supporting method and supporting system for operator - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a supporting method and device capable of judging the mental state of an operator for performing operations by voice input and performing operator supporting suited to the mental state. SOLUTION: In a voice recognition processing part 3, the pattern of the instruction word of a plant operation voice inputted beforehand is registered for each operator. During the operation, for a voice instruction word from a microphone 6, the sound pressure waveform is fetched through a data conversion part 2 to the voice recognition processing part 3 and a maximum sound pressure, a highest frequency and shortest sounding time are fetched to an operator state judgement processing part 4. The voice recognition processing part 3 divides the sound pressure waveform and performs spectrum conversion for respective sound pressure data and performs the voice recognition of the instruction word by pattern matching. Also, the operator is specified by speaker recognition. The operator state judgement processing part 4 compares the inputted maximum sound pressure, highest frequency and shortest sounding time with the registration data of the specified operator respectively and judges the mental state. A supporting processing output part 5 executes a supporting processing corresponding to instruction contents when the mental state of the operator is 'abnormal'.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an operator support system by voice input, and more particularly to a support system based on judgment of an operator's psychological state.

[0002]

2. Description of the Related Art A conventional voice input driving support system, as described in Japanese Examined Patent Publication No. 62-50850, changes the speed of voice response output according to the sounding time of voice input by an operator, or a special feature. As described in Japanese Patent Publication No. 62-48855, there is known one that performs input / output processing according to the skill level of an operator, such as automatically stopping operator guidance by voice if the working time is within a predetermined time.

[0003]

In the above-mentioned prior art, the experienced operator is quickly and directly responded to eliminate the inconvenience and the processability is improved. However, shortening of the pronunciation time etc. also occurs when the operator is in an abnormal psychological state, so in such a case, shortening the response time or omitting the guidance may lead to an operator in an abnormal state. Causes further anxiety and anxiety, which increases the possibility of erroneous operation.

It is an object of the present invention to provide a support method and device for determining the psychological state of an operator who operates by voice input and enabling the operator assistance suitable for the psychological state.

[0005]

SUMMARY OF THE INVENTION It is an object of the present invention to provide a method for assisting an operator who operates equipment such as a plant by voice input to determine whether the operator's psychological state is normal from a predetermined signal component of the voice input. Determine whether or not
This is achieved by providing predetermined support for guiding a correct operation when it is determined that the operation is not normal.

In the judgment of the psychological state, the maximum value of the sound pressure, the maximum value of the frequency and the minimum value of the sounding time of the command word by the voice input are extracted, and the registration data stored in advance for each operator is used. Each is compared, and when two or more comparison results indicate that voice input is higher, the psychological state of the operator is determined to be abnormal (not normal).

[0007] Further, an operation procedure corresponding to an operating state of a plant or the like to be operated is stored in advance, and the predetermined support refers to the corresponding operation procedure to judge whether the operation by the voice input is correct or not and correct. In this case, the next operation procedure is guided, and in the case of an error, the processing of the command is prohibited, and the corresponding operation procedure is also guided.

The recognition of the command word and the operator by the voice input is performed by matching a pattern acquired by spectrum conversion of the sound pressure waveform with a pattern registered in advance.

[0009]

In general, when tension increases due to impatience or anxiety, the voice becomes loud, the sound pressure of the voice signal becomes large, and the frequency also increases. In addition, it becomes quicker to shorten the pronunciation time of the same word. Particularly, in the operation of a plant or the like, this tendency becomes more remarkable as the operator becomes unskilled when the plant is abnormal or is started / stopped.

The present invention has been made by paying attention to the change of the voice signal depending on the psychological state of the operator, and the maximum sound pressure, the maximum frequency and the shortest sounding time in advance in the normal psychological state of the command word necessary for the operation. Each data of is registered and compared with that of a command word by voice input during driving, and, for example, when two or more of sound pressure, frequency and sounding time of voice input exceed a threshold value, the psychological state of the operator. Is judged to be abnormal.

According to the configuration of the present invention, the command word which is the operation content and the operator who is the speaker are voice-recognized from the voice input, and the respective data registered for each operator and the voice extracted from the voice input. The operator's psychological state is determined by comparing the data, and when the operator's psychological state is not normal, the operating procedure stored in advance in relation to the driving state and the voice-recognized command word is referred to, and the next operating procedure or Guidance on correct operating procedures.

According to the present invention, when tension is imposed by operation of a plant or the like, for example, when an operation procedure at the time of plant abnormality is stored and the operator's psychological state is abnormal, a command word by the voice is used. Since it assists in guiding the operator to the correct operation while judging whether the operation is correct or not, it is possible for an inexperienced operator to perform an accurate driving operation. Further, by providing the operator support system according to the present invention, the safety of the plant or the like can be secured.

[0013]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the overall configuration of an operator support system according to an embodiment of the present invention. The operator assistance processing device 1 inputs a voice signal from a microphone 6 that converts the voice of the operator into an electric signal. The voltage waveform of the audio signal shows a sound pressure waveform. A data conversion unit 2 having means 21 to 23 for acquiring the maximum sound pressure, the maximum frequency and the shortest sounding time from the voice word of the input command, and voice recognition for performing word recognition and speaker (operator) recognition based on the sound pressure waveform. The processing unit 3, the operator state determination unit 4 that determines the psychological state of the operator based on the sound pressure, the frequency, and the sounding time, determines the operation support content according to the voice recognition result and the psychological state of the operator, and the speaker, the monitor, etc. The support processing output unit 5 that outputs the support content to the output device 7 of FIG. Furthermore, the plant status is monitored via PI / O8.

FIG. 2 is a block diagram showing the structure of the voice recognition processing section. The voice recognition processing unit 3 refers to the sound pressure data dividing unit 31 that divides the sound pressure waveform of one voice word into a plurality, the sound pressure spectrum conversion unit 32 that converts the divided data into a spectrum, and the voice word registration dictionary file 34. , A voice recognition pattern matching unit 33 for recognizing a voice word from a sound pressure spectrum of an input signal, a speaker voice registration file 36 for each voice word
With reference to, a speaker recognition pattern matching unit 35 that performs speaker recognition from the sound pressure spectrum and a recognition result output unit 37 that outputs the recognition result of the voice word and the speaker.

FIG. 3 is a block diagram showing the construction of the operator status judging section. The operator state determination unit 4 inputs the sound pressure, frequency and sounding time from the data conversion unit 2,
A sound pressure comparison unit 42 that compares the sound pressure data of the input signal with the data stored in the sound pressure information data file 46 and outputs "1" when the input signal is larger, the frequency data of the input signal and the frequency information data file. The frequency comparison unit 43 that compares the stored data of 47 and outputs “1” when the input signal is higher, compares the sounding time data of the input signal with the stored data of the sounding time information data file 48, and The pronunciation time comparison unit 4 which outputs "1" when the length is shorter
4. A psychological state determination unit 45 that performs a logical operation based on “1” / “0” of the outputs of the comparison units 42 to 44 to determine the psychological state of the operator (normal / abnormal).

FIG. 4 is a block diagram showing the structure of the support processing output unit. The support process output unit 5 is the support process determination unit 51.
The operator's instruction (command) recognized by voice and the psychological state determination result at that time, and the plant state management unit 52.
Of the plant, and referring to the operation registration data file 53, the support content that matches those input conditions is determined.

In the operation registration file 53, for example, the support contents in a logical format of "if A, then C" are registered. A is an operating condition such as a plant abnormality, and C is an operation procedure at that time. further,
The operation target of the voice word (valve 1 and the like) is registered in A, and the operation procedure related to it is registered in C.

Next, the operation of this embodiment having such a configuration will be described. First, the registration processing of the voice recognition processing unit and the operator state determination unit will be described.

FIGS. 5, 6 and 7 are waveform charts for explaining a well-known spectral data generation method which is performed in the voice recognition processing section in common with the registration processing and the driving processing. This generation method is described, for example, in "Computer audio processing (Takeshi Yasuiin, Masayuki Nakajima; Akiba Publishing; 1980.
Year) ".

FIG. 5 shows a sound pressure waveform (schematic diagram) of one voice word pronounced as “valve 1”, and FIG. 5 (a) is a voltage waveform diagram output from the microphone 6. In voice recognition, pattern matching is performed for each sound pressure data in order to increase the processing speed. Therefore, from the sound pressure waveform of (a) to (b)
Waveform division shown in is performed. In this example, “B, A, R, U,
The number of divisions (= 16) is equal to or larger than the number (= 9) of sound pressure data of “B, U, I, T, I”, and a sufficient overlapping area is provided in the front and rear portions of the division. As a result, the sound pressure data is collectively included in any of the divided areas, and the reduction in the recognition rate can be prevented.

FIG. 5C shows an enlarged waveform diagram (schematic diagram) of the sound pressure waveform of one division. As shown, 1
The same waveform corresponding to the sound pressure data is continuously included. One wavelength of this same waveform is extracted by a frame called "window", and time / time is calculated by the fast Fourier transform (FTT).
Frequency conversion is performed. FIG. 6 shows a waveform of the sound pressure waveform after FTT. Furthermore, inverse fast Fourier transform (IFTT)
And FTT to generate spectral data. Figure 7
FIG. 5 shows the spectrum data generated from FIG.
The generated spectrum data of the division unit is pattern-matched with the spectrum data registered in the registration dictionary file 34 in advance, and this is repeated for the number of divisions (= 16) to perform voice recognition of one voice word. In addition,
Duplication of sound pressure data in adjacent divided areas is sorted out.

In this embodiment, the registration processing of the voice recognition processing section is performed by the configuration shown in FIG. 2 (b). The registration of the registration dictionary file is performed by dividing the first voice word indicating the name of the operation target (for example, “valve 1”) and the second voice word indicating the operation content (for example, “open”) into the spectral data. . FIG. 8 shows a format of registration data of the registration dictionary file.

Further, the registration result of the first voice word is classified for each speaker and registered in the speaker voice information file 36. FIG.
Shows the format of the registration data of the speaker voice information file. As shown, the spectral data of the registered operator is stored for the same first phonetic word. The registration data by the same speaker is obtained by averaging spectral data obtained by inputting one word a plurality of times in a normal state.

Next, the registration process of the operator status judgment unit will be described. This processing is performed by the configuration of FIG. There is generally not much difference in the length of the first phonetic word registered by the operator. In the data conversion unit 2 during the registration process, the maximum sound pressure among the plurality of first voice words for registration,
The highest frequency and the shortest pronunciation time are extracted. The data distribution unit 41 of the operator status determination unit 4 registers the sound pressure information data file 46, the frequency information data file 47, and the sounding time information data file 47 in association with the registered operator ID. 10 (a) to (c)
Shows the format of these registration data.

Next, the operation of the operator support system will be described with reference to the flow chart of FIG.

When it is necessary to operate a specific device during the operation of the plant, the operator gives a command through the microphone 6 to, for example, "valve 1" and "close" (s10).
1). Between the first voice word and the second voice word in one command, in order to recognize the breaks between the two voice words, a short pause period is provided for separating the words. The voltage signal of the input command voice word is converted into sound pressure data of the first voice word and the second voice word by the data converter 2, and the maximum voice pressure of the first voice word, the maximum frequency and the pronunciation time. It is extracted (s102). The sound pressure data of the first voice word and the second voice word are subjected to pattern matching by the divided spectrum as described above in the voice recognition processing unit 3 to perform voice recognition of the command word. Furthermore, speaker pattern matching is performed using the spectrum of the first voice word, and the operator who issued the command is recognized (s103).

Next, the operator state judging section 4 fetches the maximum sound pressure, the maximum frequency and the pronunciation time of the first voice word from the data converting section 2 and the speaker recognition result from the voice recognition processing section 3, and applies the result. Sound pressure comparison, frequency comparison and sounding time comparison based on operator registration data are performed (s104),
The operator status is determined based on the comparison results (s105).

In general, when the operator is in an abnormal psychological state due to tension or the like, the sound pressure of the voice word is high, the frequency is high, and the pronunciation time is short, as compared with the normal psychological state. However, since there are errors due to individual differences and surrounding conditions, in this embodiment, the sound pressure is 2 which is the maximum value during normal times.
The frequency is 10% of the maximum value under normal conditions, and the sounding time is 30% of the average time during normal times as threshold values. When this value is exceeded, an output (1) indicating "abnormal" is output and other values are output. If it is not output (0). The psychological state determination unit 45 includes the comparison units 4
Outputs 2 to 44 are input, and if there are two or more inputs, it is determined as "operator state: abnormal". A specific output such as sound pressure comparison may be set as a necessary condition.

Next, if the judgment result of the operator state is "normal", the support processing output section 5 performs the output processing according to the voice command received from the voice recognition processing section 3 (s10).
8). On the other hand, if the determination result is "abnormal", the plant state is determined based on the predetermined data of the plant state detection unit (s106). If the plant is normal, s10
Then, the processing shifts to 8 and the output processing according to the voice command is performed. But,
If the plant is abnormal, the operation registration file 53 is referred to, and support processing of the corresponding predetermined content is performed (s107).

The contents of the support processing are on a case-by-case basis according to the configuration and dynamic characteristics of the plant. For example, if the plant is composed of system 1 and system 2, and system 1 is abnormal, "when system 1 is abnormal, valve 1, valve 2, valve 3
Are registered in the operation registration file 53. When both the system 1 and the system 2 are normal, the processing content as instructed by the voice recognition, “valve 1, closed”, is transmitted to the valve 1 via the PI / O 8.
Is output to the operation end of.

On the other hand, if the psychological state of the operator is judged to be "abnormal" and an abnormality occurs in system 1, refer to "close valve 1, valve 2, valve 3 in this order" in operation registration file 53. Then, the monitor is instructed to "close valve 2" next to the output of "valve 1 closed". If the command at this time is voice-recognized as “system 2, input”, the “command error” can be confirmed by referring to the operation registration file 53, and “close valve 1” can be checked through the monitor or speaker. Instruct and prohibit input for system 2. Furthermore, it is possible to give guidance to the operator, such as "Please be calm and operate according to the instructions."

According to the present embodiment, the psychological state of the operator is judged from the voice input, and when the operator loses his normal spirit due to the plant abnormality or the like, the operation procedure is guided or the erroneous operation is canceled to make the correct operation. Since it is possible to instruct the operation, it is possible to accurately support the operation of an unskilled person or the like and prevent the occurrence of a serious accident in the plant.
In the above example, the support processing is performed only when both the operator status and the plant status are abnormal, but even if the plant is normal, the necessary support processing may be performed when the operator status is abnormal. Good.

[0034]

According to the present invention, when the operator is extremely nervous or frustrated due to a plant abnormality or the like, the psychological state of the operator can be judged from the voice input of the operator who operates the driving machine. When the user is in a psychological state, the necessary operation support can be provided, and even an unskilled operator can easily perform the operation.

According to the present invention, the operation at the time of abnormal or transient response of the plant can be supported according to the psychological state of the operator, so that there is an effect that the safe operation of the plant can be secured.

[Brief description of drawings]

FIG. 1 is a configuration diagram of an operator support system according to an embodiment of the present invention.

FIG. 2 is a configuration diagram of a voice recognition processing unit.

FIG. 3 is a configuration diagram of an operator status determination processing unit.

FIG. 4 is a configuration diagram of a support processing output unit.

FIG. 5 is a schematic diagram of a sound pressure waveform for explaining waveform division of a voice word.

FIG. 6 is a waveform conversion diagram of FET processing of one sound pressure data.

FIG. 7 is a spectrum diagram of one sound pressure data.

FIG. 8 is a format diagram of registration data (spectrum) of a first voice word and a second voice word of a registration command.

FIG. 9 is a format diagram of registration data (spectrum) of a registration operator linked to a first voice word.

FIG. 10 is a format diagram of registration data of an operator status determination processing unit.

FIG. 11 is a flowchart showing an example of a support operation of the operator support system.

[Explanation of symbols]

1 ... Operator support processing device, 2 ... Data conversion unit, 3 ...
Speech recognition processing unit, 4 ... Operator status determination unit, 5 ... Support processing output unit, 6 ... Microphone, 7 ... Output device, 8 ...
PI / O, 31 ... Sound pressure data division unit, 32 ... Sound pressure spectrum conversion unit, 33 ... Voice recognition pattern matching unit, 3
4 ... Voice word registration dictionary file, 35 ... Speaker recognition pattern matching unit, 36 ... Speaker voice registration file, 37 ...
Recognition result output unit, 41 ... Data distribution unit, 42 ... Sound pressure comparison unit, 43 ... Frequency comparison unit, 44 ... Sound generation time comparison unit, 45
... psychological state determination unit, 46 ... sound pressure information data file, 4
Reference numeral 7 ... Frequency information data file, 48 ... Pronunciation time information data file, 51 ... Support process determination unit, 52 ... Plant state management unit, 53 ... Operation registration data file.

 ─────────────────────────────────────────────────── ─── Continuation of front page (72) Inventor Hiroyoshi Muronoi 1-5-20 Kaigan, Minato-ku, Tokyo Within Tokyo Gas Co., Ltd. (72) Inventor Tetsuya Hori 1-5-20 Kaigan, Minato-ku, Tokyo Tokyo Gas Co., Ltd.

Claims (7)

[Claims] 1. A method of assisting an operator for operating equipment such as a plant by voice input, determining whether or not the psychological state of the operator is normal from a predetermined signal component of the voice input, and determining that the operator's psychological state is not normal. In this case, a method for assisting an operator, which is characterized by performing predetermined support for guiding a correct operation. 2. The determination of the psychological state according to claim 1, wherein the maximum value of the sound pressure, the maximum value of the frequency, and the minimum value of the sounding time of the command word by the voice input are extracted, and for each operator in advance. A method for assisting an operator, comprising comparing the stored registration data with each other, and determining that the psychological state of the operator is abnormal (not normal) when the voice input is higher than two comparison results. 3. The operation procedure according to claim 1, wherein an operation procedure according to an operating state of a plant or the like to be operated is stored in advance, and the predetermined support refers to the corresponding operation procedure, and a command by the voice input. A method for assisting an operator, comprising determining whether a word is correct, guiding the next operation procedure if the word is correct, prohibiting processing of the command if the word is incorrect, and guiding the corresponding operation procedure. 4. The command word and operator recognition according to claim 1, 2 or 3, is performed by matching a pattern acquired by spectrally converting a sound pressure waveform with a pattern registered in advance. An operator support method characterized by the above. 5. A microphone for inputting an operation of equipment such as a plant by voice, an output device such as a speaker or a monitor for outputting support contents, and an operator support process for determining support contents according to a command word recognized from voice input. An operator support system including a device, which has a registration file for storing a plurality of command word patterns and a speaker pattern linked to the command words, extracts the command word pattern from voice input, and stores the registration file. The voice recognition processing unit that pattern-matches with the command word and the speaker, and has a data file that stores each data of the maximum sound pressure, the maximum frequency and the shortest sounding time of the command word in advance for each operator name, and the voice signal Based on the maximum sound pressure, the maximum frequency, and the shortest pronunciation time data extracted from the command words and the operator name by the speaker recognition, The operator state determination processing unit that determines the psychological state of the operator based on the comparison result by comparing each value of the operator file and the determination result of the command and the operator state that are voice recognized from the voice input, An assisting system for an operator, comprising: an assisting process output unit that provides a predetermined assist for guiding a correct operation when it is determined that the mental state is not normal. 6. The support process output unit according to claim 5, wherein the operation procedure according to an abnormal state of a plant or the like to be operated is stored in advance, and the abnormal state of the plant or the like is detected. It is characterized by determining whether the operation by the voice input is correct or incorrect by referring to the operation procedure, guiding the next operation procedure when the operation is correct, and assisting the corresponding operation procedure when the operation is incorrect. Operator support system. 7. The voice recognition processing unit according to claim 5, further comprising means for dividing the sound pressure waveform of the voice input into a plurality of parts, and generating the pattern by spectrum conversion for each divided waveform. A support system for operators.