JPH06289927A - Multimedia process monitor/control device - Google Patents

Abstract

PURPOSE:To facilitate the analysis the cause of an abnormal operation by simultaneously reproducing data at the time of detecting abnormality. CONSTITUTION:A picture signal taken by a monitor camera 1 is inputted to a picture processor 17, is displayed on a multiwindow display 7 and is stored in a main storage device 16. A sound signal collected by a microphone 2 is inputted to a sound processor 18, is reproduced by a speaker 12 and is stored in the main storage device 16. Process data transmitted from a process data controller 6 is displayed on the display 7 and is stored in the main storage device 16. When abnormality is detected in a monitored object, data before and after the detection of abnormality are saved in an auxiliary storage device 10 among respective pieces of data stored in the main storage device 16. Saved data area reproduced by the display 7 and the speaker while they are mutually synchronized. Thus, the cause of abnormality can easily be analyzed.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multimedia process monitoring / controlling device used in various industrial fields.

[0002]

2. Description of the Related Art Conventionally, various types of multimedia process monitoring devices have been proposed. For example, a paper by Masayuki Tani et al. “Man-machine interface for plant operation monitoring using direct manipulation to video” (The Institute of Electrical Engineers of Japan, D, 111, No. 12, 1923, 1023-1)
Pp. 030) and Japanese Patent Application Laid-Open No. 5-334438, "Method of mapping object position in camera image and method of object identification", which has already been filed by the present applicant. Also, as an example of the multimedia / process monitoring / control device,
There is a plant monitoring system shown in FIG.

As shown in the figure, a camera 1 and a microphone 2 are installed to monitor the situation at the plant site by video and sound, and the video signal imaged by the camera 1 is sent to an image recording device 3. The voice signal collected by the microphone 2 is sent to the voice recording device 4. The image recording device 3 is
The input video signal is sent to the superimposing device 5 as it is, and the video signal is recorded and reproduced as necessary. The voice recording device 4 receives the input voice signal as it is from the speaker 1
Send to 2 for playback and record audio signal. The process data controller 6 controls input / output of process data regarding the plant.

The computer 8 includes a CPU 9 and an auxiliary storage device 1.
0, the console 11, the process data input / output unit 20, etc., and temporarily stores the process data input / output by the process data controller 6 in the auxiliary storage device 10 via the process data input / output unit 20. Send to the superimposing device 5. The superimposing device 5 superimposes the video signal and the process data and sends it to the multi-window display 7 for display. As a result, the operator operates the console 11 while operating the display 7 and the speaker 1.
2 allows process monitoring.

FIG. 21 shows an example in which trend data is displayed as a graph 13 on the display 7 shown in FIG.
Is the image additionally displayed with the image 14 taken by the camera 1. The operator monitors these displays 7, and when an abnormality occurs, the operator can search the plant system diagram or the process data displayed on the display 7 for an abnormal portion and perform processing such as restoration.

[0006]

However, these management systems have the following drawbacks. (1) If a recovery process is executed for an abnormal portion when an abnormality occurs, detailed cause analysis becomes difficult, and the cause analysis for the abnormality has been limited to the occasion. This is because image data, audio data, and process data are recorded independently, so it is easy to individually reproduce the data before and after the occurrence of an abnormality,
This is because it is difficult to reproduce each data in synchronization with each other. (2) Because of the multimedia configuration, a large-capacity storage device is required to store image / sound data, and the processing becomes large-scale.

(3) Although various data collected from the monitoring target are related to each other, they are not effectively used for determining the presence or absence of abnormality. (4) The operator must constantly monitor the display when trying to detect the presence or absence of an abnormality in the input data. If an abnormality occurs when the operator takes his eyes off the display, the response to the abnormality will be delayed. It may end up. (5) In addition, if an abnormality occurs while the operator is completely away from the monitoring device such as the display, it is not possible to warn an intruder in the monitoring area about the occurrence of the abnormality.

(6) When an operator inputs a voice with a microphone, there is a trouble that the voice cannot be input without performing an input operation each time. (7) A large number of sensors may be installed in the monitoring target to detect the state quantity related to the monitoring target. In that case, the detected state quantity is simply displayed as a numerical value on the screen. Will not be displayed effectively. The present invention has been made to solve each of the above problems, and an object of the present invention is to add various functions utilizing multimedia and to provide high-performance multimedia process monitoring / control with excellent usability. To provide a device.

[0009]

In order to achieve the above object, the first invention displays an image obtained by picking up an image of a monitoring target and process data input / output with respect to the monitoring target on a display, and In a multimedia process monitoring / control device that reproduces audio collected from a monitoring target with a speaker, a means for cyclically storing each of the above data in the main storage device, and if an abnormality is detected, a constant value before and after that A means for evacuating each data stored in the main memory during the period to the auxiliary memory, and a means for reproducing each data evacuated in the auxiliary memory in synchronization with the reading display and the speaker. And

According to a second aspect of the present invention, in the first aspect of the present invention, the process data input is compared with the latest process data already input and stored in the main storage device to determine whether they are the same or not. The present invention is characterized by comprising means for detecting a change in the state of data and means for storing the process data in the main memory only when a change in the state of the process data is detected.

According to a third aspect of the present invention, in the first and second aspects of the present invention, storage means for storing a plurality of stages of reference data from a normal state to an abnormal state for each of image, sound and process data, and each input data. A means for calculating the degree of abnormality of input data from the similarity obtained by comparing data and reference data, and comparing the degree of abnormality of input data with a preset threshold value to determine whether the input data is normal or abnormal And a means for outputting an abnormality detection signal in the case of an abnormality.

According to a fourth aspect of the present invention, in the third aspect of the present invention, storage means for storing in advance a voice pattern for anomaly occurrence notification for each anomaly degree of the input data, and the anomaly degree of the input data when the anomaly detection signal is output And a means for reading out a sound pattern for notifying an abnormality occurrence from the storage means and reproducing it by a speaker.

According to a fifth aspect of the present invention, in the third aspect, means for storing a warning voice message in the storage means for each abnormality degree of the input data input through the microphone, and an abnormality detection signal are output. Then, a means for reading a warning voice message corresponding to the degree of abnormality of the input data from the storage means and reproducing it by the speaker installed on the monitored side is provided.

According to a sixth aspect of the present invention, in the first to fifth aspects of the present invention, there are provided voice data storage means for sequentially storing voice data input from the operator microphone, latest voice data and voice data before it. The present invention is characterized by including a calculating means for calculating a correlation value and a means for fetching voice data input from the operator microphone into an internal voice data processing section when the correlation value is equal to or less than a predetermined value.

According to a seventh aspect of the present invention, in the first to sixth aspects of the invention, a plurality of sensors installed on the monitoring target, and detection corresponding to the monitoring target screen displayed on the display based on the sensor position and the sensor detection amount. It is characterized by comprising means for creating a two-dimensional distribution graph of the quantity and means for superimposing and displaying the two-dimensional distribution graph on the monitor screen of the display.

[0016]

According to the first aspect of the invention, the image obtained by picking up the monitoring target, the process data input / output with respect to the monitoring target, and the voice collected from the monitoring target are stored in the main storage device in real time and cyclically. It When an abnormality is detected in the monitoring target, each data stored in the main storage device is temporarily saved in the auxiliary storage device for a certain period before and after that, and each saved data is read and synchronized with the display and speaker. And then played.

In the second aspect of the invention, the process data input / output with respect to the monitoring target and the latest process data already input and stored in the main storage device are compared to determine whether they are the same. In this way, the process data is stored in the main memory only when the state change of the process data is detected. This prevents the same process data from being continuously stored in the main storage device.

In the third aspect of the invention, reference data of a plurality of stages from a normal state to an abnormal state is stored in advance for each of image, sound and process data, and each input data and the reference data can be compared and calculated. The degree of abnormality of the input data is calculated from the similarity. next,
The abnormality degree of the input data is compared with a preset threshold value to determine whether the input data is normal or abnormal.

In the fourth aspect of the invention, the sound pattern for notifying the occurrence of abnormality is stored in advance in the storage means for each abnormality degree of the input data, and when the abnormality detection signal is output, it corresponds to the abnormality degree of the input data. The voice pattern for transmitting the abnormality occurrence is read from the storage means and reproduced by the speaker. As a result, the operator can visually recognize the occurrence and the degree of the abnormality.

In the fifth aspect of the invention, when a warning voice message for each abnormality degree of input data is input through the microphone, it is stored in the storage means. As a result, when the abnormality detection signal is output, the warning voice message corresponding to the abnormality degree of the input data is read from the storage means and reproduced by the speaker installed on the monitored side, and the intruder in the monitored area is reproduced. Is given a warning that an abnormality has occurred.

According to the sixth aspect of the invention, the voice data input from the operator microphone is sequentially stored in the voice data storage means, and the correlation value between the latest voice data and the voice data before that is calculated. When the obtained correlation value is less than or equal to the predetermined value, the voice data input from the operator microphone is taken into the internal voice data processing unit.

In the seventh invention, a plurality of sensors are installed in the monitoring target, and a two-dimensional distribution graph of the detection amount corresponding to the monitoring target screen displayed on the display is created from the sensor position and the sensor detection amount. . Further, the created two-dimensional distribution graph is superposed and displayed on the monitor target screen of the display.

[0023]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the overall configuration of a first embodiment according to the first invention. In the figure, the monitoring camera 1 captures an image of a monitoring target and sends an image signal to the image processing device 17 in the computer 8. The microphone 2 collects a voice from the monitoring target and sends a voice signal to the voice processing device 18 in the computer 8. The process data controller 6 is connected to the computer 8, controls the data input / output with respect to the process to be monitored, and sends the data to the process data input / output unit 20 in the computer 8. Multi-window display 7
Displays the image signal sent from the image processing device 17 in the computer 8.

The speaker 12 reproduces the audio signal sent from the audio processor 18 in the computer 8. The computer 8 is composed of a CPU 9, a main storage device 16, an auxiliary storage device 10, an image processing device 17, a voice processing device 18, a process data input / output unit 20 and a console 11 which are connected to a bus 19. to manage. That is, computer 8
Displays the image signal input from the surveillance camera 1 to the image processing device 17 and the process data obtained from the controller 6 on the display 7, and stores both data in the main storage device 16. Also, the calculator 8 is the microphone 2
The audio signal input from the audio processing device 18 to the speaker 12 is sent to the speaker 12 and stored in the main storage device 16.

FIG. 2 is a block diagram showing the internal structure of the image processing apparatus 17. The image processing device 17 includes an A / D converter 21, an image data compression / decompression device 22, an image memory 23, and a D / A converter 24. A / D converter 21
Converts the image signal from the surveillance camera 1 into a digital signal and sends it to the image data compression / decompression device 22 and the image memory 23.

The image data compression / decompression device 22 compresses the image data from the A / D converter 21 and outputs it to the bus 19, and at the same time, restores the compressed image data input from the bus 19 at the time of data reproduction to restore the image memory 23. Send to. The input image data is stored in the image memory 23, and the image data is sent to the D / A converter 24 at a predetermined timing. The D / A converter 24 converts the image data composed of a digital signal into an analog image signal and outputs it to the display 7.

FIG. 3 is a block diagram showing the internal structure of the voice processing device 18. The audio processing device 18 includes an A / D converter 25, an audio data compression / decompression device 26, and a D / A converter 2.
7, voice data memory 28. A / D converter 25
Converts the audio signal from the microphone 2 into a digital signal and sends it to the audio data compression / decompression device 26. The audio data compression / decompression device 26 sends the audio data from the A / D converter 25 to the D / A converter 27 as it is, compresses the audio data, and outputs the compressed audio data to the bus 19.
Reference numeral 6 restores the compressed audio data input from the bus 19 at the time of data reproduction and sends it to the D / A converter 27.

In the apparatus having the above structure, the controller 6
When an abnormality is detected from the process data or the like input to the above, each data stored in the main storage device 16 is saved in the auxiliary storage device 10 during a certain period before and after the abnormality detection. Then, each saved data is sent to the display 7 and the speaker 12 for reproduction at the same time and in synchronization.

FIG. 4 is an explanatory diagram showing a procedure for storing each data in the main storage device 16. The data storage area 29 in the main memory 16 is an area in which image data, audio data, and process data are stored as one set for each frame.
The data management area 30 in the main storage device 16 stores the main storage addresses of image data, audio data, and process data stored in the data storage area 29 for each set (frame), and stores the latest storage frame number and The final storage frame number is stored. The initial value of the latest storage frame number is 1, and the head address of the data storage area 29 is stored in the image storage address corresponding to this frame 1. The storage processing in these areas 29 and 30 is performed as follows.

First, the image data is stored at the address of the data storage area 29 indicated by the image re-storing address corresponding to the latest storage frame number shown in the data management area 30. After storage, the data storage area 2 is added to the audio storage address area corresponding to the same frame number in the data management area 30.
The address next to the last image data stored in 9 is stored. Next, the audio data is stored in the address of the data storage area 29 indicated by the audio storage address corresponding to the same frame number. After the storage, the next address of the last voice data stored in the data storage area 29 is stored in the corresponding process data storage address area in the same frame number in the data management area 30. Next, the data storage area 29 indicated by the process data storage address corresponding to the same frame number
Store the process data at the address of.

After storing the process data, the main memory storage device 1
The data storage limit address of 6 and the next address of the last process data are compared. As a result, if one set of image data, audio data, and process data can be stored,
The latest storage frame number in the data management area 29 is updated, and the next address of the last process data just compared is stored in the image data storage address area corresponding to that number. If the result of comparison cannot be stored, the latest storage frame number in the data management area 29 is set to 1 and the final frame number is stored in the final frame number area. In this way, when the data is stored up to the last frame, the next frame number is returned to, and each data is managed so as to be cyclically stored in the data storage area 29.

In this embodiment, when an abnormality is detected in the process to be monitored and the operator inputs a command to reproduce data from the console 11 or the like, the abnormality is detected and the data is saved from the main memory 16 to the auxiliary memory 10. The respective data before and after the generated abnormality are restored by the image processing device 17 and the audio processing device 18 and sent to the display 7 and the speaker 12 to be reproduced. At this time, since each data is reproduced in synchronization with each other, it is possible to analyze the cause of the abnormality from various aspects. As a result, it becomes possible to perform the handling process when an abnormality occurs more efficiently and accurately than in the past. Further, it becomes possible to accurately re-examine the validity of the restoration operation.

FIG. 5 is a block diagram showing the overall configuration of the second embodiment according to the first invention. In this embodiment, since the monitoring target and the apparatus main body on the operator side are separated from each other, they are connected by a transmission line 38 via time division multiplexing transmission devices (TDM) 32 and 40. That is, in the figure, the image signal from the surveillance camera 1 and the audio signal from the microphone 2 are respectively the moving image compression unit 3 in the TDM 32.
3 and the audio compression unit 34, and is converted into a digital signal and compressed. Similarly, the process data controller 6 also includes the process data input / output unit 3 in the TDM 32.
6 is connected to input and output process data such as measurement data and device setting data. The compressed image data, audio data and process data are transferred to the transmission control unit 37.
Are multiplexed by time division and output to the transmission line 38.

Next, the TDM 40 and the computer 50 connected to the other end of the transmission line 38 will be described with reference to FIG. FIG. 6 is a block diagram showing the TDM 40 and the computer 50 in detail. The data input to the TDM 40 via the transmission path 38 is separated by the transmission control unit 41. The separated image data is sent to the moving image compression / decompression unit 47 via the time division multiplexing bus 46, similarly, the audio data is sent to the audio compression / decompression unit 48, and the process data is sent to the process data transmission / reception unit 49. To be

The moving image compression / decompression unit 47 decompresses the input image data, and the frame memory 54 in the computer 50 is expanded.
Send to. The voice compression / decompression unit 48 decompresses the input voice data and sends it to the external speaker 12 for reproduction, and the voice input to the microphone 39 is input. Also,
The moving image compression / decompression unit 47 and the audio compression / decompression unit 48 are the digital interface (IF) 5 in the computer 50.
Since it is connected to 5, the image / sound data can
It is also possible that the image data and audio data are fetched in 0 and transmitted from the computer 50 to the TDM 40.

Further, the process data transmitting / receiving unit 49 is connected to the process data input / output unit 56 in the computer 50,
Process data is transmitted and received between each other. The frame memory 54, digital IF 55, and process data input / output unit 56 are connected to the CPU 51, main storage device 52, and auxiliary storage device 53 via a bus. In addition, the multi-window display 7 and the large display 58 are connected to the frame memory 54 to display images to be monitored, process data and the like.

FIG. 7 is a block diagram showing the internal structure of the digital IF 55. The digital IF 55 has a control unit 6
1,62 and FIFO (First In Fires)
It is composed of input / output buffers 63 and 64 having a function of (tOut), and is accessible from both the TDM 40 and the CPU 51 of the computer 50. That is, when accessing from the TDM 40, when the CPU 51 accesses using the synchronous clock of the TDM 40, the control units 61 and 62 are operated by the access clock of the CPU 51 to access the input / output buffers 63 and 64.

FIG. 8 is a block diagram showing the internal structure of the moving picture compression / decompression unit 47 in the TDM 40. Video compression
The decompression unit 47 uses the time division multiplex interface (IF) 6
5, serial controller 66, switch (MUX) 67, decompressor 68, D / A converter 69, compressor 70, A / D converter 71. By switching switch 67, Time division multiplexing IF6 from division multiplexing bus 46
5 is selected, and either the data input via the computer 50 or the data input from the computer 50 is selected, and then the expansion units 68, D
The analog output is performed via the / A converter 69. The internal structure of the audio compression / decompression unit 48 is the same as that of the moving image compression / decompression unit 47 described above.

In the second embodiment configured as described above, the audio / image data compressed in the TDM 32 is temporally continuously input to the digital IF 55 in the computer 50 via the transmission line 38 and the TDM 40. Then, the data is once stored in the input / output buffers 63 and 64 in the digital IF 55 and then stored in the main storage device 52 by the access of the CPU 51. The storage procedure and the saving to the auxiliary storage device 53 at the time of abnormality are the same as those in the first embodiment, and thus the description thereof is omitted.

When the data saved in the auxiliary storage device 53 is to be reproduced, the switch 67 for each of the moving image compression / decompression unit 47 and the audio compression / decompression unit 48 in the TDM 40.
, The data path is switched to the digital input from the computer 50, and the image / sound data read from the auxiliary storage device 53 is transferred to the multi-window display 7 or the large display 58 and the speaker 12.
It is reproduced in synchronization with. As a result, also in this embodiment, the same effect as in the first embodiment can be obtained.

Next, a third embodiment according to the second invention will be described. This embodiment relates to the processing of the CPUs 9 and 51 when storing process data in the main storage devices 16 and 52 in the computers 8 and 50 in the first and second embodiments. FIG. 9 is an explanatory diagram showing the operation of the third embodiment. In the figure, the data storage areas 29 of the main storage devices 16 and 52
The image data, audio data, and process data collected in frame units are all stored in each frame.
For the first frame 1, image data, audio data, and process data are all stored.

For the next frame 2, compare the collected process data with the process data of frame 1,
It is determined whether they are the same. If they are the same, only image data and audio data are stored for frame 2 without storing process data. Hereinafter, also for the frame 3, the process data is compared with the frame 1. In the illustrated example, the case of the frame 3 is the same as that of the frame 1, so the storage is omitted. In the next frame 4, a change is recognized in the process data and the process data is stored for the first time.

From then on, the change from the newly stored process data of frame 4 is monitored. For the frame in which the process data is not stored, since the process data is the same as the stored process data due to the last change before that, the data management area 30
The address of the data storage area 29 in which the same process data is stored is written in the process storage address of. As a result, the data is later stored in the auxiliary storage device 10,
When saving to 53, each data can be saved frame by frame by reading the process data whose storage is omitted from another frame.

Since the relationship between the data management area 30 and the data storage area 29 is the same as that of the first embodiment shown in FIG. 4, its explanation is omitted. In this embodiment, as described above, the process data is stored in the data storage area 29 only when there is a change in the process data, so that a large amount of image data can be stored, and the data storage area 29 is effectively used. It will be. Especially for images,
The more the encoded image data is, the higher the image quality becomes, the more the information amount of the image data used for the cause analysis at the time of the occurrence of the abnormality increases, and the easier the reason for the abnormality is clarified.

Next, a fourth embodiment according to the third invention will be described. This embodiment is the same as the second embodiment except that the abnormality of the monitoring target and its degree are automatically detected by the processing of the CPU 51 based on the image / voice and the process data input to the computer 50. Figure 1
0 is a block diagram conceptually showing the processing operation of the fourth embodiment. In the figure, when image data and audio data are input, reference image data 81, reference audio data 82 and calculation units 84 and 85 prepared in advance.
Are respectively compared and calculated.

The reference image data 81 and the reference audio data 82 are data created by dividing the normal state to the abnormal state into N stages. Here, as the normal data, data collected from a state of actually operating normally is used, and as the abnormal data, data actually generated in the past or data artificially created assuming an abnormality is used. Further, when the process data is input, it is compared and calculated with the reference process data 83 prepared in advance in the calculation unit 86. The reference process data 83 includes an upper limit value and a lower limit value of reference numerical values.

Further, in the arithmetic units 84 and 85, the sum of the absolute values of the differences for each sample is compared between the input current data and each reference data, and which pattern is closer to each other is calculated by the correlation calculation. Since the calculation results of the calculation units 84 and 85 may be approximate, DCT (discrete cosine transform) is applied to the image data.
The calculation time can be shortened by performing the calculation using only the low-frequency component or the direct-current component of the above, and calculating the voice data using only the band-divided low-frequency data. Further, in the calculation unit 86, the input current process data and the reference value of each reference process data are compared and calculated.

The degree of similarity with the reference data at each stage obtained as a comparison calculation result in each of the calculation units 84 to 86 is sent to the weighting units 87 to 89 to weight each reference data before the abnormality detection unit. Sent to 90. The abnormality detection unit 90 determines the abnormality level of the input data from the weighted similarity of each reference data, outputs it as the abnormality detection level, and further compares the abnormality detection level of the input data with a preset threshold value. If the abnormal level of the input data exceeds the threshold value, it is considered that an abnormality has occurred in the monitoring target and an abnormality detection signal is output. The abnormality detection signal becomes a trigger signal for saving data in the auxiliary storage device 53 via the OR circuit 91.

FIG. 11 is an explanatory diagram showing the procedure of the processing operation of FIG. 10 described above. In the figure, the calculation unit obtains the correlation value between the input image / audio data and each reference data, and outputs the correlation information value indicating the number and the degree of the reference data having high correlation among them. Further, it is determined whether the input process data is within the upper and lower limit range of the process data. The next weighting unit weights the mutual importance (reliability) of the image data, the audio data, and the process data, and the reference data.

The abnormality detection section outputs the abnormality level stepwise based on the three types of weighted data, compares the reference level set in advance as a threshold value, and detects the abnormality when the threshold value is exceeded. Output a signal. The data stored in the main storage device 52 is saved in the auxiliary storage device 53 in response to the output of the abnormality detection signal or the request for fetching in accordance with an instruction from the operator. As described above, in this embodiment, the presence or absence and the degree of abnormality of the monitoring target can be automatically determined from the input image / sound / process data, so that the burden on the operator is reduced and the reliability is improved. Is improved.

Next, a fifth embodiment according to the fourth invention will be described. In this embodiment, when an abnormality is detected in the fourth embodiment, the operator is notified by voice of the degree of the abnormality. FIG. 12 is a block diagram showing the configuration of the fifth embodiment. In the figure, a voice information memory 92 for storing voice data 1 to N is provided in the computer 50, and the computer 50 and the TDM 4 are also provided.
A serial interface (IF) 93 such as RS-232C is connected to the transmission control unit 41 in 0. The voice data 1 to N are voice patterns to be notified to the operator corresponding to the detected abnormal level.

The voice compression / decompression unit 4 in the TDM 40
The configuration of 8 is common to the block diagram of FIG. 8, and the serial control unit 66 transmits the transmission control unit 4 via the time division multiplexing IF 65.
1 is communicated with, and various mode settings and statuses are notified. In these configurations, when the abnormality detection unit 90 (shown in FIG. 10) in the computer 50 detects an abnormality from the input data, the trigger signal is transmitted via the serial IF 93 to the TDM.
It is sent to the serial control unit 66 via the transmission control unit 41 in the 40 and the time division multiplexing IF 65.

Next, when the serial control unit 66 which receives the trigger signal switches the switch 67, the data path to the voice decompression unit 68 is switched to the digital output from the computer 50. At the same time, the computer 50 reads the voice data corresponding to the abnormality detection level from the abnormality detection unit 90 from the voice information memory 92, and the digital IF 5
5, sent to the voice decompression unit 68 via the switch 67. As a result, the audio data representing the degree of abnormality is converted into D / A converter 6
It is sent to the speaker 12 via 9 to be reproduced, and the operator is informed of the occurrence and degree of abnormality by voice.

As described above, in this embodiment, even if an abnormality occurs while the operator is looking away from the multi-window display 7 or the large display 58,
It is possible to immediately check the occurrence of an abnormality and the extent of the abnormality, and to immediately take corrective action, improve the usability of the apparatus and reduce the burden on the operator.

Next, a sixth embodiment according to the fifth invention will be described. In this embodiment, when an abnormality is detected in the fourth embodiment, an intruder in the monitoring area is given a warning consisting of a voice message according to the degree of the abnormality. FIG. 13 is a block diagram showing the configuration of the sixth embodiment. In the figure, a voice information memory 92 for storing voice data 1 to N in a computer 50
Is provided, and the computer 50 and the transmission control unit 41 in the TDM 40 are connected by a serial IF 93. The voice data 1 to N are voice messages reproduced in the monitoring area corresponding to the detected abnormal level.

The voice compression / decompression unit 4 in the TDM 40
8 is a switch (MUX) 94, 95 newly connected to the configuration of FIG.
From the computer 50 side to the serial IF 93 and the transmission control unit 4
1. Switching is performed by an instruction sent to the serial control unit 66 via the time division multiplexing IF 65. Further, the serial control unit 66 uses the time division multiplexing IF 65 to transmit the transmission control unit 41.
It communicates with and is notified of various mode settings and status. In these configurations, first, the switch 94 is switched to switch the microphone 39 to the A / D converter 71 and the audio compressor 7.
0, a switch 94, and a data input path to the digital IF 55.

Next, a plurality of voice messages are input using the microphone 39, and the voice information memory 9 in the computer 50 is input.
2 as audio data 1 to N. This voice message is sent to intruders in the surveillance area when an abnormality occurs.
This is to give a different warning by voice depending on the degree of abnormality. When a plurality of voice messages corresponding to the degree of abnormality are recorded, the switching unit 94 is switched again, and the data from the TDM 32 on the monitoring target side input to the time division multiplexing IF 65 via the transmission control unit 41 is input to the computer 50. Input in.

By switching the switch 95, either the voice message read from the voice information memory 92 or the voice data input to the microphone 39 is transferred to the time division multiplexing IF 65 and the transmission control unit 41. Is transmitted to the transmission line 38 via. That is, T on the monitored side
A voice message or an input voice from the microphone 39 can be reproduced from the speaker 31 connected to the DM 32.

In these configurations, when the abnormality detecting section 90 (shown in FIG. 10) in the computer 50 detects an abnormality from the input data, the trigger signal is sent via the serial IF 93.
It is sent to the serial control unit 66 via the transmission control unit 41 and the time division multiplexing IF 65 in the TDM 40. Next, when the serial control unit 66 that receives the trigger signal switches the switch 95, the data path is set so that the digital output from the computer 50 is sent to the transmission path 38 via the time division multiplexing IF 65 and the transmission control unit 41. It is formed.

At the same time, the computer 50 reads the voice message corresponding to the abnormality detection level from the abnormality detection unit 90 from the voice data 1 to N in the voice information memory 92 and sends it to the voice compression / decompression unit 48 via the digital IF 55. input. Next, the voice message is sent to the transmission line 38 via the switch 95, the time division multiplexing IF 65, and the transmission control unit 41, and reproduced by the speaker 31 connected to the TDM 32 on the monitored side. Further, similarly to the fifth embodiment, if the switch 67 is switched when an abnormality is detected, a voice message is reproduced from the speaker 12, so that the operator can recognize the abnormality occurrence and its degree by voice. it can.

As described above, in this embodiment, when an abnormality occurs, a voice message having contents corresponding to the degree of the abnormality is automatically notified to an intruder in the monitoring area as a warning. As a result, even if an abnormality occurs when the operator leaves the apparatus, the intruder in the monitored area is reliably alerted, the burden on the operator is reduced, and the safety and reliability of the system are improved. When the operator is near the microphone 39, the operator can directly send a message such as a warning by voice to the speaker 31 on the monitoring target side by switching the switch 95 or the like.

Next, a seventh embodiment according to the sixth invention will be described. FIG. 14 is a block diagram showing the configuration of the seventh embodiment, and FIG. 15 is a block diagram showing the internal configuration of the main parts of FIG. In this embodiment, as shown in FIG. 14, a voice recognition unit 96 is newly installed in the voice compression / decompression unit 48 of the configuration of the sixth embodiment, so that the microphone 39
The switch 95 is automatically switched by detecting the input of voice data from the. Since other configurations and operations are the same as those in FIG. 13, the same reference numerals are given to the same portions and the description thereof will be omitted.

The voice recognition unit 96 is configured as shown in FIG. 15, and the voice data input from the microphone 39 via the A / D converter 71 is converted into the current voice data buffer 9.
After input to 7, the reference audio data buffer 9
8 and the correlation calculation unit 99. The audio data input to the reference audio data buffer 98 is then sent to the correlation calculator 99. Here, the correlation calculation unit 99
The currently input audio data is input as the current audio data, and the audio data input before is input as the reference audio data.

The reference voice data can also be created by weighted averaging a plurality of temporally continuous current voice data. The correlation calculation unit 99, to which these two data are input, calculates the correlation value by performing the correlation calculation of these two data. That is, if there is no input from the microphone 39, both data substantially match and the correlation value increases. Further, when voice input is started, both data are different and the correlation value becomes low. The weighting factor to the correlation calculator 99 is input for adjustment.

Further, the correlation calculation unit 99 monitors the obtained correlation value, and if it is less than a preset value, the microphone 3
The switch 95 is switched so that the input from 9 is valid, and when it becomes larger than the set value, the voice data from the computer 50 side is validated. As a result, the operator simply inputs a voice from the microphone 39, accepts the voice input without any operation, stores the voice data 1 to N in the voice information memory 92 of the computer 50, or directly inputs the voice. Thus, a message such as a warning can be reproduced in real time from the speaker 31 on the monitoring target side.

When no sound is input from the microphone 39, the switch 95 automatically returns to the original state and the sound data collected by the microphone 2 in the monitored object is reproduced by the speaker 12. Although the third to sixth inventions have been described with reference to the embodiment having the second embodiment as a basic configuration, the invention can be similarly applied based on the configuration of the first embodiment.

Next, an eighth embodiment according to the seventh invention will be described. In this embodiment, a storage warehouse of a chlorine cylinder, which is a dangerous substance, is monitored, and the inside of the warehouse is monitored by a camera to obtain a monitoring image shown in FIG. The circle shape surrounding the numbers in the figure indicates the cylinder position. Further, in FIG. 17, a plurality of chlorine concentration sensors for detecting a chlorine leak are arranged in the storage warehouse, and the arrangement of the chlorine concentration sensors is indicated by a cross mark in FIG. 17 corresponding to the monitoring image frame of FIG. is there. When chlorine leak is detected by these sensors, the operator recognizes that it is dangerous and instructs the computer constituting the apparatus to display the danger information.

Then, the computer creates the contamination concentration distribution data in the warehouse based on the chlorine concentration from the sensor, and further obtains the concentration contour line from the concentration distribution. FIG. 18 is a contour line showing the concentration distribution. The image of FIG. 19 is obtained by superimposing the image showing the contour lines on the surveillance image of FIG. By displaying this image on the display, the operator can grasp the occurrence of chlorine leakage and the concentration and spread of the contamination at a glance, and the subsequent countermeasures can be facilitated. In addition, although the chlorine concentration distribution in the closed chamber is visualized in the embodiment, the oxygen concentration distribution and the electromagnetic field intensity distribution in a facility where a high-voltage current is flowing can be visualized.

[0069]

As described above, according to the first invention,
When an abnormality is detected, the data before and after the abnormality can be reproduced in synchronization with the display and the speaker. As a result, it becomes easy to analyze the cause of the abnormality that has occurred.

According to the second aspect of the invention, the process data is stored in the main memory only when a change in the state of the process data is detected, so that the same process data is continuously stored in the main memory. The main memory device is effectively operated, the image data amount is relatively increased and the image quality is high, and the cause analysis at the time of trouble becomes easy.

According to the third aspect of the invention, reference data of a plurality of stages from a normal state to an abnormal state is prepared in advance for each of image, sound and process data, and each input data is compared with the reference data. After calculating the similarity and calculating the abnormality degree of the input data, the abnormality degree of the input data is compared with a preset threshold value to determine whether the input data is normal or abnormal. In this way, by automatically detecting the abnormality of the monitoring target based on the data collected from the monitoring target, the burden on the operator is reduced and the abnormality determination can be performed objectively.

According to the fourth aspect of the invention, a voice pattern for informing abnormality is stored in advance in the storage means for each abnormality degree of the input data, and when the abnormality detection signal is output, the abnormality degree of the input data is output. The audio pattern for notifying the occurrence of abnormality corresponding to is read from the storage means and reproduced on the speaker. Even when an abnormality occurs when the operator takes his eyes off the display, the operator can swiftly respond to the abnormality that has occurred and the degree of the abnormality is notified by voice.

According to the fifth aspect of the invention, a warning voice message for each abnormality degree of input data is input to the storage means in advance via the microphone, so that when an abnormality occurs, a warning corresponding to the abnormality degree is generated. The voice message can be reproduced by the speaker installed on the monitored side. Therefore, even if an abnormality occurs while the operator is completely away from the monitoring device such as the display, an intruder in the monitoring area is automatically warned of the abnormality and the safety of the device is improved. Reliability and reliability are improved.

According to the sixth aspect of the invention, the start of voice input by the operator is detected by monitoring the change in voice input from the microphone, and the input voice data is started. This eliminates the need for the operator to operate the device prior to the voice input to the microphone, thus improving the operability.

According to the seventh aspect of the present invention, by displaying the detection amounts of the plurality of sensors installed on the monitoring target on the monitoring target screen of the display as a two-dimensional distribution graph,
The condition of the monitoring target can be grasped in two dimensions, improving the usability of the device.

[Brief description of drawings]

FIG. 1 is a block diagram showing an overall configuration of a first embodiment according to the first invention.

FIG. 2 is a block diagram showing an internal configuration of the image processing apparatus of FIG.

3 is a block diagram showing an internal configuration of the voice processing device of FIG. 1. FIG.

FIG. 4 is an explanatory diagram showing a procedure of storing data in the main storage device of FIG.

FIG. 5 is a block diagram showing an overall configuration of a second embodiment according to the first invention.

6 is a block diagram showing an internal configuration of a main part of FIG.

7 is a block diagram showing an internal configuration of a main part of FIG.

8 is a block diagram showing an internal configuration of a main part of FIG.

FIG. 9 is an explanatory diagram showing a processing procedure of a third embodiment according to the second invention.

FIG. 10 is a block diagram conceptually showing a processing operation of the fourth example according to the third invention.

11 is an explanatory diagram showing a specific processing method of FIG.

FIG. 12 is a block diagram showing the configuration of a fifth embodiment according to the fourth invention.

FIG. 13 is a block diagram showing the configuration of a sixth embodiment according to the fifth invention.

FIG. 14 is a block diagram showing the configuration of a seventh embodiment according to the sixth invention.

15 is a block diagram showing an internal configuration of a main part of FIG.

FIG. 16 is an image displayed in the eighth embodiment according to the seventh invention.

17 is an explanatory diagram corresponding to the image in FIG.

FIG. 18 is an image showing contour lines created in the eighth embodiment.

FIG. 19 is an image displayed in the eighth example.

FIG. 20 is a block diagram showing a conventional example.

FIG. 21 is a display example of a conventional display.

FIG. 22 is a display example of a conventional display.

[Explanation of symbols]

 1 Monitoring Camera 2 Microphone 6 Process Data Controller 7 Multi Window Display 8 Computer 9 CPU 10 Auxiliary Storage Device 12 Speaker 16 Main Storage Device 17 Image Processing Device 18 Audio Processing Device 19 Bus 20 Process Data Input / Output Unit 21 A / D Converter 22 Image data compression / decompression device 23 Image memory 24 D / A converter 25 A / D converter 26 Audio data compression / decompression device 27 D / A converter 28 Audio data memory 29 Data storage area 30 Data management area 31 Speaker 32 Time division multiplexing Transmission device (TDM) 33 Moving image compression unit 34 Audio compression unit 36 Process data input / output unit 38 Transmission line 39 Microphone 40 Time division multiplexing transmission device (TDM) 41 Transmission control unit 46 Time division multiplexing bus 47 Moving image compression / decompression unit 48 Audio compression / decompression unit 4 9 Process Data Transmitter / Receiver 50 Computer 51 CPU 52 Main Memory 53 Auxiliary Memory 54 Frame Memory 55 Digital Interface (IF) 56 Process Data Input / Output Section 58 Large Display 61, 62 Control Section 63, 64 Input / Output Buffer 65 Time Division Multiplexing Interface (IF) 66 Serial control unit 67 Switching device (MUX) 68 Expansion unit 69 D / A converter 70 Compression unit 71 A / D converter 81 Reference image data 82 Reference audio data 83 Reference process data 84 to 86 Calculation unit 87 -89 Weighting unit 90 Abnormality detection unit 91 OR circuit 92 Voice information memory 93 Serial interface (IF) 94, 95 Switcher 96 Voice recognition unit 97 Current voice data buffer 98 Reference voice Data buffer 99 Correlation calculator

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Shiro Nakatoge 1 Fujimachi, Hino City, Tokyo Inside Fujifacom Control Co., Ltd. Electric Co., Ltd. (72) Inventor, Yuji Katada, Fujimachi, Hino City, Tokyo, 1st, Fujifacom Control Co., Ltd. (72) Inventor, Takashi Karasawa, Fujimachi, Hino City, Tokyo, 1

Claims (7)

[Claims] 1. A multimedia process monitoring / control for displaying an image obtained by imaging a monitoring target and process data input / output with respect to the monitoring target on a display, and reproducing a voice collected from the monitoring target by a speaker. In the device, a means for cyclically storing each of the above data in the main storage device, and when an abnormality is detected, saves each data stored in the main storage device to the auxiliary storage device for a certain period before and after that. A multimedia process monitoring / controlling apparatus comprising: a means and a means for reproducing each data saved in the auxiliary storage device in synchronization with a display and a speaker. 2. The multimedia / process monitoring / controlling apparatus according to claim 1, wherein the input process data is compared with the latest process data already input and stored in the main storage device to determine whether they are the same. A multimedia multimedia device characterized by comprising: a means for determining a change in process data status by detecting a change in the process data; and a means for storing process data in the main memory only when a change in process data status is detected. Process monitoring and control equipment. 3. The multimedia / process monitoring / controlling apparatus according to claim 1, further comprising a storage unit that stores reference data of a plurality of stages from a normal state to an abnormal state for each of image, voice and process data. A method of calculating the abnormality degree of the input data from the similarity obtained by comparing each input data and the reference data, and comparing the abnormality degree of the input data with a preset threshold value. A multimedia / process monitoring / controlling device comprising: means for determining whether or not there is an abnormality and outputting an abnormality detection signal when there is an abnormality. 4. The multimedia process monitoring / controlling apparatus according to claim 3, wherein a storage means for storing in advance an abnormality occurrence notification voice pattern for each abnormality degree of the input data, and an abnormality detection signal are output. A multimedia / process monitoring / controlling apparatus comprising: a means for reading out an abnormality occurrence notification voice pattern corresponding to an abnormality degree of input data from a storage means and reproducing the same by a speaker. 5. The multimedia process monitoring / controlling apparatus according to claim 3, wherein a warning voice message for each abnormality degree of the input data input via the microphone is stored in the storage means, When the detection signal is output, a means for reading a warning voice message corresponding to the degree of abnormality of the input data from the storage means and reproducing it by the speaker installed on the monitored side is provided. Process monitoring and control equipment. 6. The multimedia process monitoring / controlling apparatus according to claim 1, further comprising a voice data storage unit for sequentially storing voice data input from a microphone for an operator. A calculating means for calculating a correlation value between the voice data and the voice data before it, and a means for taking the voice data input from the operator microphone into the internal voice data processing section when the correlation value is equal to or less than a predetermined value, A multimedia / process monitoring / controlling device characterized by being equipped with. 7. The multimedia / process monitoring / controlling apparatus according to claim 1, wherein a plurality of sensors installed on a monitoring target and a sensor position and a sensor detection amount are displayed on a display. A multimedia feature characterized by comprising: means for creating a two-dimensional distribution graph of the detection amount corresponding to the displayed monitoring target screen; and means for displaying the two-dimensional distribution graph on the monitoring target screen of the display. Process monitoring and control equipment.