JPH0579951A - Monitoring system - Google Patents

Abstract

PURPOSE:To synchronously display not only the process data but also the image and voice data during reproduction in a process control monitoring system. CONSTITUTION:A data base 104 storing the process measurement data and an image/voice record section 108 recording at least one of the sound data and the image data are provided, the measurement data, sound data, and image data are correspondingly stored, the data base 104 reproduces the measurement data, and the image/voice record section 108 reproduces at least one of the sound data and the image data. A work station 103 synchronizing the image/ voice record section 108 and the data base 104, a display 111 outputting the measurement data and the image data, and a speaker 112 outputting the sound data are provided.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION The present invention relates to process data obtained in a system for monitoring and controlling power plants, chemical plants, hospitals, steel, railways, water and sewage management, images from ITV cameras and audio microphones, and sound information. The present invention relates to a system for recording and displaying video, sound, and process data in association with each other when an analysis is required such as an abnormality.

[0002]

2. Description of the Related Art Generally, in a monitoring system such as a process control monitoring system, a monitor device is installed in a central control room, an ITV camera is installed on the process side, and the state of the process is monitored by an image taken by the camera. The method of visually monitoring the process status is adopted by displaying on. The image and sound are recorded on a recording medium such as a video tape. The video and sound are rewound and played back in the event of an abnormality.

On the other hand, the process data (measurement data) sent from the process every moment is displayed on the monitor and meters of the central control room, and is also stored in the database in the system to analyze abnormalities and analysis. It can be retrieved from the database when needed. This system is introduced in the plant operation history display system of JP-A-60-93518.

However, since the recording mediums for the image information and sound information and the process data are different, it has conventionally been necessary to separately reproduce or analyze when reproducing or analyzing. For example, when an abnormality occurs on the plant, the instrument detects it and a buzzer sounds. Then, in the overall process diagram, find the corresponding device, determine the cause and countermeasure, and take action. It took a lot of effort to estimate the cause and the faulty device because many related data and images were needed. In analysis using video, a method of looking at the video in advance to find the vicinity of the abnormality and then examining the vicinity in the process data, or after finding the abnormal point in the process data, wind the video up to that point. The method of returning and reproducing is used.

However, in general, an ITV for monitoring a plant or the like
Since there are multiple cameras and the images from them are recorded in multiple videos, in order to view the images from each camera in association with each other during analysis, such as during abnormal conditions, rewind all of those videos to the desired location and play them back. The burden on the operator was heavy.

On the other hand, desired data from the database is
It is difficult to properly extract the image information without using such image information, and in many cases, after outputting a huge amount of information with a printer, an operator had to analyze it.

[0007]

The conventional system has the following problems.

(1) Since the process data cannot be referred to at the same time when the video and sound information is reproduced, even if the information is obtained from the video, it is troublesome and time-consuming to retrieve the process data thereafter.

(2) Even if the operator displays the process data on a trend graph and knows when the operator wants to refer to the image, it takes time and effort to display the image. For this reason, it was not possible to quickly grasp the actual situation at the site.

(3) Even when a process data such as an abnormal value is searched for, it takes a lot of time to display the image related to the process data.

(4) When displaying the recorded process data, particularly when displaying a large amount of the recorded data in fast-forward, a heavy load was placed on the computer.

(5) Due to the limitation of the data display method, it was not possible to meet the demand for detailed and skipped requests. When analyzing in detail, if the related video and sound are also referred to in slow playback, it is more useful for analysis, but this function does not exist.

(6) An important factor that determines the operation of the process is an operation command from an operator. Since these are not reproduced, what operation changed the process state,
I did not understand.

(7) Even if the operator memorized the command operated, the command could not be searched. Therefore, it was necessary to analyze the process data and the like to estimate the time when the command was issued.

(8) Since there is no relation between the process data and the video information, even if an abnormality is found on the video, what is displayed in that video and what data is output from that video It is only known to trained veterans, and to those who do not know which process equipment the data relates to.

(9) Since the place for displaying the image and the place for displaying the process data were separated, the operator had to move his or her line of sight and could not see the ever-changing data and the image.

(10) The conventionally used video tape has a problem in its reproducing ability in terms of quick access to image data. On the other hand, if an optical disk is used, quick access is possible, but since the video data becomes very large, a large capacity disk is required for recording.

It is an object of the present invention to provide a monitoring system capable of reproducing measurement data and video or sound in synchronization with each other.

[0019]

In order to achieve the above-mentioned object, a first storage means for storing measurement data relating to an object to be monitored.
Storage means, and second storage means for storing at least one of sound data and video data relating to the monitored object, the measurement data, the sound data, and The video data is, in a surveillance system stored together with information for associating, first reproduction means for reproducing the measurement data, and at least the sound data and the video data. Second reproducing means for reproducing one side, a control section for synchronizing the first reproducing means with the second reproducing means based on the correspondence information, and a first outputting means for outputting the measurement data. Outputting means for outputting the sound data and video data
A second output means having at least one of the output means for outputting the data.

[0020]

The first storage means for storing the measurement data on the monitored object and the second storage means for storing at least one of the sound data and the video data on the monitored object. In the surveillance system in which the measurement data and the sound data and the video data are stored in association with each other, the first reproducing means reproduces the measurement data. To do. The second reproducing means reproduces at least one of the sound data and the video data. The control unit synchronizes the first reproducing means with the second reproducing means by the information of the correspondence. The first output means outputs the measurement data.
The second output means outputs at least one of the sound data and the video data. In this way, the measurement data is output in synchronization with the reproduction of the image or sound.

Detailed Description of the Invention

[0022]

Embodiments of the present invention will be described below with reference to FIGS. 1 to 31.

FIG. 1 shows the configuration of the plant control and monitoring system of this embodiment. A device (hereinafter, simply referred to as a control target device) 101, which is a monitoring target at a factory site, sends process data indicating an operation status at each time to a cable 135.
Through the control computer 102, which is the first input means. The control computer 102 analyzes the process data and sends a control signal to the controlled device 101 through the cable 136. In addition, the LAN 12 through the cable 137
Flow the process data to 0 and pass L through cable 138.
It receives an operator command from the AN 120 and processes it. As described above, the main functions of the control computer 102 are collection of process data, output of process data to the LAN, input of operator commands from the LAN, and control target device 10.
1 is the output of the process control signal.

The LAN 120 is a cable such as Ethernet for transmitting signals such as operator commands and process data. The LAN 120 includes an output cable 137 from the control computer 102, an input cable 138 to the control computer 102, and an output cable 1 from the database 104.
43, an input cable 144 to the data base 104,
Output cable 14 from workstation 103
0, input cable 13 to workstation 103
At 9, the device is connected.

The database 104, which is the first and third storage means and the first reproducing means, receives the process data and the like flowing in the LAN 120 through the cable 144 and records them together with the time t output by the internal clock. .. Further, when a data read command is input through the cable 144, the data designated to be read is sent to the LAN 120 through the cable 143.

A plurality of ITV cameras 110 are equipped with a camera control device capable of remotely controlling pan, tilt, zoom, etc. by inputting a control signal, and a voice microphone that moves together with the cameras. The camera 110 is the control target device 10.
The video and audio of No. 1 are sent to the switcher 109 through the cables 130 and 131. The switcher 109 sends a camera control signal input from the workstation 103 to the camera 110 through the cable 132. ITV camera 11
0 is a second input means.

As the video / audio recording section 108 which is the second storage means and the second reproducing means, a random accessable one such as an optical disk is used. It may be something like a video tape, but in this case the data search is sequential, so it takes time to search and display the data. All video and audio from the ITV camera 110 are input through the cable 133 through the switcher 109. Further, the workstation 103, which is the control unit, is the switcher 10
When a read command is input through the cable 145 through the cable 9, the specified video and audio information is output to the switcher 109 through the cable 134.

The switcher 109 sends a plurality of input video and audio signals to the workstation 103 via a cable 141.
Is a video / audio changeover switch for sending through the cable, and a signal transmission destination changeover switch for sending a camera control signal and a recorded video calling signal output from the workstation 103 through the cable 142 to the camera 110 and the video / audio recording unit 108. ..

The work station 103 has first and third output means for display 111, a speaker 112, and input means (measurement data output designating means, selection target) from the operator as output means to the operator. Is connected to an input device such as a keyboard 106, a mouse 105, and a touch panel 107 as a means for selecting () and a means for designating a search value of measurement data). Also, the cable 13
LAN 140 and cables 141 and 142 at 9 and 140
To connect to the switcher 109. Workstation 1
03 processes the process data input from the cable 139 to create a display screen, and displays it on the display 111 together with the video data input from the cable 141. On the other hand, the audio data input by the cable 141 is output by the speaker 112. The speaker 112 and the display for display 111 are second output means. In addition, a key input from the operator via the keyboard 106 and a mouse 105
Input from the input device such as the touch panel 107 is processed in the workstation 103, and is output as a control code of the control target device 101 through the cable 140, a switching instruction to the video / audio switching switcher 109, and control of the camera 110. It is output by the cable 142 as a calling code to the code, video and audio recording unit 108.

The operator monitors the system status indicated by the images, characters and graphics displayed on the display 111, and performs necessary operations and commands on the mouse 105,
This is performed using the keyboard 106 and the touch panel 107. In the explanation below, as an input device from the operator,
The touch panel 107 is used. Of course, other devices are also acceptable.

Next, referring to FIG.
The internal structure of 03 is shown. 201 is a CPU, 202 is a main memory, 203 is an IO, 204 is a graphics screen frame buffer for displaying process data on the display 111, 205 is a digitizer for converting an input video signal into a digital signal, and 206 is a video frame buffer. 207 are blending circuits for blending the graphics screen and the video.

FIG. 3 shows the structure of the video / audio recording unit 108. A CPU that receives various commands from the workstation 103, processes them, and issues recording and reproducing commands.
301, main memory 30 used for video buffering, etc.
2. An AD / DA converter 303 for converting a digital signal from the ITV camera 110 into an analog signal to be sent to the workstation 103 and a video / audio recording / reading unit 304.

FIG. 4 shows a display screen in the process control monitoring system. The display screen is
An overall process configuration diagram 401, a moving image display area 402 that mainly displays video from an ITV camera, a trend graph 403 that displays process data from the control target device 101, a clock 406, switches, help information, etc. are displayed. It comprises a work area 404, a process data display meter 405, and a menu area 407. In the menu area 407, a camera switch button 408, a selection target designation button 40 in the image and process data
9, switching between monitoring mode and playback mode, standard playback / slow playback switching button 410, simple editor call or graph selection button 411, data selection button 412 to be displayed, and help button 4
13 is displayed. Here, as the device for displaying the process data from the controlled device 101, there may be a data table, a scaler, or the like. Further, there may be a plurality of the data display means mentioned above on the display.

FIG. 5 shows the trend graph 403 displaying the process data in more detail. The trend graph 403 includes a data display unit 501, a data item display unit 502, a time cursor 503, a time axis 504, a data value cursor 505, and time axis movement buttons 506 and 507.

The process data is displayed as a graph on the data display section 501 and the data item display section 50.
The name is displayed on 2. Correspondence between data and its name is performed by line thickness and color or line type.

The time cursor 503 is used to display all the data currently displayed on the display (for example, the meter 405).
Of the data value, the image 402, the time of the clock 406, the point on the time cursor 503 of the trend graph 403) generated or recorded are represented using the time axis 504. That is, the time cursor 503 of the trend graph 403 is a time display unit that represents the time when the data currently displayed is recorded.

If the time at which the data to be displayed occurs is not within the time axis 504 currently being displayed, the time axis 504 indicates that the value of the time being displayed is in the right direction (the direction in which the time is returned, hereinafter the reverse direction). The value of the time is displayed in the time axis 504 by moving in the left direction (the direction in which time is advanced, hereinafter referred to as the forward direction). Also time axis 50
4 can be enlarged or reduced, and can be enlarged or reduced by designating a section. As a result, the time axis 504 can be expanded in the section to be viewed in detail, and the section other than that can be reduced.

The time axis move button 507 moves the value of the time displayed on the time axis 504 to the right and displays the time before the displayed time. On the other hand, button 508
Moves to the left to display the time after the displayed time.

The data value cursor 505 is used to search for process data. When the data value cursor is moved to the search value after selecting the process data to be searched, the time axis 504 and the time cursor 503 move, and the time cursor 503 comes at the time when the selected data takes the search value. Like

In the following embodiments, the trend graph will be used exclusively as the data display means for displaying the process data on the display. Of course, data display means other than the trend graph may be used.

The process monitoring system of the present invention has the following functions.

(1) By the operation of reproducing the recorded video, not only the video and audio are reproduced, but also the process data at the time when the video is taken is called and displayed.

(2) Trend graph time cursor 50
By specifying the time using the 3rd time display means,
The video and audio at the time when the data was recorded and the process data at that time are called and displayed.

(3) By designating process data and its search value, the data is searched, the data is called and displayed, and at the same time, the image at the time when the data was recorded and the image at that time. Call and display other process data.

(4) When the recorded video is played back, the display frequency with respect to the time of the process data changes depending on the playback speed.

(5) By pre-designating the display frequency relating to the time of the process data, the playback speed of the video and audio that matches the display frequency is determined, and played back and displayed.

(6) The operation information is recorded by the operator, and the operation of the operator is also reproduced when the image is reproduced.

(7) By recording the operation information from the operator and designating the operation data of the operator, the operation is searched for, and the image at the time of the operation and the process data are called and displayed.

(8) A selection target which the operator can select using a touch panel is defined in the image, and the operator selects it when the image is reproduced to display the related process data.

(9) A selection target that can be selected by the operator using the touch panel is defined in the image, and the operator selects it when the image is reproduced, thereby highlighting the related process data.

(10) In the video, an operator defines a selection target that can be selected by using the touch panel, and the operator selects the selection target at the time of reproducing the video to display a selection menu related to related process data. When one item is selected from the menu, the process data of the selected item is displayed.

(11) A selection target that can be selected by the operator using the touch panel is defined in the video, and the operator selects the selection target at the time of playing the video to display the related process data on the selection target in the video. To display.

(12) A selection target that the operator can select using the touch panel is defined in the video, and the operator selects the selection target at the time of playing the video to display the related process data on the video by computer graphics. Overlay with.

(13) A selection target that can be selected by the operator using the touch panel is defined in the video, and the operator selects the selection target at the time of reproducing the video to emphasize other selection targets in the related video. indicate.

(14) A selection target that can be selected by the operator using the touch panel is defined in the video, and the additional information of the selection target is displayed by the selection of the selection target when the video is reproduced.

(15) A selection target is defined in the image,
When the operator selects the process data at the time of playing the video, the video is switched to the video related to the selected process data and the selection target in the video is displayed.

(16) Define the selection target in the video,
When the operator selects the process data at the time of playing the video, the video is switched to the video related to the selected process data, and the selection target in the video is displayed and the data value thereof is displayed over the selection target.

(17) A selection target is defined in the image,
When the operator selects the process data at the time of playing the video, the video is switched to the video related to the selected process data, and the selection target in the video is displayed and the data value is displayed by computer graphics in an overlapping manner.

Hereinafter, the above-mentioned functions for reproducing the recorded process data, video data and audio data will be described in detail.

The function 1 will be described with reference to FIGS. 1 to 11. By selecting the mode switching button 410 using the touch panel, the recorded information standard reproduction mode is set. During playback, recording is performed on an optical disc different from the one used for playback. At this time, as shown in FIG.
The video controller 603 is displayed in the work area 404. As shown in FIG. 7, the video controller includes a forward double speed playback button 705, a forward normal speed playback button 704, a backward double speed playback button 701, a backward normal speed playback button 702, and a video stop button 70.
Have 3. If slow mode reproduction is selected with the mode switching button 410, the reverse 1/2 speed reproduction button 70 is used instead of the reverse 2 speed button or the forward 2 speed button.
6. Display the forward 1/2 speed button play button 707. However, here, the reproduction at the standard speed means the reproduction at the same speed as when the video and audio are recorded, and the forward direction means the direction in which the time advances. Therefore, for example, reverse double speed reproduction means that reproduction is performed at a speed twice as high as that when recording is performed in the direction opposite to the direction in which time advances. Here, the standard mode and the slow mode are provided in the reproduction mode when reproducing the recorded information, but it is not necessary to divide them in this way.

By pressing the forward standard speed reproduction button 704 on the touch panel, the video data and the audio data are reproduced at the normal speed at the forward direction and displayed on the video display unit 402. At this time, the time cursor 503 in the trend graph is moved in accordance with this video so that the process data at the time when the displayed video is recorded comes to the time cursor 503. When the time cursor 503 reaches a certain position, the process data is called from the database 104, and the time value displayed on the time axis 504 moves leftward (or rightward),
Process data at a new time not on the current time axis 504 is displayed. In addition to this, other process data display units, such as the meter 405, display the values that the data had when the image was taken, as well as the image. As described above, not only the video and audio are reproduced by the operation of reproducing the video, but also the process data at the time when the video is captured is called and displayed.

With this, it is possible to refer to the process data at that time while watching the image. Also, by using other play buttons, fast forward, reverse play, slow play, etc. of video information can be performed with reference to process data, and assistance for diagnosing operating conditions and issuing control commands,
Useful for finding and analyzing abnormal points.

Next, a method for realizing this embodiment will be described.

First, the data structure and recording method of the process data, video and audio data of this embodiment will be described. In FIG. 8, data 800 represents the structure of process data sent from the control device 101 to the control computer. Generally, since one cable inputs multiple types of data,
It has a structure such as a header 801, which indicates the beginning of process data, a data type 802, a data number 803, and data 804 to 806 which are process data. The control computer 102 sends a plurality of data of this format coming from each cable to the LAN 120. The database 104 decomposes the flowed process data and
The data 82 together with the time t of the clock in the base 104
Record in an array with a structure of 0s. Here, 821 is a data index, 822 is a data name, 823 is a time, 8
24 is process data. Thus, the database 104 has a table corresponding to the type of process data, and records the latest data together with the time t after the last element of the array which is the element of the table.

On the other hand, the process data block calling instruction from the workstation 103 is sent to the database 10.
4 is input, the data having the structure shown in the data 810 of FIG. 8 is sent to 103. Here, a header 811 indicating the beginning of the process data, a data type 812, the number of data 813, process data data 814 to 816, time data 817 of the data 814, time data 818 of the data 815, and data 816. Is composed of time data 819. Of course, the length of this data, the interval of time data, etc. differ depending on the type of block call instruction.

Next, recording of video and audio data will be described. The structure of video / audio data to be recorded is 830. Generally, since video data from a plurality of cameras is recorded, the video / audio data has an index 831 (disk No.) and a data name 832 (camera No. or boiler No.). 834 is the time when the sound is recorded, 833 is the sound information, 835 is the time when the video is recorded, and 836 is the video information. Here, as shown in the figure, the video and audio may not be recorded separately, but may be recorded together. When recording together, time information will be shared.

A method for recording and reproducing the above video and audio data will be described with reference to FIG. In the present invention, the procedure of three steps shown in 901 to 903 for video recording is first executed by the CPU 201 in the workstation 103. After this procedure is executed, recording starts at 904. In video recording,
When the system is started up or when the playback mode is ended and the mode is returned to the recording mode, the entire video screen is first recorded, and as shown at 905 below, the video is recorded at 906 only when the recording conditions are satisfied. To do. As for audio information, the capacity required for recording is relatively smaller than that of video information, so recording will be performed at any time. Hereinafter, recording and reproduction of video information will be described exclusively.

In the video recording target determination procedure 901, the target to be recorded is determined. Specifically, it is one of the following methods.

(1) All the camera image screens are recorded. The realization method is to record all video signals from the camera.

(2) An area including a portion for outputting process data, a moving portion, a changing portion, etc. is designated in advance, and only that area is to be recorded. Data 84
Is the data structure of the video data 836 in this case.
The elements of the data 840 are the image data 84 to be recorded.
6, and its position information, that is, the coordinate values 841 and 842 of the captured image data, the image data size (spatial size of the screen) 843 and 844, and the closest full screen data before that are recorded. The time (or index) 845 when The realization method is that the ITV camera records the entire screen when zooming, chilling, panning, etc.
When the camera stops after such a camera operation, the video data 836 is sent to the workstation 103, image analysis is performed, and the area including the recording target is determined. This area may be rectangular, for example, for simplicity. When this area is determined, positional information such as the coordinate value and size of the area is sent to the video / audio recording unit 108, and the CPU 301
Takes only this area from the image sent from the camera and records it. After that, only this area is recorded until the camera operation is performed again. During reproduction, the CPU 301 blends the video data at time 845 with the recorded data 846 to generate a full screen.

In the video recording condition determination procedure 902, the conditions for recording the video are determined. Specifically, it is one of the following conditions.

(1) Record at a set time interval. This has a clock in the workstation 103 or the CPU 201, 301 in the video / audio recording means 108. In the case of the former, a video recording command to record at regular intervals,
It is sent to the voice recording unit 108. In the latter case, only the recording start command is sent to the video / audio recording unit 108, and the CPU 30
1 manages the recording time.

(2) The video from each camera is recorded when the difference from the previous screen exceeds a certain threshold value. This is because the CPU 301 calculates the difference value between the video information of the screen stored in the main memory 302 in the video / audio recording unit 108 and the current video information, and sends a recording command to the video / audio reading unit 304 according to the calculated value. ..

(3) If each process data exceeds a certain value peculiar to the data, it is recorded. This is the process data coming into the workstation 103 to the CPU2.
The video / audio recording unit 108 is instructed to record the image of the camera taking the image regarding the abnormal data.

(4) Record each process data when the difference from the previous data exceeds a certain value peculiar to the data. The implementation method is the same as in (3).

(5) Weighted average of each process data,
That is, if the weight is wi (wi ≧ 0) and each process data is di, then e = Σwi * di is recorded when it exceeds a certain value. Realization method is (3)
Is the same as.

(5) Recording is performed at a predetermined time interval, and when any of the above conditions is satisfied, recording is performed at a shorter time interval, and when the condition is not satisfied, the original time interval is restored.

In the video recording method determination 903, the recording method is determined. Specifically, it is one of the following methods.

(1) The video information from the ITV camera is stored as it is.

(2) The difference from the previous screen is recorded. In this, the CPU 301 takes the difference between the video image buffered in the main storage unit 302 and the current video image. At the time of reproduction, a video to be recorded is created by adding / subtracting the difference from the time when all the recording targets are recorded to the present time.

The display of the video data recorded as described above at time t is performed according to the procedure shown in FIG. Step 1
At 001, the video data index 821 and time t
Is specified. Here, the video index is specified by the workstation 103, and the time t is specified by the workstation 103 or by the CPU 301 in the video / audio recording unit 108. The video / audio recording / reading unit 304 uses steps 1002 and 1003.
If the image at time t is not recorded as shown in FIG. 3, the image data at time s closest to t recorded before time t is read. If the video data is the video information recorded as it is in step 1004, it is used. On the other hand, if the difference is recorded, the procedure 10
At 05, the video information that is closest to the time t and is not the difference value is read out, and the main memory 302 in the video / audio recording unit 108 is read.
In step 1006, and the difference from that is taken in step 1006 to create a video. If the video is the entire camera image, display it. If not, it is displayed after being combined with the background in 1008.

CPU 301 in video / audio recording section 108
When a playback command specifying a playback direction and speed is sent from the workstation 103, the display time data t internally held is advanced by the following equation.

T = t + a * w where w is the video reading speed at the standard playback speed,
a is positive when the playback direction is forward, negative when it is backward,
The coefficient has an absolute value of 2 at double speed and 1 at standard speed. In the video display during reproduction, when the time data 835 exceeds the time data 835 in the case of forward reproduction, the video data 836 is sent to the workstation 103, and in the case of reverse reproduction, this time data t is next to 835. When it is less than the time data, the video data 836 is sent. When the workstation 103 requests the generation time of the image being displayed, this time t is sent to 103.

FIG. 11 shows a processing procedure for realizing the first function based on the above recording / reproducing system. Step 11
In 01, the reproduction mode is selected from the menu. At this time, the workstation 103 displays the control button indicated by 603 in FIG. In process 1102, the workstation 103 processes an input signal from a pointing device such as a touch panel and detects the type of button by checking the input position. At this time, in order to show that the button has been pressed, the color of the pressed button is changed as shown in FIG. 6 to be displayed again on the display, and the reproduction direction and speed are determined. In process 1103, the time t at which the process data to be displayed next is generated is determined based on the determined direction and speed.

Specifically, there are the following two methods.

(1) Inquire about the time t at which the video and audio recorded on the video / audio recording unit 108 is recorded.

(2) The time t represented by the following equation is set as the time to be displayed next.

T = t + a * v Here, v is the time for rewriting all the data currently being displayed once, a is positive when the reproduction direction is the forward direction, and negative when the reproduction direction is the reverse direction, The coefficient has an absolute value of 2 at double speed and 1 at standard speed. However, the time to rewrite the data changes depending on the other load of the computer, so (1)
The method of is also used together. By using this, it is possible to advance the time of the next display information by the advance time of the video and audio information for the time required for the workstation 103 to display.

In the process 1104, it is judged whether the process data to be displayed at the time t is the data buffered in the workstation 103, and if they are sufficient, they are displayed. However, the case of being sufficient is the case where the process data at time t is buffered, or the data at time t is not present but the data before and after that is buffered. When only the data before and after is buffered, the data closest to t is substituted or data is created by linearly interpolating the data before and after. If not enough, in process 1105, the workstation 103 determines the range of data to be read as display data from the data base 104 based on the display speed and direction, and in process 1106,
The type of process data to be displayed and the range of data to be read are sent to the database 104 via the LAN, and the process data requested by the database 104 is sent to the workstation 103. In process 1107,
Video or audio information is displayed or output, and in process 1108, each process data sent by the workstation 103 is displayed as a trend graph or a meter to display individual process data recorded in the main memory 202. It is displayed together with the video information and audio information by the way. Next, the second function will be described with reference to FIGS. 1 to 5 and 12. The time cursor 503 is moved left and right by moving the cursor 503 left and right while pressing the cursor 503 with a finger using the touch panel 107. At this time, as shown in FIG. 12, the time cursor 503 directly in the trend graph 403 is displayed.
By moving the time cursor 1201 in another trend graph 403 to the time pointed by the time cursor 503, the video at the time determined by the time cursor 503 is displayed in the call video display area 402. .. At this time, the meter 405 and the like in FIG. 2 also display the data of the time pointed by the time cursor 503. Of course, it is also possible to specify a time that is not currently displayed on the time axis of the trend graph 403 by moving the time axis movement buttons 506, 50.
It can be done by using 7. By specifying the place to refer to the process data displayed as above,
The video at the time when the data was recorded and other process data at that time can be referred to. As a result, the operator can directly display the time when he / she wants to refer to the process data by looking at the trend graph 403 and the like to display the image.

As a result, it is possible to refer to a specific state of the site by referring to the process data.

A method for realizing this embodiment will be described below with reference to FIG. The algorithm of FIG. 11 is processing 1301.
To detect the time t pointed by the time cursor, and process 1
The only difference is that the time t 302 is already buffered in the workstation 103. Processing 1
In 301, the coordinate value of the input signal from a pointing device such as a touch panel is displayed on the workstation 10.
The CPU 201 in 3 performs processing to redraw the time cursor 503 on the coordinates, and at the same time, the time t indicated by the time cursor 503 is calculated from the coordinate value. If the data at time t is not buffered in the workstation 103, steps 1105 and 110 in the first embodiment will be described below.
6, and data, video, and audio are displayed in steps 1106 and 1107 below.

The third function will be described. As shown in FIG. 14, a data item 1401 in the data item display section in the trend graph 403 is displayed using the touch panel 107.
After selecting, the data value cursor 505 is moved to the value to be searched and the search value is determined. At this time, the time cursor 503 moves to the time when the selected data takes the value indicated by the data value cursor 505, and another trend graph 4
Also in 03, the time cursor 1402 moves to that point and the image at that point is displayed on the image display unit 402.
At this time, the meter 405 in FIG.
The data at the time pointed by the sol 503 is displayed. Here, the search is performed only once in the opposite direction with respect to the time axis. When further searching is desired, the time axis moving button 506 is pressed to search in the opposite direction. On the other hand, when searching in the forward direction, the button 507 is pressed to perform the search in the forward direction. When a value is searched for in the process data displayed as above, the search result is displayed, and it is possible to refer to the video at the time when the displayed data was recorded and other process data at that time. it can.

A method for realizing this embodiment will be described below with reference to FIG. In process 1501, the workstation 103 processes the coordinate value of the input signal from the pointing device such as the touch panel 107, and the data item display unit 50.
The data selected as the search target in 2 and the search value indicated by the control value cursor 505 are determined. Next, in process 1502, a search direction, that is, a forward search or a backward search with respect to the time axis is determined. This is, for example,
Basically, the backward search is performed once, and when the forward button 507 of the time axis movement button is further pressed, the search is performed in the forward direction and further when the backward button 506 of the time axis movement button is pressed. It is assumed that the search is performed in the opposite direction. The workstation 103 determines whether this button has been pressed.
Will do. In the process 1503, the database 10
In step 1504, a search command including the search target, search value, data creation time currently displayed, search direction, etc. is issued to step 4.
In the first step, the search value and the display time that were found first are determined. Since the following steps 1104 to 1109 of the first embodiment are the same, description thereof will be omitted.

With this function, it is possible to compare and analyze other process data values with video information, and to call out abnormal values that occur very rarely, under the condition that a certain process data takes a constant value. Become.

Embodiments for the fourth function are shown in FIGS.
This will be described in 7 and 18. In FIG. 16, when the forward double speed button 705 is selected when the video information is reproduced, the time axis 504 in the trend graph 403 displays the time in the double range, and the currently displayed process data is It is redisplayed according to the new time axis, and the data that was not displayed is read from the database and displayed according to the time axis. Next, the video display unit 402 displays the standard speed of 2
The image is displayed at double speed, and the time cursor 503 moves. In this way, during double speed playback, the trend graph 4
It is possible to display the data of a longer time in 03, and it is possible to overview the change of the data with time. Such a display is convenient for data retrieval.

On the other hand, in FIG. 17, when the 1/2 speed playback button 707 is selected, the time in the range half the time when the time axis 504 is the standard speed is displayed. At this time, since the data can be displayed in more detail, the data that was not displayed at the standard speed is read from the database and is redisplayed together with the existing data. That is, when the video is played back, the method of calling the process data and the display method are changed depending on the playback speed. As a result, when the reproduction speed is increased, data for a long time can be displayed on the trend graph 403, which facilitates data search, overview, and the like. Further, when the reproduction speed is fast when calling the process data, the interval of the generation time of the called data becomes large, but the roughness of the calling becomes inconspicuous by using this display. On the other hand, if you want to slow down the playback speed,
Data can be displayed in detail. Therefore, when detailed analysis is desired, the process data can be displayed in detail only by slow-playing the video.

As a result, the display frequency of the process data with respect to the time is changed depending on the reproduction speed, so that the load on the computer can be suppressed to some extent.

A method of realizing this embodiment will be described with reference to FIG. In step 1102, the direction and speed at which the video information and audio information are reproduced are determined by the input from the operator. In step 1801, the workstation 103 determines the process data display method and the call method based on the determined speed. The display method determines the time axis display unit in the trend graph 403, etc., that is, how often the time is cut, and the calling method is
Determine the time interval between data within the calling block and the length of time in a single calling block. If the data buffered in step 1104 is insufficient, then in step 1105 the determined time interval and time length are coded and sent to the database. Based on the code sent in step 1105, the database reads the block data for the time interval and the time length and sends it to the workstation 103. Hereinafter, the workstation displays data based on the determined display method. This part corresponds to step 11 of the above embodiment.
Since it is the same as 04 to 1109, it is omitted.

The fifth function will be described. In FIG. 19, a trend graph 40 is displayed as a method of displaying process data.
In the section 1901 of the time axis of 3, the time axis 504 is 1/2
To the section 190 and the section 190
In No. 3, it is doubled. At this time, the interval between the generation times of the process data to be displayed is twice as long as the section 1902 in the section 1901 and ½ as long as the section 1902 in the section 1903. Therefore, in the section 1901, the same display as in the case of the double speed reproduction in the previous embodiment is displayed, in the section 1902, the same display as the standard speed reproduction is displayed, and in the section 1903.
Shows the same display as 1/2 speed playback. In this case, if the standard forward playback is performed by the video controller 603 using the button 704, the video is displayed in the video display area 4
02, it is displayed at double speed when the time cursor 503 is in the section 1901, at standard speed when it is in the section 1902, and at half speed when it is in the section 1903. That is, by setting the method of displaying the process data in advance, when reproducing, the speed at the time of reproducing the image is set according to the display method, and the reproduction is performed at that speed.

As a result, not only can the operator specify the display method of the data to be displayed, but the image can be played back slowly when the data is desired to be viewed in detail and at the point where the data is desired to be skipped.

A method for realizing this embodiment will be described with reference to FIG. In step 2001, input from the operator reduces the size,
Specify the time axis section to be expanded. In step 2002, the operator selects either enlargement or reduction for the section. These designations and selections may be performed by, for example, a menu, or by designating a section using a touch panel or the like and then grasping an end point of the section and extending or contracting the section. At this time, in step 2003, the time axis is displayed again, and if necessary, a block is called from the database to display the process data again. At this time, the workstation 103 determines the reproduction speed of each section and stores it in the main memory 202. Next, reproduction is started, and the display time t is determined in step 1103. After the CPU 201 determines the section including the time t, if it is a section different from the previous section, in step 2004, a reproduction command such as a reproduction speed and a direction is transmitted to the video / audio recording unit 108. Hereinafter, the steps 1104 to 1109 of the above-described embodiment are the same.

The sixth function will be described. In FIG. 21, when the video information is reproduced, not only the process data but also the operation information from the operator is reproduced. At this time, the video and process data displayed on the display at that time are displayed and the mouse cursor 21
The operator's input indicated by 01 or the like is reproduced and displayed. At this time, by newly selecting the image to be displayed in the image display area 402 as indicated by 2102, it is possible to refer to the image information which is caused by the operation of the operator and which cannot be seen at the time of recording. Further, it is possible to display process data and the like which were not displayed by the same operation. As a result, for example, an abnormality caused by an operation error of the operator can be found quickly. In addition, the control operation education can be greatly enhanced.

Further, the operation information of the operator is reproduced, and it is possible to know whether the process operation status change is due to the operation instruction of the operator. Further, by recording and reproducing such an operation command, it can be used for explaining the operation procedure, and can be used to monitor the operation system of the educational system and the operator.

The seventh function is to input the operation information from the operator who wants to search, to search the input operation information, and to call and display the operation information and video information, the voice information and the process data at that time. It Thereby, the information search can be performed by using the operation performed by the operator as a mark.

Therefore, since it is possible to retrieve the operation command of the operator, it is possible to retrieve the change of the process data and the change of the image caused by the operation of the operator.

A method for realizing the above two embodiments will be described below. In FIG. 8, the data 850 is the database 104.
The screen information to be recorded in FIG. Screen information 85
0 indicates a time 851, a camera name 852 that captures an image displayed in the moving image display area 202, process data names 853 to 855 displayed on the trend graph 403, and a meter 4
05 and other data display means 85
It consists of 6 mag. This data is displayed by the operator in the moving image display area 4
02 The video displayed on 02 and the trend graph 4
When the data displayed in 03 is changed, added, or deleted, the workstation 103 sends it to the database 104.

The data structure of the operation data input by the operator is the same as the data structure 820 of the process data in FIG. However, instead of the process data value 824, an operation command input as operation data (a command generated by the workstation 103 processing the coordinate value input by the operator using a pointing device) is entered. The workstation 103 also sends this data to the database 104 when an operation command is issued.

Reproduction is the same as the algorithm shown in FIG. However, in step 1108, the process data was created by interpolating the data to be displayed with the data value at the time closest to the display time t, or by interpolating the preceding and following data, but the operator operation data is executed in the forward direction. In the case of reproduction, the display time t is over the recording time of the operation data, and in the case of backward reproduction, it is below the recording time of the operation data. Further, the content of the screen information data recorded at time 851 is displayed when the display time t becomes 851 or more in the forward reproduction, and when the display time t becomes 857 or less in the reverse reproduction. ..

The search is the same as the algorithm shown in FIG. However, after the display time t is determined in step 1504, the screen information data closest to t before t is first called in step 1106, then the process data to be displayed is determined and the process data is called.

The following embodiments relate to the related display of video and process data when reproducing video, audio and process data in all the above embodiments.

The eighth function is to select the window of the boiler displayed in the moving image display area 402 in FIG.
01 is defined, and when this is selected, a graphic indicating that the selection target is selected is displayed, and the process data name 2202 generated therefrom is displayed in the process data item in the trend graph 403. , Process data 2203 is displayed in a graph. As described above, by selecting the selection target in the image using the pointing device, the related process data is displayed. Here, of course, even if the selection target is not the window of the boiler, it may be registered in the control computer as the selection target in advance. In addition to displaying the data on the trend graph 403, the data can be displayed on the meter 405 or the like, but for simplicity, only the case of displaying the data on the trend graph 403 will be shown.

The ninth function is to define the upper pipe of the boiler displayed in the moving image display area 402 in FIG. 23 as the selection target 2301, and when this is selected, a graphic indicating that the selection target has been selected. Is displayed,
When the process data 2302 related to the selection target is the vapor pressure already displayed in the trend graph 403, the vapor pressure 2302 of the process data item is highlighted and the graph 2303 is highlighted and selected by the operator. It indicates that the data is related data of the selected selection target. That is, when the data regarding the selection target in the selected video is already displayed, the data is highlighted to indicate that the selection has been made.

The tenth function is to define the left pipe of the boiler displayed in the moving image display area 402 in FIG. 24 as the selection target 2401. When this is selected, a graphic indicating that it has been selected is displayed. Is displayed, and if there are multiple process data related to the selection target,
A selection menu 2402 having process data as an item is displayed right next to the selection target in the moving image, and the process data to be referred to is selected from the selection menu using a pointing device to display the data in the trend graph 403. .. That is, when there are a plurality of data related to the selection target in the selected video, the selection menu is displayed so that the operator can select the data to be referred to.

In the eleventh function, the main body of the boiler displayed in the moving image display area 402 in FIG. 25 is defined as a selection target, and when this is selected, a graphic showing that the selection target is selected. 2501 is displayed, and process data 2502 to 2504 related to it are displayed.
It is displayed by superimposing where each data of the moving image is related. That is, by selecting the selection target in the image using the pointing device, the related process data is displayed at the related location in the image.

In the twelfth function, the whole of the boiler displayed in the moving image display area 402 in FIG. 26 is defined as a selection target, and when this is selected, graphics 2601 showing that it has been selected. Is displayed, and the temperature distribution data related to the selected object is called up and displayed on the image by computer graphics 2602.
By selecting the selection target in the image using the pointing device, the process data displayed by computer graphics is overlaid on the selection target.

In the thirteenth function, the entire boiler displayed in the moving image display area 402 in FIG. 27 is defined as the selection target, and when this is selected, the graphics 2701 indicating that the selection target is selected. Is displayed, and the graphics 2702 is displayed in the fuel supply unit that is closely related to the selection target. That is, the selection target in the image is
By using the pointing device to make a selection, the selection target in the video that is related to the selection target is displayed.

In the fourteenth function, the entire boiler displayed in the moving image display area 402 in FIG. 28 is defined as a selection target, and when this is selected, graphics 2801 indicating that the selection target is selected. Is displayed, and additional information 2802 such as the control method and maintenance information regarding the selection target is read from the database and displayed on the video. That is, the selection target in the image is
By selecting using the pointing device, additional information such as the operation method, control method, and maintenance information of the selection target is displayed.

As described above, with the functions 8 to 14, the process data and the device shown in the image information can be associated with each other, and the process data can refer to the related device in the image or the process by the device in the image. You can refer to the data. Thus, for example, even a trained operator can easily operate and monitor the device by viewing the image and data.

Next, the process data is used to display information in the image.

In FIG. 29, the fifteenth function is to select the process data item 2302 in the trend graph 403. By highlighting the process data item 2302, the process data item is selected. Further, graphics 2301 indicating that there is a selection target related to the process data in the image display area 402 is displayed. That is, a graphic showing which selection target the process data is related to in the image is displayed. The 16th function is shown in FIG.
In (1), by selecting the process data item 2302 in the trend graph 403, the process data 3001 is displayed in the image 402 in a superimposed manner on the selection target to which the process data is related.

The seventeenth function, in FIG. 31, is to select the process data item 2302 in the trend graph 403 to display the process data in the image 402 in the computer graphics 3101 as a selection target to which the process data is related. Overlap and display.

A method of realizing the above-described embodiments of the functions 8 to 16 will be described with reference to FIG. The shape model of the controlled device 3201 to be monitored is recorded in the workstation 103. A part of this shape model is defined as a selection target that receives an input from an operator. This shape model may be three-dimensional data such as a CAD model, a process design drawing, or a simple rectangular area defined by the operator looking at the image from the camera 110. In order to determine the position and size in the image to be selected, the ITV camera 110 records the angle of view information, the vertical angle information, and the horizontal angle information in the database 104 together with the time. Or ITV
The CPU 201 in the workstation 103 calculates the angle of view information, vertical angle information, and horizontal angle information from the camera control command sent to the camera and the initial setting of the ITV camera, and sends the result to the database 104 and records it together with the time. Since the ITV camera and control target device do not move, the position and size of the selection target in the image can be known by combining the initial position of the camera, the camera information to be recorded, and the shape model.

ITV camera 1 showing process device 3201
10 is the vertical angle information 3211 and the horizontal angle information 3
By giving 212 and the zoom value, the image 320
Create 2 to 3204. Here, the image of the device 3201 displayed in each image is 3205 depending on the zoom value,
3206 and 3207. Scaling of the selection target inside the computer is performed according to each zoom value. Here, when a simple rectangular area is used as the selection area, the selection target corresponding to the image 3202 is 3208, the image 3203 is 3209, and the image 3204 is 3210. All scaling is linear and can be done easily.

When an operator makes a selection with respect to the selection object defined in this way, or when some message command is sent from another selection object, the selection object is displayed, relevant data is output, or the like. Define to activate the action.

The data structure of this selection target is shown by the data 860 in FIG. 861 and 862 are the size of the selection target, 8
63 and 864 are positions, 865 is a motion to be activated when selected by an operator, or a pointer to a motion table is input, and 866 is related text information or the like. This makes it possible to associate the device in the video with the process data and related information. It is also possible to associate devices in the video. Further, it is possible not only to simply display the process data and the selection target, but also to execute a pre-defined instruction when selected.

In this way, the process data can be displayed on the device in the image, and the operator can see the moving image and the process data without moving the line of sight. Also, by displaying this in computer graphics, the operator can intuitively judge the data value. Furthermore, it is possible to avoid unnecessary recording of an image by setting a condition such as a background in the image that does not need to be recorded at any time or a condition of the image recording time. In this way, video, audio,
By synchronizing and reproducing the process data, it becomes possible to understand the process status more easily and quickly detect an abnormality. By selecting the process data that the operator wants to refer to on the video or directly selecting the video to refer to on the process data display, more direct operation becomes possible, and process monitoring, operability, and reliability are ensured. Enhance sex. Further, it is possible to realize a process data search using video data and a video data search using process data.

Summarizing the above, the following effects can be obtained.

(1) Preview when setting process data value By searching and displaying video and process data from past data, it is possible to see what the process will look like when the operator sets the process data to a certain value. Can be previewed.

(2) Comparison in Operation Monitoring The operation status of the process being monitored is recorded, and the operation status of the process can be grasped by comparing the operation status with video, audio, and process data.

(3) Determination of process data set value In order to set a certain process data value to a desired value, it is necessary to also set a related data value. In this way, when it is necessary to set a plurality of data values, it is possible to give the operator a guideline for determining the set values by referring to past data and video and audio.

(4) Search and analysis of abnormal time point The search for an abnormal time point and the detection of a failure point can be carried out more effectively by using the synchronous reproduction of the past process data and video and audio.

(5) Educational simulator It can be used as an educational simulator by reproducing the operating procedure of the operator.

In this embodiment, the time is recorded in order to synchronize the measurement data with the video or sound data, but the present invention is not limited to this.
For example, a serial number may be attached to the measurement data and the video data and the serial number may be used as a key to synchronize the measurement data with the video or sound data. .

Regarding the reproduction of the video data or the sound data, it has been described in the above embodiment that the speed of reproduction is canceled or delayed, but the present invention is not limited to this. However, it may be stopped (temporarily stopped) as a reproducing method instead of being stopped. At this time, as a method of stopping the operation, a method by operating an operator or recording an alarm and stopping at the time when the alarm is generated may be performed. At this time, there is an advantage that the screen at the time of occurrence of the accident can be quickly searched for when analyzing the cause of the accident.

Furthermore, the present invention is not limited to the moving image by the ITV camera as described above, and can also handle the still image by the still camera.

[0136]

As described above, according to the present invention, it is possible to provide the monitoring system capable of reproducing the measurement data and the image or the sound in synchronization with each other.

[Brief description of drawings]

FIG. 1 is a system configuration diagram.

FIG. 2 is a configuration diagram of a workstation.

FIG. 3 is a configuration diagram of a video / audio recording unit.

FIG. 4 is an explanatory diagram of an example of a display screen.

FIG. 5 is an explanatory diagram of an example of a trend graph displayed on a display.

FIG. 6 is an explanatory diagram of a display on the display according to the embodiment of the present invention.

FIG. 7 is an explanatory diagram of a video controller that determines the reproduction direction and speed of video and audio.

FIG. 8 is an explanatory diagram of a data structure of process data, video data, and the like used in this embodiment.

FIG. 9 is a flowchart for recording video and audio in the video / audio recording unit.
It's a chart.

FIG. 10 is a flowchart for displaying a recorded image.

FIG. 11 is a flowchart for realizing an embodiment of the present invention.

FIG. 12 is an explanatory diagram of a display on the display according to the embodiment of the present invention.

FIG. 13 is a flow chart for realizing an embodiment of the present invention.

FIG. 14 is an explanatory diagram of a display on the display according to the embodiment of the present invention.

FIG. 15 is a flow chart for realizing an embodiment of the present invention.

FIG. 16 is an explanatory diagram of a display on the display according to the embodiment of the present invention.

FIG. 17 is an explanatory diagram of a display on the display according to the embodiment of the present invention.

FIG. 18 is a flowchart for realizing an embodiment of the present invention.

FIG. 19 is an explanatory diagram of a display on the display according to the embodiment of the present invention.

FIG. 20 is a flowchart for realizing an embodiment of the present invention.

FIG. 21 is an explanatory diagram of a display on the display according to the embodiment of the present invention.

FIG. 22 is an explanatory diagram of a display on the display according to the embodiment of the present invention.

FIG. 23 is an explanatory diagram of a display on the display according to the embodiment of the present invention.

FIG. 24 is an explanatory diagram of a display on the display according to the embodiment of the present invention.

FIG. 25 is an explanatory diagram of a display on the display according to the embodiment of the present invention.

FIG. 26 is an explanatory diagram of a display on the display according to the embodiment of the present invention.

FIG. 27 is an explanatory diagram of a display display in the example of the present invention.

FIG. 28 is an explanatory diagram of a display on the display according to the embodiment of the present invention.

FIG. 29 is an explanatory diagram of a display on the display according to the embodiment of the present invention.

FIG. 30 is an explanatory diagram of a display on the display according to the embodiment of the present invention.

FIG. 31 is an explanatory diagram of a display display in the example of the present invention.

FIG. 32 is an explanatory diagram of a method of determining a selection target in the control unit according to the embodiment of the present invention.

[Explanation of symbols]

101 ... Control target device, 102 ... Control computer, 103
... workstation, 104 ... database, 108
… Video and audio recorder, 109… Switcher, 110… ITV
Cameras, 402 ... Moving image display area, 403 ... Trend graph, 502 ... Data item display section, 503 ... Time cursor, 504 ... Time axis.

 ─────────────────────────────────────────────────── ─── Continued front page (72) Inventor Koichiro Tanikoshi 4026 Kuji Town, Hitachi City, Hitachi, Ibaraki Prefecture Hitachi Research Laboratory, Ltd. (72) Shinya Tanifuji 4026 Kuji Town, Hitachi City, Ibaraki Prefecture Hitachi, Ltd. Hitachi, Ltd. (72) Inventor Atsuhiko Nishikawa 5-2-1 Omika-cho, Hitachi City, Ibaraki Hitachi Ltd. Omika Plant, Hitachi, Ltd. (72) Inventor ▲ Hiro ▼ Atsuhiko Ta, 5-chome, Omika-cho, Ibaraki Prefecture No. 1 stock company Hitachi Ltd. Omika factory

Claims (28)

[Claims] 1. A first storage means for storing measurement data on a monitored object, and a second storage means for storing at least one of sound data and video data on the monitored object.
Is a monitoring system in which the measurement data and the sound data and the video data are stored together with information for associating with each other. A first reproducing means for reproducing the audio data, a reproducing means (second reproducing means) for reproducing at least one of the sound data and the video data, a second reproducing means, and a first reproducing means. At least among the output means for outputting the sound data and the output means for outputting the video data. A second output means having one of them, and outputs the measurement data in synchronism with reproduction of video or sound. 2. The monitoring system according to claim 1, further comprising means for designating a reproduction direction or a reproduction speed of at least one of the sound data and the video data, and the second reproduction means. A monitoring system characterized by reproducing in a specified reproduction direction and reproduction speed. 3. The monitoring system according to claim 1 or 2, wherein first input means for receiving measurement data on the monitored object, input means for receiving sound data on the monitored object, and video data. And a second input unit having at least one of the input units for receiving the monitoring system. 4. The monitoring system according to claim 1, 2 or 3, wherein the first output means displays the measurement data. 5. The monitoring system according to claim 1, 2, 3 or 4, wherein the first output means displays a time. 6. The monitoring system according to claim 1, further comprising measurement data output designating means for designating measurement data to be output and time, wherein the control unit is provided at a time point when the designated measurement data is recorded. A monitoring system characterized by instructing the first output means to output the video or audio at the closest point. 7. The monitoring system according to claim 6, wherein the first output means displays the measurement data as a trend graph having a time axis, and the measurement data output designating means displays the trend graph. The monitoring system is characterized in that the time is designated by using the time axis, and the first output means displays the measurement data at the designated time. 8. The monitoring system according to claim 6 or 7, wherein the measurement data output designating means receives a search value of the measurement data, and the control section sets a time when the designated measurement data takes the search value. The monitoring system is characterized in that the first output means for searching outputs the time and the data value at the time. 9. The monitoring system according to claim 2, 3, 4, 5, 6, 7 or 8, wherein the control unit determines the display frequency of the measurement data according to the reproduction speed of the image or sound, Alternatively, the measurement monitoring control system is characterized in that the measurement data is displayed at a determined display frequency in synchronization with the reproduction of sound. 10. The monitoring system according to claim 6, 7, 8 or 9, wherein said measurement data output designating means receives the display frequency of the measurement data, and said first output means follows the designated display frequency. A monitoring system, wherein the measurement data is displayed, the control unit determines a reproduction speed from a designated display frequency, and the first output unit outputs video and sound at the determined speed. 11. The monitoring system according to claim 7, 8, 9 or 10, wherein said measurement data output designating means accepts designation of a time axis section of a trend graph and designation of reduction or enlargement in that section. The monitoring system is characterized in that the control section is fast in a reduced section and slow in an enlarged section, and determines the reproduction speed of video and sound according to the time axis display. 12. The monitoring system according to claim 1, wherein the third storage means stores the operation input information of the operator, and the operation input information of the operator is output in synchronization with the reproduction of the image or sound. 3. A monitoring system having three output means. 13. The monitoring system according to claim 12, further comprising means for accepting designation of operation input information, wherein the control unit searches for a time when the designated operation is input and a video and sound closest to the time. The third output means outputs the time and its operation input information, and the second output means displays a video and a sound. 14. The monitoring system according to claim 1, further comprising: a storage unit that stores information about a monitored object in the video that can be selected and information about the association between the monitored object and the measurement data. And a means for selecting the selection target, wherein the control unit selects the measurement data related to the selected selection target based on the stored information of the monitoring target object and the association information, A monitoring system, wherein the output means displays relevant measurement data. 15. The monitoring system according to claim 1, further comprising a storage unit that stores information about association between a selection target and measurement data, and a unit that selects one of the measurement data. The monitoring system is characterized in that a selection target related to the selected measurement data is selected according to the association information, and the second output means displays the selected selection target. 16. The monitoring system according to claim 14, wherein when there is a plurality of measurement data related to the selection target, the second output means displays a related measurement data selection menu in the vicinity of the selection target, A monitoring system comprising means for accepting designation of at least one item from the menu and means for displaying the accepted measurement data. 17. The monitoring system according to claim 14, wherein the second output means displays the measurement data related to the selection target in a trend graph. 18. The monitoring system according to claim 14, wherein the second output means displays at least one measurement data related to the selection target near the selection target. 19. The monitoring system according to claim 14, wherein the second output means displays the measurement data related to the selection target in graphics and displays the measurement data in an overlapping manner on the image. 20. The surveillance system according to claim 1, wherein information regarding a surveillance target in a video that can be selected and information relating to a surveillance target related to the surveillance target are stored. The control unit has a storage unit and a unit that selects the selection target in the video when the video is played back, and the control unit relates to the selected selection target based on the information of the stored monitoring target information and the association information. A selection target is selected, and the second output means indicates that the selection target has been selected on the video, and displays the selection target in at least one video related to the selection target. Monitoring system. 21. The storage system according to claim 1, wherein information about a monitoring target object in a video that can be selected and information about association with additional information related to the monitoring target object are stored. And a means for selecting the selection target in the video when the video is played back, and the control unit selects the selection target related to the selected selection target by the stored information of the monitored target and the association information. A monitoring system, wherein a target is selected, and the second output means indicates on the video that the selection target has been selected and displays additional information related to the selection target. 22. The monitoring system according to claim 15, wherein the second output means has means for displaying measurement data in a trend graph and selecting the displayed data item. The monitoring system is characterized in that the means provides a display indicating that the selection target related to the data has been selected in the image. 23. The monitoring system according to claim 15, wherein the second output means has means for displaying measurement data in a trend graph and selecting the displayed data item, and the control section is When the selection target related to the data is not in the video, the selection target related to the data is switched to the video in which the selection target related to the data is selected in the video. A monitoring system characterized by displaying. 24. The monitoring system according to claim 1, further comprising means for accepting designation of a recording target in an image, designation of a recording condition for determining when to record, and designation of a recording method of obtained image data. The monitoring system is characterized in that the control unit controls the second storage means by specifying a recording target, a recording condition, and a recording method. 25. The monitoring system according to claim 24, further comprising means for designating a recording target in advance, and said accepting means targeting only the recording target designated in advance. 26. The monitoring system according to claim 24, wherein the recording condition is to record when a difference from the previous screen exceeds a certain threshold value. 27. The monitoring system according to claim 24, wherein the recording condition is to record when the value of the measurement data exceeds a predetermined threshold value. 28. The monitoring system according to claim 24, wherein the recording condition is recorded when the difference value between each measurement data and the previous data exceeds a threshold value determined for each. Monitoring system.