JP2970964B2 - Monitoring device - Google Patents

Abstract

PURPOSE:To synchronously display not only the process data but also the image and voice data during reproduction in a process control monitoring system. CONSTITUTION:A data base 104 storing the process measurement data and an image/voice record section 108 recording at least one of the sound data and the image data are provided, the measurement data, sound data, and image data are correspondingly stored, the data base 104 reproduces the measurement data, and the image/voice record section 108 reproduces at least one of the sound data and the image data. A work station 103 synchronizing the image/ voice record section 108 and the data base 104, a display 111 outputting the measurement data and the image data, and a speaker 112 outputting the sound data are provided.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to process data obtained in a system for monitoring and controlling a power plant, a chemical plant, a hospital, steel, a railway, water and sewage management, and the like, video and sound information from an ITV camera and a voice microphone. The present invention relates to a system for recording video, sound, and process data in association with each other when an analysis is necessary, such as when an abnormality occurs.

[0002]

2. Description of the Related Art Generally, in a monitoring system such as a process control monitoring system, a monitor device is installed in a central control room, an ITV camera is installed on a process side, and a picture of the process is monitored by the camera. A method of visually monitoring the status of the process by displaying the information on the screen is adopted. The video and the sound are recorded on a recording medium such as a video tape. The video and the sound are rewound and reproduced at the time of abnormality or the like.

On the other hand, process data (measurement data) sent from the process every moment is displayed on monitors and meters in the central control room, and is stored in a database in the system so that abnormalities and analysis can be performed. It can be retrieved from the database when needed. This system is introduced in Japanese Patent Application Laid-Open No. 60-93518 in a plant operation history display system.

However, because the recording media of the video information and the sound information and the process data are different from each other, conventionally, when reproducing or analyzing, it has been necessary to reproduce or analyze separately. For example, when an abnormality occurs on the plant, the instrument detects it and a buzzer sounds. After that, the corresponding equipment is searched for in the overall process diagram, the cause and the countermeasure are determined, and a measure is taken. Here, the estimation of the cause and the faulty device required a great deal of work because many related data and images were required. In the analysis using video, a method is used in which a video is searched in advance to find the vicinity where an abnormality occurs, and then the vicinity is examined using process data.After finding an abnormal point in the process data, the video is wound up to that point. The method of returning and reproducing is used.

However, in general, ITVs for monitoring plants and the like
Since there are multiple cameras and the video from them is recorded in multiple videos, in order to view the video from each camera in association with analysis at the time of analysis, such as when there is an abnormality, rewind and playback all of those videos to the desired location The burden on the operator was great.

On the other hand, desired data is stored in a database,
It is difficult to extract the information without using such video information, and in many cases, after outputting a huge amount of information with a printer, an operator has been analyzing the information.

[0007]

The conventional system has the following problems.

(1) Since process data cannot be referred to at the time of reproducing video and audio information, even if information is obtained from the video, it takes time and effort to search for the process data thereafter.

(2) Even if the process data is displayed on a trend graph or the like and the operator knows when to view the video, it takes time and effort to display the video. For this reason, it was not possible to quickly grasp the actual situation at the site.

(3) Even when process data such as an abnormal value is searched, it takes time and effort to display an image related to the process data.

(4) When displaying recorded process data, if a large amount of recorded data is displayed at a fast forward speed, a great deal of load is imposed on the computer.

(5) Due to the limitation in the data display method, it was not possible to respond to a request to see in detail or to skip to see. When analyzing in particular, if the related video and sound are referred to in slow reproduction, it is more useful for analysis, but this function is not provided.

(6) As an important factor in determining the operation of the process, there is an operation command of the operator, but since these are not reproduced, what operation has changed the state of the process,
I did not understand.

(7) Even if the operator remembers the command operated, he or she could not search for the command, so it was necessary to analyze the process data and the like to estimate the point at which the command was given.

(8) Since there is no association between the process data and the video information, even if an abnormality is found on the video, what the video shows is, and what data is output therefrom. Only the trained veteran could tell if it was, but no one could tell which process unit the data was associated with.

(9) Since the place for displaying the image and the place for displaying the process data are separated, the operator must move his / her gaze, and the data and the image, which change every moment, cannot be seen at the same time.

(10) A video tape used conventionally has a problem in its reproduction capability in terms of quick access to video data and the like. On the other hand, if an optical disk is used, quick access is possible, but the video data becomes very large, so a large-capacity disk is required for recording.

It is an object of the present invention to provide a monitoring system capable of synchronously reproducing measured data and video or sound.

[0019]

In order to solve the above-mentioned problems , the present invention provides video information and sound data relating to an object to be monitored.
Data for playback of at least one of the
A storage unit that stores and processes data about the storage
Reproducing the reproduction information stored in the storage unit;
An output unit for outputting the stored process data in the section, before
The reproduction of the reproduction information by the recording / output unit and the process data
And a control unit for synchronizing the data output.
A monitoring device is provided.

[0020]

The first storage means stores measurement data relating to the monitored object, and the second storage means stores at least one of sound data and video data relating to the monitored object. Wherein the measurement data, the sound data and the video data are stored in association with each other, wherein the first reproducing means reproduces the measurement data. I do. The second reproducing means reproduces at least one of the sound data and the video data. The control unit synchronizes the first reproducing unit with the second reproducing unit based on the associated information. The first output means outputs the measurement data.
The second output means outputs at least one of the sound data and the video data. Thus, the measurement data is output in synchronization with the reproduction of the video or sound.

[0021]

[0022]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to FIGS.

FIG. 1 shows the configuration of the plant control and monitoring system of this embodiment. The device (hereinafter, simply referred to as a control target device) 101 which is a monitoring target at a factory site transmits process data indicating an operation status at each time to a cable 135.
To the control computer 102 as the first input means. The control computer 102 analyzes the process data and sends a control signal to the control target device 101 through the cable 136. In addition, LAN12 through cable 137
0, and process data is passed through the cable 138.
An operator command received from the AN 120 is received and processed. As described above, the main functions of the control computer 102 are collection of process data, output of process data to the LAN, input of operator commands from the LAN, control target device 10
1 is the output of the process control signal.

The LAN 120 transmits signals such as operator commands and process data via a cable such as Ethernet. The LAN 120 includes an output cable 137 from the control computer 102, an input cable 138 to the control computer 102, and an output cable 1 from the database 104.
43, an input cable 144 to the database 104,
Output cable 14 from workstation 103
0, input cable 13 to workstation 103
At 9, it is connected to each device.

The database 104 serving as the first and third storage means and the first reproduction means receives process data and the like flowing through the LAN 120 through the cable 144 and records the data together with the time t output from the clock provided therein. . When a data read command is input via the cable 144, the data designated to be read is sent to the LAN 120 via the cable 143.

The plurality of ITV cameras 110 are provided with a camera control device capable of remote control such as panning, tilting, and zooming by inputting a control signal, and an audio microphone that operates together with the cameras. The camera 110 is the control target device 10
1 is transmitted to the switcher 109 through the cables 130 and 131. The switcher 109 sends a camera control signal input from the workstation 103 to the camera 110 via the cable 132. ITV camera 11
0 is a second input means.

The video / audio recording unit 108 serving as the second storage means and the second reproduction means uses an optical disk or the like capable of random access. A video tape may be used, but in this case, since the data search is sequential, it takes time to search and display the data. All video and audio from the ITV camera 110 pass through the switcher 109 and are input from the cable 133. Also, the workstation 103 as a control unit is provided with the switcher 10.
When a read command is input through the cable 145 through the cable 9, the designated video and audio information is output to the switcher 109 through the cable 134.

The switcher 109 transmits a plurality of video and audio to be input to the workstation 103 via a cable 141.
And a signal transmission destination switch for transmitting a camera control signal and a recorded video calling signal output from the workstation 103 through the cable 142 to the camera 110 and the video and audio recording unit 108. .

The workstation 103 has first and third output means as display means 111, a speaker 112, and input means (measurement data output designation means, selection target, And means for designating a search value of measurement data) are connected to input devices such as a keyboard 106, a mouse 105, and a touch panel 107. Also, the cable 13
9 and 140, the LAN 120 and the cables 141 and 142
To connect to the switcher 109. Workstation 1
03 processes the process data input from the cable 139 to create a display screen, and displays it on the display 111 together with the video data input from the cable 141. On the other hand, audio data input by the cable 141 is output by the speaker 112. The speaker 112 and the display 111 are a second output unit. Also, key input from the keyboard 106 by the operator or the mouse 105
Input from an input device such as the touch panel 107 is processed in the workstation 103, output as a control code of the control target device 101 via the cable 140, a command to switch to the video / audio switcher 109, and control of the camera 110. The data is output via a cable 142 as a call code to the code, video, and audio recording unit 108.

The operator monitors the status of the system indicated by the images, characters, and graphics displayed on the display 111, and inputs necessary operations and instructions to the mouse 105,
This is performed using the keyboard 106 and the touch panel 107. In the following description, as an input device from the operator,
The touch panel 107 is used. Of course, other devices are also acceptable.

Next, in FIG.
03 shows the internal structure. 201 is a CPU, 202 is a main memory, 203 is an IO, 204 is a graphics screen frame buffer for displaying process data on a display 111, 205 is a digitizer for converting an input video signal into a digital signal, and 206 is a video frame buffer. , 207 are blend circuits for blending the graphics screen and the video.

FIG. 3 shows the configuration of the video / audio recording unit 108. CPU that receives and processes various instructions from the workstation 103 and issues recording and reproduction instructions
301, main memory 30 used for buffering of video and the like
2. It comprises an AD / DA converter 303 for converting a signal from the ITV camera 110 into a digital signal and an analog signal to be sent to the workstation 103, and a video / audio recording / reading unit 304.

FIG. 4 shows a display screen in the process control monitoring system. The display screen is
A process overall configuration diagram 401, a moving image display area 402 for mainly displaying video from an ITV camera, a trend graph 403 for displaying process data from the controlled device 101, a clock 406, a switch, help information, and the like are displayed. It comprises a work area 404, a process data display meter 405, and a menu area 407. In the menu area 407, a camera switching button 408, a selection target designation button 40 in the video and the process data are displayed.
9. Switching between monitoring mode and playback mode, switching button 410 for standard playback and slow playback, simple editor call or selection button 411 for displaying graph, selection button 412 for displaying data, and help button 4
13 or the like is displayed. Here, a data table, a scaler, or the like may be used to display the process data from the control target device 101. Further, there may be a plurality of the above data display means on the display.

FIG. 5 shows the trend graph 403 for displaying process data in more detail. The trend graph 403 includes a data display unit 501, a data item display unit 502, a time cursor 503, a time axis 504, a data value cursor 505, and time axis move buttons 506 and 507.

The process data is displayed as a graph on the data display unit 501 and the data item display unit 50
Display the name in 2. The correspondence between the data and the name is determined by the thickness of the line and the color or type of the line.

The time cursor 503 indicates all data (for example, the meter 405) currently displayed on the display.
The time at which the data value, the video 402, the time of the clock 406, and the point on the time cursor 503 of the trend graph 403) are generated or recorded using the time axis 504. That is, the time cursor 503 of the trend graph 403 is a time display unit indicating the time at which the currently displayed data is recorded.

If the time at which the data to be displayed is not within the currently displayed time axis 504, the value of the displayed time is shifted to the right (to return the time; By moving in the left or right direction (hereinafter referred to as the forward direction), the value of the time is displayed in the time axis 504. Time axis 50
Numeral 4 can be enlarged or reduced, and can be enlarged or reduced by designating a section. As a result, the time axis 504 can be expanded in a section to be viewed in detail, and can be reduced in a section other than the section.

A time axis shift button 507 shifts the time value displayed on the time axis 504 to the right, and displays a time earlier than the displayed time. On the other hand, button 508
Moves to the left and displays a time later than the displayed time.

The data value cursor 505 is used to search for process data. After the process data to be searched is selected, when the data value cursor is moved to the search value, the time axis 504 and the time cursor 503 move, and the time cursor 503 comes at the time when the selected data takes the search value. Become like

In the following embodiment, a trend graph is used exclusively as data display means for displaying process data on a display. Of course, data display means other than the trend graph may be used.

The process monitoring system of the present invention has the following functions.

(1) In the operation of reproducing the recorded video, not only the video and audio are reproduced, but also the process data at the time when the video was shot is called and displayed.

(2) Time cursor 50 of the trend graph
By specifying the time using the third time display means,
The video and audio at the time when the data is recorded and the process data at that time are called and displayed.

(3) By designating process data and its search value, the data is searched, the data is called and displayed, and the video at the time when the data is recorded and the video at that time are displayed. Recall and display other process data.

(4) When the recorded video is reproduced, the display frequency of the process data with respect to time changes depending on the reproduction speed.

(5) By specifying the display frequency related to the time of the process data in advance, at the time of reproducing the video, the reproduction speed of the video and audio is determined according to the display frequency, and the reproduction and display are performed.

(6) The operation information is recorded from the operator, and the operation of the operator is reproduced when the video is reproduced.

(7) The operation information is recorded from the operator, and by specifying the operation data of the operator, the operation is searched, and the video and the process data at the time when the operation is performed are called and displayed.

(8) A selection target that can be selected by the operator using the touch panel is defined in the video, and the operator selects the selected item when the video is reproduced, thereby displaying the relevant process data.

(9) A selection target that can be selected by the operator using the touch panel is defined in the video, and the operator selects it during video playback, thereby highlighting the relevant process data.

(10) A selection object that can be selected by the operator using the touch panel is defined in the video, and when the video is reproduced, the operator selects it to display a selection menu related to the relevant process data. When one item is selected from the menu, the process data of the selected item is displayed.

(11) A selection target that can be selected by the operator using the touch panel is defined in the video, and when the video is played back, the operator selects it, so that the relevant process data is displayed on the selection target in the video. To be displayed.

(12) A selection object that can be selected by the operator using the touch panel is defined in the video, and the operator selects it when the video is reproduced, so that the relevant process data is displayed on the video by computer graphics. To overlap.

(13) Selection items that can be selected by the operator using the touch panel are defined in the video, and when the video is reproduced, the operator selects them to emphasize other selection targets in the related video. indicate.

(14) A selection target that can be selected by the operator using the touch panel is defined in the video, and when the video is reproduced, the operator selects it, thereby displaying additional information of the selection target.

(15) Selection targets are defined in the video,
When the operator selects the process data at the time of reproducing the video, the video is switched to the video related to the selected process data, and the selection target in the video is displayed.

(16) Selection targets are defined in the video,
When the operator selects the process data at the time of reproducing the video, the video is switched to the video related to the selected process data, the selection target in the video is displayed, and its data value is displayed on the selection target.

(17) Selection targets are defined in the video,
When the operator selects the process data at the time of reproducing the video, the video is switched to the video related to the selected process data, and the selection target in the video is displayed, and the data value is superimposed and displayed on the computer graphics.

Hereinafter, the above functions for reproducing the recorded process data, video data, and audio data will be described in detail.

Function 1 will be described with reference to FIGS. When the mode switching button 410 is selected using the touch panel, a recording information standard reproduction mode is set. During reproduction, recording is performed on an optical disk different from the one used for reproduction. At this time, as shown in FIG.
The video controller 603 is displayed in the work area 404. As shown in FIG. 7, the video controller includes a forward double speed play button 705, a forward normal speed play button 704, a reverse double speed play button 701, a reverse normal speed play button 702, and a video stop button 70.
Have three. If slow mode playback is selected by the mode switching button 410, the reverse 1/2 speed playback button 70 is used instead of the reverse 2x speed button and the forward 2x speed button.
6. A forward 1/2 speed button playback button 707 is displayed. However, reproducing at the standard speed here means reproducing at the same speed as when recording video and audio, and the forward direction is a direction in which time advances. Therefore, for example, double speed reproduction in the reverse direction means that reproduction is performed at twice the speed of recording in the direction opposite to the time direction. Here, when the recorded information is reproduced, the reproduction mode is provided with the standard mode and the slow mode.

By pressing the forward standard speed playback button 704 on the touch panel, video data and audio data are played back at the forward standard speed and displayed on the video display unit 402. At this time, the time cursor 503 in the trend graph moves in accordance with this image, and the process data at the time when the displayed image is recorded is brought onto the time cursor 503. When the time cursor 503 reaches a certain point, the process data is called from the database 104, and the time value displayed on the time axis 504 moves leftward (or rightward).
The process data at a new time that is not on the current time axis 504 is displayed. In addition to this, other process data display units such as the meter 405 also display the values taken by the data when the video was taken, and display the values one after another. As described above, not only the video and audio are reproduced by the operation of reproducing the video, but also the process data at the time when the video is captured is called and displayed.

Thus, it is possible to refer to the process data at that time while watching the video. In addition, by using other playback buttons, fast forward, reverse playback, slow playback, etc. of video information can be performed with reference to the process data, diagnosis of driving conditions, assistance for issuing control commands,
Useful for finding and analyzing abnormal points.

Next, a method of realizing this embodiment will be described.

First, a data structure and a recording method of process data, video data, and audio data of this embodiment will be described. In FIG. 8, data 800 represents the structure of process data sent from the control device 101 to the control computer. Generally, multiple types of data are input with one cable,
A header 801 indicating the beginning of the process data, a data type 802, a number of data 803, and data 804 to 806 as process data are obtained. The control computer 102 sends a plurality of data of this type coming from each cable to the LAN 120. The database 104 decomposes the flowing process data, and
Data 82 together with the time t of the clock in the base 104
Record in an array with 0 structure. Here, 821 is a data index, 822 is a data name, 823 is time, 8
24 is process data. As described above, the database 104 has a table corresponding to the type of process data, and records the latest data together with the time t after the last element of the array which is an element of the table.

On the other hand, a command for calling a block of process data from the workstation 103
In the case of inputting to 4, the data having the structure shown in the data 810 of FIG. Here, a header 811, which indicates the start of the process data, a data type 812, a data number 813, data 814 to 816 as process data, time data 817 of the data 814, time data 818 of the data 815, and data 816 And time data 819. Of course, the length of the data, the interval of the time data, and the like differ depending on the type of the block call instruction.

Next, recording of video and audio data will be described. First, the structure of the video / audio data to be recorded is as shown in 830. Generally, since video data from a plurality of cameras is recorded, it has an index 831 (disk number) and data name 832 (camera number or boiler number) of video / audio data. Reference numeral 834 denotes the time at which the audio was recorded, 833 denotes the audio information, 835 denotes the time at which the video was recorded, and 836 denotes the video information. Here, as shown in the figure, the video and the audio may not be separately recorded but may be recorded together. If they are recorded together, time information will be shared.

A method for recording and reproducing such video and audio data as described above will be described with reference to FIG. In the present invention, a three-step procedure indicated by reference numerals 901 to 903 is first executed by the CPU 201 in the workstation 103 for video recording. After the execution of this procedure, recording starts at 904. In video recording,
When the system starts up or when the playback mode ends and the recording mode returns to the recording mode, the entire video screen is recorded first, and as shown by 905, the video is recorded at 906 only when the recording conditions are satisfied, as indicated by 905 below. I do. Audio information is recorded as needed because the capacity required for recording is relatively small compared to video information. Hereinafter, recording and reproduction of video information will be described exclusively.

In the video recording object determination procedure 901, it is determined where the object to be recorded is. Specifically, one of the following methods is used.

(1) All of the camera video screens are to be recorded. The realization method is to record all video signals from the camera.

(2) An area including a part for outputting process data, a moving part, a changing part, and the like is specified in advance, and only that area is recorded. Data 840
Is the data structure of the video data 836 in this case.
The element of the data 840 is the image data 84 to be recorded.
6, and its position information, that is, the coordinate values 841 and 842 of the photographed image data, the size of the image data (spatial size of the screen) 843 and 844, and the closest full screen data before that are recorded. Time (or index) 845. The realization method records the entire screen when the ITV camera performs zoom, tilde, pan, and the like.
When the camera stops after such a camera operation, the video data 836 is sent to the workstation 103, the image is analyzed, and the area including the recording target is determined. For simplicity, this area may be, for example, a rectangle. When this area is determined, position information such as the coordinate value and size of the area is sent to the video and audio recording unit 108, and the CPU 301
Extracts only this area from the image sent from the camera and records it. Thereafter, only this area is recorded until the camera operation is performed again. At the time of reproduction, the CPU 301 blends the video data at the time 845 with the recorded data 846 to generate an entire screen.

In the video recording condition determination procedure 902, a video recording condition is determined. Specifically, one of the following conditions is satisfied.

(1) Recording is performed at predetermined time intervals. It has a clock in the workstation 103 or the CPUs 201 and 301 in the video and audio recording means 108. In the case of the former, a command to record at regular intervals is recorded in the video,
The data is sent to the audio recording unit 108. In the latter case, only the recording start instruction is sent to the video / audio recording unit 108 and the CPU 30
1 manages the recording time.

(2) In the video from each camera, if the difference from the previous screen exceeds a certain threshold value, it is recorded. The CPU 301 calculates a difference value between the video information of the screen stored on the main memory 302 in the video and audio recording unit 108 and the current video information, and sends a recording command to the video and audio reading unit 304 based on the calculated value. .

(3) If each process data exceeds a certain value inherent to the data, it is recorded. This is because the process data coming into the workstation 103 is transferred to the CPU2.
01, and instructs the video / audio recording unit 108 to record the video of the camera taking the video relating to the data having the abnormality in particular.

(4) Each process data is recorded when the difference from the previous data exceeds a certain value inherent to the data. The realization method is the same as (3).

(5) Weighted average of each process data,
That is, when the weight is wi (wi ≧ 0) and each process data is di, the data is recorded when e = Σwi * di exceeds a certain value. Realization method is (3)
Is the same as

(5) Recording is performed at a predetermined time interval. If any of the above conditions is satisfied, recording is performed at a shorter time interval. If the condition is not satisfied, the original time interval is restored.

In the video recording method determination 903, a recording method is determined. Specifically, one of the following methods is used.

(1) The video information from the ITV camera is stored as it is.

(2) The difference from the previous screen is recorded. The CPU 301 calculates the difference between the video buffered in the main storage unit 302 and the current video. At the time of reproduction, a video to be recorded is created by adding or subtracting a difference from a point in time when all the recording targets have been recorded to the present.

The display of the video data recorded as described above at time t is performed according to the procedure shown in FIG. Step 1
001, the index 821 of the video data and the time t
Is specified. Here, the video index is specified by the workstation 103, and the time t is specified by the workstation 103 or by the CPU 301 in the video and audio recording unit 108. The video / audio recording / reading unit 304 performs steps 1002 and 1003
If the video at the time t is not recorded as shown in (1), the video data at the time point s closest to t recorded before the time t is read. If the video data is video data directly recorded in step 1004, it is used. On the other hand, if the difference is recorded, the procedure 10
At 05, the video information closest to the time t and having no difference value is read out from the main memory 302 in the video and audio recording unit 108.
In step 1006, a difference is calculated to create a video. If the video is all camera images, it is displayed. Otherwise, it is displayed after being synthesized with the background in 1008.

CPU 301 in video / audio recording unit 108
When a playback command specifying the playback direction and speed is sent from the workstation 103, the display time data t contained therein is advanced by the following equation.

T = t + a * w where w is the video reading speed at the standard playback speed,
a is positive when the playback direction is forward, negative when the playback direction is reverse,
This is a coefficient whose absolute value is 2 at double speed and 1 at standard speed. The video display at the time of reproduction is such that in the case of forward reproduction, the video data 836 is transmitted to the workstation 103 when the time data t exceeds the time data 835, and in the case of reverse reproduction, the time data t is When the time falls below the time data, the video data 836 is sent. When there is a request from the workstation 103 for the generation time of the video being displayed, this time t is sent to 103.

FIG. 11 shows a processing procedure for realizing the first function based on the above recording and reproducing methods. Step 11
At 01, the reproduction mode is selected in the menu. At this time, the workstation 103 displays a control button 603 in FIG. In process 1102, the workstation 103 processes an input signal from a pointing device such as a touch panel, and detects the type of button by checking the input position. At this time, in order to indicate that the button has been pressed, the color of the pressed button is changed and displayed again on the display as shown in FIG. 6, and the reproduction direction and speed are determined. In the process 1103, a time t at which process data to be displayed next is generated is determined based on the determined direction and speed.

Specifically, the following two methods are used.

(1) The video and audio recording unit 108 is inquired about the time t at which the video and audio being displayed are recorded.

(2) The time t represented by the following equation is the next time to be displayed.

T = t + a * v Here, v is the time to rewrite all the data currently displayed once, a is positive when the reproduction direction is forward, and negative when the reproduction direction is reverse, This is a coefficient whose absolute value is 2 at double speed and 1 at standard speed. However, the time to rewrite data varies with other loads on the computer, so (1)
Method is also used. By using this, it is possible to advance the time of the next display information by the time that the video and audio information advance for the time required for the workstation 103 to display.

In step 1104, it is determined whether the process data to be displayed at time t is sufficient to be buffered in the work station 103, and if sufficient, the data is displayed. However, the sufficient case means that the process data at the time t is buffered, or that there is no data at the time t but the data before and after the data is buffered. When only the preceding and following data is buffered, the data is created by substituting the data closest to t or by linearly interpolating the preceding and following data. If it is not sufficient, in step 1105, the workstation 103 determines the range of data to be read as display data from the database 104 based on the display speed and direction, and in step 1106,
The type of process data to be displayed and the range of data to be read are sent to the database 104 via the LAN, and the process data requested from the database 104 is sent to the workstation 103. In processing 1107,
The video and audio information are displayed or output, and in a process 1108, each process data sent from the workstation 103 is displayed on the main memory 202 as a trend graph or a meter. It is displayed together with video information and audio information in a manner. Next, the second function will be described with reference to FIGS. 1 to 5 and FIG. The time cursor 503 moves left and right while pressing the cursor 503 with a finger using the touch panel 107, so that the cursor 503 moves left and right. At this time, as shown in FIG. 12, the time cursor 503 in the trend graph 403 is directly displayed.
Is moved to a point in time when the user wants to refer, the time cursor 1201 in the other trend graph 403 moves to the time pointed by the time cursor 503, and the video at the time determined by the time cursor 503 is displayed in the call video display area 402. . At this time, the meter 405 and the like in FIG. 2 also display the data at the time pointed by the time cursor 503. Of course, the time not displayed on the time axis of the trend graph 403 is also specified by the time axis movement buttons 506 and 50.
7 or the like. By specifying where you want to refer to the process data displayed as described above,
It is possible to refer to the video at the time when the data was recorded and other process data at that time. Thus, an image can be displayed by directly specifying a point in time when the operator wants to refer to the process data or the like by looking at the trend graph 403 or the like.

Thus, it is possible to refer to a specific site situation by referring to the process data.

Hereinafter, a method for realizing this embodiment will be described with reference to FIG. The algorithm in FIG.
To detect the time t pointed to by the time cursor, and
Only the determination as to whether the time t of 302 is already buffered in the workstation 103 is different. Processing 1
At 301, the coordinate value of an input signal from a pointing device such as a touch panel is
The CPU 201 in the CPU 3 performs processing to redraw the time cursor 503 on the coordinates, and calculates the time t indicated by the time cursor 503 from the coordinate values. If the data at time t is not buffered in the workstation 103, the procedures 1105 and 110 in the first embodiment will be described below.
Step 6 and display data, video and audio in steps 1106 and 1107.

Next, the third function will be described. As shown in FIG. 14, a data item 1401 in a data item display unit in a trend graph 403 is displayed using a touch panel 107.
Is selected, the data value cursor 505 is moved to a value to be searched, and the search value is determined. At this time, the time cursor 503 moves to the time when the selected data takes the value indicated by the data value cursor 505, and the other trend graph 4
Also in 03, the time cursor 1402 moves to that point, and the image at that point is displayed on the image display unit 402.
At this time, the meter 405 in FIG.
The data at the time pointed by the Sol 503 is displayed. Here, the search is performed only once in the reverse direction with respect to the time axis. If the user wants to search further, the user presses the time axis movement button 506 to perform the search in the opposite direction. On the other hand, when searching in the forward direction, pressing the button 507 causes the search to be performed in the forward direction. When a value is searched for in the process data displayed as described above, a search result is displayed, and it is possible to refer to the video at the time when the displayed data is recorded and other process data at that time. it can.

The method of realizing this embodiment will be described below with reference to FIG. In processing 1501, the workstation 103 processes coordinate values of an input signal from a pointing device such as the touch panel 107 and the like.
In step 2, the data selected as the search target and the search value indicated by the control value cursor 505 are determined. Next, in a process 1502, a search direction, that is, a forward search or a backward search with respect to the time axis is determined. This is, for example,
Basically, the backward search is performed once, and when the forward button 507 of the time axis move button is pressed, the search is performed in the forward direction. When the reverse button 506 of the time axis move button is pressed, the search is further performed. The search is performed in the reverse direction. Whether this button has been pressed is determined by the workstation 103
Do. In processing 1503, the database 10
In step 1504, a search command including a search target, a search value, a currently displayed data creation time, a search direction, and the like is issued.
Determines the search value and the display time that were found first. Hereinafter, the procedures are the same as those in steps 1104 to 1109 of the first embodiment, and a description thereof will be omitted.

This function makes it possible to compare and analyze other process data values with video data under the condition that certain process data takes a constant value, and to call abnormal values that occur very rarely. Become.

An embodiment for the fourth function is shown in FIGS.
It will be described in 7 and 18. In FIG. 16, when the forward double speed button 705 is selected at the time of reproducing the video information, the time axis 504 in the trend graph 403 displays the time in the twice range, and the process data currently displayed is The data is redisplayed according to the new time axis, and the data not displayed is read from the database and displayed according to the time axis. Next, the standard speed 2
The image is displayed at double speed, and the time cursor 503 moves. As described above, at the time of double-speed playback, the trend graph 4
Within 03, it is possible to display data for a longer time, and it is possible to overview changes in data over time. Such a display is convenient for data retrieval.

On the other hand, in FIG. 17, when the 1/2 speed reproduction button 707 is selected, the time axis 504 displays a time in a range that is half the time when the standard speed is set. At this time, since the data can be displayed in more detail, the data not displayed at the standard speed is read from the database and displayed again together with the already existing data. That is, when reproducing a video, the method of calling and displaying the process data is changed depending on the reproduction speed. Thus, when the reproduction speed is increased, data for a long time can be displayed on the trend graph 403, so that data search and overview can be easily performed. Also, when the reproduction speed is high when the process data is called, the interval of the generation time of the called data becomes large. However, the use of this display makes the calling coarser. On the other hand, if you want to slow down the playback speed,
Data can be displayed in detail. Therefore, at a point where detailed analysis is desired, detailed process data can be displayed only by slow reproduction of the video.

Thus, the display frequency of the process data with respect to time is changed depending on the reproduction speed, so that the load on the computer can be suppressed to some extent.

A method for realizing this embodiment will be described with reference to FIG. In step 1102, the direction and speed of reproducing video information and audio information are determined by input from the operator. In step 1801, the workstation 103 determines a display method and a calling method of process data based on the determined speed. The display method determines the display unit of the time axis in the trend graph 403 or the like, that is, how often the time is ticked, and the calling method is
Determine the time interval between data in the calling block and the length of time in a single calling block. If the data buffered in step 1104 is not enough, in step 1105 the determined time interval and time length are coded and sent to the database. The database reads out block data for a time interval and a time length based on the code sent in step 1105 and sends it to the workstation 103. Hereinafter, the workstation performs data display based on the determined display method. This part is the procedure 11 of the above embodiment.
Since it is the same as 04 to 1109, the description is omitted.

The fifth function will be described. In FIG. 19, a trend graph 40 is used as a method of displaying process data.
In the time axis section 1901 of 3, the time axis 504 is
And in the section 1902,
In No. 3, it is doubled. At this time, the interval between the generation times of the process data to be displayed is twice as large as the interval 1902 in the interval 1901, and is の times as large as the interval 1902 in the interval 1903. Therefore, in the section 1901, the display is the same as that of the previous embodiment at the time of the double speed reproduction, and in the section 1902, the display is the same as that of the standard speed reproduction.
In this example, the display is the same as that of 1/2 speed reproduction. In this case, if the video controller 603 performs standard forward playback using the button 704, the video is displayed in the video display area 4.
At time 02, when the time cursor 503 is in the section 1901, it is displayed at double speed, when it is in the section 1902, it is displayed at standard speed, and when it is in the section 1903, it is displayed at 1/2 speed. That is, by setting the method of displaying the process data in advance, at the time of reproduction, the speed at the time of video reproduction is set according to the display method, and reproduction is performed at that speed.

As a result, not only can the operator specify the display method of the data to be displayed, but also the image can be reproduced slowly when the user wants to view the data in detail, and can be reproduced when the user wants to skip the data.

A method for implementing this embodiment will be described with reference to FIG. In step 2001, reduction by input from the operator
Specify the section of the time axis to be expanded. In step 2002, the operator selects either enlargement or reduction for the section. These designations and selections may be made by, for example, a menu, or, as in this example, by designating a section with a touch panel or the like, and then grasping an end point of the section to extend and contract the section. At this time, the time axis is redisplayed in step 2003, and if necessary, a block call is made from the database to redisplay the process data. At this time, the workstation 103 determines the reproduction speed of each section and stores it in the main memory 202. Next, reproduction is started, and in step 1103, the display time t is determined. After the section including the time t is determined by the CPU 201, if the section is different from the previous section, a playback command such as a playback speed and a direction is sent to the video / audio recording unit 108 in step 2004. The subsequent steps are the same as steps 1104 to 1109 in the above embodiment.

The sixth function will be described. In FIG. 21, when reproducing the video information, not only the process data but also the operation information from the operator is reproduced together. At this time, the image and the process data that were being displayed on the display at that time are displayed, and the mouse cursor 21 is displayed.
The input of the operator indicated by 01 or the like is reproduced and displayed. At this time, by newly selecting a video to be displayed in the video display area 402 as indicated by reference numeral 2102, it is possible to refer to video information which has been generated by the operation of the operator and has not been seen at the time of recording. Also, process data and the like that have not been displayed by the same operation can be displayed. As a result, it is possible to quickly find an abnormality caused by, for example, an operation error of the operator. Also, a great effect can be obtained in control operation education.

Further, by reproducing the operation information of the operator, it is possible to determine whether the change in the process operation status is caused by the operation command of the operator. In addition, by recording and reproducing such an operation command, it can be used for explaining the operation procedure, and it is possible to monitor the educational system and the operation status of the operator.

A seventh function is to input operation information from an operator who wants to search, search for the input operation information, and retrieve and display the operation information and video information, audio information and process data at that time. You. Thus, the search for information can be performed using the operation performed by the operator as a mark.

Therefore, since the operation command of the operator can be searched, the change of the process data and the change of the image caused by the operation of the operator can be searched.

A method for realizing the above two embodiments will be described below. In FIG. 8, data 850 is the database 104
3 shows screen information to be recorded. Screen information 85
0 indicates a time 851, a camera name 852 for taking a picture to be displayed on the moving image display area 202, a process data name 853 to 855 for a trend graph 403, a meter 4
05 or data name 85 displayed on other data display means
6 mag. This data is stored in the moving image display area 4 by the operator.
02 to switch the video to be displayed,
When the data to be displayed in 03 is changed, added, or deleted, the workstation 103 sends it to the database 104.

The data structure of the operation data input from the operator is the same as the data structure 820 of the process data in FIG. However, instead of the process data value 824, an operation command input as operation data (a command generated by the workstation 103 processing coordinate values input by an operator using a pointing device) is input. The workstation 103 sends this data to the database 104 when the operation command is issued.

Reproduction is the same as the algorithm shown in FIG. However, in the procedure 1108, the process data is created by interpolating the data to be displayed with the data value at the time closest to the display time t or the data before and after the displayed data. However, the execution of the operator operation data is performed in the forward direction. In the case of reproduction, it is assumed that the display time t exceeds the recording time of the operation data, and in the case of reverse reproduction, it is assumed that the display time t falls below the recording time of the operation data. The contents of the screen information data recorded at the time 851 are displayed when the display time t is equal to or longer than the time 851 in forward reproduction, and is equal to or less than the time 857 in reverse reproduction. .

The search is the same as the algorithm shown in FIG. However, after the display time t is determined in step 1504, the screen information data closest to t before t is first called in step 1106, and then the process data to be displayed is determined and the process data is called.

The following embodiments relate to all the above-described embodiments, in which video, audio, and process data are reproduced when related display of the video and process data is performed.

The eighth function is to select the boiler window displayed in the moving image display area 402 in FIG.
01, and when this is selected, a graphic indicating that the selected object is selected is displayed, and the process data name 2202 generated therefrom is displayed in the process data item in the trend graph 403. , Process data 2203 is displayed in a graph. As described above, the relevant process data is displayed by selecting the selection target in the video using the pointing device. Here, needless to say, even if the selection target is not a boiler window, the selection target may be registered in advance in the control computer as the selection target. In addition to displaying data on the trend graph 403, it is also possible to display the data on the meter 405 or the like.

The ninth function is to define the upper pipe of the boiler displayed in the moving image display area 402 in FIG. 23 as a selection target 2301, and when this is selected, a graphic indicating that the selection target has been selected is displayed. Is displayed,
If the process data 2302 related to the selection target is the vapor pressure already displayed in the trend graph 403, the vapor pressure 2302 of the process data item is highlighted, and the graph 2303 is highlighted and selected by the operator. Indicates that the relevant data is the selected relevant data. That is, when data related to the selection target in the selected video is already displayed, the data is highlighted to indicate selection.

The tenth function is that the left pipe of the boiler displayed in the moving image display area 402 in FIG. 24 is defined as a selection target 2401, and when this is selected, a graphics indicating that it is selected is displayed. Is displayed and if there is more than one process data related to the selection,
A selection menu 2402 with process data as an item is displayed immediately beside the selection target in the moving image, and the process data to be referred to is selected from the selection menu using a pointing device to display the data in the trend graph 403. . That is, when there is a plurality of data items related to the selection target in the selected video, a selection menu is displayed, and the operator can select data to be referred to.

The eleventh function is to define the main body of the boiler displayed in the moving image display area 402 in FIG. 25 as a selection target, and when this is selected, a graphics indicating that the selection target is selected is displayed. 2501 is displayed, and associated process data 2502 to 2504 is displayed.
The data of the moving image is displayed in a superimposed manner at a relevant place. That is, by selecting a selection target in the video using the pointing device, the relevant process data is displayed at a relevant location in the video.

The twelfth function is to define the whole of the boiler displayed in the moving image display area 402 in FIG. 26 as a selection target, and when this is selected, a graphics 2601 indicating that it is selected Is displayed, the temperature distribution data related to the selected object is called, and is displayed on the image by the computer graphics 2602.
By selecting a selection target in the video using a pointing device, a display of process data in computer graphics is superimposed on the selection target.

A thirteenth function is that, in FIG. 27, the entire boiler displayed in the moving image display area 402 is defined as a selection target, and when this is selected, a graphics 2701 indicating that the selection target has been selected is displayed. Is displayed, and graphics 2702 is displayed in the fuel supply unit closely related to the selection target. That is, the selection target in the video is
By selecting using the pointing device, the selection target in the video related to the selection target is displayed.

The fourteenth function is that, in FIG. 28, the entire boiler displayed in the moving image display area 402 is defined as a selection target, and when this is selected, a graphics 2801 indicating that the selection target has been selected is displayed. Is displayed, and additional information 2802 such as a control method and maintenance information on the selected object is read out from the database and displayed on the video. That is, the selection target in the video is
When a selection is made using the pointing device, additional information such as an operation method, a control method, and maintenance information of the selection target is displayed.

As described above, the function 8 to the function 14 can associate the process data with the device shown in the video information, and can refer to the related device in the video by the process data and the process by the device in the video. Can reference data. Thus, for example, even a non-trained operator can perform simple operation and monitoring of the apparatus by viewing images and data.

Next, displaying information in a video using process data will be described.

The fifteenth function is that, in FIG. 29, when a process data item 2302 in the trend graph 403 is selected, the process data item 2302 is highlighted to indicate that the process data is selected. A graphics 2301 is displayed in the video display area 402, indicating that there is a selection target related to the process data. That is, graphics indicating which selection target the process data has in the video are displayed. The sixteenth function is shown in FIG.
In, by selecting the process data item 2302 in the trend graph 403, the process data 3001 is superimposed and displayed on the video 402 with a selection target associated with the process data.

The seventeenth function is that, in FIG. 31, by selecting a process data item 2302 in the trend graph 403, the process data is selected by the computer graphics 3101 in the video 402 as a selection target related to the process data. Display them in layers.

With reference to FIG. 32, a description will be given of a method of realizing the above-described embodiments of the functions 8 to 16 with reference to FIG. The shape model of the control target device 3201 to be monitored is recorded in the workstation 103. A part of this shape model is defined as a selection subject to receive input from the operator. The shape model may be three-dimensional data such as a CAD model, a process design drawing, or a simple rectangular area defined by an operator viewing an image from the camera 110. In order to determine the position and size in the video to be selected, the ITV camera 110 records angle-of-view information, vertical angle information, and horizontal angle information in the database 104 together with time. Or ITV
The angle of view information, vertical angle information, and horizontal angle information are calculated by the CPU 201 in the workstation 103 from the camera control command sent to the camera and the initial setting of the ITV camera, and the result is sent to the database 104 and recorded with the time. Since the ITV camera and the control target device do not move, the position and size of the selection target in the image can be known by combining the initial position of the camera, the camera information to be recorded, and the shape model.

ITV camera 1 showing process device 3201
10 is the vertical angle information 3211 and the horizontal angle information 3
By providing 212 and zoom values, the image 320
Create 2204 from 3204. Here, the image of the device 3201 projected on each image is 3205 according to the zoom value,
3206 and 3207. The scaling of the selection inside the computer is performed according to each zoom value. Here, when a simple rectangular area is used as the selection area, the selection target corresponding to the image 3202 is 3208, 3209 for the image 3203, and 3210 for the image 3204. The scaling is easy because it is all linear.

When the operator selects a selection target defined in this way, or when a message command is sent from another selection target, the selection target is displayed or related data is output. It is defined to start the operation.

The data structure to be selected is shown by data 860 in FIG. 861 and 862 are the size of the selection target, 8
63 and 864 are positions, 865 is an operation to be activated when selected by the operator, or a pointer to an operation table, etc., and 866 is related text information or the like. This makes it possible to associate the device in the video with the process data and related information. It is also possible to associate the devices in the video. Further, it is possible to not only display the process data and the selection target, but also execute a predefined command when the selection is made.

As described above, the process data can be displayed on the device in the video, and the operator can see the moving image and the process data without moving the line of sight. Further, by displaying this in computer graphics, the operator can intuitively judge the data value. Furthermore, it is possible to avoid unnecessary recording of a video such as a background in a video that does not need to be recorded as needed, and setting a condition of a video recording time. Thus, video, audio,
By synchronizing and reproducing the process data, it is possible to more easily understand the status of the process and quickly find an abnormality. In addition, the operator can select the process data to be referred to on the video, or select the video to be referred directly on the process data display, so that the operation can be performed more directly. Enhance the nature. Further, a search for process data using video data and a search for video data using process data can be realized.

The following effects can be summarized as follows.

(1) Preview at the time of setting process data value When an operator sets process data to a certain value, the process of the process is searched and displayed from past data by searching for and displaying video and process data. You can preview.

(2) Comparison in Operation Monitoring The process status can be grasped by comparing the operation status of the monitored process with the recorded video, audio, and process data of the operation status.

(3) Determination of Process Data Set Value In order to set a certain process data value to a desired value, it is necessary to set related data values. As described above, when a plurality of data values need to be set, the operator can be given guidance for determining the set values by referring to past data, video, and audio.

(4) Search and Analysis of Abnormal Time Point Search for abnormal time point and detection of a failure point can be performed more effectively by using synchronous reproduction of past process data and video and audio.

(5) Educational Simulator By reproducing the operation procedure of the operator, the simulator can be used as an educational simulator.

In this embodiment, the time is recorded in order to synchronize the measurement data with the video or audio data. However, the present invention is not limited to this.
For example, a serial number may be assigned to the measurement data and the video data, and the serial number may be used as a key to synchronize the measurement data with the video or audio data. .

In the above embodiment, the reproduction speed of the video data or the sound data is described as being reduced or slowed down. However, the present invention is not limited to this. Instead, the still image may be played back (paused) as a reproduction method. At this time, as a method of stopping, a method by operating an operator or an alarm may be recorded and stopped when an alarm occurs. In this case, there is a merit that the screen at the time of occurrence of the accident can be quickly searched, for example, when analyzing the cause of the accident.

Further, the present invention is not limited to the above-described moving image by an ITV camera, but can also handle still images by a still camera.

[0136]

As described above, according to the present invention, it is possible to provide a monitoring system capable of synchronizing and reproducing measurement data and video or sound.

[Brief description of the drawings]

FIG. 1 is a system configuration diagram.

FIG. 2 is a configuration diagram of a workstation.

FIG. 3 is a configuration diagram of a video / audio recording unit.

FIG. 4 is an explanatory diagram of an example of a display screen.

FIG. 5 is an explanatory diagram of an example of a trend graph displayed on a display.

FIG. 6 is an explanatory diagram of display display in the embodiment of the present invention.

FIG. 7 is an explanatory diagram of a video controller that determines a video and audio reproduction direction and speed.

FIG. 8 is an explanatory diagram of a data structure of process data, video data, and the like used in the present embodiment.

FIG. 9 is a flowchart for recording video and audio in a video and audio recording unit.
It is a chart.

FIG. 10 is a flowchart for displaying a recorded video.

FIG. 11 is a flowchart for realizing an embodiment of the present invention.

FIG. 12 is an explanatory diagram of display display in the embodiment of the present invention.

FIG. 13 is a flowchart for realizing an embodiment of the present invention.

FIG. 14 is an explanatory diagram of display display in the embodiment of the present invention.

FIG. 15 is a flowchart for realizing an embodiment of the present invention.

FIG. 16 is an explanatory diagram of display display in the embodiment of the present invention.

FIG. 17 is an explanatory diagram of display display in the embodiment of the present invention.

FIG. 18 is a flowchart for realizing an embodiment of the present invention.

FIG. 19 is an explanatory diagram of display display in the embodiment of the present invention.

FIG. 20 is a flowchart for realizing an embodiment of the present invention.

FIG. 21 is an explanatory diagram of display display in the embodiment of the present invention.

FIG. 22 is an explanatory diagram of display display in the embodiment of the present invention.

FIG. 23 is an explanatory diagram of display display in the embodiment of the present invention.

FIG. 24 is an explanatory diagram of display display in the embodiment of the present invention.

FIG. 25 is an explanatory diagram of display display in the embodiment of the present invention.

FIG. 26 is an explanatory diagram of display display in the embodiment of the present invention.

FIG. 27 is an explanatory diagram of display display in the embodiment of the present invention.

FIG. 28 is an explanatory diagram of display display in the embodiment of the present invention.

FIG. 29 is an explanatory diagram of display display in the embodiment of the present invention.

FIG. 30 is an explanatory diagram of display display in the embodiment of the present invention.

FIG. 31 is an explanatory diagram of display display in the embodiment of the present invention.

FIG. 32 is an explanatory diagram of a method of determining a selection target in the control unit according to the embodiment of the present invention.

[Explanation of symbols]

101: Controlled device, 102: Control computer, 103
... workstation, 104 ... database, 108
... Video and audio recording unit, 109 ... Switcher, 110 ... ITV
Camera, 402: moving image display area, 403: trend graph, 502: data item display section, 503: time cursor, 504: time axis.

──────────────────────────────────────────────────の Continued on the front page (72) Inventor Koichiro Tanigoshi 4026 Kuji-cho, Hitachi City, Ibaraki Prefecture Within Hitachi Research Laboratory, Hitachi, Ltd. Inside the laboratory (72) Inventor Atsuhiko Nishikawa 5-2-1 Omikacho, Hitachi City, Ibaraki Prefecture Inside the Omika Plant, Hitachi, Ltd. (72) Inventor ▲ Hiro ▼ Atsuhiko 5-2-1 Omikacho, Hitachi City, Ibaraki Prefecture (56) References JP-A-3-176795 (JP, A) (58) Fields investigated (Int. Cl. ⁶ , DB name) G01M 19/00 G01D 21/00 G05B 23 / 02 302 G08B 31/00

Claims (19)

(57) [Claims] 1. Video information and sound data relating to an object to be monitored
Information for playback of at least one of
A storage unit storing the process data to be reproduced, and reproducing the reproduction information stored in the storage unit,
An output for outputting the process data stored in the storage unit
Unit for reproducing the information for reproduction by the output unit and the process
And a control unit for synchronizing data output.
Monitoring equipment. 2. Video information and sound data relating to an object to be monitored
Information for playback of at least one of
From the storage medium storing the process data to be reproduced.
Information and data for reading the process data.
A data reading unit for reproducing the reproduction information and the process data.
And an output unit for outputting the information for reproduction by the output unit and the process.
And a control unit for synchronizing data output.
Monitoring equipment. 3. The monitoring device according to claim 1, wherein the storage unit further stores the reproduction information in the process.
The controller stores association information for associating the data with the data, and the controller outputs the output information based on the association information.
Unit for reproducing the reproduction information and the process data
A monitoring device for synchronizing the output with the monitoring device. 4. The monitoring device according to claim 3, wherein the association information includes the reproduction information and the process information.
Information on the time when the data was recorded in the storage unit.
A monitoring device, comprising: 5. The method as claimed in claim 1 , wherein said first , second, third, and fourth aspects
2. The monitoring device according to claim 1, wherein a designation of a reproduction direction or a reproduction speed of reproduction information of at least one of the sound data and the video information is received.
The control unit has a designation unit for attaching the designation unit according to the designation received by the designation unit.
And monitoring the output unit . 6. Video information and sound data relating to a monitoring target.
At least one of the reproduction information and the monitoring target
A storage unit that stores data regarding the specific information from the video information and the sound data.
The specifying unit for receiving the specification and the specific information for which the specifying unit has received the specification are stored in the storage unit.
Stored in the storage unit at a time closest to the stored time.
And a display unit for displaying the data.
Monitoring device. 7. Video information and sound data relating to a monitoring target.
At least one of the reproduction information and the monitoring target
From the storage medium storing the data relating to the reproduction.
Information and data on the monitored object
A data readout unit, and specific information from the video information and the sound data;
The specifying unit for receiving the specification and the specific information for which the specifying unit has received the specification are stored in the storage unit.
Stored in the storage unit at a time closest to the stored time.
And a display unit for displaying the data.
Monitoring device. 8. Video information and sound data relating to a monitoring target.
At least one of the reproduction information and the operation input information
And a storage unit storing the operation input information while reproducing the reproduction information.
An output unit for outputting, reproduction of the reproduction information by the output unit, and the operation input
It has a control unit that synchronizes with the output of information.
Monitoring device. 9. Video information and sound data relating to an object to be monitored
At least one of the reproduction information and the operation input information
From the storage medium storing the information for reproduction and the operation.
A data reading unit for reading the SakuIri force information, reproduces the reproduction information, the operation input information
An output unit for outputting, reproduction of the reproduction information by the output unit, and the operation input
It has a control unit that synchronizes with the output of information.
Monitoring device. 10. The monitoring device according to claim 8 , further comprising a designation unit for accepting designation of the operation input information, wherein the control unit is configured to control the reproduction stored in the storage unit. Information
Thus, the reproduction recorded in the storage unit at the time closest to the input time of the operation input information for which the specification unit has received the specification.
Information for reproduction, and the retrieved reproduction information and the finger
The time at which the setting unit has received the designation of the operation input information ,
A monitoring device , wherein the monitoring unit outputs the data from the output unit . 11. The method as claimed in claim 1 , wherein said first , second, third, and fourth aspects are selected.
The monitoring device according to claim 1 , wherein the storage unit or the storage medium stores information on a monitoring target in video information that can be a selection target, and information on an association between the monitoring target and process data , The monitoring device further includes a unit for selecting the selection target, and the control unit uses the stored information on the monitoring target and the information on the association to store process data related to the selected selection target. Select and select the selected process data
A monitoring device that outputs the data from an output unit . 12. The method according to claim 1 , wherein the first , second, third, and fourth aspects are the same.
2. The monitoring device according to claim 1, further comprising: means for selecting one of the process data, wherein the storage unit or the storage medium stores information on an association between a selection target and the process data , The control unit, based on the association information, selects the selected program.
Select the selection associated with Seth data is the selected
A monitoring device that outputs the selected object from the output unit . 13. The monitoring device according to claim 11 , wherein when there are a plurality of process data related to the selection target, the output unit displays a relevant process data selection menu near the selection target. The monitoring device further has means for receiving designation of at least one item from the menu , and the output unit further includes a process data corresponding to the designation.
A monitoring device that outputs data . 14. A method as claimed in claim 1 , wherein said first , second, third, and fourth aspects are the same.
Or in the monitoring device according to item 1, the storage unit or the storage medium may store information relating to a monitored subject in a video that can be selected and the association of information with the monitoring target that is associated with the monitoring target The monitoring apparatus further includes a unit for selecting the selection target in the video at the time of outputting the video, and the control unit performs the selection based on the stored monitoring target information and the association information. Selecting a selection target related to the target, the output unit indicates that the selection target is selected on the video, and displays the selection target in at least one video related to the selection target. Monitoring device . 15. The method according to claim 1 , 2, 3, or 4.
Or in the monitoring device according to item 1, the storage unit or the storage medium may store information relating to a monitored subject in a video that can be selected and the association of information with additional information associated with the monitored The monitoring apparatus further includes a unit for selecting the selection target in the video at the time of outputting the video, and the control unit performs the selection based on the stored information on the monitoring target and the information of the association. A monitoring device for selecting a selection target related to a target, wherein the output unit indicates that the selection target is selected on a video, and outputs additional information related to the selection target. 16. A method according to claim 1 , 2, 3, or 4.
2. The monitoring device according to claim 1 , wherein a recording target in the video is specified, a recording condition for determining when to record is specified, and the obtained video information and sound data are
Has a receiving means for receiving designation of at least one of recording of reproducing information out, the control unit has been in designation the accepting means accepts
It, the monitoring apparatus characterized by storing the said reproduction information in the storage unit. 17. The monitoring device of claim 16, comprising means for specifying a recording target in advance, said accepting means, monitoring devices, characterized in that the target only recorded pre-specified. 18. Video information and sound data relating to a monitoring target.
Data for reproduction of at least one of the
A storage unit storing process data to be monitored, and at least one process from the process data
A designation unit for accepting designation of data; and the control unit, based on the reproduction information,
The process data accepted by the user is stored in the storage unit.
The time stored in the storage unit at the time closest to the time
Search the raw information, and replace the searched reproduction information with the above
Along with the process data for which the
A monitoring device characterized by outputting from an output unit. 19. Video information and sound data relating to a monitoring target.
Data for reproduction of at least one of the
From the storage medium that stores the process data to be monitored,
Read out the reproduction information and the process data.
A data reading unit for reading out at least one process from the process data
And Keru with received designation data designating unit, the control unit, from among the reproduction information, the designation unit
The process data accepted by the user is stored in the storage unit.
The time stored in the storage unit at the time closest to the time
Search the raw information, and replace the searched reproduction information with the above
Along with the process data for which the
A monitoring device characterized by outputting from an output unit.