JP2020202449A - Control apparatus - Google Patents

Abstract

To display an image from a desired proper time from an image obtained by photographing an external apparatus.SOLUTION: A control apparatus comprises: recording control means for controlling record means to record an image signal obtained by photographing a first external apparatus; acquisition means for acquiring first timing information indicating timing when the first external apparatus executes a predetermined operation; and display control means for controlling display means to display an image obtained by reproducing the image signal. The display control means controls the display means to display information indicating timing which the first timing information corresponds to.SELECTED DRAWING: Figure 8

Description

The present invention relates to control devices, control systems, control methods, programs and recording media.

Patent Document 1 discloses a video playback device that controls playback of a video material based on associated information data that defines the information content of the video material.

Japanese Unexamined Patent Publication No. 2002-281457

However, in Patent Document 1, it is difficult to obtain a reproduction position based on the timing at which the external device to be monitored is controlled.

An object of the present invention is to make it possible to display an image from an image taken by an external device from a desired appropriate time.

According to one aspect of the present invention, the control device indicates a recording control means for causing the recording means to record a video signal obtained by photographing the first external device, and a timing at which the first external device executes a predetermined operation. The display control means includes an acquisition means for acquiring the first timing information and a display control means for displaying the image obtained by reproducing the video signal on the display means, and the display control means corresponds to the first timing information. Information indicating the timing is displayed on the display means.

According to the present invention, it is possible to display an image from an appropriate time desired from the images captured by the external device.

It is a figure which shows the functional configuration example of a control system. It is a figure which shows the hardware configuration example of a control device. It is a flowchart which shows the control method of a control system. It is a figure which shows the example of the event list. It is a figure which shows the example of the correspondence table. It is a figure which shows the example of a script program. It is a figure which shows the example of the display of a control system. It is a figure which shows the example of the display of a control device. It is a figure which shows the example of the operation period of a device. It is a figure which shows the display example of the image at the time of a normal state and an event occurrence. It is a figure which shows the configuration example of the condition monitoring system. It is a figure which shows the configuration example of a control device. It is a figure which shows the image displayed on the display device.

Hereinafter, preferred embodiments of the present invention will be described in detail, exemplary, with reference to the drawings. However, the components described in this embodiment are merely examples, and the scope of the present invention is not limited to them.

(Embodiment 1)
FIG. 1 is a diagram showing a functional configuration example of the control system 130 according to the first embodiment. The control system 130 includes a monitoring device 100, a recording device 101, a control device 102, a plurality of monitoring cameras 103, a plurality of devices 104, and a programmable logic controller (PLC) 105.

The PLC 105 controls the operation of the device 104. The device 104 is an external device, for example, a robot. The monitoring device 100 monitors the status and control information of the device 104 to be monitored in the factory or the plant directly or via the PLC 105. The device 104 and the PLC 105 may be the same device. Further, the monitoring device 100 controls the device 104 as needed.

The monitoring device 100 has a storage unit 110. The storage unit 110 acquires the state signal and control information of the device 104 from the device 104 by direct communication or communication via the PLC 105, and obtains event data associated with the state signal and control information of the device 104. accumulate. For the event data, the event information (device name and event name) of the device 104 defined in advance, the time information when the status signal changes (event occurrence time and event end time), and the operation of the device 104 are controlled. Contains control information. The control information includes the value of the control information and the control command time when the value of the control information changes. The storage unit 110 constantly temporarily records information on the control command time for a certain period of time. Here, the device name is device information for uniquely identifying each of the plurality of devices 104 to be monitored. Further, the event name is event identification information for uniquely identifying each of a plurality of events that occur in the device 104 to be monitored, which is detected by the monitoring device 100.

Further, the storage unit 110 can transfer the event data to the control device 102 by requesting the event data. Further, the storage unit 110 notifies the control device 102 of whether or not the event data has been updated, and transmits event data update information (hereinafter referred to as update event data) including update meta information in response to a request in response to the notification. It can be performed.

Further, the monitoring device 100 has a display unit 111. The display unit 111 displays a live state diagram by graphical animation in which icons indicating the states of the device 104 and the device 104 are installed according to the state signal of the device 104 acquired by the storage unit 110. In addition, the display unit 111 can accept a click event for the icon and start a predetermined program or script.

The plurality of surveillance cameras 103 are photographing devices that are installed for monitoring the equipment 104 and the production line in a factory or a plant and capture images of the equipment 104 and the production line. The recording device 101 is a recording unit, and acquires and records images captured by a plurality of surveillance cameras 103.

The recording device 101 receives camera information (information identifying one of a plurality of surveillance cameras 103 (for example, camera name)) and recording information of recording start and recording stop to the control device 102, and the surveillance camera Record 103 images. The camera information and the recorded information are configured in a format such as XML (Extension Markup Language). The recording device 101 receives camera information, playback time information, and playback control signals (for example, play, stop, frame advance, fast forward, frame rewind, and rewind) from the control device 102 in a format such as XML, and is a surveillance camera. 103 images are transmitted.

Each of the plurality of surveillance cameras 103 is, for example, a camera that supports pan, tilt, and zoom. In this case, the recording device 101 receives the camera information and the camera control signal from the control device 102, and controls the surveillance camera 103 specified by the camera information by the camera control signal.

The recording device 101 and the control device 102 may be connected via a network, or may be configured by the same computer.

The control device 102 is a device that links the monitoring device 100 and the recording device 101. The control device 102 realizes the functions of the event data display unit 120, the acquisition unit 121, the association unit 122, the acquisition unit 123, and the display unit 124 when the computer executes the program.

The monitoring device 100 and the control device 102 may be connected via a network or may be configured by the same computer. Further, the monitoring device 100, the recording device 101, and the control device 102 may be configured by the same computer.

The control device 102 includes an event data display unit 120, an acquisition unit 121, an association unit 122, an acquisition unit 123, and a video display unit 124.

The event data display unit 120 outputs an event data transfer request to the storage unit 110 of the monitoring device 100, for example, in an SQL statement, acquires event data from the storage unit 110, lists the acquired event data, and displays the event list. To do. The user can select one of the events that have occurred in the past from the displayed event list. The event data display unit 120 outputs the event data of the selected event to the acquisition unit 121. Further, the event data display unit 120 may determine whether or not the event data is updated in the monitoring device 100 based on the number of the acquired event data described above, or input the event update information of the storage unit 110 to determine. You may. Further, when the event data display unit 120 determines that the update has occurred, the event data display unit 120 requests the monitoring device 100 for the update event data, acquires the update event data from the monitoring device 100, and outputs the update event data to the acquisition unit 123.

The acquisition unit 121 inputs event data from the event data display unit 120. Then, the acquisition unit 121 outputs the event information (for example, the device name and the event name) to the association unit 122, and acquires the camera information associated with the event information from the association unit 122. After that, the acquisition unit 121 outputs the camera information and the time information (for example, the event start time and the event end time) included in the event data to the video display unit 124. The acquisition unit 121 can input event data from the outside, not limited to the event data display unit 120.

The association unit 122 extracts the camera name of the surveillance camera 103 associated with the event information input from the acquisition unit 121 or the acquisition unit 123, and outputs the extracted camera name to the acquisition unit 121 or the acquisition unit 123. The association unit 122 extracts the camera name of the surveillance camera 103 by using the correspondence table 500 of FIG. 5 and the script program 600 of FIG. 6, as will be described later.

Further, the association unit 122 can input event information (device name and event name) from the outside, extract the camera name of the surveillance camera 103 associated with the event information, and output the camera name to the outside. ..

The acquisition unit 123 inputs event data from the event data display unit 120. Then, the acquisition unit 123 outputs the event information (device name and event name) to the association unit 122, and acquires the camera information associated with the event information from the association unit 122. After that, the acquisition unit 123 gives a recording start instruction and a recording end instruction of the surveillance camera 103 to the recording device 101 based on the camera information and the event time information included in the event data.

In addition, the acquisition unit 123 can input camera information and recording start and recording end information from the outside, and give a recording start instruction and a recording end instruction of the surveillance camera 103 to the recording device 101.

The image display unit 124 is a display control unit, acquires an image specified based on camera information and time information from the acquisition unit 121 from the recording device 101, reproduces the acquired image, and displays the display device of FIG. It is displayed on 211. Further, the image display unit 124 can input camera information and time information from the outside, acquire an image specified based on the input camera information and time information from the recording device 101, and display the acquired image. Further, the video display unit 124 outputs a reproduction control signal or a camera control signal for the surveillance camera 103 selected by the user among the plurality of surveillance cameras 103 to the recording device 101.

By linking the monitoring device 100 and the recording device 101, the control device 102 associates the event data collected by the monitoring device 100 with the monitoring camera 103 controlled by the recording device 101. The recording device 101 records the video captured by the surveillance camera 103 on the recording medium. When the control device 102 selects one of the past events using the event list displayed by the user, the control device 102 acquires the image of the surveillance camera 103 associated with the event data of that event from the recording device 101. The control device 102 not only reproduces the acquired video from the beginning, but also allows the user to select an arbitrary time from the playback start time candidates based on the control information included in the event data, so that the video is captured from the user's desired position. Can be played back, and meaningful images can be played back quickly. Further, the control device 102 can synchronously reproduce the video at the time of the event and the video at the normal time based on the control information, and can reduce the labor of the user for abnormality analysis. Here, the control information is information that the device 104 to be monitored refers to and controls the operation of the device 104.

FIG. 2 is a block diagram showing a hardware configuration example of the control device 102 of FIG. The control device 102 is a computer and is connected to the network line 210, the display device 211, and the input device 212. The control device 102 includes a control unit 201, a ROM 202, a RAM 203, an external storage device 204, an input interface 205, a bit move unit (BMU) 206, a VRAM 207, a network interface 209, and a system bus 212.

The control unit 201 realizes the function of the control device 102 of FIG. 2 by executing the program stored in the ROM 202, the RAM 203, or the external storage device 204. Further, the control device 102 may realize the functions of the monitoring device 100 and the control device 102 of FIG. 1, or may realize the functions of the monitoring device 100, the recording device 101, and the control device 102 of FIG. ..

That is, the monitoring device 100, the recording device 101, and the control device 102 of FIG. 1 may be realized by a single computer or may be distributed to a plurality of computers. When realized by a plurality of computers, the plurality of computers are connected by a network such as a local area network (LAN) that can communicate with each other.

The control unit 201 is, for example, a central processing unit (CPU) and controls the control device 102. The ROM (Read Only Memory) 202 stores programs and parameters. The RAM (Random Access Memory) 203 stores programs and data supplied from an external storage device 204 or the like. The external storage device 204 is, for example, a hard disk drive (HDD). The external storage device 204 may be a solid state drive (SSD) composed of a flash memory, or a hybrid drive or a memory card in which a hard disk and a flash memory are used in combination. The external storage device 204 stores a program such as an OS (Operating System).

The input interface 205 is connected to an input device 212 such as a pointing device or a keyboard for inputting data. The BMU 206 controls data transfer between memories (eg, between VRAM 207 and another memory) and between memory and each I / O device (eg, network interface 209). The VRAM (Video RAM) 207 stores an image to be displayed on the display device 211. The VRAM 207 outputs an image to the display device 211 according to a predetermined specification. The display device 211 is a display unit and displays an image thereof. The network interface 209 is connected to the network line 210. The system bus 212 connects each component of the control device 102.

FIG. 3 is a flowchart showing a control method of the control system 130. The control system 130 includes a monitoring device 100, a recording device 101, and a control device 102.

In step S200, the association unit 122 of the control device 102 associates an event that is expected to occur in the plurality of devices 104 with the surveillance camera 103 that shoots the device 104 of the event among the plurality of surveillance cameras 103. Pre-register. This association will be described later with reference to FIGS. 5 and 6. Further, the control device 102 and the monitoring device 100 share this pre-registered association. The monitoring device 100 may perform pre-registration, and then the control device 102 may share the pre-registered association.

Next, in step S101, the storage unit 110 of the monitoring device 100 acquires the status signal and control information of the device 104 to be monitored, either directly or via the PLC 105. Next, in step S102, the storage unit 110 of the monitoring device 100 generates event data 400 of FIG. 4A, which is associated with the acquired state signal and control information and is defined in advance, and stores the event data 400. .. Details of this step S102 will be described later with reference to FIG. 4A.

Next, in step S103, the display unit 111 of the monitoring device 100 displays the states of the device 104 and the device 104 as the live state diagram 720 of FIG. 7, which will be described later, based on the state signal acquired by the storage unit 110. Since this live phase diagram is not an image of the surveillance camera 103 acquired via the recording device 101, it is represented by a graphical animation.

Next, in step S104, the storage unit 110 of the monitoring device 100 outputs the information that the event data has been updated to the event data display unit 120 of the control device 102. This output is done when the event occurs and when the event ends. Specifically, when the storage unit 110 inputs a status signal indicating the occurrence of an event from the PLC 105 or the like, the storage unit 110 generates event data including an event name, a device name, and an event occurrence time, and outputs step S104. After that, when the storage unit 110 receives the status signal indicating the end of the event from the PLC 105 or the like, the storage unit 110 adds the event end time to the above event data and outputs the step S104. Further, when a control command is issued from the PLC 105 or the like before or after the event occurs, the storage unit 110 adds control information to the event data and outputs the step S104.

In step S201, the event data display unit 120 determines that the event data has been updated when the output of step S104 is output, and proceeds to step S202. The event data display unit 120 may inquire the monitoring device 100 whether or not the event data is updated at a fixed timing after the pre-registration in step S200 is performed.

In step S202, the event data display unit 120 requests the storage unit 110 for the event data newly stored or updated in step S102 (hereinafter, referred to as update event data). In step S203, the storage unit 110 outputs the update event data to the event data display unit 120 in response to the request. The event data display unit 120 inputs the update event data and outputs the update event data to the acquisition unit 123.

In step S204, the acquisition unit 123 analyzes the update event data. Based on this analysis result, the acquisition unit 123 extracts the device name, the event name, the event occurrence time and / or the event end time from the update event data. The item order of the update event data is, for example, the order of the device name, the event name, the event occurrence time, and the event end time, and is preset by the user. After that, the acquisition unit 123 outputs the extracted device name and event name to the association unit 122.

In step S205, the association unit 122 specifies the camera name of the surveillance camera 103 associated with the device name and the event name input from the acquisition unit 123. The correspondence table 500 of FIG. 5 or the script program 600 of FIG. 6 is used for this identification. After that, the association unit 122 outputs the camera name of the specified surveillance camera 103 to the acquisition unit 123.

In step S206, the acquisition unit 123 generates a recording start command or a recording end command based on the event occurrence time or event end time extracted in step S204 in response to the update of the event data. For example, the acquisition unit 123 generates a recording start command including the event occurrence time. In addition, the acquisition unit 123 generates a recording start command including the event end time. After that, the acquisition unit 123 outputs the camera information including the camera name of the surveillance camera 103 input from the association unit 122 and the generated recording start command or recording end command to the recording device 101. That is, the acquisition unit 123 is a recording control unit, and outputs a recording start command or a recording end command for causing the recording device 101 to record a video for a predetermined period corresponding to the event occurrence time and the event end time.

In step S301, the recording device 101 starts recording the video of the surveillance camera 103 specified by the input camera information from the recording start time predetermined time before the event occurrence time included in the recording start command. Further, the recording device 101 ends the recording of the video of the surveillance camera 103 specified by the input camera information at the recording end time after a predetermined time from the event end time included in the recording end command. In this way, the recording device 101 records and stores the video captured by the surveillance camera 103 on the recording medium from the recording start time to the recording end time.

The recording device 101 constantly temporarily records the live image from the surveillance camera 103 for a certain period of time in the buffer. As a result, the recording device 101 acquires the video recorded from the recording start time before the event occurrence time by the predetermined time to the recording end time after the event end time by the predetermined time from the buffer and records it on the recording medium. Perform post recording. The recording method of the recording device 101 is not limited to pre-post recording. For example, the recording device 101 starts temporary recording of the video in the buffer after inputting the recording start command including the event occurrence time, and records the video in the buffer after inputting the recording end command including the event end time. Post-recording may be performed. It is sufficient that the recording device 101 can confirm the state during the past event occurrence by recording the video from at least the event occurrence time. It is preferable that the recording device 101 records an image including an image taken at least from the event occurrence time to the event end time. That is, the recording start time is preferably before the event occurrence time, and the recording end time is preferably after the event end time. The buffer in which the video is temporarily recorded and the recording medium (for example, a hard disk drive) in which the video is recorded are different. The video temporarily recorded in the buffer is automatically overwritten with another video or deleted after a certain period of time. The video recorded on the recording medium is not automatically erased even after a certain period of time has passed. The video recorded on the recording medium is erased according to, for example, a user operation.

The above steps S101 to S104, S200 to S206 and S301 are repeatedly processed in a fixed cycle. In step S201, the event data display unit 120 may determine whether or not the event data is updated based on the number of event data input from the storage unit 110 when the event list is generated in step S207 described later. Good. Specifically, the event data display unit 120 displays the event data when the number of event data input during the current event list generation is larger than the number of event data input during the previous event list generation. Determine that there was an update.

Next, the video reproduction process of the control system 130 will be described with reference to steps S207 to S214.

In step S207, the event data display unit 120 of the control device 102 generates the event list 401 shown in FIG. 4B based on the event data 400 shown in FIG. 4A stored in the storage unit 110.

Next, in step S208, the event data display unit 120 displays the event list 702 of FIG. 7 on the display device 211 based on the event list 401. The event list 401 generated in step S207 may be updated at predetermined timings set in advance, or the timing at which the storage unit 110 outputs the update event data to the event data display unit 120 in step S203. You may go every time.

Next, in step S209, the user uses the input device 212 to select one event data from the plurality of event data in the event list 702. The event data display unit 120 outputs the selected event data to the acquisition unit 121. Hereinafter, the event data output by the event data display unit 120 to the acquisition unit 121 is referred to as designated event data. For example, the user selects one event data 730 from the plurality of event data displayed in each line of the event list 702 in FIG. 7. Then, the event data display unit 120 reverses the display of the line of the selected event data 730 as shown in FIG.

Next, in step S210, the acquisition unit 121 analyzes the designated event data and extracts the device name, event name, event occurrence time, event end time, and / or control information from the designated event data. After that, the acquisition unit 121 outputs the extracted device name and event name to the association unit 122. The item order of the designated event data is, for example, the order of the device name, the event name, the event occurrence time, and the event end time, and is preset by the user. Although the acquisition unit 121 extracts the event occurrence time and the event end time from the event data, only the event occurrence time may be extracted.

Next, in step S211 the association unit 122 identifies the camera name of the surveillance camera 103 associated with the device name and the event name input from the acquisition unit 121. After that, the association unit 122 outputs the camera name of the specified surveillance camera 103 to the acquisition unit 121. The association unit 122 specifies the camera name of the surveillance camera 103 by using the correspondence table 500 of FIG. 5 or the script program 600 of FIG.

Next, in step S212, the acquisition unit 121 is from the recording start time to the recording end time including the video from the event occurrence time to the event end time extracted in step S210 based on the camera name input from the association unit 122. Generate a video acquisition command. After that, the video display unit 124 outputs the video acquisition command to the recording device 101.

Next, in step S213, the recording device 101 outputs the video from the recording start time to the recording end time including the video from the event occurrence time to the event end time to the video display unit 124 in response to the video acquisition command. To do.

Next, in step S214, the video display unit 124 reproduces the input video and displays the video 800 of FIG. 8 in the windows 701 of FIGS. 7 and 8 of the display device 211. Further, the video display unit 124 can reproduce a video based on the control information in the event data and display the video on the display device 211. The display of this image will be described later with reference to FIGS. 7 and 8.

FIG. 4A is a diagram showing an example of event data 400. In step S102, the storage unit 110 stores the event data 400. FIG. 4B is a diagram showing an example of the event list 401. In step S207, the event data display unit 120 generates the event list 401 shown in FIG. 4B based on the event data 400 shown in FIG. 4A stored in the storage unit 110.

Next, a method in which the event data display unit 120 generates the event list 401 will be described. As shown in FIG. 4A, the storage unit 110 of the monitoring device 100 generates the event data 400 of FIG. 4A associated with the status signal and the control information of the device 104, and stores the event data 400. ..

The event data 400 has device names D410 and D438, event names D421 and D440, event occurrence times D414 and D442, event end times D416 and D444, and control information D418 to D436. The control information D418 to D436 each have a control command time and a value of the control information after the change. Here, the value of the control information is a value for the device (robot) 104 that refers to this value to determine the type of operation. For example, when picking a work of a different type, the device 104 determines the type of work to be picked based on the value of the control information. The device 104 stacks red workpieces when the control information value is 1, stacks blue workpieces when the control information value is 2, and stacks blue workpieces when the control information value is 2. End the operation.

The event data display unit 120 transmits a request for event data 400 to the storage unit 110. Then, the storage unit 110 outputs the event data 400 to the event data display unit 120 in response to the request. The event data display unit 120 stores the events in the order of items set in advance by the user (for example, the order of the device name 4100, the event name 4101, the event occurrence time 4102, and the event end time 4103) based on the event data 400. Generate Listing 401.

The event occurrence time 4102 is, for example, an abnormality occurrence time. The event end time 4103 is, for example, an abnormal end time.

In the event list 401, event data is arranged in the order of the device name 4100, the event name 4101, the event occurrence time 4102, and the event end time (event return time) 4103. The event data display unit 120 displays the event list 401 as the event list 702 of FIG. 7 on the display device 211.

The event names D412 and D440 of the event data 400 indicate an abnormality name indicating an abnormality of the device 104, but if the event name is registered as an event name that the user wants to record in advance, it is not an abnormality event. You may. For example, the event names D412 and D440 may be event names that notify each time a predetermined number of products are manufactured by the device 104. In this case, the event end time is the same time as the event start time.

FIG. 5 is a diagram showing an example of a correspondence table 500 registered by the association unit 122 of the control device 102. The association unit 122 identifies the camera name of the surveillance camera 103 from the device name and the event name extracted from the update event data by the acquisition unit 123 using the correspondence table 500.

The correspondence table 500 is predetermined by the user and represents the camera name of the surveillance camera 103 associated with the device name and the event name. For example, when the device name is "first device" and the event name is "transport abnormality", the camera names of the surveillance camera 103 are two cameras, "first camera" and "second camera". The name may be specified.

The camera name of the surveillance camera 103 in step S205 and the camera name of the surveillance camera 103 in step S211 are specified as follows.

In step S204, the acquisition unit 123 extracts the device name and the event name from the update event data. In step S205, the association unit 122 specifies the camera name of the surveillance camera 103 associated with the device name and the event name extracted by the acquisition unit 123 by using the correspondence table 500.

In step S210, the acquisition unit 121 extracts the device name and the event name from the designated event data. In step S211 the association unit 122 identifies the camera name of the surveillance camera 103 associated with the device name and the event name extracted by the acquisition unit 121 by using the correspondence table 500.

The association unit 122 has specified the camera name of the surveillance camera 103 from the combination of the device name and the event name, but the association unit 122 is not limited to this. For example, the association unit 122 may specify the camera name of the surveillance camera 103 from only one of the device name and the event name. In addition, the association unit 122 can specify the camera name of the surveillance camera 103 by sorting or SELECT of the SQL server, for example.

The camera name is, for example, the IP address of each of the plurality of surveillance cameras 103. The camera name is not limited to this as long as it is camera information for recognizing each of the plurality of surveillance cameras 103.

The association unit 122 may specify the camera name of the surveillance camera 103 by using the script program 600 of FIG. 6 instead of the correspondence table 500 of FIG.

FIG. 6 is a diagram showing an example of the script program 600 used by the association unit 122. The script program 600 is a conditional branch switch statement written in the C language of the programming language, and represents the camera name of the surveillance camera 103 associated with the device name and the event name. For example, when the device name is "first device" and the event name is "assembly abnormality", the camera name of the surveillance camera 103 is "first camera". The script program 600 is not limited in the language in which the program code is described and the description method thereof.

FIG. 7 is a diagram showing an example of the window 700, the window 701, and the event list 702 displayed on the display device 211. The window 700 has a control button 710 and a live phase diagram 720.

In step S103, the display unit 111 displays the window 700 on the display device 211. In step S208, the event data display unit 120 displays the event list 702 on the display device 211. In step S214, the video display unit 124 displays the window 701 on the display device 211.

Hereinafter, a method of displaying the window 701 using the event list 702 will be described. The event data display unit 120 displays the event list 702. The user selects event data 730 from a plurality of event data in the event list 702. Each of the plurality of event data indicates a period from the event occurrence time to the event end time. Then, the event data display unit 120 reverses the display of the line of the selected event data 730. The association unit 122 identifies, for example, the camera name of the surveillance camera 7201 associated with the selected event data 730. The surveillance camera 7201 is a live phase diagram showing one of the plurality of surveillance cameras 103.

At the display timing of the event list 702, the event end time of the event data in the event list 702 is blank for the event that has not ended yet. Further, at the display timing of the event list 702, the line of the event data in the event list 702 is blinking and displayed for the event that is still ongoing.

The video display unit 124 reproduces the video from the recording start time to the recording end time including the video captured by the specified surveillance camera 7201 in the period from the event occurrence time to the event end time in the event data 730, and the video is reproduced. Is displayed in window 701.

Window 701 is displayed by a window separate from window 700, but is not limited thereto. For example, the window 700 may switch the display from the live state diagram 720 to the image of the window 701. Further, the window 701 may be displayed in a part of the area (picture-in-picture) in the window 700.

When the association unit 122 specifies the camera names of the plurality of surveillance cameras 103, the image display unit 124 reproduces and displays a plurality of images captured by the specified plurality of surveillance cameras 103. In that case, the image display unit 124 may display a plurality of images in the plurality of windows 701, or may display a plurality of images in the plurality of divided areas in one window 701.

The display unit 111 displays the window 700. The window 700 has a control button 710 and a live phase diagram 720. The control button 710 has an operation start button 7100 and an operation stop button 7101.

The operation start button 7100 is a button for starting the processing of the monitoring device 100. The operation stop button 7101 is a button for stopping the processing of the monitoring device 100. The user can press the operation start button 7100 or the operation stop button 7101 by clicking the mouse. Of the operation start button 7100 and the operation stop button 7101, the button with a pattern like the operation start button 7100 is a button clicked by the user with the mouse. A button with a pattern such as the stop button 7101 is a button that has not been clicked by the user with the mouse. In the example of FIG. 7, the display of the control button 710 indicates that the monitoring device 100 is in the process of processing.

The display unit 111 displays the live state diagram 720. The live state diagram 720 has icons 7203 to 7208 whose display changes according to the state signal of the device 104 collected by the storage unit 110, and is a diagram for monitoring the device 104 and the production line.

The live phase diagram 720 also has icons 7200-7202. The icons 7200 to 7202 are icons representing a plurality of surveillance cameras 103. The plurality of surveillance cameras 103 photograph the plurality of devices 104 and the production line. The plurality of surveillance cameras 103 include, for example, a first surveillance camera 103 and a second surveillance camera 103. The plurality of devices 104 include, for example, a first device 104 and a second device 104.

The icon 7203 is an icon representing the first device 104 on which the inspection camera 7205 is mounted. The icon 7204 is an icon representing a second device 104 equipped with an inspection camera. The icon 7206 is an icon representing a main transport unit that transports the base component 740 and the finished product 760 in the production line. The icon 7207 is an icon representing a base component supply unit for supplying the base component 740 in the production line. The icon 7208 is an icon representing an assembly component supply unit for supplying the assembly component 750 in the production line.

The user can specify the camera information of the selected surveillance camera 103 by selecting one of the plurality of icons 7200 to 7202 representing the plurality of surveillance cameras 103. In that case, the display unit 111 outputs the designated camera information to the video display unit 124. The video display unit 124 acquires the live video captured by the surveillance camera 103 corresponding to the camera information from the recording device 101, and displays the live video. As a result, the user can easily grasp the state of the device 104.

FIG. 8 is an enlarged view of the window 701 of FIG. The video display unit 124 displays the video 800, the playback bar 801, the event start time 802, the event end time 803, and the playback date and time 804 in the window 701. Further, the video display unit 124 displays a playback knob 805, a rewind button 806, a frame rewind button 807, a play / pause button 808, a frame advance button 809, and a fast forward button 810 in the window 701. Further, the video display unit 124 displays the control command time jump units 81 to 814 on the playback bar 801 in the window 701. The video 800 is a video from the recording start time to the recording end time, and includes a video from the event occurrence time to the event end time.

The video display unit 124 digitally starts one of pan, tilt, and zoom for the video 800 being played back in the window 701 when the user specifies it, and returns to the original display of the video 800 when it is redesignated. The playback display switching button may be displayed.

Further, the video display unit 124 starts any one of optical pan, tilt, and zoom with the surveillance camera 103 when the user specifies the live video being reproduced, and when the video is redesignated, the original video 800 Return to display You may display the live display switching button.

FIG. 9 is a diagram showing an example of an operation corresponding to the event data of the first device of FIG. 4A. The time t flows from left to right and indicates from the recording start time to the recording end time. Time 901 is the event occurrence time. Time 902 is the event end time. The operation periods 903, 904, 905, 906, 907, and 908 are operation periods from the time when the start of the operation of the device (robot) 104 is controlled to the time when the end of the operation of the device 104 is controlled. Times 911, 913, 915, 917 and 919 are times when the start of operation of the device 104 is controlled. Times 910, 912, 914, 916 and 918 are times when the end of operation of the device 104 is controlled.

For example, the operation period 905 is a period from the time 913 when the start of the operation of the device 104 is controlled to the time 914 when the end of the operation of the device 104 is controlled. The start time 913 is the control command time for starting the operation of the control information D424 in FIG. The end time 914 is the control command time for the end of the operation of the control information D426 of FIG.

The operation period 906 is a period from the time 915 when the start of the operation of the device 104 is controlled to the time 916 when the end of the operation of the device 104 is controlled. The start time 915 is the control command time for starting the operation of the control information D428 in FIG. The end time 916 is the control command time for the end of the operation of the control information D430 in FIG.

The operation periods 903 to 908 correspond to the control command time jump units 81 to 816 in FIG. 8, respectively. The user selects an arbitrary position from the control command time jump units 81 to 816 on the playback bar 801 so that the video display unit 124 receives, for example, a control command for starting the operation of the device 104 that appears immediately before the event occurrence time 901. Video 800 can be played back from time 915. The image display unit 124 reproduces and displays the image 800 from the time based on the time when the operation of the device 104 is controlled among the images recorded in the recording device 101.

When the user selects the control command time jump unit 811, the video display unit 124 reproduces and displays the video 800 having the operation period 903 corresponding to the selected control command time jump unit 811. When the user selects the control command time jump unit 812, the video display unit 124 reproduces and displays the video 800 having the operation period 904 corresponding to the selected control command time jump unit 812. Similarly, when the user selects any of the control command time jump units 813 to 816, the video display unit 124 displays the video 800 of the operation period 905 to 908 corresponding to the selected control command time jump units 813 to 816, respectively. Play and display.

Further, the video display unit 124 can repeatedly reproduce the video 800 having the operation period 903 to 908 corresponding to any of the selected control command time jump units 81 to 816, and can repeatedly display the video 800.

In addition, the video display unit 124 can perform skip playback in which a part of the video 800 is skipped. The video display unit 124 sets the operation period 906 of the device 104 including the event occurrence time 901 in the video 800 during that period, for example, the control command time 916 of the operation end of the device 104 and the control of the start of operation. During the command time 917, it is possible to skip and play. The video display unit 124 can continuously reproduce and display the video 800 having a plurality of operation periods 903 to 908.

FIG. 10 is a diagram showing an example of the window 1000 displayed by the video display unit 124. The window 1000 is a window for comparing the image 1004 when the event occurs and the image 1003 when the event occurs. The normal image 1003 is an image when no event has occurred. The image display unit 124 displays a normal label 1001, an event occurrence label 1002, a normal image 1003, and an event occurrence image 1004 in the window 1000. In addition, the video display unit 124 has a common playback bar 1005, playback knob 1006, rewind button 1007, frame rewind button 1008, play / pause button 1009, frame advance button 1010, and fast forward button 1011 in the window 1000. indicate.

Further, the image display unit 124 displays the reproduction bar 1012 for the image 1003 and the selected reproduction area 1013 of the reproduction bars 1012 in the window 1000. The reproduction bar 1012 is the same as the reproduction bar 801 of FIG. The reproduction area 1013 corresponds to the control command time jump unit 813 of FIG. 8 and the normal operation period 905 of FIG. The image display unit 124 reproduces and displays the selected image 1003 of the normal operation period 905.

Further, the image display unit 124 has a reproduction bar 1014 for the image 1004 and a selected reproduction area 1015 of the reproduction bars 1014 in the window 1000. The reproduction bar 1014 is the same as the reproduction bar 801 of FIG. The reproduction area 1013 corresponds to the control command time jump unit 814 of FIG. 8 and the operation period 906 when the event occurs (abnormal time) of FIG. The video display unit 124 reproduces and displays the video 1004 having the operation period 906 when the selected event occurs.

The video display unit 124 simultaneously reproduces and displays the video 1003 having the normal operating period 905 and the video 1004 having the operating period 906 when the event occurs. The image display unit 124 can simultaneously reproduce and display images of at least two periods out of the images of a plurality of operation periods 903 to 908.

The image display unit 124 may be able to transition from the window 701 of FIG. 8 to the window 1000 of FIG. 10, or may display the window 1000 by other methods. Window 1000 shows an example of two images 1003 and 1004, but the number of images is not limited. The labeling of the video can be freely determined by the user.

The user can select the playback area 1013 or 1015 by clicking on the playback bar 1012 or 1014. In addition, the user can select the reproduction area 1013 or 1015 by selecting from the display of the list of the video group of the operation period of the device 104. Further, the plurality of images 1003 and 1004 may be displayed in different windows.

For example, it is assumed that an abnormality in the device 104 is detected, an image of the device 104 in which the abnormality is detected is recorded, and the user reproduces the image to analyze the cause. At this time, in reality, the movement of the device 104 is deviated from the normal state, and although the device 104 shows a sign of failure, the user seems to be moving without any problem just by looking at the image. It can look like this and miss the problem. Therefore, the image display unit 124 reproduces the normal image 1003 and the abnormal image 1004 in accordance with the control information in the window 1000 shown in FIG. As a result, the images 1003 and 1004 are reproduced from the timing when the control command is output to the device 104, and the user can easily recognize the abnormality of the movement of the device 104 and the timing deviation. For example, the image display unit 124 reproduces the image 1004 of the operation period 906 of the device 104 including the event occurrence time 901 of FIG. 9 and the image 1003 of the operation period 904 or 905 of the device 104 in the normal state side by side.

Further, FIG. 9 is a diagram corresponding to event data of the first device shown in FIG. 4 (a). In FIG. 4A, the operation period 906 of the device 104 corresponds to the period from the control command time of the operation start of the control information D428 to the control command time of the operation end of the control information D430. The image 1004 when the event occurs is an image of the operation period 906 of the device 104.

On the other hand, the normal image 1004 is an image of the operation period 905 of the device 104. In FIG. 4A, the operation period 905 of the device 104 corresponds to the period from the control command time of the operation start of the control information D424 to the control command time of the operation end of the control information D426.

The video display unit 124 may display the labeled and color-coded areas on the reproduction bars 1012 and 1014 according to the value of the control information so that the common operation portions of the video 1003 and 1004 can be easily identified.

As described above, the event data display unit 120 acquires the update event data from the storage unit 110. The association unit 122 identifies the device name included in the update event data and the camera name of the surveillance camera 103 associated with the event name. After that, the acquisition unit 123 issues a recording start command and a recording end command for the video of the surveillance camera 103 to the recording device 101 based on the specified camera name and the event start time and event end time included in the update event data. Output. As a result, when an event occurs in the device 104, the recording device 101 can surely record the image at the time when the event occurs.

Further, the event data display unit 120 displays an event list 702 that lists the event data 400 of the storage unit 110. After that, when the user selects the event data 730 from the event list 702, the acquisition unit 121 acquires the event data 730 from the event data display unit 120 as the designated event data. The association unit 122 identifies the device name included in the designated event data and the camera name of the surveillance camera 103 associated with the event name. After that, the video display unit 124 acquires and reproduces the video 800 specified based on the event start time and the event end time included in the specified camera name and the designated event data from the recording device 101. As a result, the image display unit 124 can quickly reproduce the image 800 at the time of the event occurrence desired by the user from the past images of the device 104.

Further, when determining the reproduction position, the image display unit 124 can quickly reproduce the image 800 from the desired reproduction position by selecting from the reproduction start candidates based on the control information included in the designated event data.

Further, the video display unit 124 can synchronize the video 1003 and 1004 in which the device 104 is performing the same operation by playing the plurality of video 1003 and 1004 in accordance with the timing when the control information changes. This makes it easier for the user to find anomalies that may have been overlooked in the past.

The video display unit 124 can obtain the playback position of the video 800 based on the timing at which the device 104 to be monitored is controlled. The image display unit 124 can be used at the timing when the device 104 inputs the control command even when the change on the image 800 or the change in sound is small or when the device 104 does not move even though the control command is received. Based on this, the video 800 can be reproduced from a desired reproduction position. The image display unit 124 can quickly reproduce the image 800 from a meaningful place.

According to the present embodiment, the control device 102 can reproduce the video from an appropriate time in the past video of the device 104 based on the control information. Since the control device 102 can reproduce the video at a meaningful time such as the operating period of the device 104 in the video, the burden on the user can be reduced.

(Embodiment 2)
FIG. 11 is a schematic diagram showing an outline of a device condition monitoring system. The condition monitoring system includes one or more devices 1101a to 1101c, one or more imaging devices 1102a to 1102c, a recording device 1103, and a control device 1104. It is assumed that communication is performed between the devices by a communication method conforming to the Ethernet (registered trademark) standard. The communication method between devices is not limited to the above. It is also possible to use a plurality of communication methods. Hereinafter, the devices 1101a to 1101c will be referred to as a device 1101, and the imaging devices 1102a to 1102c will be referred to as an imaging device 1102. The input device 1105 and the display device 1106 are connected to the control device 1104.

The device 1101 is a device whose operation is controlled in response to a control signal from the control device 1104. For example, device 1101a is a robot arm for moving a work from one space A to the other space B. There may be a plurality of devices 1101 that are controlled in response to the control signal from the control device 1104. For example, the device 1101b is a belt conveyor that conveys the work to the space A in FIG. The device 1101c is a belt conveyor that conveys the work from the space B. Hereinafter, the robot arm 1101a will be described as the device 1101.

The image pickup device 1102 is an image pickup device capable of capturing an image pickup range related to the operation of the device 1101 and acquiring a video signal. The image pickup apparatus 1102 is provided with a buffer memory and can store the acquired video signal for a predetermined period of time. The imaging device 1102 includes an imaging device 1102a, an imaging device 1102b, and an imaging device 1102c. The image pickup apparatus 1102a images the operating range of the apparatus 1101. The image pickup apparatus 1102b images the work before being moved by the apparatus 1101 arranged in the space A. The image pickup apparatus 1102c images the work after being moved to the space B by the apparatus 1101. When each image pickup apparatus is distinguished from each other, the imaging apparatus 1102a, 1102b, 1102c are indicated by reference numerals. When the control common to the entire image pickup apparatus is shown, it is referred to as the image pickup apparatus 1102.

The image pickup device 1102 outputs the acquired video signal to the control device 1104 each time. Further, the imaging device 1102 outputs a video signal corresponding to the period corresponding to the request to the control device 1104 in response to the request from the control device 1104. The recording device 1103 is a recording device that records the video signal acquired by the imaging device 1102. In addition, the recording device 1103 records information indicating the timing at which the device 1101 executes a predetermined operation in association with the video signal during the period in which the recorded video signal is acquired by the imaging device 1102.

The control device 1104 is a control device that controls the entire condition monitoring system. The control device 1104 controls the operations of the device 1101, the image pickup device 1102, and the recording device 1103.

FIG. 12 is a diagram showing a configuration example of the control device 1104. The control device 1104 includes a control unit 1201, a system memory 1202, a temporary memory 1203, a network interface 1204, an input interface 1205, and a display control memory 1206.

The control unit 1201 is a processor that controls the operation of the control device 1104 by executing a program. It is assumed that the control unit 1201 is, for example, a Central Processing Unit (CPU). The control unit 1201 may be composed of two or more processors. Further, the control unit 1201 may include an electronic circuit capable of executing one or more of the functions described later. The control unit 1201 also controls other devices of the condition monitoring system.

The system memory 1202 is a non-volatile storage medium that stores a program executed by the control unit 1201 and parameters used for the operation of the control unit 1201. The temporary memory 1203 is a readable and writable storage medium that temporarily stores data used by the control unit 1201 to control the operation of the control device 1104.

The network interface 1204 is an interface that communicates with the device 1101, the image pickup device 1102, and the recording device 1103. The network interface 1204 is an interface that connects to the recording device 1103 and transmits / receives video signals and control timing information to / from the recording device 1103.

The network interface 1204 is an interface that connects to the device 1101 and transmits / receives information regarding control commands and execution timing of operations. The display control memory 1206 is a storage medium for storing image data to be displayed on the display device 1106. The display device 1106 reads out the image data of the display control memory 1206 and displays the image. The display control memory 1206 may be an output interface that outputs image data to the display device 1106 instead of a storage medium on which the display device 1106 can read the image data.

The input interface 1205 is a reception unit, which is an interface that connects to the input device 1105 and receives instructions from the user. It is assumed that the input device 1105 is an input member such as a mouse or a keyboard. Further, the input device 1105 may be a touch panel provided on the display device 1106.

The control unit 1201 is a drive control unit, and executes a drive control process for outputting a control command (instruction) for causing the device 1101 to execute a predetermined operation to the device 1101 via the network interface 1204. The control command is information indicating the content of control in which the device 1101 operates. The control instruction may include information indicating the time or timing at which control is executed. This time or timing is the timing at which the control unit 1201 outputs a control command.

The control unit 1201 executes an acquisition process for acquiring a state signal indicating the state of the device 1101 from the device 1101 via the network interface 1204. The state signal is, for example, an output value of a temperature sensor (not shown) provided in the device 1101, a detected value of a load applied to the device 1101, or a drive current value of the device 1101. The control unit 1201 executes a recording control process of recording the acquired status signal as log data in the recording device 1103 via the network interface 1204. The log data is data in which the time and the value of the status signal are associated with each other.

The control unit 1201 executes a recording control process for recording the control timing information indicating the timing at which the device 1101 executes a predetermined operation in the recording device 1103 via the network interface 1204. The control unit 1201 records information indicating the timing at which a control command instructing the apparatus 1101 to execute a predetermined operation is output as control timing information. The predetermined operation is an operation in which the device 1101 which is a robot arm holds the work in the space A (pick operation), lifts the work (up operation), and moves the held work to the space B side (move operation). , And the operation of arranging the work at the target position of the space B (place operation). The predetermined operation may be any of the above-mentioned operations. The predetermined operation can be arbitrarily determined depending on the type of the device 1101 and the contents of the system.

When the control unit 1201 outputs information indicating the timing for executing a predetermined operation to the device 1101, the information indicating the timing may be recorded as the control timing information. Further, the control device 101 may receive information indicating the timing of executing a predetermined operation from the device 101 via the network interface 1204 and record it in the recording device 1103 as control timing information. That is, the control timing information may be such that the user (administrator) can use the timing related to the timing at which the device 1101 executes a predetermined operation as the reproduction timing of the video signal at the time of reproducing the video signal described later. ..

The control unit 1201 acquires a video signal from the image pickup apparatus 1102 and executes a determination process for determining whether or not an abnormal state has occurred. The control unit 1201 analyzes the live view image acquired by the image pickup apparatus 1102, and determines that an abnormal state has occurred when a predetermined determination condition is satisfied. For example, it is assumed that the determination condition is that the work moved to the space B protrudes from a predetermined arrangement position by a predetermined amount or more. The control unit 1201 analyzes the live view image acquired from the image pickup apparatus 1102c, and determines that an abnormal state has occurred when the determination condition is satisfied. The control unit 1201 may determine the occurrence of an abnormal state based on whether or not the state signal is included in a predetermined range.

The control unit 1201 records the video signals acquired by the imaging devices 1102a, 1102b, and 1102c for a predetermined period including the timing at which the determination is made in response to the determination that the abnormal state has occurred, in the recording device 1103. Execute recording control to record in. In the present embodiment, the control unit 1201 causes the image pickup apparatus 1102 and the recording apparatus 1103 to execute the pre-recording process and the post-recording process. The pre-recording process is a process in which the control unit 1201 records (pre-recording) a video signal acquired in a period retroactive from a predetermined pre-period from the timing when it is determined that an abnormal state has occurred. The post recording process is a process in which the control unit 1201 records (post recording) a video signal acquired during a predetermined post period from the timing when it is determined that an abnormal state has occurred. The period to be recorded, which is the sum of the pre-period and the post period, is referred to as a recording period in the present embodiment. The pre-recording process and the post-recording process may be either one or the other.

In the recording control, the video signal recorded in the recording device 1103 is input from the imaging device 1102 to the recording device 1103. The control unit 1201 transmits an instruction to the image pickup device 1102 to output the video signal acquired during the recording period among the video signals stored in the buffer memory of the image pickup device 1102 to the recording device 1103. Further, the control unit 1201 transmits an instruction to the recording device 1103 so as to record the video signal input from the image pickup device 1102.

In the recording control, the control unit 1201 records the information (event information) indicating the details of the abnormal state that triggered the recording of the video signal in the recording device 1103 in association with the corresponding video signal. The event information indicates the status corresponding to the judgment condition when the abnormal state is determined.

Next, the operation when the user confirms the video signal recorded in the recording device 1103 will be described. By checking the video signals before and after the occurrence of the abnormality recorded in the recording device 1103, the user can analyze the cause of the occurrence of the abnormality and examine countermeasures.

The control unit 1201 displays a graphical user interface (GUI) for selecting a video signal recorded on the recording device 1103 on the display device 1106. The GUI includes information indicating the name of the abnormal state that triggered the abnormal state and the time when the abnormal state occurred as information indicating the video signal recorded in the recording device 1103.

The user operates the displayed GUI using the input device 1105 to select a video signal to be displayed on the display device 1106. The control unit 1201 generates image data for display based on the selected video signal.

FIG. 13 is a schematic diagram showing an image 1300 displayed on the display device 1106 based on the selected video signal.

The control unit 1201 generates image data so as to include at least the area 1301 and the area 1302. An image based on the selected video signal is displayed in the area 1301. In the area 1302, a user interface image (UI image) indicating the timing of the displayed video signal during the recording period of the selected video signal is displayed. At this time, if the device 1101 includes a control timing for executing a predetermined operation in the recording period of the selected video signal, the control unit 1201 displays an index (mark) 1303 indicating the control timing on the UI image. ..

Here, in the area 1302, the entire recording period of the video signal, the index 1304 indicating the timing of the video signal being reproduced during the recording period, and the control timing at which the device 1101 executes a predetermined operation during the recording period. The index 1303 indicating the above and the time display bar indicating the above are displayed. The time display bar indicates the period for which the video signal is reproduced. Indicator 1304 indicates the timing at which the displayed image corresponds. The index 1303 may also display information indicating the corresponding operation. In the present embodiment, when the user moves the mouse cursor to the index 1303, information indicating the corresponding operation is displayed in a pop-up. Further, the control unit 1201 may display the index 1305 indicating the timing at which the abnormal state that triggered the recording of the video signal being displayed is generated in the time display bar.

When a predetermined input is made by the user with respect to the index indicating the control timing of the displayed UI image, the control unit 1201 starts the reproduction of the video signal from the time corresponding to the control timing. The predetermined input is an input by clicking the mouse in accordance with the index. The predetermined input may be double-clicked.

According to the process of the control device 1103 described above, it is possible to display information indicating the timing of a predetermined operation of the device 1101 executed within the recording period at the time of reproducing the video signal. This makes it easy for the user to specify the timing at which the device 1101 operates and execute the reproduction of the video signal. When an abnormality occurs in the device 1101 or the system in which the device 1101 operates, there is a high possibility that the abnormality will occur before or after the device 1101 executes some operation in analyzing the cause. According to the control of the present embodiment, the user can easily confirm the situation at the timing when the operation of the device 1101 is executed.

The control timing shown in the area 1302 may indicate the operation of the plurality of devices 1101. For example, the control device 1104 can control a plurality of devices 1101. The plurality of devices 1101 include a robot arm 1101a for moving the work from the space A to the other space B, a belt conveyor 1101b for transporting the work to the space A, and a belt conveyor 1101c for transporting the work from the space B. At this time, the control unit 1201 displays the index corresponding to the control timing of each device in the area 1302 so as to be identifiable. For example, the control unit 1201 displays the index in different colors for each device. For example, the control unit 1201 displays the timing at which the respective conveying speeds of the belt conveyors 1101a and 1101c change as the control timing. For example, the index 1303 in FIG. 13 is an index indicating the timing related to the control of the robot arm 1101a, and the index 1306 is an index indicating the timing when the speed change operation of the belt conveyor 1101b is executed. Indicators 1303 and 1306 are index images. As a result, the user can determine the device 1101 that is likely to be related to the abnormal state that has occurred, and can confirm the image at the timing of the operation that is likely to be related.

Further, the control unit 1201 selects and displays the control timing of the device 1101 included in the angle of view of the selected video signal when the reproduction of the video signal is instructed. For example, in the video signal captured by the imaging device 1102a, since the plurality of devices 1101 described above are all included in the angle of view, an index indicating all control timings of the plurality of devices 1101 is displayed in the area 1302. However, when the video signal captured by the image pickup apparatus 1102b is selected, the belt conveyor 1101c is not included in the angle of view of the selected video signal. Therefore, the control unit 1201 generates image data so as to display an index indicating the control timing of the robot arm 1101a and the belt conveyor 1101b and not to display the index indicating the control timing of the belt conveyor 1101c. By controlling in this way, the user can easily confirm the image related to the operation of the device 1101 included in the confirmed image.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or recording medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. The recording medium is a computer-readable recording medium on which the above program is recorded. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

It should be noted that all of the above embodiments merely show examples of embodiment in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner by these. That is, the present invention can be implemented in various forms without departing from the technical idea or its main features.

100 Surveillance device, 101 Recording device, 102 Control device, 103 Surveillance camera, 104 equipment, 120 Event data display unit, 121 Acquisition unit, 122 Association unit, 123 Acquisition unit, 124 Video display unit

Claims (18)

A recording control means for causing the recording means to record a video signal obtained by photographing the first external device,
An acquisition means for acquiring first timing information indicating the timing at which the first external device executes a predetermined operation, and
A display control means for displaying an image obtained by reproducing the video signal on the display means,
With
The display control means is a control device characterized in that the display means displays information indicating a timing corresponding to the first timing information. Equipped with a reception means that accepts input from users
The display control means
An index image showing the first timing information is displayed on the display means,
The control device according to claim 1, wherein the image is displayed at a time corresponding to the predetermined operation of the external device in response to receiving a predetermined input for the index image by the user. The display control means includes a period for reproducing the video signal, a timing corresponding to the image displayed on the display means in the period of the video signal, and a timing corresponding to the first timing information. The control device according to claim 1 or 2, wherein a time display bar indicating the above is displayed on the display means. The recording control means is characterized in that, in response to the occurrence of a predetermined event in the first external device, the recording means is made to record a video signal for a predetermined period including the occurrence time of the event. The control device according to any one of claims 1 to 3. A drive control means for outputting an instruction to execute the predetermined operation to the first external device is provided.
The control device according to any one of claims 1 to 4, wherein the first timing information indicates a timing at which the drive control means outputs the instruction. The acquisition means acquires second timing information indicating the timing at which a second external device different from the first external device executes a predetermined operation.
The display control means is characterized in that information indicating the timing corresponding to the second timing information is displayed on the display means so as to be distinguishable from the information indicating the timing corresponding to the first timing information. The control device according to any one of 1 to 5. The control device according to claim 1, wherein the display control means causes the display means to display the image at a time corresponding to the predetermined operation of the external device. The display control means displays an image from the time when the start of the predetermined operation of the first external device is controlled to the time when the end of the predetermined operation of the first external device is controlled. The control device according to claim 7, wherein the control device is displayed on the screen. The display control means causes the display means to continuously display images for a plurality of periods from the time when the start of the operation of the external device is controlled to the time when the end of the operation of the external device is controlled. The control device according to claim 1 or 2. The display control means is from the time when the start of the predetermined operation of the first external device is controlled to the time when the end of the predetermined operation of the first external device is controlled when an event occurs. And the time when the end of the predetermined operation of the first external device is controlled from the time when the start of the predetermined operation of the first external device is controlled when the event does not occur. The control device according to claim 8 or 9, wherein the images up to the above are displayed on the display means at the same time. The first external device and
Recording device and
Display device and
Control device and
It is a control system equipped with
A recording control means for causing the recording device to record a video signal obtained by photographing the first external device, and
An acquisition means for acquiring first timing information indicating the timing at which the first external device executes a predetermined operation, and
A display control means for displaying an image obtained by reproducing the video signal on the display device,
With
The display control means is a control system characterized in that the display device displays information indicating the timing corresponding to the first timing information. A recording control process for recording a video signal obtained by photographing the first external device on the recording device, and
An acquisition step of acquiring first timing information indicating the timing at which the first external device executes a predetermined operation, and
A display control step of displaying the image obtained by reproducing the video signal on the display device, and
With
The display control step is a control method characterized in that information indicating a timing corresponding to the first timing information is displayed on the display device. Equipped with a reception process that accepts input from users
The display control step is
An index image showing the first timing information is displayed on the display device,
The control method according to claim 12, wherein the image is displayed at a time corresponding to the predetermined operation of the external device in response to the user receiving a predetermined input to the index image. The display control step includes a period of the video signal to be reproduced, a timing corresponding to the image displayed on the display device in the period of the video signal, and a timing corresponding to the first timing information. The control method according to claim 12 or 13, wherein the time display bar indicating the above is displayed on the display device. The recording control step is characterized in that, in response to the occurrence of a predetermined event in the first external device, the recording device records a video signal for a predetermined period including the occurrence time of the event. The control method according to any one of claims 12 to 14. A drive control step for outputting an instruction to cause the first external device to execute the predetermined operation is provided.
The control method according to any one of claims 12 to 15, wherein the first timing information indicates a timing at which the drive control step outputs the instruction. A program for causing a computer to function as each means of the control device according to any one of claims 1 to 10. A computer-readable recording medium on which the program according to claim 17 is recorded.

Images