WO2019021342A1

WO2019021342A1 - Display and display method

Info

Publication number: WO2019021342A1
Application number: PCT/JP2017/026667
Authority: WO
Inventors: 孝一折戸; 良輔 ▲高▼橋
Original assignee: 三菱電機株式会社
Priority date: 2017-07-24
Filing date: 2017-07-24
Publication date: 2019-01-31
Also published as: CN110089110A; JPWO2019021342A1; CN110089110B; JP6355878B1

Abstract

A programmable display according to the present invention is provided with: a storage unit for storing a master video serving as a reference video from among a plurality of videos to be synchronously played back, a slave video serving as a video the playback speed of which is adjusted from among the plurality of videos, a start event serving as an event for starting synchronous playback of a portion of the master video and a portion of the slave video, and an end event serving as an event for ending the synchronous playback; a data processing unit for adjusting the setting of the playback speed of the slave video from the start event to the end event so that the video in the slave video from the start event to the end event is played back synchronously with the video in the master video from the start event to the end event; and a display unit for synchronously playing back the master video and the adjusted slave video from the start event to the end event.

Description

Display and display method

The present invention relates to a display for displaying an image and a display method.

In recent years, a display for displaying a video has various display functions such as a function of changing a display speed of the video and a function of simultaneously displaying a plurality of videos, and can display the video in various modes.

The moving image reproducing apparatus described in Patent Document 1 captures the video on the master side and the video on the slave side at the same frame rate, and the operation speed of the specific operation on the master side and the operation speed of the specific operation on the slave side And compare. Then, the moving image reproduction apparatus described in Patent Document 1 controls the reproduction speed on the slave side based on the comparison result of the operation speed, thereby synchronously reproducing the video on the master side and the video on the slave side.

JP 2014-225772 A

However, in Patent Document 1 which is the above-mentioned prior art, since it is necessary to shoot the video on the master side and the video on the slave side at the same frame rate, there is a problem that the restriction on capturing the video increases. there were.

The present invention has been made in view of the above, and it is an object of the present invention to obtain a display capable of synchronously reproducing a plurality of videos in which moving image frames are obtained under various conditions.

In order to solve the problems described above and to achieve the object, according to the present invention, in the display device, the master video serving as the reference video among the plurality of videos to be synchronously played back and the playback speed among the plurality of videos are adjusted. And a start event which is an event for starting synchronous reproduction of a part of the master video and a part of the slave video, and an end event which is an event for ending the synchronous reproduction. Storage unit. In the display device of the present invention, the end of the video in the slave video between the start event which is a part of the slave video and the end event, and the start of the start event which is a part of the master video. A data processing unit is provided which adjusts the setting of the reproduction speed of the slave image from the start event to the end event so that the image in the master image until the event is synchronously reproduced. The display unit according to the present invention further includes a display unit that synchronously reproduces the master image and the slave image after adjustment from the start event to the end event.

The display according to the present invention has an effect of being able to synchronously reproduce a plurality of videos for which moving image frames are obtained under various conditions.

A block diagram showing a configuration of a programmable display according to an embodiment of the present invention Flow chart showing the processing procedure of the event setting process according to the embodiment Flow chart showing the processing procedure of the recording process according to the embodiment A diagram showing a configuration of video data at the time of recording according to the embodiment Flow chart showing processing procedure of video reproduction processing according to the embodiment A diagram showing a configuration of video data at the time of synchronous reproduction according to the embodiment A figure showing an example of hardware constitutions of a data processing part concerning an embodiment

Hereinafter, a display and a display method according to an embodiment of the present invention will be described in detail based on the drawings. The present invention is not limited by the embodiment.

Embodiment FIG. 1 is a block diagram showing a configuration of a programmable display according to an embodiment of the present invention. A programmable display 10, which is an example of a display, is connected to an external device 30 and an imaging device 40.

An example of the external device 30 is a control device such as a programmable logic controller (PLC). When the external device 30 is a PLC, the external device 30 sends information collected from other devices such as a sensor or a robot to the programmable display 10. The external device 30 may be another device such as a sensor or a robot. Below, the case where the external device 30 is PLC is demonstrated. Therefore, information detected by another device such as a sensor or a robot is sent to the programmable display 10 via the PLC.

The external device 30 sends an event indicating the start of the process and an event indicating the end of the process to the programmable display 10. In the following description, an event indicating the start of the process is referred to as a start event V1, and an event indicating the end of the process is referred to as an end event V2. The external device 30 generates the start event V1 when the first trigger condition is satisfied, and generates the end event V2 when the second trigger condition is satisfied. The start event V1 and the end event V2 are information indicating that a specific event has occurred during the process. Examples of the start event V1 and the end event V2 which are event information are device values or sensor information on a monitoring target monitored by the external device 30. The start event V1 and the end event V2 occur when an event such as contact on, contact off, statement transition, or detected numerical transition occurs. The start event V1 and the end event V2 are represented by the on / off state of a bit from the external device 30 to the programmable display 10 to indicate presence / absence of occurrence.

The programmable display 10 is a device that displays various information such as a value of an internal memory of the PLC, information acquired by the PLC from a sensor or a robot, and an image captured by the imaging device 40.

The programmable display 10 according to the embodiment synchronously reproduces, based on the start event V1 and the end event V2 sent from the external device 30, a plurality of videos in which frames are acquired under various conditions. Specifically, the programmable display 10 adjusts the reproduction speed of the slave video so that the slave video to be described later and the master video to be described later are synchronously reproduced from the start event V1 to the end event V2. In the embodiment, a case will be described in which the programmable display 10 is a display used in the field of FA (Factory Automation) aiming to automate a production process of a factory.

The imaging device 40 images various monitored objects such as products, manufacturing devices, or materials in a factory, and sends an image, which is imaging data, to the programmable display 10. The monitoring target is a controlled device when the external device 30 is a control device, and is a sensed device when the external device 30 is a sensing device such as a sensor. The imaging device 40 may record an image obtained by imaging the monitoring target.

The imaging device 40 captures a master video, which is a reference video among the videos to be synchronously reproduced, and a slave video, which is a video whose reproduction speed is adjusted, among the videos to be synchronously reproduced. Which video is to be the master video and which is to be the slave video is designated by the operator on the programmable display 10 at the time of synchronous reproduction. The imaging device 40 sends the captured video to the imaging device connection IF (I / F: Interface) 13. An example of the video that the imaging device 40 sends to the imaging device connection IF 13 is a video captured by a camera or a video.

The number of imaging devices 40 for capturing a master video and a slave video may be one or more. That is, the master video and the slave video may be captured by the same imaging device 40 or may be captured by different imaging devices 40. Also, the master video and the slave video may be captured simultaneously or may be captured at different dates and times. The master video and the slave video are captured by different imaging devices 40 when they are captured simultaneously. Also, the master video and the slave video may be captured from the same angle, or may be captured from different angles.

Also, the master video and the slave video may be videos with different frame rates, or may be videos with the same frame rate. The frame rate is the number of frames per unit time.

In addition, the imaging length, which is the imaging time, may be different between the video portion of the master video and the video portion of the slave video that are synchronously reproduced, or the imaging time may be the same. . The video portion of the master video to be synchronously reproduced is a video from a master start frame MS described later to the master end frame ME, and the video portion to be synchronously reproduced of the slave video is a slave end from a slave start frame SS described later It is an image up to the frame SE.

In the following description, there is one imaging device 40, and the same imaging device 40 captures a master video and a slave video at different dates and times. In the embodiment, the master video captured by the imaging device 40 is a video obtained when the monitoring target is operated in the first operation, and the slave video captured by the imaging device 40 is the second monitoring target. It is assumed that it is an image at the time of operating by operation.

When the external device 30 sends the detection value detected by the sensor to the programmable display 10, the external device 30 detects the occurrence of the start event V1 or the end event V2 when the detection value detected by the sensor exceeds the reference value. To the programmable display 10. Also, when the external device 30 sends information of the motion detected by the robot such as the robot arm to the programmable display 10, when the robot performs a specific motion, the external device 30 detects the occurrence of the start event V1 or the end event V2. And send to the programmable display 10. When the external device 30 operates according to a program such as a ladder program, when the external device 30 executes a program corresponding to a specific operation, the external device 30 detects the occurrence of the start event V1 or the end event V2 and performs programmable display To the container 10.

The programmable display 10 includes an imaging device connection IF 13 that receives an image sent from the imaging device 40. The video sent from the imaging device 40 is a video set as a master video or a slave video. The programmable display 10 further includes an external device connection IF 12 that receives the start event V1 and the end event V2 sent from the external device 30. The programmable display 10 further includes an input unit 11 for receiving an instruction from the operator, and a data processing unit 14 for adjusting the video playback speed. The programmable display 10 further includes a display unit 17 for displaying an image, a display information generation unit 16 for generating information to be displayed on the display unit 17, and a storage unit 15 for storing the image sent from the data processing unit 14. And have.

The external device connection IF 12 is an interface for communicating with the external device 30. The external device connection IF 12 is connected to the external device 30, and receives the start event V1 and the end event V2 sent from the external device 30. An example of the external device connection IF 12 is a Universal Serial Bus (USB) connector or a Recommended Standard (RS) -232C connector. The external device connection IF 12 sends the received start event V1 and end event V2 to the data processing unit 14.

The imaging device connection IF 13 is an interface for communicating with the imaging device 40. The imaging device connection IF 13 is connected to the imaging device 40 and receives an image sent from the imaging device 40. An example of the imaging device connection IF 13 is a USB connector or Ethernet (registered trademark). The imaging device connection IF 13 sends the received video to the data processing unit 14.

The input unit 11 is a mouse, a keyboard or a touch panel. When the operator inputs an instruction to the mouse, the keyboard or the touch panel, the input unit 11 receives the instruction and inputs the instruction to the data processing unit 14.

The input unit 11 is an interface for receiving the following instructions (1) to (7) by the operator, and inputs the accepted instruction to the data processing unit 14.
(1) instruction to specify the type of start event V1 (2) instruction to specify the type of end event V2 (3) instruction to start video recording (4) instruction to end video recording (5) synchronous playback Instruction to specify master image (6) Instruction to specify slave image to be synchronized playback (7) Instruction to start synchronized playback of master image and slave image

When the data processing unit 14 receives an instruction to specify the type of start event V1 from the input unit 11, the data processing unit 14 sets a start event V1 that is the start timing of synchronized reproduction of video. Further, when the data processing unit 14 receives an instruction to specify the type of the end event V2 from the input unit 11, the data processing unit 14 sets an end event V2 which is an end timing of synchronous reproduction of video. The data processing unit 14 stores the set start event V1 and end event V2 in the storage unit 15.

The data processing unit 14 includes a RAM (Random Access Memory) 140, and the RAM 140 temporarily stores an image sent from the imaging device 40. The RAM 140 has a ring buffer function, and when the data storage area in the RAM 140 is filled with video, the data processing unit 14 sequentially deletes the oldest video and stores new video preferentially.

When the data processing unit 14 receives an instruction to start video recording from the input unit 11, the data processing unit 14 starts video recording sent from the imaging device 40 and stores the video in the RAM 140. Specifically, the data processing unit 14 converts the video signal into electronic data by executing processing such as AD (Analog-to-Digital: analog-to-digital) conversion on the video signal of the video, thereby generating video data. Do. Then, the data processing unit 14 stores the generated video data in the storage unit 15.

The data processing unit 14 detects the start event V1 when the same start event V1 as the set start event V1 is sent from the external device 30 during recording, and the frame acquired at the timing when the start event V1 is detected. Is stored in the RAM 140. The data processing unit 14 detects an end event V2 when an end event V2 identical to the set end event V2 is sent from the external device 30 during recording, and is acquired at the timing when the end event V2 is detected. The frame position of the selected frame is stored in the RAM 140.

The frame position is information of a frame number indicating which frame the frame is from the head of the video data. In the following description, although the case where the programmable display 10 uses a frame number as a frame position will be described, the programmable display 10 may use an elapsed time of an image as a frame position instead of the frame number. In the following description, the frame position of the frame acquired at the timing when the start event V1 is detected is referred to as the start frame position. Further, the frame position of the frame acquired at the timing when the end event V2 is detected is referred to as an end frame position. The start frame position is the position of the first frame of the synchronously reproduced video portion, and the end frame position is the position of the last frame of the synchronously reproduced video portion.

The data processing unit 14 causes the storage unit 15 to store the video portion in the RAM 140 for which the recording has been completed. The data processing unit 14 causes the storage unit 15 to sequentially store the latest video portions that are video portions in the RAM 140 and are not stored in the storage unit 15. In this case, the data processing unit 14 stores the recorded video, the start frame position, and the end frame position in the storage unit 15 in association with each other. The video to be stored in the storage unit 15 by the data processing unit 14 includes a frame at the start frame position and a frame at the end frame position.

When the data processing unit 14 receives an instruction to end the video recording from the input unit 11, the data processing unit 14 ends the process of storing the video in the RAM 140. Then, when the data processing unit 14 stores all latest video portions in the RAM 140 in the storage unit 15, the data processing unit 14 ends the process of storing the video in the storage unit 15. Thereby, the storage unit 15 stores a series of video from the start to the end of the recording.

When the data processing unit 14 receives an instruction to specify a slave video from the input unit 11, the data processing unit 14 reads the specified slave video from the storage unit 15. The data processing unit 14 sets the frame at the start frame position among the frames of the read video as the slave start frame SS. The slave start frame SS is a frame of the first timing at which the start event V1 has occurred in the slave video, and is a frame for starting synchronous reproduction. In other words, the slave start frame SS is the first frame of slave video on which synchronous reproduction is performed.

Further, the data processing unit 14 sets the frame at the end frame position among the frames of the read video as the slave end frame SE. The slave end frame SE is a frame of the second timing at which the end event V2 has occurred in the slave video, and is a frame for ending the synchronous reproduction. In other words, the slave end frame SE is the last frame of the slave video on which synchronous reproduction is performed. The processing to be executed on the monitoring target from the slave start frame SS to the slave end frame SE is processing for one tact. The processing for one tact is processing included within the scope of one sequence processing that the PLC executes. An example of processing for one tact is processing from detection of the start event V1 by the sensor to detection of the end event V2 by the sensor.

When the data processing unit 14 receives an instruction to specify a master video from the input unit 11, the data processing unit 14 reads the specified master video from the storage unit 15. The data processing unit 14 sets the frame at the start frame position among the frames of the read video as the master start frame MS. The master start frame MS is a frame of the third timing at which the start event V1 has occurred in the master video, and is a frame for starting synchronous reproduction. In other words, the master start frame MS is the first frame of the master video on which synchronous reproduction is performed.

Further, the data processing unit 14 sets a frame at an end frame position among the frames of the read video as a master end frame ME. The master end frame ME is a frame of the fourth timing at which the end event V2 occurs in the master video, and is a frame for ending synchronous reproduction. In other words, the master end frame ME is the last frame of the master video on which synchronous reproduction is performed.

When the data processing unit 14 receives an instruction to specify a plurality of slave videos from the input unit 11, the data processing unit 14 sets a slave start frame SS and a slave end frame SE to each of the specified plurality of slave videos.

When the data processing unit 14 receives from the input unit 11 an instruction to start synchronous reproduction of the master video and the slave video, the slave 14 and the master video are synchronously reproduced between the start event V1 and the end event V2. Adjust the video playback speed settings. At this time, the data processing unit 14 ends the slave from the slave start frame SS so that the reproduction time from the slave start frame SS to the slave end frame SE is the same as the reproduction time from the master start frame MS to the master end frame ME. Adjust the playback speed setting up to the frame SE. In this case, the data processing unit 14 adjusts the setting of the reproduction speed of the slave video so that the slave video is reproduced at a constant frame rate. The data processing unit 14 sends, to the display information generation unit 16, the master video and the slave video whose playback speed has been adjusted.

When the data processing unit 14 receives from the input unit 11 an instruction to specify the master video and the slave video to be synchronously reproduced, the data processing unit 14 may determine that the instruction to start the synchronous reproduction of the master video and the slave video is given. . In this case, the data processing unit 14 adjusts the reproduction speed of the slave video without receiving the instruction to start the synchronous reproduction.

The storage unit 15 is a non-volatile storage device. An example of the storage unit 15 is a non-volatile memory such as a flash memory or a drive such as an HDD (Hard Disk Drive). The storage unit 15 stores a master video and a slave video, which are videos obtained by capturing the monitoring target. In addition, the storage unit 15 stores a start event V1 which is an event for starting synchronous reproduction of a part of the master video and a part of the slave video, and an end event V2 which is an event for ending the synchronous reproduction.

The display information generation unit 16 generates screen data for the display unit 17 to display an image based on the master image sent from the data processing unit 14 and the slave image whose setting of the reproduction speed is adjusted. . An example of a video generated by the display information generation unit 16 is a video for simultaneously displaying a master video and a slave video whose playback speed has been adjusted. In this case, the display information generation unit 16 causes the master video to be displayed in the first area of the video display screen, and generates the video in which the slave video whose reproduction speed has been adjusted is displayed in the second area. The display information generation unit 16 sends the generated video to the display unit 17.

The display unit 17 is a functional unit that displays various information such as data, an image, and a video. An example of the screen data generated by the display information generation unit 16 is video data. The display unit 17 according to the embodiment synchronously reproduces the master video and the slave video whose playback speed has been adjusted. The display unit 17 can apply a liquid crystal panel or an organic electro luminescence (EL: Electro Luminescence) panel. The programmable display 10 may have a touch panel in which the input unit 11 and the display unit 17 are integrally formed.

Next, the operation of the programmable display 10 will be described. FIG. 2 is a flowchart showing the procedure of the event setting process according to the embodiment. Here, a setting procedure of the start event V1 and the end event V2 used when the programmable display 10 adjusts the reproduction timing of the video recorded of the monitoring target will be described. Although the number of events set by the programmable display 10 may be any number, here, a processing procedure in the case where the programmable display 10 sets two events will be described.

When the event setting is started, when the designation information specifying the start event V1 is inputted by the operator, the input unit 11 of the programmable display 10 receives the designation information and sends it to the data processing unit 14. Thereby, in step S11, the data processing unit 14 sets the start event V1 and stores the set start event V1 in the storage unit 15.

When the designation information specifying the end event V2 is input by the operator, the input unit 11 receives the designation information and sends it to the data processing unit 14. Accordingly, in step S12, the data processing unit 14 sets the end event V2 and stores the set end event V2 in the storage unit 15.

The input unit 11 may receive first any information of information in which the start event V1 is designated and information in which the end event V2 is designated. The data processing unit 14 may set any event of the start event V1 and the end event V2 first, or may store any event in the storage unit 15 first.

FIG. 3 is a flowchart showing the processing procedure of the recording process according to the embodiment. Since the master video and the slave video have the same recording processing procedure, the slave video recording processing procedure will be described here.

When imaging of the monitoring target is started, the imaging device 40 sends the captured slave image to the imaging device connection IF 13 of the programmable display 10. Then, the imaging device connection IF 13 sends the slave video to the data processing unit 14. At this time, the data processing unit 14 reads the start event V1 stored in the storage unit 15. In other words, the data processing unit 14 reads the set start event V1 from the storage unit 15.

The data processing unit 14 records a video by storing the video sent from the imaging device 40 in the RAM 140. Also, when detecting the start event V1, the external device 30 sends the start event V1 to the programmable display 10. If the start event V1 is a device value transition, the external device 30 detects the start event V1 and sends it to the programmable display 10 when the device value set in the start event V1 transitions.

The data processing unit 14 detects the start event V1 when the same start event V1 as the set start event V1 is sent from the external device 30 during recording. Then, in step S21, the data processing unit 14 specifies the start frame position when the start event V1 is detected among the images captured by the imaging device 40 and stored in the RAM 140, and the specified start frame position Are stored in the RAM 140. As described above, the data processing unit 14 stores the start frame position of the frame acquired at the timing when the start event V1 is detected in the RAM 140. Then, the data processing unit 14 stores the start frame position at the time of detecting the start event V1 and the video in the storage unit 15 in association with each other.

When the end event V2 is a transition of the device value, the external device 30 detects the end event V2 and sends it to the programmable display 10 when the device value set in the end event V2 transitions.

The data processing unit 14 detects an end event V2 when an end event V2 identical to the set end event V2 is sent from the external device 30 during recording. Then, in step S22, the data processing unit 14 specifies the end frame position when the end event V2 is detected among the images captured by the imaging device 40 and stored in the RAM 140, and the specified end frame position Are stored in the RAM 140. Thus, the data processing unit 14 stores the end frame position of the frame acquired at the timing when the end event V2 is detected in the RAM 140. Then, the data processing unit 14 stores the end frame position when the end event V2 is detected and the video in the storage unit 15 in association with each other. Thus, the data processing unit 14 stores the video stored in the RAM 140, the start frame position, and the end frame position in the storage unit 15 in association with each other.

Here, the configuration of the video data of the video stored in the storage unit 15 will be described. FIG. 4 is a diagram showing a configuration of video data at the time of recording according to the embodiment. FIG. 4 shows the concept of the video data stored in the storage unit 15 in the flow of FIG. Although the case where the programmable display 10 stores one master video 101M and one slave video 101S in the storage unit 15 will be described here, the programmable display 10 includes two or more slave video 101S. May be stored in the storage unit 15. In addition, some of the slave video 101S and the master video 101M are not limited to the recorded video in the storage unit 15, but may be real-time video in which the process of the actual production line is captured. The real time video is a video before being stored in the storage unit 15.

The storage unit 15 stores a master video 101M in a state in which the start frame position 52, the end frame position 53, and the video data are associated with each other. In addition, the storage unit 15 stores the slave video 101S in a state in which the start frame position 62, the end frame position 63, and the video data are associated with each other.

The master video 101M includes a first frame F0 which is the first frame in the video, a final frame F1 which is the last frame in the video, and other frames. The top frame F0 is a frame at the recording start timing 51 which is the timing when the recording of the master video 101M is started. Further, the final frame F1 is a frame at the recording end timing 54 which is a timing when the recording of the master video 101M is ended.

Other frames in the master video 101M include a master start frame MS which is a frame of the start frame position 52 and a master end frame ME which is a frame of the end frame position 53. The master start frame MS is the first frame to be synchronously reproduced, and the master end frame ME is the last frame to be synchronously reproduced.

The slave video 101S also includes a first frame F10 which is the first frame in the video, a final frame F11 which is the last frame in the video, and other frames. The leading frame F10 is a frame at the recording start timing 61, which is the timing at which recording of the slave video is started. Further, the final frame F11 is a frame at the recording end timing 64 which is a timing when the recording of the slave video is ended.

The other frames in the slave video 101S include a slave start frame SS which is a frame of the start frame position 62 and a slave end frame SE which is a frame of the end frame position 63. The slave start frame SS is the first frame to be synchronously reproduced, and the slave end frame SE is the last frame to be synchronously reproduced.

Among the frames in the slave video 101S, the frame at the first timing at which the start event V1 occurs is the slave start frame SS, and the frame at the second timing at which the termination event V2 occurs in the slave video 101S is the slave termination frame It is SE.

In addition, the frame at the third timing at which the start event V1 occurs among the frames in the master video 101M is the master start frame MS, and the frame at the fourth timing at which the termination event V2 occurs in the master video 101M is the master It is an end frame ME.

Note that, when storing the master video 101M in the storage unit 15, the data processing unit 14 may add information indicating that it is the start frame position 52 to the master start frame MS. Further, when storing the master video 101M in the storage unit 15, the data processing unit 14 may add information indicating that the end frame position 53 is to the master end frame ME. Further, when storing the slave video 101S in the storage unit 15, the data processing unit 14 may add information indicating that it is the start frame position 62 to the slave start frame SS. In addition, when storing the slave video 101S in the storage unit 15, the data processing unit 14 may add information indicating that it is the end frame position 63 to the slave end frame SE.

As shown in FIG. 4, the slave start frame SS is acquired at a later timing than the master start frame MS, and the slave end frame SE is acquired at an earlier timing than the master end frame ME. This is because the operation to be monitored is faster when the slave video 101S is captured than when the master video 101M is captured. In other words, as the monitoring target executes the earlier operation, the time between the start event V1 and the end event V2 becomes shorter.

As described above, since the recording time defined by the start event V1 and the end event V2 is different between the slave video 101S and the master video 101M, if the slave video 101S and the master video 101M are reproduced simultaneously as they are, the monitoring target is It will be played at different operating speeds. The programmable display 10 according to the embodiment adjusts the playback speed of the slave video 101S when the monitoring target is played back at different operating speeds for the slave video 101S and the master video 101M, thereby the slave video 101S and the master video 101S. Synchronized reproduction with the image 101M. In other words, the programmable display 10 adjusts the playback speed of the slave video 101S such that the operating speed of the monitoring target in the master video 101M and the operating speed of the monitoring target in the slave video 101S become the same.

FIG. 5 is a flowchart showing a processing procedure of video reproduction processing according to the embodiment. When the designation information specifying the master video 101M is input by the operator, the input unit 11 of the programmable display 10 receives the designation information and sends it to the data processing unit 14. Accordingly, in step S31, the data processing unit 14 selects and reads the designated master video 101M from the storage unit 15. The master video 101M is a video serving as a reference of synchronous reproduction processing.

Then, in step S32, the data processing unit 14 adds the master start frame MS specified at the detection of the start event V1 and the master end frame ME specified at the detection of the end event V2 to the frames included in the master video 101M. Set At this time, the data processing unit 14 sets the master start frame MS based on the start frame position 52 in the storage unit 15 and sets the master end frame ME based on the end frame position 53 in the storage unit 15.

Further, when the designation information specifying the slave image 101S is input by the operator, the input unit 11 of the programmable display 10 receives the designation information and sends it to the data processing unit 14. Thereby, in step S33, the data processing unit 14 selects and reads out the designated slave video 101S from the storage unit 15. The slave video 101S is a video whose playback speed is adjusted.

Then, in step S34, the data processing unit 14 adds the slave start frame SS specified at the detection of the start event V1 and the slave end frame SE specified at the detection of the end event V2 to the frames included in the slave video 101S. Set At this time, the data processing unit 14 sets the slave start frame SS based on the start frame position 62 in the storage unit 15 and sets the slave end frame SE based on the end frame position 63 in the storage unit 15.

Thus, the programmable display 10 executes the process of step S32 after step S31, and executes the process of step S34 after step S33. The programmable display 10 may execute the processing of step S33 and step S34 prior to the processing of step S31 and step S32.

After steps S31 to S34, the data processing unit 14 reads the slave video 101S and the master video 101M from the storage unit 15. Then, in step S35, the data processing unit 14 adjusts the setting of the reproduction speed of the slave video 101S so that the reproduction time to the slave start frame SS is the same as the reproduction time to the master start frame MS.

Specifically, the data processing unit 14 sets the slave so that the reproduction time from the reproduction start of the slave image 101S to the slave start frame SS is the same as the reproduction time from the reproduction start to the master image 101M to the master start frame MS. The reproduction speed of frames from the start of reproduction of the video 101S to the slave start frame SS is adjusted. The frame at the start of reproduction of the slave video 101S is the top frame F10, and the frame at the start of reproduction of the master video 101M is the top frame F0. Therefore, the data processing unit 14 adjusts the setting of the reproduction speed of the slave video 101S so that the reproduction time from the first frame F10 to the slave start frame SS is the same as the reproduction time from the first frame F0 to the master start frame MS. Do.

Thereafter, in step S36, the data processing unit 14 adjusts the setting of the reproduction speed of the slave video 101S so that the reproduction time until the slave end frame SE becomes the same as the reproduction time until the master end frame ME.

Specifically, the data processing unit 14 makes the reproduction time from the slave start frame SS to the slave end frame SE of the slave video 101S the same as the reproduction time from the master start frame MS to the master end frame ME of the master video 101M. The setting of the reproduction speed of the frames from the slave start frame SS to the slave end frame SE of the slave video 101S is adjusted so that

Furthermore, in step S37, the data processing unit 14 adjusts the setting of the playback speed of the slave video 101S so that the playback time until the playback end of the slave video 101S is the same as the playback time until the playback end of the master video 101M. Do.

Specifically, the data processing unit 14 is a slave such that the reproduction time from the slave end frame SE of the slave video 101S to the reproduction end is the same as the reproduction time from the master end frame ME of the master video 101M to the reproduction end. The setting of the reproduction speed of the frame from the slave end frame SE of the image 101S to the reproduction end is adjusted. The frame at the end of reproduction of the slave video 101S is the final frame F11, and the frame at the end of reproduction of the master video 101M is the final frame F1. Therefore, the data processing unit 14 adjusts the setting of the reproduction speed of the slave video 101S so that the reproduction time from the slave end frame SE to the final frame F11 is the same as the reproduction time from the master end frame ME to the final frame F1. Do.

The data processing unit 14 sends the master image 101M and a slave image 102S described later, which is an image after adjusting the setting of the reproduction speed of the slave image 101S, to the display unit 17 through the display information generation unit 16. Thereby, the display unit 17 aligns and reproduces the master video 101M and the slave video 102S. Thus, the programmable display 10 synchronously reproduces the master video 101M and the slave video 102S every one tact.

FIG. 6 is a diagram showing the configuration of video data at the time of synchronous reproduction according to the embodiment. FIG. 6 illustrates the concept of video data in the case where the programmable display 10 performs synchronous reproduction using the video data described in FIG. 4.

The data processing unit 14 reads the master video 101M from the storage unit 15. In addition, the data processing unit 14 reads the slave video 101S, adjusts the reproduction speed of the slave video 101S by the processing described in FIG. 5, and thereby generates the slave video 102S.

The data processing unit 14 displays from the slave start frame SS to the slave end frame SE so that the slave start frame SS and the master start frame MS are simultaneously displayed, and the slave end frame SE and the master end frame ME are simultaneously displayed. Adjust the playback speed setting. The slave video after this adjustment is the slave video 102S.

Specifically, the data processing unit 14 sets the top frame F0 of the master video 101M and the top frame F10 of the slave video 102S as frames to be simultaneously displayed at timing 71. Timing 71 is timing to start synchronous reproduction of the master video 101M and the slave video 102S.

Further, the data processing unit 14 sets the master start frame MS and the slave start frame SS as frames to be simultaneously displayed at timing 72. The master start frame MS and the slave start frame SS at timing 72 are frames at the timing when the start event V1 occurs.

Further, the data processing unit 14 sets the master end frame ME and the slave end frame SE as frames to be simultaneously displayed at timing 73. The master end frame ME and the slave end frame SE at timing 73 are frames at the timing when the end event V2 occurs.

In addition, the data processing unit 14 sets the final frame F1 of the master video 101M and the final frame F11 of the slave video 102S as frames to be simultaneously displayed at timing 74. Timing 74 is timing to end synchronous reproduction of the master video 101M and the slave video 102S.

Then, from timing 71 to timing 72, the display unit 17 reproduces the master video 101M at a constant first frame rate, and reproduces the slave video 102S at a constant second frame rate. Specifically, the display unit 17 reproduces the head frame F0 to the master start frame MS at a fixed first frame rate, and simultaneously reproduces the head frame F10 to the slave start frame SS at a fixed second frame rate. .

In addition, from the timing 72 to the timing 73, the display unit 17 reproduces the master video 101M at a constant third frame rate, and reproduces the slave video 102S at a constant fourth frame rate. Specifically, the display unit 17 reproduces from the master start frame MS to the master end frame ME at a fixed third frame rate, and at the same time, from the slave start frame SS to the slave end frame SE at a fixed fourth frame rate Reproduce.

Also, from timing 73 to timing 74, the display unit 17 reproduces the master video 101M at a constant fifth frame rate, and reproduces the slave video 102S at a constant sixth frame rate. Specifically, the display unit 17 reproduces from the master end frame ME to the final frame F1 at a constant fifth frame rate, and simultaneously reproduces from the slave end frame SE to the final frame F11 at a constant sixth frame rate .

Here, the hardware configuration of the data processing unit 14 will be described. FIG. 7 is a diagram illustrating an example of the hardware configuration of the data processing unit according to the embodiment. The data processing unit 14 can be realized by the control circuit 300 shown in FIG. 7, that is, the processor 301 and the memory 302. An example of the processor 301 is a CPU (Central Processing Unit, central processing unit, processing unit, arithmetic unit, microprocessor, microcomputer, processor, also referred to as DSP) or a system LSI (Large Scale Integration). An example of the memory 302 is a RAM, a read only memory (ROM), or a flash memory.

The data processing unit 14 is realized by the processor 301 reading out and executing a program for executing the operation of the data processing unit 14 stored in the memory 302. Further, it can be said that this program causes a computer to execute the procedure or method of the data processing unit 14. The memory 302 is also used as a temporary memory when the processor 301 executes various processes.

Thus, the program executed by the processor 301 is a computer-executable computer-program product having a computer-readable non-transitory recording medium including a plurality of instructions for performing data processing. is there. The program executed by the processor 301 causes the computer to execute data processing of a plurality of instructions.

Also, the data processing unit 14 may be realized by dedicated hardware. Further, a part of the function of the data processing unit 14 may be realized by dedicated hardware and a part may be realized by software or firmware.

As described above, the programmable display 10 according to the embodiment reproduces the slave video 101S from the start event V1 to the end event V2 in accordance with the master video 101M from the start event V1 to the end event V2. Thereby, even when the monitoring targets operate at different speeds in a plurality of images, the programmable display 10 can align and compare the operation speeds of the monitoring targets at each process at the time of reproduction.

In addition, when the master video 101M is a video during normal operation of the monitoring target and the slave video 101S is a video during abnormal operation of the monitoring target, the programmable display 10 displays the video during normal operation and the abnormal operation. And can be reproduced synchronously. Also, the programmable display 10 may set an image when the monitoring target is operated at a speed different from that at the time of the normal operation, as the slave image 101S. Even in this case, the programmable display 10 can synchronously reproduce the master video 101M and the slave video 101S. Therefore, even if an irregular situation occurs in the process, the programmable display 10 can make the operator easily confirm the difference detection or the situation grasp.

In the embodiment, the programmable display 10 adjusts the reproduction speed of the slave video 101S based on the start event V1 and the end event V2. For this reason, the programmable display 10 can perform synchronous reproduction even when the frame rate at the time of recording differs between the master video 101M and the slave video 101S. Further, the programmable display 10 can synchronously reproduce the plurality of slave images 101S even when the frame rates of the plurality of slave images 101S are different. Further, the programmable display 10 can perform synchronous reproduction even when the imaging time differs between the master video 101M and the slave video 101S.

In the embodiment, the programmable display 10 can synchronously reproduce the slave video 101S captured by the plurality of imaging devices 40 on the master video 101M. In other words, the programmable display 10 can synchronously reproduce the master image 101M and the slave image 101S when the subject to be monitored is photographed from different angles. Therefore, even when the channel assigned for each slave video 101S is switched, the programmable display 10 can ensure the correlation between the slave video 101S and the master video 101M. In addition, since the programmable display 10 can compare the processes of the manufacturing line to be monitored from various angles, it is possible to facilitate detection of differences that are hard to notice with only one image in one direction.

Further, in the embodiment, since the programmable display 10 can manage a plurality of images based on the start event V1 and the end event V2, when extracting the reference frame that is the reference data to be subjected to the synchronous reproduction. Calculation amount and calculation time can be suppressed. The reference frames serving as reference data for performing synchronous reproduction are the master start frame MS, the master end frame ME, the slave start frame SS, and the slave end frame SE.

Further, in the embodiment, since the programmable display 10 can manage a plurality of videos based on the start event V1 and the end event V2, synchronized playback is easily performed even when a large amount of video data is acquired. Can be performed.

Further, in the embodiment, since the programmable display 10 performs synchronous reproduction without using a time stamp, it is not necessary to manage the master video 101M and the slave video 101S on the same time axis. Therefore, even if the frame rate of the master video 101M or the slave video 101S is deviated due to a failure of the imaging device 40, the programmable display 10 can execute synchronous reproduction. That is, the programmable display 10 can execute synchronous reproduction of the master video 101M and the slave video 101S acquired on different time axes.

Although the embodiment has described the case where the programmable display 10 synchronously reproduces a video based on two events of the start event V1 and the end event V2, the programmable display 10 has three or more events. The video may be synchronously played back on the basis of. When synchronously reproducing the video based on the three events, the programmable display 10 synchronously reproduces the front part of the video based on the first first event and the second second event. The latter part of the video is synchronously reproduced based on the second event of the eye and the third event of the third.

Also, when the slave video 101S is a real-time video, the programmable display 10 displays the second to third events based on the display mode of the slave video 101S and the master video 101M from the first event to the second event. , Adjust the playback speed of the slave video 101S. In this case, the data processing unit 14 calculates a reproduction method of the slave image 101S for synchronously reproducing the slave image 101S and the master image 101M from the first event to the second event, and the reproduction method as the calculation result is the slave image 101S. Apply to playback from the second event to the third event. That is, during the period from the first event to the second event, the programmable display 10 collects information on the playback speed, frame rate, and playback time of the master video 101M and the slave video 101S, and based on the collection result, 3. Adjust the playback speed of the slave video 101S from the third event to the fourth event.

When the slave video 101S or the master video 101M is a real-time video, the programmable display 10 may adjust the playback speed of the slave video 101S based on the frame rate and the recording time of the master video 101M and the slave video 101S. . In this case, the programmable display 10 acquires in advance the frame rate and the recording time of the master video 101M and the slave video 101S. Then, the programmable display 10 calculates the playback speed at the time of synchronized playback based on the acquired frame rate and recording time, and based on the calculation result, the slave video from the start event V1 to the end event V2 Adjust the 101S playback speed. The master video 101M may be a real-time video, and the master video 101M and the slave video 101S may be real-time video.

The programmable display 10 may also acquire one of the master video 101M and the slave video 101S from another device other than the imaging device 40. Also, the programmable display 10 may synchronously reproduce a plurality of sounds based on two events of the start event V1 and the end event V2. In this case, the programmable display 10 synchronously reproduces the master audio, which is the reference audio of the audio to be synchronously reproduced, and the slave audio, which is the audio of which the reproduction speed is adjusted, of the video to be synchronously reproduced. In this case, the programmable display 10 adjusts the playback speed of the slave audio so that the operating speed of the monitoring target in the master voice and the operating speed of the monitoring target in the slave voice become the same. The programmable display 10 synchronously reproduces the master voice and the slave voice based on information that the voice has, such as a voice waveform, instead of the frame.

Also, the programmable display 10 may synchronously reproduce measurement data other than video and audio. Also in this case, the programmable display 10 synchronously reproduces the measurement data of the master and the measurement data of the slave based on the information that the measurement data has, such as the waveform of the measurement data, instead of the frame.

Moreover, although the embodiment described the case where the programmable display 10 executes the processing of steps S31 to S37, the programmable display 10 may omit the processing of step S35 or step S37.

As described above, according to the embodiment, the playback speed of the slave video 101S is adjusted based on the start event V1 and the end event V2 so that the master video 101M and the slave video 101S are synchronously played back. It becomes possible to synchronously reproduce a plurality of videos for which moving image frames have been acquired under conditions.

The configuration shown in the above embodiment shows an example of the contents of the present invention, and can be combined with another known technique, and one of the configurations is possible within the scope of the present invention. Parts can be omitted or changed.

Reference Signs List 10 programmable display 11 input unit 12 external device connection IF 13 imaging device connection IF 14 data processing unit 15 storage unit 16 display information generation unit 17 display unit 30 external device 40 imaging device 51 51 61 recording start timing, 52, 62 start frame position, 53, 63 end frame position, 54, 64 recording end timing, 101M master video, 101S, 102S slave video, F0, F10 top frame, F1, F11 last frame, ME master End frame, MS master start frame, SE slave end frame, SS slave start frame.

Claims

Among a plurality of images to be synchronously reproduced, a master image which is a reference image, a slave image which is an image whose reproduction speed is adjusted among the plurality of images, a part of the master image and a part of the slave image A storage unit for storing a start event which is an event for starting synchronous reproduction of the frame, and an end event which is an event for ending the synchronous reproduction;
The image in the slave image between the start event which is a part of the slave image and the end event, and the master image between the start event and the end event which is a part of the master image A data processing unit for adjusting the setting of the playback speed of the slave video between the start event and the end event so that the video and the video are synchronously played back;
A display unit for synchronously reproducing the master video and the slave video after adjustment from the start event to the end event;
An indicator characterized by comprising.
The data processing unit
The moving image frame is a moving image frame at a second timing at which the end event occurs in the moving image frame in the slave image from the slave starting frame which is a moving image frame at a first timing in which the start event occurs among moving image frames in the slave image Among the frames in the master video, the playback time up to the slave end frame has the end event generated in the video frame in the master video from the master start frame which is the video frame of the third timing at which the start event occurs. Adjust the setting of the playback speed to be the same as the playback time up to the master end frame, which is a video frame of the fourth timing,
The display according to claim 1, characterized in that:
The data processing unit
Adjusting the setting of the reproduction speed from the slave start frame to the slave end frame every tact of the manufacturing process;
The display according to claim 2, characterized in that:
The one tact process is a process included in the range of one sequence process performed by the programmable logic controller.
The display according to claim 3, characterized in that:
The one tact is a process from detection of the start event by a sensor to detection of the end event by the sensor.
The display according to claim 3, characterized in that:
The data processing unit
Detecting the first to fourth timings based on information sent from an external device;
The slave start frame is set based on the first timing, the slave end frame is set based on the second timing, the master start frame is set based on the third timing, and the fourth timing is set. Setting the master end frame based on
The display according to claim 2, characterized in that:
The master image and the slave image are images having different frame rates or recording times.
The display according to any one of claims 1 to 6, characterized in that:
The slave video is a video captured simultaneously with the master video,
The display according to any one of claims 1 to 7, characterized in that:
The slave video is a video captured at a time different from the master video.
The display according to any one of claims 1 to 7, characterized in that:
The slave image is an image captured at an angle different from that of the master image.
The display according to any one of claims 1 to 9, characterized in that.
The master image or the slave image is a real time image,
The display according to any one of claims 1 to 10, characterized in that:
The display unit displays the master video and the slave video while switching.
The display according to any one of claims 1 to 10, characterized in that:
Obtaining a master video as a reference video among the plurality of videos to be synchronously reproduced and a slave video as the video whose playback speed is adjusted among the plurality of videos;
A setting step of setting a start event which is an event for starting synchronous reproduction of a part of the master video and a part of the slave video, and an end event which is an event for ending the synchronous reproduction;
From the start event, the video in the slave video between the start event and the end event and the video in the master video between the start event and the end event are synchronously reproduced from the start event An adjusting step of adjusting the setting of the reproduction speed of the slave video until the end event;
A reproduction step of synchronously reproducing the master image and the adjusted slave image from the start event to the end event;
A display method characterized in that