TWI454141B - Data synchronous regeneration device, data synchronous regeneration method and data synchronization control program - Google Patents

Description

Data synchronous regeneration device, data synchronous regeneration method and data synchronization control program

The present invention relates to a data synchronization reproducing device and data synchronization for synchronizing and displaying image data of a plurality of image frames including a camera to be monitored by a camera, and process data including a plurality of numerical data acquired from a monitoring target in time series. Regeneration method and data synchronization control program.

As a data synchronous reproduction device, for example, when an abnormality occurs in a factory, the factory monitors the numerical data and the image frame of the factory when the abnormality occurs, and stores the numerical data and the image frame in the memory device as Another file, and when the resume information screen is developed to catch up with the abnormality, the numerical data and the image frame at the time of the abnormality are simultaneously displayed, and the prior art for fine factory monitoring has been proposed (for example, refer to the patent literature). 1).

(previous technical literature)

(Patent Literature)

Patent Document 1: Japanese Laid-Open Patent Publication No. Hei 5-34181.

However, in the data synchronous reproducing apparatus of the above-mentioned Patent Document 1, there is a problem that the factory numerical data and the image frame are displayed and stored only when an abnormality occurs in the factory, and thus it is impossible to confirm that the abnormality occurs. Numerical data and image frames in normal operation.

In particular, in the data synchronous reproducing apparatus of the above-mentioned Patent Document 1, there is a problem in that, in the case where the numerical data and the image frame at the time of confirming the abnormality are to be confirmed, the recording and addition are performed based on the respective numerical data. The information is synchronously reproduced on the day of each image. Therefore, when the monitoring object is added, or the numerical data and image frame during normal operation are also stored, the number of files and the file memory capacity are increased. When synchronously reproducing image frames and numerical data, it becomes necessary to have a very large capacity memory device.

Further, in the data synchronous reproduction device of Patent Document 1, there is a problem that synchronization is performed by adding time-of-day information based on the logarithmic data and the image frame, so that MPEG-4 cannot be used for image storage. Image frame compression and decoding technology; and because the common image data reproduction program such as the Quick Time (registered trademark) player cannot be used during image reproduction, the development cost is increased.

The following problems are solved, that is, when the image frame and the numerical data are synchronously reproduced, the reproduction is performed while checking the data of each numerical data and the information attached to the time of each image frame, so that high-speed synchronous reproduction or prompting is performed. It will be very difficult.

The present invention has been made in view of the above problems, and an object of the present invention is to provide a data synchronization frame and a data synchronous reproduction device which can be used for image frame data included in image data and synchronously reproduced in process data. Method and data synchronization control program.

In order to achieve the above object, a first feature of the data synchronous reproducing apparatus of the present invention includes: an image data storage unit that captures a plurality of image frames obtained by monitoring a camera, and adds a frame number to each image frame, and The frame period of the image frame is memorized together as image data; the process data storage unit is configured to add a record number to each piece of data from the plurality of pieces of data acquired by the monitoring object in time series, and together with the sampling interval of the foregoing data. The memory is used as the process data; the image data reproduction unit outputs and reproduces the image data stored in the image data storage unit to the monitor, and outputs the frame number of the reproduced image frame; and the data synchronous reproduction control unit The frame number outputted from the image data reproducing unit is converted into the record number of the data based on the frame period of the image frame and the sampling interval of the data, and the data having the converted record number is Regenerated from the aforementioned monitor.

Furthermore, in the second aspect of the data synchronous reproduction device of the present invention, the data synchronous reproduction control unit performs an operation of designating the data in a specific time series in the process data displayed on the monitor. And converting the record number of the specified data to the frame number according to the sampling interval of the data and the frame period of the image frame, and outputting the converted frame number to the image material regeneration The portion of the image data having the converted frame number is reproduced.

Furthermore, the third feature of the data synchronous reproducing apparatus of the present invention has: a synchronous image frame selecting unit for specifying, in the process data displayed on the monitor, externally inputting data to be memorized as another file. In the range, according to the sampling interval of the foregoing data and the frame period of the image frame, the record number of the data of the specified range is converted into the frame number of the image frame, and the foregoing data of the specified range is selected. The image frame of the synchronization range and the synchronization storage control unit are configured to store the data of the range specified by the external input and the image frame of the range selected by the synchronization image frame selection unit as the same file.

Furthermore, a fourth feature of the data synchronous playback device of the present invention includes a screen arrangement setting unit for setting, on the monitor, an image data reproduction area for displaying the image data by the image data reproduction unit according to an external input. And displaying the process data reproduction area of the process data by the data synchronization control unit, and simultaneously setting the number and size of the image data to be reproduced within the predetermined condition in the image data reproduction area.

Furthermore, the first feature of the data synchronization reproducing method of the present invention has a plurality of image frames obtained by capturing a monitoring target by a camera, and adding a frame number to each image frame, and a frame period of the image frame. Memorizing together as a video data; adding a record number to each data from the plurality of data acquired by the monitoring object in time series, and storing the step as a process data together with the sampling interval of the foregoing data; When reproducing the monitor and outputting the frame number of the reproduced image frame, the frame number outputted is converted into the record of the data according to the frame period of the image frame and the sampling interval of the data. The step of numbering and reproducing the aforementioned material having the converted record number on the aforementioned monitor.

Furthermore, a second feature of the data synchronous reproducing method of the present invention is: when the data is specified in a specific time series in the process data displayed on the monitor, the predetermined recording number of the data is specified according to the foregoing The sampling interval of the data and the frame period of the image frame are converted into the frame number, and the image data having the converted frame number is reproduced.

Furthermore, the third feature of the data synchronous regeneration method of the present invention is characterized in that, in the process data displayed on the monitor, when the data range to be memorized as another file is specified, the foregoing data of the specified range is specified. The recording number, according to the sampling interval of the foregoing data and the frame period of the image frame, the steps of converting the frame number of the image frame and selecting the image frame of the range synchronized with the data of the specified range And the step of storing the aforementioned information in the specified range and the image frame of the selected range as the same file.

Furthermore, a fourth feature of the data synchronous reproducing method of the present invention is characterized in that: according to an external input, the image data reproducing area for displaying the image data by the image data reproducing unit is set on the monitor, and The data synchronization reproduction control unit displays the process data reproduction area of the process data, and sets the number and size of the image data to be reproduced within the predetermined condition in the image data reproduction area.

Furthermore, the first feature of the data synchronization control program of the present invention is executed by a computer: a plurality of image frames obtained by capturing a monitoring object by a camera, and a frame number is added to each image frame, and the frame of the image frame is arranged. The cycle is a step of memorizing the image data; the plurality of materials obtained from the monitoring object according to the time series are added with a record number for each data, and the step of storing the data as a process data together with the sampling interval of the foregoing data; And when the monitor is reproduced and the frame number of the reproduced image frame is output, the frame number outputted is converted into the record of the data according to the frame period of the image frame and the sampling interval of the data. The step of numbering and reproducing the aforementioned material having the converted record number on the aforementioned monitor.

Furthermore, the second feature of the data synchronization control program of the present invention is re-executed: in the foregoing process data displayed on the monitor, when the foregoing data is specified in a specific time sequence, the record number of the specified data is According to the sampling interval of the above-mentioned data and the frame period of the image frame, the frame number is converted into the frame number, and the image data having the converted frame number is reproduced.

Furthermore, the third feature of the data synchronization control program of the present invention is repeated: in the process data displayed on the monitor, when the data range to be memorized as another file is specified, the sampling interval according to the foregoing data is as described above. a frame period of the image frame, a step of converting the record number of the data of the specified range into the frame number of the image frame, and selecting an image frame of a range synchronized with the data of the specified range; The above-mentioned data of the specified range and the above-mentioned image frame of the selected range are recorded as the same file.

Furthermore, the fourth feature of the data synchronization control program of the present invention is repeatedly executed: according to an external input, setting, on the monitor, an image data reproduction area for displaying the image data by the image data reproduction unit; The data synchronization reproduction control unit displays the process data reproduction area of the process data, and sets the number and size of the image data to be reproduced within the predetermined condition in the image data reproduction area.

The data synchronization reproduction device, the data synchronization reproduction method, and the data synchronization control program according to the present invention are not based on time information, but are based on the frame number of the image frame included in the image data and the record of the data included in the process data. The number is used to synchronize the image frame with the data, so there is no need to attach time information to the data and image frames.

Therefore, the number of files and the memory capacity of the file can be reduced, and the memory and the synchronous reproduction can be performed for a longer period of time than the conventional memory device having the same memory capacity.

In addition, since the image frame compression and decoding technology such as MPEG-4 can be used for image storage, a general image data reproduction program such as a Quick Time (registered trademark) player can be used for image reproduction. Reduce development costs.

Moreover, it is possible to simultaneously reproduce data and image frames at high speed without using the time information, or to prompt.

(First embodiment)

Hereinafter, a first embodiment of the data synchronous reproduction device, the data synchronous reproduction method, and the data synchronization control program of the present invention will be described.

<Composition description>

Fig. 1 is a view showing the construction of an online data management system including the data reproducing apparatus of the present invention.

The online data management system is a hot rolling mill system that processes steel, stainless steel, and aluminum (steel) which are rolled and heated by a rolling mill to a hot rolled coil having a thickness of several millimeters (mm) to several tens of mm. Or in order to finish the processing of the disk manufactured by the hot rolling mill or the cold rolling mill into the final product, and perform pickling, annealing, plating, painting, cutting, etc., to process the image data obtained by each monitoring object in the production line factory system, and A system for synchronous display and management of the process data acquired from each monitoring object on a monitor.

In Fig. 1, the online data management system is composed of: the data synchronous reproducing device of the present invention, a plurality of industrial television (ITV) cameras 21 to 2 m (m is a natural number), a camera switcher 3, and a digital position. The image conversion unit 4, a programmable logic controller (PLC), and the like control devices 51 to 5n (n is a natural number) are connected, and a plurality of ITV cameras 21 to 2m are used to take a roll or press to drive these. A system for monitoring the image data obtained by the operation state and the control state of each of the monitoring objects 61 to 6p (p is a natural number) such as a motor, a piston, and the like, and the process data acquired by the control devices 51 to 5n.

Here, for example, the data synchronization reproduction device 1 and the digital image conversion unit 4 are connected by a LAN (Local Area Network) 7 such as an Ethernet (registered trademark), and on the other hand, data synchronization is reproduced. The device 1 and the control devices 51 to 5n are connected by another control network 8 which is separate from the LAN 7. Further, in the case where the control devices 51 to 5n can be connected to the LAN 7, the control network 8 can be replaced with the LAN 7.

Further, the data synchronous reproduction device 1 of the present invention is constituted by a computer device such as a monitor (display) 12, a keyboard 13, a mouse 14, and the like connected to the device body 11. Further, the data synchronous reproduction device 1 is used by the digital image conversion unit 4 to convert image data of a plurality of ITV cameras 21 to 2m converted from analog video signals into digital video signals, and plural images obtained from the control devices 51 to 5n. The process data of the numerical data is input therein, and is usually displayed on the monitor 12 or is subjected to a synchronous display processor. The configuration or operation of the data synchronizing apparatus 1 of the present invention, the video data, the composition of the process data, and the like are described later.

The ITV camera 21 to 2m is an analog video signal such as a PAL (Phase Alternation by Line) or an NTSC (National Television System Committee), in which a plurality of imaging objects 61 to 6p are imaged. It can be a fixed camera, or a camera that can perform remote operations such as horizontal movement, tilt, zoom, etc. by transmitting control signals. In addition, in the ITV cameras 21 to 2m, the microphone may or may not be provided. In addition, the ITV cameras 21 to 2m may be provided for each of the objects to be monitored 61 to 6p, or a plurality of units may be provided.

The camera switcher 3 outputs a video signal of a predetermined number (for example, four in the first embodiment) displayed on the monitor 12 from a plurality of ITV cameras 21 to 2 m video signals. In addition, in the case where the number of video signals displayed on the monitor 12 is not limited, or in the case where the number of ITV cameras 21 to 2 m is less than the predetermined number, the digital video conversion unit 4 or the data synchronous reproduction device 1 is provided. When the function is switched, the camera switch 3 is of course omitted.

The digital video conversion unit 4 compresses and encodes an analog video signal of an NTSC or PAL format outputted from a plurality of ITV cameras 21 to 2m into digital video data such as MPEG-4, and outputs it to the data synchronous reproduction device 1. Further, when the video signals obtained by capturing the inside of the plurality of ITV cameras 21 to 2m are A/D-converted, and the image data of the digital form can be output, the digital image conversion unit 4 can be omitted.

Each of the control devices 51 to 5n controls the actor of each of the monitoring objects 61 to 6p such as a motor, a heater, and a hydraulic press by a program, and each of the objects to be monitored via the respective I/O (input/output) units 511 to 5n1. 61 to 6p, including the process data of the numerical data obtained by the time series detection of the predetermined sampling interval described later.

Fig. 2 is a block diagram showing a configuration example of the data synchronous reproduction device 1 of the present invention.

As described above, the data synchronous reproduction device 1 is constituted by a computer device in which the device body 11 is connected to a monitor (display) 12, a keyboard 13, a mouse 14, and the like.

As shown in FIG. 2, the apparatus main body 11 includes a memory unit 113 such as a CPU 111, a memory 112, an HDD, or a large-capacity memory, an external device interface (hereinafter abbreviated as I/F) 114, and a control network I/. The F portion 115, the LAN I/F portion 116, and the like are connected via the internal bus bar 117.

The CPU 111 functions as a video data reproducing unit of the present invention by executing a video data reproducing program stored in various applications stored in the program storage unit 113a of the storage unit 113, and simultaneously executes a data synchronization control program. The function of the data synchronization reproduction control unit, the synchronized image frame selection unit, the synchronous storage control unit, and the screen arrangement setting unit of the present invention is exhibited.

The memory 112 is used as a calculation area for temporarily storing or unfolding, etc., when the CPU 111 executes various applications.

The memory unit 113 has a program storage unit 113a and a data storage unit 113b. The program memory unit 113a stores an OS (operating system) and a video data reproducing program that causes the CPU 111 to function as the video data reproducing unit of the present invention, and causes the CPU 111 to function as the data synchronous reproduction control unit of the present invention. An application such as an image frame selection unit, a synchronous storage control unit, and a data synchronization control program for the screen layout setting unit function. Further, in the first embodiment, as the video data reproducing program, a video data reproducing program for the device-specific planning or a general-purpose video data reproducing program such as a Quick Time (registered trademark) player can be used.

The data storage unit 113b is a video data storage unit of the present invention, and stores image data of a plurality of image frames including a plurality of ITV cameras 21 to 2m for imaging objects 61 to 6p and performing A/D conversion, and simultaneously As the process data storage unit of the present invention, the process data including the plurality of numerical data obtained by the control devices 51 to 5n from the plurality of monitoring objects 61 to 6p in time series is stored.

Further, in the first embodiment, the program storage unit 113a and the data storage unit 113b are provided in one memory unit 113, but the program storage unit 113a and the data storage unit 113b may of course be configured as respective memory units. .

The external device I/F unit 114 is an interface for the monitor 12 and an external device such as the keyboard 13 and the mouse 14 to be connected to the device body 11.

The control network I/F unit 115 is an interface for the control network 19 to be connected to the device body 11.

The LAN I/F unit 116 is an interface for connecting the LAN 7 to the device body 11.

3A to 3C are diagrams showing an example of the file format of the image data and the process data stored in the data storage unit 113b of the data synchronous reproduction device 1, and the data structure of the image data and the process data, respectively.

As shown in FIG. 3A, in the data storage unit 113b of the memory unit 113 of the data synchronous playback device 1, an image in which a plurality of ITV cameras 21 to 2m capture a plurality of image data of a plurality of monitoring objects 61 to 6p is used. The data D1 and the process data D2 including the plurality of numerical data obtained by the control devices 51 to 5n from the plurality of monitoring objects 61 to 6p in time series are stored in two blocks S1 and S2 in one data file Df. Here, the image data D1 and the process data D2 may of course be stored in another block of another file.

Fig. 3B is a data structure diagram showing the image data D1 stored in the data storage unit 113b.

As shown in FIG. 3B, the image data D1 is a digital image data obtained by converting the analog video signal output from the plurality of ITV cameras 21 to 2m into the MPEG-4 encoding method by the digital image conversion unit 4, and is an image. The header information D11 is composed of a plurality of image frames D12 of the frame numbers #1 to #q (q is a natural number) after the target objects 61 to 6p are captured by the respective ITV cameras 21 to 2m. Further, for convenience of explanation, although only one image data D1 is shown, actually, a plurality of images have image data D1 every 21 to 2 m per ITV camera.

In the case of the first embodiment, as shown in FIG. 3B, the image header information D11 has at least a frame period Tf of each unit frame time belonging to the image frame D12 constituting the image data D1, and The total number of frames q of the plurality of image frames D12 constituting the image data D1. In addition, the image header information D11 may of course be set to include camera identification information for the ITV cameras 21 to 2m for identifying the output image data D1, or for including the image data D1 from the same ITV camera 21 to 2m. The way in which the daytime information of the image data D1 is different. Furthermore, of course, the complex information can be set to include link information for linking the image header information D11 with the corresponding process data D2.

Here, in the first embodiment, the frame period Tf of the image frame D12 is set to, for example, one frame = 1/30 second. In addition, 1/30 second is an example. Of course, the frame period Tf of the image frame D12 is 1/60 second and 1/50 second.

The image frame D12 is a data compression-encoded image of the analog video of the analog video captured by the ITV camera 21 to 2 m by the digital image compression unit MPEG-4 or the like; of course, other compression coding forms, For example, H.264 encoding method, or Motino-JPEG without inter-frame prediction. In short, as long as it is transmitted by the digital image data transmitted by LAN7. In addition, it is of course possible to provide time information such as decoding time stamp (DTS) or reproduction time information (PTS) for general decoding and reproduction for a plurality of video frames D12. However, in the first embodiment, the synchronization information processing with the process data does not use the time information such as the decoding time stamp (DTS) and the reproduction time information (PTS).

As described above, in the first embodiment, the video material D1 does not have the time information, but even if it has the time information, it is not used in the synchronous reproduction with the process data D2. Therefore, in the first embodiment, at the time of the reproduction processing of the image data D1 and the process data D2, the sampling start time Ts1 stored in the process data header information D21 of the process data D2 corresponding to the image data D1 is regarded as The start time of the image data D1 is the reproduction start information of the frame number #1 image frame D12. Then, the frame reproduction start time of the frame number #2 to the image frame D12 after the image frame D12 of the frame number #1 is based on the reproduction start time of the image frame D12 which is the frame number #1. The sampling start time Ts1 to be used and the frame number # of each video frame D12 are calculated as will be described later. Therefore, in the first embodiment, the synchronization reproduction processing with the process data D2 does not require the time information of each of the image frames D12, and is not used even if there is time information or the like (reference). Thereby, since the time information of each of the image frames D12 is not used, only the frame number is reproduced, so that the high-speed reproduction and presentation can be performed.

The 3C chart is a data structure diagram showing the process data D2 stored in the data storage unit 113b.

The process data D2 is Binary data composed of the numerical data D22 obtained by the control devices 51 to 5n from the plurality of monitoring objects 61 to 6p in time series for each predetermined sampling interval Ts2, as shown in FIG. 3C. The process data header information D21 and the plurality of numerical data D22 of the record numbers #1 to r (r is a natural number) acquired from the objects to be monitored 61 to 5n by the plurality of station control devices 51 to 5n.

In the case of the first embodiment, the process data header information D21 stores, at a minimum, a sampling start time Ts1 belonging to the start time of the process data D2 acquisition, and a sampling interval Ts2 and an interval belonging to the interval at which the numerical data D22 is acquired. The total number of records r of the plurality of numerical data D22 obtained by the objects to be monitored 61 to 6p at the predetermined sampling interval Ts2. Here, in the first embodiment, the sampling interval Ts2 of the process data header information D21 is, for example, 1 millisecond (mSec), which is an example, and may of course be 1 msec.

Further, in the numerical data D22 obtained from the objects to be monitored 61 to 6p in time series, the numerical data D22 of the number #1 is recorded from the process data D2 in the manner of recording numbers #1 to #r, respectively, based on the sampling interval Ts2. Start with the record number. Then, each numerical data D22 is composed of one or a plurality of items of item numbers #1 to #s (s is a natural number) as shown in FIG. 3C. For example, item #1 is the temperature sensing value and item #2 is the pressure sensing value. Further, in the present invention, since the item is not related to the main part of the invention, it may be omitted.

Thus, the plurality of image frames D12 constituting the image data D1 and the plurality of numerical data D22 constituting the process data D2 are not synchronized at the time of recording. Furthermore, 1/30 second of the image data frame period Tf is inconsistent with 1 m second of the numerical data sampling interval Ts2. Therefore, the data synchronous playback device 1 of the first embodiment does not use the time information of the video data D1 such as the decoding time stamp (DTS) and the reproduction time information (PTS), and the time information when the data of the process data D2 is acquired. Therefore, in order to perform synchronous reproduction processing for the image frame D12 constituting the image data D1 and the numerical data D22 constituting the process data D2, it is necessary to perform synchronization processing at the time of synchronous reproduction.

Therefore, in the data synchronous reproduction device 1 of the first embodiment, the frame number and the record number are used without using or referring to the time information, and the image frame and the numerical data are used for the synchronous reproduction processing.

<Action Description>

Next, the material synchronization display processing in the data synchronous playback device 1 of the first embodiment will be described with reference to a flowchart.

(Describes the action of synchronous reproduction from image data D1 to process data D2)

First, the process data synchronous reproduction operation based on the image data in the material reproduction device 1 of the first embodiment will be described with reference to a flowchart.

Fig. 4 is a flow chart showing the procedure of data synchronization display processing by the material synchronous reproduction device 1 of the first embodiment.

First, the user operates the keyboard 13 or the mouse 14 of the data synchronization reproducing device 1 to click (select) the synchronous reproduction button (not shown), and synchronizes the input of the image data D1 with the process data D2 to the device body 11. In the reproduction instruction (S400, YES), the CPU 111 starts the image data reproduction program and the data synchronization control program stored in the program storage unit 113a, and starts the synchronous reproduction processing.

In other words, the CPU 111 of the apparatus main body 11 first functions as the image data reproducing unit by executing the image data reproducing program, and reads the image header information D11 and the process data D2 of the image data D1 stored in the data storage unit 113b. The process data header information D21 is temporarily stored in the memory 112, and the frame period Tf included in the image header information D11 and the sampling interval Ts2 included in the process material header information D21 are acquired (S410). Here, as described above, the CPU 111 acquires, for example, 1/30 second as the frame period Tf and acquires, for example, 1 m second as the sampling interval Ts2.

Further, before the CPU 111 performs the processing in S400, it is of course possible to obtain the frame period Tf and the sampling interval Ts2 in advance by the processing of S410. Furthermore, in the case where the frame period Tf included in the image header information D11 is known, the CPU 111 does not acquire the frame period Tf included in the image header information D11, but only acquires the header information D21 included in the process data. The sampling interval is Ts2.

Next, the CPU 111 functions as the video data reproducing unit of the present invention by the execution of the video data reproducing program, and uses a timer (not shown) of the OS to take 1/ of the frame period Tf belonging to the video data D1. A timer event is generated at a 30 second interval (S420).

When the timer 111 generates a timer event signal (S430, YES) at intervals of Tf1/30 seconds belonging to the frame period, the CPU 111 functions as the video data reproducing unit of the present invention, and reads and stores it in the data storage unit 113b. When the image frame D12 is encoded by the MPEG-4 format or the like, the image frame D12 is decoded by the MPEG-4 format or the like, and is reproduced on the monitor 12, and is decoded by the image frame D12. At the same time, the frame number of the reproduced image frame D12 is output (S440). At this time, when the user specifies the image frame in which the reproduction is started by using the frame number or the like, the reproduction is started from the designated image frame. Further, the designation of the image frame D12 is not limited to the frame number or the like, and may be specified by the cursor on the display screen displayed on the monitor 12 by the keyboard 13 or the mouse 14. When the image frame D12 is not specified, the image frame D12 is started from the image data D1, that is, the image frame D12 of the frame number #1 is started to be reproduced. Further, at this stage, the image data D1 is reproduced asynchronously with the process data D2.

At the same time, the CPU 111 functions as the data synchronization reproduction control unit of the present invention by executing the data synchronization control program according to the present invention. First, the frame number of the currently reproduced video frame D12 and the frame period Tf are obtained. (S450). Here, in the S450 processing, similarly to the processing of S410, the CPU 111 uses the known frame period Tf in the case where the frame period Tf included in the image header information D11 is known.

Next, the CPU 111 calculates the "relative time from the start image frame" by the following formula (1) based on the frame number of the image frame D12 currently being reproduced and the frame period Tf (S460). As the data synchronous regeneration control unit of the present invention,

"Relative time from the starting image frame" = frame number × frame period Tf...(1)

Then, the CPU 111 obtains the "relative time from the start image frame" from the above equation (1), by the process data D2 temporarily stored in the memory 112 or the like as shown in the following equation (2). The sampling interval Ts2 is divided, and the record number (S470) of the numerical data D22 synchronized with the currently displayed image frame D12 is calculated as the data synchronous reproduction control unit of the present invention.

"Record number of the data to be synchronized" = "relative time from the starting image frame" ÷ sampling interval Ts2...(2)

Then, the CPU 111 selects and displays the numerical data D22 corresponding to the record number from the process data D2 based on the "record number of the numerical data to be synchronized" obtained by the above formula (2) (S480). Thereby, the CPU 111 can synchronize and reproduce the synchronized numerical data D22 temporally synchronized with the currently displayed image frame D12 via the frame number of the image frame D12 and the record number of the numerical data D22 without referring to the time information. . At this time, for example, when the process data D2 is displayed in a graphical form or the like, the CPU 111 converts the numerical data D22 of the record number into a graphic form or the like, and updates it by including the value displayed in the range of the graphic display. The display of the graphic. Further, in the case where the numerical data D22 is composed of a plurality of items #1 to #s, a plurality of items #1 to #s are reflected in the figure.

Then, the CPU 111 ends the reproduction by the user inputting the synchronous reproduction suspension instruction or the final image frame D12 of the image data D1 currently being reproduced (S490, NO), and each time belongs to the frame period Tf. The timer event signal is generated at intervals of 1/30 second (Yes in S430), and the synchronous reproduction processing of the image frame D12 and the numerical data D22 corresponding to the record number is performed by the processing of the above S440 to S480.

Therefore, in the data synchronous playback device 1 of the first embodiment, the frame number of the video data D1 is associated with the record number of the numerical data D22 without referring to the time information, and the image frame constituting the image data D1 is made. D12, and is reproduced in synchronization with the numerical data D22 constituting the process data D2.

Then, when the user inputs the synchronous reproduction suspension instruction or the reproduction of the video image D1 final video frame D12 currently being reproduced (S490, YES), the CPU 111 ends the synchronous reproduction processing.

(Specific example of synchronous reproduction from image data D1 to process data D2)

Next, a specific example of synchronous reproduction from the image data D1 to the process data D2 will be described.

Fig. 5 is a view showing a specific example of the case where the synchronous reproduction processing is performed from the image data D1 to the process data D2. In addition, FIG. 5 omits the image header information D11 of the image data D1 and the process data header information D21 of the process data D2.

As shown in Fig. 5, in the data synchronous reproduction device 1 of the first embodiment, for example, the image frame of the frame number #3 of the third frame is counted from the start image frame in the image data D1. In the case of the first embodiment, since the frame period Tf is 1/30 second, the CPU 111 calculates "relative time from the start image frame D12 of the frame number #1" by the above formula (1). 3 × 1/30 = 0.1 second. Then, since the CPU 111 calculates the sampling interval Ts2 to be 1 msec, the calculation "the recording number from the starting value data" is 0.1 second/1 msec = 100, and the numerical data D22 of the recording number #100 is reproduced.

Therefore, according to the data synchronous reproduction device 1 of the first embodiment, the image frame D12 constituting the image data D1 and the numerical data D22 constituting the process data D2 are not synchronized during recording, and the image frame D12 is not shown. The frame period Tf and the sampling time Ts2 of the numerical data D22 do not use the time information of the image frame D12 and the numerical data D22, but use the frame number of the image frame D12 and the record number of the numerical data D22, that is, It is possible to synchronously reproduce the image frame D12 and the numerical data D22.

As a result, according to the data synchronous reproduction device 1 of the first embodiment, in the synchronous reproduction processing of the image data D1 and the process data D2, since it is not necessary to refer to the time information of both materials, it is not necessary to analyze and search for the image data D1. The syntax (construction) or header information of the process material D2 can perform synchronous reproduction processing at high speed.

Further, in the description of the first embodiment, it has been described that the timer event is generated every time in the frame period Tf by the OS timer, and the equations (1) and (2) are calculated from the start. The relative time of the image frame, and the record number of the numerical data D22 to be synchronously reproduced is calculated one by one according to the "relative time from the starting image frame", but the present invention is not limited thereto; 1) and (2), in the case where the record number of the numerical data D22 to be synchronously reproduced is calculated at a time, after each occurrence of the timer event by the OS timer in each frame period Tf, the synchronous reproduction should be performed simultaneously. The record number of the numerical data D22 may of course be incremented by +1. According to this, as long as the recording is performed by the processing of S450 to S470 to calculate the recording number of the numerical data D22 to be synchronously reproduced initially, it is not necessary to perform the processing of S450 to S470, and the initial calculation should be synchronously reproduced. The recording number of the numerical data D22 is incremented by +1, so that the synchronous reproduction processing of the image frame D12 constituting the image data D1 and the numerical data D22 constituting the process data D2 can be made possible, and the processing can be performed at a higher speed and higher speed. Negative and lighter synchronized regeneration processing becomes possible.

(Describe the synchronous regeneration operation from the process data D2 to the image data D1)

Next, the synchronous reproduction processing from the process material D2 to the video material D1 in the data synchronous playback device 1 according to the first embodiment will be described with reference to a flowchart.

Fig. 6 is a flow chart showing the sequence of data synchronization display processing by the data synchronous reproducing apparatus 1 of the first embodiment.

The user displays the numerical data D22 in the process data D2 by specifying a scroll bar or a cursor or the like in the display area of the process data D2 displayed on the monitor 12, for example, by the mouse 14 or the keyboard 13 or the like. When the numerical data D22 and the image frame D12 are synchronized and instructed to input the apparatus main body 11 (S600, YES), the CPU 111 functions as a data synchronous reproduction control unit by executing a data synchronization control program; The record number of the specified numerical data D22 and the sampling interval Ts2 of the slave process header information D21 (S610).

At the same time, the CPU 111 functions as a data synchronization reproduction control unit, and acquires the frame period Tf from the image header information D11 of the image data D1 (S620). Further, in the case where the sampling unit Ts2 and the image frame period Tf are stored, and the CPU 111 is known, it is of course possible to omit the acquisition processing.

Next, the CPU 111 functions as a data synchronous reproduction control unit, and calculates the process data D2 as shown in the following equation (3) based on the sampling interval Ts2 and the record number of the numerical data D22 specified by a scroll bar or a cursor. "relative time from the starting value data" (S630).

"Relative time from the starting value data" = current record number × sampling interval Ts2...(3)

Next, the CPU 111 divides the "relative time from the starting value data" calculated by the equation (3) by the frame period Tf of each frame time of the image frame D12 as shown in the following formula (4). The "frame number of the image frame to be synchronized" which is synchronized with the current numerical data is calculated (S640).

"The frame number of the image frame should be synchronized" = from the start relative time ÷ frame period Tf...(4)

Then, the CPU 111 functions as a video data reproducing unit, and reads the image frame D12 corresponding to the frame number from the video data D1 stored in the data storage unit 113b based on the frame number calculated by the above formula (4). The regeneration is performed (S640). As a result, according to the first embodiment, it is possible to specify the synchronization processing of the image frame D12 of the numerical data D22 by the recording number of the numerical data D22 and the frame number of the video frame D12.

(Specific example of synchronous reproduction from process data D2 to video data D1)

Next, a specific example of synchronous reproduction from the process material D2 to the video material D1 will be described.

Fig. 7 is a view showing a specific example when the process data D2 is synchronously reproduced from the image data D1. Further, in the same manner as in the fifth drawing, the image header information D11 indicating the image data D1 and the process data header information D21 of the process data D2 are omitted.

In other words, in the data synchronous playback device 1 of the first embodiment, for example, as shown in FIG. 7, in the process data D2, the user displays the mouse in the display area of the process data D2 displayed on the monitor 12 by the mouse. 14 or the keyboard 13 or the like moves the roll bar or the cursor, etc., and the numerical data D22 of the record number #134 of No. 134 from the start value D22 is set as the designated material, for example.

As described above, in the first embodiment, since the sampling interval Ts2 is 1 mm second, the relative time from the starting value data is 134 × 1 m seconds = 134 m seconds by the above formula (3). Then, the frame number of the image frame D12 which should be synchronized with the 134th numerical data D22, since the frame period Tf is 1/30 second, is: 134 m second ÷ (1/30 second by the above formula (4) ) = 4.02 ≒ 4, so the image frame D12 of the fourth frame number #4 from the start image frame D12 is synchronously reproduced as the numerical data D22 of the record number #134.

Therefore, according to the data synchronous reproduction device 1 of the first embodiment, it is not necessary to use the time information of the image frame D12 or the numerical data D22, and the frame number of the image frame D12 and the record number of the numerical data D22 are used, as long as By specifying the numerical data D22, the image frame D12 corresponding to the image data D1 can be synchronously reproduced.

Further, in the data synchronous playback device 1 of the first embodiment, the plurality of numerical data obtained by the control devices 51 to 5n from the plurality of monitoring objects 61 to 6p in time series are stored in the storage unit 113b of the storage unit 113. The process data D2, but the data contained in the process data D2, is not limited to numerical data, and may also be bit data or text data.

(Display screen specific example)

Next, a specific example of the display screen of the monitor 12 displayed on the data synchronous playback device 1 of the first embodiment will be described.

Fig. 8 to Fig. 12 are diagrams showing specific examples of the display screens 121 to 125 of the monitor 12 of the data synchronous reproduction device 1 of the first embodiment. Further, the display screens 121 to 125 shown in Figs. 8 to 12 are configured by the CPU 111 of the data synchronous playback device 1 of the first embodiment to execute the data synchronization control program stored in the program storage unit 113a. An example of a display screen in which the setting unit performs a function and is set or changed. In other words, the display screens 121 to 125 shown in FIGS. 8 to 12 are functions in which the CPU 111 functions as a screen arrangement setting unit, and the video data reproduction area in which the video material is displayed in accordance with the user's instruction is set on the monitor 12, and An example of a display screen for displaying the process data reproduction area and setting and displaying the number and size of image data to be reproduced within a predetermined condition in the image data reproduction area.

In the display screen 121 shown in FIG. 8, the image data setting area 121a is provided on the upper side of the display screen 121 as a screen arrangement setting unit function, and the process data display is provided on the lower side of the display screen 121. Area 121b. Then, on the display screen 121, four image data display windows 121a1 to 121a4 are provided in the image data display area 121a, and are displayed on the four image data display windows 121a1 to 121a4, and the camera switcher 3 is instructed by the user or automatically Switching, from the plurality of ITV cameras 21 to 2m, for example, four ITV cameras of camera numbers #1 to #4 (camera 1 to 4) capture the obtained moving image data D1.

Further, in the display screen 121, the process data D2 of the monitoring target object captured by the four ITV cameras of the machine numbers #1 to #4 (camera 1 to 4) switched by the camera switcher 3 is reproduced. The process data display area 121b. Here, the horizontal axis in the horizontal direction on the map of the process data display area 121b represents the time axis, and the vertical axis in the vertical direction on the figure represents the value (level) of the numerical data D22 constituting the process data D2.

Then, the image data D1 reproduced in the four image data display windows 121a1 to 121a4 and the process data D2 reproduced in the process data display area 121b are respectively used by the CPU 111 functioning as the data synchronous reproduction control unit, as described above. The frame number of the image frame D12 is synchronized with the record number of the numerical data D22 to be reproduced.

Further, in the process material display area 121b of the display screen 121, in order to convert the process data D2 at the time of reproduction, a process data time change bar (bar) 121b2 that moves in the horizontal direction is displayed on the map. The user can operate the process data time change bar 121b2 by operating the cursor (not shown) on the display screen 121 by the keyboard 13 or the mouse 14, and the time axis of the process data D2 can be enlarged/reduced or enlarged.

Further, in Fig. 8, 121b3 is a drawing start position specifying bar for designating a start position of a process data extraction record (snapshot) displayed in the process material display area 121b; 121b4 is for specifying display in the process data display area. The extraction of the record end position specifying bar at the end of the recording of the process data of 121b; the extraction record start position designation bar and the extraction record end position designation bar are designated bars using the extraction record (snapshot) of the second embodiment to be described later, The first embodiment is not used. The designated bars are moved in the process material display area 121b by the user using the mouse 14 or the keyboard 13 or the like as described above.

In the display screen 122 shown in FIG. 9, in order to enlarge and display the process data D2, the enlarged state of the process material display area 122b is displayed on the entire display screen 122. Therefore, in the display screen 122 shown in FIG. 9, the image data display area 121a existing on the upper half of the center of the display screen 121 on the display screen 121 of FIG. 8 cannot be displayed, so that the display image data display area 122a overlaps The process material display area 122b and the four image data display windows 122a1 to 122a4 are provided in the material display area 122a.

Similarly to the display screen 121 shown in FIG. 8, the display screen 123 shown in FIG. 10 is provided with an image data display area 123a on the upper side in the center of the display screen 123, and on the lower side in the center of the display screen 123. The process data display area 123b is set. Then, on the display screen 123, two video material display areas 123a1 and 123a2 are provided in the video material display area 123a, and the video image data D1 of the ITV camera 21 from the camera number #1 is displayed in the video material display area 123a1, and the other is displayed. On the other hand, the video material D1 of the still image from the ITV camera 21 (camera 1) of the same camera number #1 is displayed in the video material display area 123a2. Further, on the display screen 123, in the process material display area 123b, the video image data D1 from the ITV camera 21 of the camera number #1 displayed in the video material display area 123a1 is displayed, and the display control device is displayed. The process data D2 obtained by the monitoring target image captured by the ITV camera 21 of the camera number #1. Further, the process material display area 123b is not different from the process material display areas 121b and 122b displayed on the display screens 121 and 122 of the eighth and ninth drawings.

In the display screen 124 shown in FIG. 11, similarly to the display screen 121 shown in FIG. 8, the video data display area 124a is provided on the upper side of the display screen 124, and the lower side of the display screen 124 is on the lower side. The process data display area 124b is set. Then, on the display screen 124, two video material display areas 124a1 and 124a2 are provided in the video material display area 124a in the same manner as the display screen 123 shown in FIG. 10, and the camera number # is displayed in the video material display area 124a1. On the other hand, the moving image data D1 of the ITV camera 21 of 1 is displayed on the image data display area 124a2, and the ITV camera 21 (camera 1) from the same camera number #1 is displayed, and the images of the four still images separated by the shooting time are displayed. Information D1.

Further, when the process data D2 includes, for example, a character data such as a rolled material batch number which is being rolled, the character data 124C is displayed in the process data display area 124b.

Further, the process material display area 124b is not different from the process material display areas 121b to 123b displayed on the display screens 121 to 123 of Figs. 8 to 10 .

The display screen 125 shown in FIG. 12 is similar to the display screen 121 shown in FIG. 8, and the image data display area 125a is provided on the upper side of the center of the display screen 125, and is disposed on the lower side of the center of the display screen 125. The process data display area 125b. Then, on the display screen 125, for example, a camera image designation button 125a1 for selecting two camera images from four camera images is displayed in the image data display area 125a, and the camera image designation button 125a1 is selected by the camera image designation button 125a1. The moving image data of the two ITV cameras 21 and 23 (Camera 1, 3) of the camera numbers #1 and #3 are displayed on the image data display areas 125a1 and 125a3. In this manner, the window size of the image data display areas 125a2 and 125a3 of the display screen 125 is smaller than that of the image data display areas 123a1 and 123a2 displayed in the display screen 123 of FIG. The window of the image data display areas 123a1 to 121a4 displayed in the display screen 121 of Fig. 8 is large, so that the image data D1 is easily seen. Further, on the display screen 125, the process material display area 125b is not different from the process material display areas 121b to 124b displayed on the display screens 121 to 124 of Figs. 8 to 11 .

As described above, according to the data reproducing apparatus 1 of the first embodiment, the frame number and the frame period Tf of the image frame D12 constituting the image data D1 are used without referring to the time information of the image data D1 and the process data D2. The recording number of the numerical data D22 of the process data D2, the sampling interval Ts2, and the like can be used to synchronously reproduce the image frame D12 and the numerical data D22, and can be displayed on the monitor 12 as shown in FIGS. 8 to 12. The display screens shown on the screens 121 to 125 are displayed. As a result, according to the data synchronous reproduction device 1 of the first embodiment, the synchronous reproduction processing of the image data D1 and the numerical data D22 does not require reference to the time information of both materials, thereby reducing the number of files and the file memory capacity, and In the past, memory devices having the same memory capacity have become more likely to be remembered and synchronized for a longer period of time than in the past.

In addition, since image frame compression and decoding technology such as MPEG-4 can be used for image storage, a general-purpose image data reproduction program such as a Quick Time (registered trademark) player can be used for image reproduction. Reduce development costs.

Moreover, it is possible to simultaneously reproduce data and image frames at high speed without using the time information, or to prompt.

(Second embodiment)

In the first embodiment described above, the synchronous reproduction processing using the frame number of the image frame D12 and the image data D1 and the process data D2 of the record number of the numerical data D22 has been described without referring to the time information; In the second embodiment, the function of the first embodiment is added: by extracting the extracted record of the range data value D22 from the process data D2, that is, by specifying the range of the numerical data D22 to be recorded by the snapshot, the automatic storage is possible. The function of the image data D12 synchronized with the specified range numerical data D22. Therefore, in the data synchronous reproduction device of the second embodiment, the function of the data synchronization reproduction device 1 of the first embodiment is added with the function of the snapshot recording; and the data synchronization control stored in the program storage unit 113a is performed. In the program, it is assumed that a program that functions as a synchronous storage control unit belonging to the snapshot recording function is included. In addition, since the configuration of the data reproducing apparatus of the second embodiment is the same as that of the first embodiment, the configuration of the first embodiment shown in Figs. 1 and 2 will be described, and only the characteristic operation will be described.

In other words, in the data synchronization reproducing apparatus 1 of the second embodiment, the user moves the extracted recording start position specifying bar 121b3 by using the mouse 14 or the keyboard 13 or the like in the process data display area 121b displayed on the monitor 12. The recording end position specifying bar 121b4 is extracted, and as shown in Fig. 8, the range of the process data D2 designated for drawing from the process data D2 for additional recording is performed. Then, when the user clicks the save button (not shown) or the like, the CPU 111 executes the data synchronization control program stored in the program storage unit 113a, and first functions as a synchronous image frame selection unit, and specifies the user. The recording number of the numerical data D22 of the range is converted into an image synchronized to the range of the numerical data D22 of the designated range by the above mathematical expression (3), mathematical expression (4), and the like in the same manner as the synchronous reproduction of the first embodiment. The frame number of frame D12.

Next, the CPU 111 functions as a synchronous storage control unit by executing the data synchronization control program, so that the numerical data D22 of the range specified by the user is associated with the image frame D12 synchronized with the specified range numerical data D22 to form an image. The data file D1 is separated from the data file Df of the process data D2, that is, the numerical data D22 of the range specified by the user is extracted from the data file D1 of the image data D1 and the process data D2, and synchronized with the specified range. The form of the image data D12 of the numerical data D22 is stored in the data storage unit 113b.

Therefore, in the data synchronous playback device 1 of the second embodiment, the file can be attached to an electronic mail or the like via a mail program, and transmitted to another computer device via a network such as the Internet or an internal network.

Thereby, the other computer device that receives the value data D22 stored in the range specified by the user and the file frame D12 synchronized with the range value data D22 has the data of the first embodiment or the second embodiment. The synchronous reproduction function of the same image data D1 and the process data D2 of the synchronous reproduction device 1 can synchronously reproduce the numerical data D22 of the user-specified range and the image frame D12 synchronized with the range numerical data D22 from the received file. It is possible to perform remote diagnosis and the like.

(industrial use may be)

The invention is applicable to an online data management system for monitoring a factory.

1. . . Data synchronous regeneration device

3. . . Camera switcher

4. . . Digital image conversion unit

7. . . LAN (Local Area Network)

11. . . Device body

12. . . Monitor

13. . . keyboard

14. . . mouse

21 to 2 m. . . ITV camera (m is a natural number)

51 to 5n. . . Control the machine (n is a natural number)

61 to 6p. . . Monitoring object (p is a natural number)

111. . . CPU (image data reproduction unit, data synchronization reproduction control unit, synchronization image frame selection unit, synchronization storage control unit, screen layout setting unit)

112. . . Memory

113. . . Memory department

113a. . . Program memory

113b. . . Data memory unit (image data memory unit, process data memory unit)

114. . . External machine I/F department

115. . . Control network I/F department

116. . . LAN I/F Department

117. . . Internal bus

D1. . . video material

D2. . . Process data

D12. . . Image frame

D22. . . Numerical data

Figure 1 is a diagram showing an example of the configuration of an online data management system including the data reproducing apparatus of the present invention;

Figure 2 is a block diagram showing a configuration example of the data synchronous reproducing apparatus of the present invention;

Figure 3A is a diagram showing an example of a file format of image data and process data stored in a data memory unit of a data synchronous reproduction device;

Figure 3B is a diagram showing an example of the structure of the image data;

Figure 3C is a diagram showing the structure of the data of the process data;

Figure 4 is a flow chart showing the sequence of data synchronization display processing executed by the data synchronous reproducing apparatus of the first embodiment;

Fig. 5 is a view showing a specific example of the synchronous reproduction processing from the image data D1 to the process data D2;

Figure 6 is a flow chart showing the sequence of data synchronization display processing of the data synchronous reproduction device of the first embodiment;

Fig. 7 is a view showing a specific example of the synchronous reproduction processing from the numerical data to the image data;

Figure 8 is a view showing an example of a monitor display screen displayed on the data synchronous reproduction device of the first embodiment;

Figure 9 is a view showing another example of a monitor display screen displayed on the data synchronous reproduction device of the first embodiment;

Figure 10 is a view showing another example of a monitor display screen displayed on the data synchronous reproduction device of the first embodiment;

Figure 11 is a view showing another example of a monitor display screen displayed on the data synchronous reproduction device of the first embodiment;

Figure 12 is a view showing another example of a monitor display screen displayed on the data synchronous reproduction device of the first embodiment;

D1. . . video material

D2. . . Process data

D12. . . Image frame

D22. . . Numerical data

Claims (12)

A data synchronization reproducing device includes: an image data storage unit that captures a plurality of image frames obtained by monitoring a camera, adds a frame number to each image frame, and memorizes with the frame period of the image frame as The image data storage unit is a plurality of data obtained from the monitoring object in time series, and the recording number is added to each data, and is recorded as process data together with the sampling interval of the data; the image data reproducing unit is used for Outputting and reproducing the video data stored in the video data storage unit to a monitor, and outputting a frame number of the reproduced video frame; and a data synchronous playback control unit that outputs the frame from the video data reproducing unit The number is converted into the record number of the data according to the frame period of the image frame and the sampling interval of the data, and the data having the converted record number is reproduced on the monitor. The data synchronous reproduction device according to claim 1, wherein the data synchronous reproduction control unit performs an operation of designating the data at a specific timing in the process data displayed on the monitor. And converting the record number of the specified data to the frame number according to the sampling interval of the data frame and the frame period of the image frame, and outputting the converted frame number to the image material The reproduction unit reproduces the image data having the converted frame number. The data synchronous reproduction device according to the first or the second aspect of the patent application, wherein the complex image frame selection unit is configured to be specified by an external input in the process data displayed on the monitor. When the data range of the other file is memorized, the record number of the data of the specified range is converted into the frame number of the image frame according to the sampling interval of the foregoing data and the frame period of the image frame. Selecting an image frame of a range synchronized with the aforementioned range of the specified range; and a synchronous storage control unit for inputting the aforementioned data of the range specified by the external input and the range selected by the synchronous image frame selecting unit The image frame is remembered as the same file. The data synchronization/reproduction device according to the first or second aspect of the invention, further comprising: a screen arrangement setting unit configured to display, on the monitor, the video image reproduction unit to display the image by the external input a video data reproduction area of the data, and a process data reproduction area for displaying the process data by the data synchronization control unit, and setting the number and size of the image data to be reproduced within the predetermined condition in the image data reproduction area . A data synchronization control method includes: a plurality of image frames obtained by capturing a monitoring object by a camera, adding a frame number to each image frame, and memorizing as a video data together with a frame period of the image frame; Recording a record number from each of the plurality of materials obtained from the monitoring object in time series, and sampling interval with the aforementioned data And storing the image data as a process data; and outputting and reproducing the image data to a monitor, and outputting the frame number of the reproduced image frame, according to the frame of the image frame The sampling interval of the period and the foregoing data is converted into the record number of the above-mentioned data, and the above-mentioned data having the converted record number is reproduced in the above-mentioned monitor. The method for synchronizing data according to claim 5, wherein, in the process data displayed on the monitor, when the information is specified in a specific time series, the record number of the specified data is determined according to The sampling interval of the aforementioned data and the frame period of the image frame are converted into the frame number, and the image data having the converted frame number is reproduced. The method for synchronously controlling data according to Item 5 or Item 6 of the patent application, wherein the method further comprises: when specifying a data range to be memorized as another file in the process data displayed on the monitor The record number of the aforementioned data in the specified range is converted into the frame number of the image frame according to the sampling interval of the aforementioned data and the frame period of the image frame, and the range synchronized with the aforementioned data of the specified range is selected. The steps of the image frame; and the step of storing the aforementioned data of the specified range and the image frame of the selected range as the same file. For example, the method for synchronously controlling data according to item 5 or item 6 of the patent application scope, wherein the complex has: According to the external input, the video data reproduction area for displaying the video data and the process data reproduction area for displaying the process data by the data synchronization reproduction control unit are set by the video data reproduction unit, and In the image data reproduction area, the step of setting the number and size of the image data to be reproduced within predetermined conditions. A data synchronization control program for causing a computer to perform the following steps: shooting a plurality of image frames obtained by monitoring a camera, adding a frame number to each image frame, and storing the image as an image together with the frame period of the image frame. a step of data; adding a record number to each piece of data from the time-series sequence of the monitoring object, and storing the record as a process data together with the sampling interval of the data; and outputting and reproducing the image data for monitoring And outputting the frame number of the reproduced image frame, converting the frame number outputted according to the frame period of the image frame and the sampling interval of the data to the record number of the data, and The aforementioned data having the converted record number is reproduced in the step of the aforementioned monitor. The data synchronization control program of claim 9, wherein the data is: in the foregoing process data displayed on the monitor, when the foregoing data is specified in a specific time sequence, the foregoing record of the data is specified. The number is converted into the frame number according to the sampling interval of the foregoing data and the frame period of the image frame, so that the frame number after conversion is obtained. The above image data is subjected to a regeneration step. For example, the data synchronization control program described in claim 9 or 10, wherein the execution is performed: in the process data displayed on the monitor, when the data range to be memorized as another file is specified, according to the foregoing The sampling interval of the data and the frame period of the image frame are converted into the frame number of the image frame of the specified range, and the image of the range synchronized with the data of the specified range is selected. a step of the frame; and the step of storing the aforementioned data of the specified range and the image frame of the selected range as the same file. The data synchronization control program according to claim 9 or 10, wherein the execution is performed on the monitor according to an external input: displaying the image data of the image data by the image data reproduction unit The reproduction area and the process data reproduction area for displaying the process data by the data synchronization reproduction control unit, and the step of setting the number and size of the image data to be reproduced within the predetermined condition in the image data reproduction area.