JP7079565B2 - Information processing system, information processing system control method, and its program - Google Patents

Description

The present invention relates to an information processing system, a control method of the information processing system, and a program, and more particularly to a mechanism capable of easily identifying the cause of an error when an error occurs in a monitored area.

Generally, in a production factory, a PLC (Programmable Logic Controller) is used to control various machine tools, and when a defect occurs in a production process, the PLC raises an error alert. In addition, a network camera that monitors the production line is separately installed, and both may operate as separate systems or as one system.

Patent Document 1 discloses a mechanism for monitoring a production process of a product with a network camera. Further, Patent Document 1 discloses that when an alert is raised from a machine tool, the network camera is directed in the direction in which the alert is raised.

Japanese Unexamined Patent Publication No. 2008-17158

By the way, when an alert is raised from a machine tool as in Patent Document 1, there are cases where it is difficult to identify the cause of what actually happened and caused an error simply by pointing the network camera in that direction. There was a possibility. For example, the sensor attached to the arm of the manufacturing equipment that lifts the product and packs it in a box should have been turned on if the machine tool 101 had lifted the product 301, but it did not turn on. In the case of the error event shown, even if the cause of the error event is that the screw of the arm of the machine tool 101 is unscrewed, the error event and the direction in which the alert is raised are captured in a wide range. There was a possibility that it could not be understood from the image data of the network camera 104.

An object of the present invention is to provide a mechanism that makes it easy for a user to identify the cause of an error when an error occurs in the monitored area.

The present invention is an image pickup means for imaging a monitored area, and is an information processing system capable of communicating with an image pickup means for imaging an image to be imaged corresponding to the error when an error occurs in the monitored area. When the acquisition means for acquiring the image data captured by the image pickup means, the image data acquired by the acquisition means, and the pattern recognized in advance and the error does not occur, the image to be imaged is captured. Using the change pattern of the captured image data, among the image data acquired by the acquisition means, the content of the error event based on the information indicating the occurrence of the abnormality received from the PLC that detects the error of the imaged object. In the corresponding specific area, the specific means for specifying the part having a change different from the pattern as the part related to the cause of the error and the part specified by the specific means are zoomed and imaged by the image pickup means. The control means for controlling the display and the image data acquired by the acquisition means are controlled to be displayed, and the cause of the error specified by the specific means to the display means on the displayed image data is related to the control means. A display control means for displaying the portion in an identifiable manner and further controlling the portion related to the cause of the error so as to display the zoomed image data captured by zooming with the control means.
It is characterized by having.

Further, the present invention is an imaging means for imaging a monitored area, and when an error occurs in the monitored area, an information processing system capable of communicating with an imaging means for imaging an object to be imaged corresponding to the error. In the control method, an acquisition step in which the acquisition means of the information processing system acquires image data captured by the image pickup means, and the image data acquired by the identification means of the information processing system in the acquisition step. Of the image data acquired by the acquisition means, the image to be imaged is the pattern recognized in advance and the pattern of change in the image data obtained by imaging the object to be imaged when the error does not occur. In the specific area according to the content of the error event based on the information indicating the occurrence of the abnormality received from the PLC that detects the error of the object, the part that changes differently from the above pattern is specified as the part related to the cause of the error. The specific step, the control step in which the control means of the information processing system controls the portion specified in the specific step to zoom in and take an image, and the display control means of the information processing system are described above. In addition to controlling the display of the image data acquired in the acquisition process, the display means on the displayed image data is identifiable by displaying the location related to the cause of the error specified in the specific process, and further. It is characterized by comprising a display control step of controlling a portion related to the cause of the error so as to display the zoom image data captured by zooming by the control means.

Further, the present invention is an imaging means for imaging a monitored area, and is an information processing system capable of communicating with an imaging means for imaging an object to be imaged corresponding to the error when an error occurs in the monitored area. It is a program that can be read and executed, and is a pattern recognized in advance by the information processing system with the acquisition means for acquiring the image data captured by the imaging means and the image data acquired by the acquisition means. From the PLC that detects the error of the object to be imaged among the image data acquired by the acquisition means by using the pattern of change of the image data that imaged the object to be imaged when no error has occurred. In the specific area according to the content of the error event based on the information indicating the occurrence of the received abnormality, the specific means for specifying the part having a change different from the above pattern as the part related to the cause of the error, and the specific means. A control means for controlling the specified portion to be zoomed and imaged by the image pickup means, and a control means for displaying the image data acquired by the acquisition means, and display on the displayed image data. A portion related to the cause of the error identified by the specific means is identifiablely displayed on the means, and further, the portion related to the cause of the error is zoomed by the control means and the imaged zoom image data is displayed. It is characterized in that it functions as a display control means for controlling.

According to the present invention, when an error occurs in a monitored area, it is possible for the user to easily identify the cause of the error.

The system block diagram which shows an example of the structure of the production process monitoring system of this invention. FIG. 3 is a diagram showing an example of a hardware configuration of an information processing device applicable to the PLC client terminal 103, the relay server 105, the recording server 106, and the browsing client terminal 107 shown in FIG. 1. The figure which shows an example which applied the production process monitoring system of this invention. The figure which shows an example of a network camera database. The figure which shows an example of the display screen when the image captured by a network camera 104 is displayed by a browsing client terminal 107. The figure which shows an example of the flowchart about the whole processing in the production process monitoring system of this invention. The figure which shows an example which applied the production process monitoring system of this invention. The figure explaining the details of the process of step S605 of FIG. The figure which shows an example of the display screen when the image captured by a network camera 104 is displayed by a browsing client terminal 107. The figure explaining the details of the process of step S609 of FIG. The figure explaining the details of the process of step S612 of FIG. The figure which shows an example of the display screen when the image captured by a network camera 104 is displayed by a browsing client terminal 107. The figure explaining the details of the process of step S614 of FIG.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The embodiment described below is an example of a specific embodiment of the present invention, and is one of the specific embodiments of the configuration described in the claims.

FIG. 1 shows a production process for monitoring a production process of an object flowing through a factory line, including the machine tool 101, PLC 102, PLC client terminal 103, network camera 104, relay server 105, recording server 106, and browsing client terminal 107 of the present invention. It is a system configuration diagram which shows an example of the configuration of a monitoring system.

In the present production process monitoring system, each device is connected so as to be able to communicate with each other via a network 108 or a wireless LAN (including a mobile communication network).
The machine tool 101 assembles, processes, packs, and the like the product to be produced.

The PLC 102 controls the machine tool 101. Further, when an error occurs in the machine tool 101, the error is detected and the processing result of the PLC 102 is transmitted to the PLC client terminal 103. Further, when an abnormality occurs in the machine tool 101, a code indicating the cause of the abnormality is transmitted to the PLC client terminal 103.

The PLC client terminal 103 receives the processing result of the PLC 102 transmitted from the PLC 102 and the code indicating the cause when the abnormality occurs, and when the received information is the code indicating the cause when the abnormality occurs in the PLC 102, the PLC client terminal 103 receives the code indicating the cause when the abnormality occurs. An event indicating an error (also referred to as an error event) is transmitted to the relay server 105.

The error event is, for example, "in production process A, the container taking-out robot_grip OFF abnormality (the sensor attached to the hand of the machine tool 101 for taking out the container which is the product does not turn on)" and the like. Various errors that occur on the system.

When the relay server 105 receives an event indicating an error from the PLC client terminal 103, the relay server 105 changes the PTZ of the network camera 104 (pan to move left and right, tilt to move up and down, and telephoto or wide-angle) according to the content of the event. It decides whether to adjust the zoom) and sends the decided content to the recording server 106.

The recording server 106 sends an image data (also referred to as image capture data) acquisition request to the network camera 104 to acquire and store the image data captured by the network camera 104, or the image acquired to the browsing client terminal 107. Send data. Further, the recording server analyzes the acquired image data and determines whether the PTZ of the network camera 104 needs to be adjusted.

In this embodiment, the relay server 105 and the recording server 106 are separate devices, but as another embodiment, the relay server 105 and the recording server 106 may be the same device.

The network camera 104 is provided one or more for each production process, and images each production process. Then, the image data captured by the network camera 104 is transmitted to the recording server 106. Further, the PTZ of the network camera 104 is adjusted according to the instruction from the recording server 106.

The browsing client terminal 107 is used by the user to check the image data when an error event occurs, and displays the image data transmitted from the recording server 106 on the display.
This is the end of the description of FIG.

Next, the hardware configuration of the information processing device applicable to the PLC client terminal 103, the relay server 105, the recording server 106, and the browsing client terminal 107 shown in FIG. 1 will be described with reference to FIG.

In FIG. 2, 201 is a CPU that comprehensively controls each device and controller connected to the system bus 204. Further, in the ROM 202 or the external memory 211, a BIOS (Basic Input / Output System) which is a control program of the CPU 201, an operating system program (hereinafter, OS), and various functions necessary for realizing a function executed by the PC, which will be described later, are used. The program etc. are stored.

Reference numeral 203 denotes a RAM, which functions as a main memory, a work area, and the like of the CPU 201. The CPU 201 realizes various operations by loading a program or the like necessary for executing a process from the ROM 202 or the external memory 211 into the RAM 203 and executing the loaded program.

Further, 205 is an input controller, which controls input from a pointing device such as a keyboard (KB) 209. Reference numeral 206 denotes a video controller, which controls the display on a display such as a display 210 (whether a liquid crystal display or a cathode ray tube).

The 207 is a memory controller that can be used in an external storage device (hard disk (HD)) that stores boot programs, various applications, font data, user files, edit files, various data, etc., a flexible disk (FD), or a PCMCIA card slot. It controls access to an external memory 211 such as a compact flash (registered trademark) memory connected via an adapter.

Reference numeral 208 denotes a communication I / F controller, which connects and communicates with an external device via a network, and executes communication control processing on the network. For example, communication using TCP / IP is possible.

The CPU 201 enables display on the display 210 by, for example, executing an outline font expansion (rasterization) process in the display information area in the RAM 203. Further, the CPU 201 enables a user instruction with a mouse cursor or the like (not shown) on the display 210.

Various programs described later for realizing the present invention are recorded in the external memory 211, and are executed by the CPU 201 by being loaded into the RAM 203 as needed. Further, a setting file or the like used when executing the above program is also stored in the external memory 211, and detailed description of these will be described later. This is the end of the description of FIG.

Next, it is a figure which shows an example which applied the production process monitoring system of this invention using FIG. 3, and is the figure which looked at the production line of a certain factory from the top.

The production line shown in FIG. 3 consists of three production processes from production process A to production process C, and one network camera 104 is installed for each production process. The network camera 104 is a preset determined in advance by the user at the time of installation, and usually performs imaging.

The information of the network camera 104 installed on the production line is managed in the network camera database shown in FIG.

Further, a machine tool 101 is provided for each production process, and the machine tool 101 assembles, processes, packs, and the like the product 301. The product 301 is transported to each production process by the transport table 302. This is the end of the description of FIG.
Next, the network camera database will be described with reference to FIG.

FIG. 4 is a diagram showing an example of a network camera database managed by the relay server 105 or the external memory 211 of the recording server 106, and the network camera database is a network camera name for uniquely identifying each network camera 104. It is composed of 401, an image pickup target 402 indicating a production process taken by the network camera 104 indicated by the network camera name 401, and a preset 403 indicating a preset of the network camera 104 indicated by the network camera name 401. This is the end of the description of FIG.

Next, an example of a display screen when the image captured by the network camera 104 is displayed on the viewing client terminal 107 will be described with reference to FIG.

FIG. 5 is a diagram showing an example of a display screen displayed on the display 210 of the browsing client terminal 107 when the image captured by the network camera 104 is displayed on the browsing client terminal 107.

The display screen 500 has a production line display area 501 that displays image data in which the production line is illustrated, a live image area 502 that displays image data (live image) currently captured by the network camera 104, and an error in the production process. It is composed of a recorded video display area 503 that displays image data (recorded video) when an event occurs, and a list area 504 that displays a list of error events that have occurred in the production process.

Image data is displayed in the production line display area 501 so that the user can recognize the position of the network camera 104 currently displaying the live image.

The list area 504 includes a date and time 505 indicating the date and time when the error event occurred, a name 506 indicating the error status (displayed as "new" if the error event is newly added), a message 507 indicating the content of the error event, and an error. It is composed of a production process in which an event has occurred and a source 508 indicating a network camera 104 that images the production process. This is the end of the description of FIG.
Next, the entire process in the production process monitoring system of the present invention will be described with reference to FIG.

FIG. 6 is a diagram showing an example of a flowchart relating to the entire processing in the production process monitoring system of the present invention.

Each step of the flowchart shown in FIG. 6 and FIGS. 8, 10, 11, and 13 described later is processed by the CPU of each device.

In step S601, the PLC client terminal 103 receives the processing result of the PLC 102 transmitted from the PLC 102 and the code indicating the cause when the abnormality occurs, and the received information is the code indicating the cause when the abnormality occurs in the PLC 102. If the received information is a code indicating the cause of the occurrence of an abnormality in the PLC 102, an error event is transmitted to the relay server 105 (step S603).

In step S604, when the relay server 105 receives an error event from the PLC client terminal 103 (step S604), the relay server 105 executes the received error event analysis process (step S605). The details of the process of step S605 will be described later with reference to FIG.

It should be noted that the error event received from which PLC 102 can be determined from the IP address of the source. Further, although the PLC 102 has different error event transmission procedures and transmission data formats depending on the manufacturer, the relay server 105 can implement a reception driver for each manufacturer for the transmission procedures and transmission data formats of a plurality of corresponding PLC manufacturers. , It is possible to absorb the difference between manufacturers.

Further, in the present invention, it is possible to identify the approximate location of the error and the content of the error from the error event.

If there are a plurality of error events, in step S604, the relay server 105 acquires all the error events.

Step S604 is an example of the acquisition means in the present invention for acquiring the detection result by the detection means for detecting the occurrence of an error in the production process.

In step S606, the relay server 105 adjusts the PTZ of the network camera 104 (pan that moves left and right, tilt that moves up and down, and zoom that makes it telescopic or wide angle) determined in the process of step S605. Information on the error event including the above contents is transmitted to the recording server 106, and an adjustment instruction is given to the network camera 104 via the recording server 106.

In step S606, according to the determination means in the present invention, the production process in which the error has occurred is imaged by another imaging means other than the image pickup means for imaging the production process in which the error has occurred. It is an example of the control means for controlling the other image pickup means to take an image of the production process in which the error has occurred.

In step S607, the recording server 106 transmits a request for acquiring image data captured by the network camera 104 to the network camera 104.

In step S608, the recording server 106 acquires the image data transmitted from the network camera 104 in response to the image data acquisition request in step S607, and executes the process for the image data (step S609). The details of the process of step S609 will be described later with reference to FIG. The acquired image data is buffered for a predetermined time.

In step S610, the recording server 106 transmits the image data acquired from the network camera 104 to the browsing client terminal 107.

In step S611, the network camera 104 receives the request from the recording server 106 and executes the process according to the request (step S612). The details of the process of step S612 will be described later with reference to FIG.

In step S613, the browsing client terminal 107 receives the image data transmitted from the recording server 106, and executes a process of displaying the image data on the display 210 (step S614). The details of the process of step S614 will be described later with reference to FIG.

Step S613 is an example of the acquisition means in the present invention for acquiring image data captured by the image pickup means when an error occurs in the monitored area.
This is the end of the description of FIG.
Next, the details of the process of step S605 of FIG. 6 will be described with reference to FIGS. 7 and 8.

As described with reference to FIG. 4, since the network camera 104 that captures images is determined for each production process, if an error event occurs in the production process, the network camera 104 that targets the production process as an imaging target (imaging target 402). (The network camera indicated by the network camera name 401) captures the production process. However, depending on the content of the error, the error state may not be grasped only by the image data of the network camera 104.

For example, in the state shown in FIG. 3, when an error event occurs in the production process A, in the camera 1, the side opposite to the side imaged by the camera 1 of the machine tool a, that is, the back surface of the machine tool a is the camera 1. It becomes a blind spot and there is a possibility that the error part cannot be imaged. Further, even when an error event occurs below the transport table 302 for transporting the product 301, there is a possibility that the error portion cannot be imaged.

Therefore, the present invention provides a mechanism capable of capturing an image of an error portion even in the above-mentioned case by executing the process shown in FIG.

Specifically, when an error event occurs at a position that becomes a blind spot of the network camera 104 that captures an image of the production process in which the error event has occurred by executing the process of FIG. 8, the location where the error event occurs occurs. Since the approximate position can be specified from the error event information transmitted from the PLC client terminal 103, in such a case, as shown in FIG. 7, an error event also occurs in the network camera 104 that captures images of other production processes. Decide to image the production process.

FIG. 7 is a diagram showing an example in which the production process monitoring system of the present invention is applied. In the production line shown in FIG. 3, an error event occurs at a position that becomes a blind spot of the network camera 104 that captures the production process in which the error event occurs. The network camera 104 (camera 2 and camera 3) that captures images of other production processes also captures the production process in which an error event occurs.

The network camera 104 that captures images of other production processes also captures the production process in which an error event has occurred, so that the camera 1 can capture a place that becomes a blind spot.

Further, at this time, the network camera 104 (camera 3 in the case of FIG. 7) that captures the image of the production process farthest from the production process in which the error event occurs is controlled to be zoomed out (that is, a wide range is captured). In the present invention, even if an error event generated in the production process affects other production processes, the influence can be confirmed from the image data of the zoomed-out network camera 104.
Then, from here, each step of FIG. 8 will be described.

In step S801, the relay server 105 so that the error event acquired in step S604 captures the production process other than the production process in which the error event occurred, and the network camera 104 also captures the production process in which the error event occurs. Determine if it is an error event that should be set. That is, it is determined whether or not the error event has occurred at the position that becomes the blind spot of the network camera 104 that captures the image of the production process in which the error event has occurred. If the network camera 104 that captures the other production process is also an error event that should capture the production process in which the error event occurred, the process shifts to step S803, and the network camera 104 that captures the other production process. If it is not an error event that should be used to image the production process in which the error event has occurred, the process proceeds to step S802.

In step S802, the relay server 105 determines that the network camera 104 that captures the production process in which the error event has occurred captures the location where the error has occurred, and ends this process.

In step S803, the relay server 105 determines whether an error event has occurred in another production process based on whether a plurality of error events have been acquired in step S604 and the production process in which each error event has occurred is different. .. If it is determined that an error event has occurred in another production process, the process is transferred to step S805, and if not, the process is transferred to step S804.

In step S804, the relay server 105 determines that the network camera 104, which captures images of production processes other than the production process in which the error occurred, also captures the production process in which the error occurred.

In step S804, using the detection result acquired by the acquisition means in the present invention, an error may occur in the other imaging means for imaging a production process other than the imaging means for imaging the production process in which an error has occurred. This is an example of a determination means for determining whether to control the generated production process to be imaged.

In step S805, the relay server 105 determines from the content of the error event acquired in step S604 whether the error event occurring in the other production process is an error event indicating a serious error. The relay server 105 shifts the process to step S806 if it is an error event indicating a serious error, and shifts the process to step S804 if it is not an error event indicating a serious error.

That is, even if an error event occurs in another production process, if it is a minor error event, the network camera 104 that captures images of other production processes other than the production process in which the error occurred also has the production process in which the error occurred. Is decided to be imaged.

Whether it is a serious error event can be determined by the user by setting a weighting for each error event and specifying that the error is a serious error when the weighting value exceeds the reference value.

The serious error referred to here is, for example, a case where the machine tool 101 malfunctions and becomes uncontrollable, or the machine tool 101 is completely stopped.

In step S806, the relay server 105 determines that each network camera 104 images a production process originally set as an image pickup target (imaging target 402).

In step S801, even if an error event occurs at a position that becomes a blind spot of the network camera 104 that captures the production process in which the error event has occurred, if the error event is a minor error event, the network camera 104 that captures the other production process. It is not necessary to image the production process in which the error occurred. This is the end of the description of FIG.
Next, the details of the process of step S609 of FIG. 6 will be described with reference to FIGS. 9 and 10.

As described above, when the relay server 105 receives the error event from the PLC client terminal 103, the network camera 104, which targets the production process in which the error event occurs, captures the image of the production process. In some cases, it is difficult for the user to visually check the content of the error event and the image data of the network camera 104 that captures the production process with preset presets to identify the cause of what happened and the error occurred. exist. For example, the sensor attached to the arm of the machine tool 101 that lifts the product 301 and packs it in a box should have been turned on if the machine tool 101 had lifted the product 301, but it did not turn on. In the case of an error event indicating that, even if the cause of the error event is due to the parts of the arm of the machine tool 101 being detached, the image is taken with the error event or a preset preset. There was a possibility that it could not be understood from the image data of the network camera 104.

Therefore, in the present invention, the image data (normal system image data) when the machine tool 101 is correctly operated in the production process, which is managed in advance by the recording server 106, is read into the Plug-in application. , The pattern of change in image data (pattern of change in image data when the machine tool 101 is operating correctly) in a certain period of time during normal operation is recognized, and the production process is imaged when an error event occurs. Among the changes in the image data that have been generated, a change that is different from the change pattern of the recognized image data is detected. Then, the network camera 104 is instructed to zoom in and take an image of the position in the real space corresponding to the portion where the different change is detected. By confirming the image data captured by the zoomed network camera 104, the user can confirm the location causing the error event in more detail.

Further, in the present invention, in order to detect a change in an image, a rectangle is set in advance in the image data at a rough position that is likely to be a cause of an error based on the content of the error event. By detecting whether the change in the image data in the image is different from the recognized image data, the processing time required for specifying the change is shortened and the load on the recording server 106 is reduced.
The rectangle is set for a place where a problem is expected to occur based on the content of the error event.
An image of the result of performing the processing up to this point is shown in FIG.

FIG. 9 is a diagram showing an example of a display screen displayed on the display 210 of the browsing client terminal 107 when the image data captured by the network camera 104 is displayed on the browsing client terminal 107.

In FIG. 9, the rectangle 901 is set at a rough position in the image data in the recorded video display area 503, which is likely to cause an error. Then, in the example of FIG. 9, in the rectangle 901, the portion of the frame 902 is specified as a portion of the rectangle 901 that changes differently from the pattern of change of the image data captured in the production process at the normal time, which is recognized in advance, in a certain period of time. Indicates the state of being. After that, when zoom imaging is executed by the network camera 104 instructed by the recording server 106 to zoom the portion of the frame 902, the zoom imaged image data (zoom image data 903) is displayed in the recorded video display area 503. Will be done. As another embodiment, the image data obtained by capturing the image of the production process in the normal state is temporarily displayed in the production line display area 501 or the live video area 502, so that the production process when an error event occurs can be described. It is also possible to allow the user to compare the difference in change with the captured image data (image data displayed in the recorded video display area 503).

In step S1001, the recording server 106 determines whether an error event has been received. If the recording server 106 receives the error event, the process shifts to step S1004, and if not, the process shifts to step S1002-1.

In step S1002-1, the recording server 106 transmits the image data to the browsing client terminal 107 so that the image data acquired from the network camera 104 in step S608 is displayed in the live video area 502 of the browsing client terminal 107, and then the image data is transmitted. If the error event is not received in step S1001 while the image data is buffered for a predetermined time, the image data is deleted in step S1002.

In step S1003, the recording server 106 determines whether or not the end instruction has been received from the user, ends the present process if the end instruction is received, and returns the process to step S1001 if not.

In step S1004, the recording server 106 gives an operation instruction of the network camera 104 from the relay server 105 (specifically, the network camera 104 that captures an image of a production process other than the production process in which the error of step S804 occurred also has the error. It is determined whether or not the instruction to image the generated production process) is received. If the operation instruction of the network camera 104 is received from the relay server 105, the received operation instruction is transmitted to the network camera 104 (step S1005), and if not, the process proceeds to step S1006.

In step S1006, the recording server 106 stores (records) the buffered image data in the external memory 211 for a predetermined time before and after the occurrence of the error event.

In step S1007, the recording server 106 changes the image data of the production process when an error event occurs, and the change pattern of the image data of the image data of the production process during the normal time recognized in advance. Identify areas that are undergoing different changes.

Step S1007 is an image data acquired by the acquisition means using the image data acquired by the acquisition means in the present invention and the image data obtained by imaging the object to be imaged when the error does not occur. This is an example of a specific means for identifying a part that is likely to cause an error.

In step S1008, the recording server 106 determines whether zooming of the network camera 104 is necessary as a result of the process of step S1007. If the zoom of the network camera 104 is necessary, the process is transferred to step S1009, and if not, the process is transferred to step S1001.

In step S1009, the recording server 106 sends a request to the network camera 104 to zoom in and image the position of the production process corresponding to the location identified as having a different change in step S1007. After that, the image data captured by the network camera 104 by zooming is acquired and stored (recorded) in the external memory 211.

Step S1009 is an example of the control means in the present invention for controlling the portion specified by the specific means to be zoomed and imaged by the image pickup means.
This is the end of the description of FIG.
Next, the details of the process of step S612 of FIG. 6 will be described with reference to FIG.

In step S1101, the network camera 104 determines what the request received in step S611 does. If it is a transmission request for image data captured by the network camera 104, the image data is transmitted to the recording server 106 in step S1102, and if it is a preset position change instruction, the preset position is changed to the preset position supported by step S1103 and the image is zoomed in. , Or if it is a zoom-out instruction, zoom in or zoom out, if it is another instruction, the operation according to the content of the instruction is executed, and if it is an end instruction, this process is terminated. This is the end of the description of FIG.
Next, the details of the process of step S614 of FIG. 6 will be described with reference to FIGS. 12 and 13.

As described with reference to FIG. 5, when an error event occurs in the production process, the browsing client terminal 107 displays the image data (recorded video) at the time of the error event in the recorded video display area 503, but conventionally, an error occurs. When the image data (recorded video) when an event occurs is displayed in the recorded video display area 503 and the user is viewing the image data of the error event and the next error event occurs, the next error event is displayed in the recorded video display area 503. Since the image data when the error event of the above occurs is displayed, the browsing of the image data of the error event that the user had been browsing until then has been interrupted.

Therefore, if the user wants to continue to browse the image data of the error event whose browsing is interrupted, the corresponding error event is searched for from the list area 504 and displayed in the recorded video display area 503, and the browsing is further interrupted. It was necessary to send image data, and the operation was very complicated.

Therefore, in the present invention, as shown in FIG. 12, when the next error event occurs while the user is browsing the image data of the error event, the image data when the next error event occurs is displayed as a recorded video. When the pop-up 1201 for allowing the user to select whether to display in the area 503 is displayed on the display screen 500 and the YES button 1202 is selected according to the user operation, the image data when the next error event occurs is displayed as a recorded image. When the image data is displayed in the area 503 and the NO button 1203 is selected according to the user operation, the image data when the current error event occurs is continuously displayed in the recorded image display area 503, so that the user's complicated operation can be performed. Reduce.
Then, each step of FIG. 13 will be described from here.

In step S1301, the viewing client terminal 107 causes the recording server 106 to display the image data (live video) acquired from the network camera 104 in the live video area 502.

In step S1302, the browsing client terminal 107 determines whether the image data of the error event has been received from the recording server 106. If the image data of the error event is received from the recording server 106, the process shifts to step S1304, and if the image data of the error event is not received from the recording server 106, it is determined whether the end instruction has been received from the user ( Step S1303) If the end instruction is received, this process is terminated.

In step S1304, the browsing client terminal 107 determines whether the image data at the time of the error event is currently displayed in the recorded video display area 503. If the image data when the error event occurs is currently displayed in the recorded video display area 503, the process shifts to step S1306, and the image data when the error event occurs is displayed in the currently recorded video display area 503. If it is not displayed, the received image data is displayed in the recorded video display area 503 in step S1305.

Step S1304 controls the display control means to display the image data captured by the image pickup means when another error has already occurred when the image data is acquired by the acquisition means of the present invention. This is an example of a determination means for determining whether or not the data is being used.

Further, step S1305 is an example of the display control means for controlling the display of the image data acquired by the acquisition means in the present invention.

In step S1306, the browsing client terminal 107 forcibly receives the image data in step S1309 if the error event received from the recording server 106 is a serious error and determines whether the error event is a serious error. To display in the recorded video display area 503, or in step S1307, to let the user select whether to display the image data when the next error event shown in FIG. 12 occurs in the recorded video display area 503. Pop-up 1201 is displayed on the display screen 500.

In step S1307, it is determined that the determination means in the present invention controls the display control means to display the image data captured by the image pickup means when another error has already occurred. It is an example of a notification means for notifying a condition.

The serious error referred to here is, for example, a case where the machine tool 101 malfunctions and becomes uncontrollable, or the machine tool 101 is completely stopped.

In step S1308, the browsing client terminal 107 determines whether the YES button 1202 of the pop-up 1201 displayed in step S1307 is selected according to the user operation. When the YES button 1202 is selected according to the user operation, the process shifts to step 1309, and when the NO button 1203 is selected according to the user operation, the process shifts to step S1310 and the current error event occurs. The image data is continuously displayed in the recorded video display area 503.

In step S1311, when the viewing client terminal 107 has elapsed a predetermined time since the NO button 1203 was selected according to the user operation, or when the current error event continuously displayed in the recorded video display area 503 has occurred. When the reproduction of the image data is completed, the process proceeds to step S1309.
This is the end of the description of FIG.

As described above, according to the present invention, when an error occurs in the monitored area, it is possible for the user to easily identify the cause of the error.

The present invention can be, for example, an embodiment as a system, an apparatus, a method, a program, a storage medium, or the like, and specifically, may be applied to a system composed of a plurality of devices, or 1 It may be applied to a device consisting of two devices.

The present invention includes a software program that realizes the functions of the above-described embodiment, which is directly or remotely supplied to a system or an apparatus. The present invention also includes cases where the computer of the system or apparatus can also read and execute the supplied program code.

Therefore, in order to realize the functional processing of the present invention on a computer, the program code itself installed in the computer also realizes the present invention. That is, the present invention also includes the computer program itself for realizing the functional processing of the present invention.

In that case, as long as it has a program function, it may be in the form of an object code, a program executed by an interpreter, script data supplied to the OS, or the like.

Examples of the recording medium for supplying the program include a flexible disk, a hard disk, an optical disk, a magneto-optical disk, MO, a CD-ROM, a CD-R, a CD-RW, and the like. There are also magnetic tapes, non-volatile memory cards, ROMs, DVDs (DVD-ROM, DVD-R) and the like.

In addition, as a method of supplying the program, the browser of the client computer is used to connect to the homepage of the Internet. Then, it can also be supplied by downloading the computer program of the present invention itself or a compressed file including an automatic installation function to a recording medium such as a hard disk from the homepage.

It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from different homepages. That is, the present invention also includes a WWW server that causes a plurality of users to download a program file for realizing the functional processing of the present invention on a computer.

In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, and distributed to users, and the key information for decrypting the encryption is downloaded from the homepage to the user who clears the predetermined conditions. Let me. Then, by using the downloaded key information, it is also possible to execute an encrypted program and install it on a computer.

Further, the function of the above-described embodiment is realized by the computer executing the read program. In addition, based on the instruction of the program, the OS or the like running on the computer performs a part or all of the actual processing, and the function of the above-described embodiment can be realized by the processing.

Further, the program read from the recording medium is written to the memory provided in the function expansion board inserted in the computer or the function expansion unit connected to the computer. After that, based on the instruction of the program, the function expansion board, the CPU provided in the function expansion unit, or the like performs a part or all of the actual processing, and the function of the above-described embodiment is also realized by the processing.

It should be noted that the above-described embodiments are merely examples of embodiment of the present invention, and the technical scope of the present invention should not be construed in a limited manner by these. That is, the present invention can be implemented in various forms without departing from the technical idea or its main features.

101 Machine Tool 102 PLC
103 PLC client terminal 104 Network camera 105 Relay server 106 Recording server 107 Browsing client terminal

Claims (6)

An information processing system that can communicate with an imaging means that captures an image of an object to be imaged corresponding to the error when an error occurs in the monitored area.
An acquisition means for acquiring image data captured by the image pickup means, and
The acquisition means using the image data acquired by the acquisition means and a pattern of changes in the image data obtained by imaging the object to be imaged when the error does not occur in the pattern recognized in advance. Of the image data acquired in, changes different from the above pattern in a specific area according to the content of the error event based on the information indicating the occurrence of the abnormality received from the PLC that detects the error of the object to be imaged. Specific means to identify the location as the cause of the error, and
A control means for controlling a portion specified by the specific means to be zoomed and imaged by the image pickup means, and a control means.
In addition to controlling the display of the image data acquired by the acquisition means, the displayed image data is identifiablely displayed on the display means as a portion related to the cause of the error specified by the specific means. Further, a display control means for controlling the portion related to the cause of the error so as to display the zoom image data captured by zooming with the control means.
An information processing system characterized by being equipped with. Further, it has a PLC client terminal that receives the information indicating the occurrence of an abnormality from the PLC.
The information processing system according to claim 1, wherein when a signal corresponding to the information is received from the PLC client terminal, a specific process is executed by the specific means. The specific means was acquired by the acquisition means using a predetermined area of the image data acquired by the acquisition means and image data obtained by imaging the object to be imaged when the error did not occur. The information processing system according to claim 1, wherein a part having a high possibility of causing an error is specified from the image data. It is characterized by comprising a display control means for controlling so that the image data acquired by the acquisition means and the image data obtained by imaging the object to be imaged are displayed in parallel when the error does not occur. The information processing system according to claim 1 or 3. It is an imaging means that captures an image of a monitored area, and is a control method of an information processing system that can communicate with an imaging means that captures an image to be imaged corresponding to the error when an error occurs in the monitored area.
An acquisition step in which the acquisition means of the information processing system acquires image data captured by the image pickup means,
The change in the image data acquired in the acquisition step and the image data obtained by imaging the object to be imaged when the error does not occur in the pattern recognized in advance by the specific means of the information processing system. Among the image data acquired by the acquisition means using the pattern, in a specific area corresponding to the content of the error event based on the information indicating the occurrence of the abnormality received from the PLC that detects the error of the object to be imaged. A specific step of identifying a part that changes differently from the above pattern as a part related to the cause of the error, and
A control step in which the control means of the information processing system controls the location specified in the specific step to be zoomed and imaged by the image pickup means.
The display control means of the information processing system controls to display the image data acquired in the acquisition step, and at the same time, the cause of the error specified in the display means by the display means on the displayed image data. A display control step of displaying the part related to the error in an identifiable manner, and further controlling the part related to the cause of the error to be zoomed by the control means and displaying the imaged zoom image data.
A control method for an information processing system, characterized in that the information processing system is provided with. An image pickup means that captures an image of a monitored area, and if an error occurs in the monitored area, a program that can be read and executed by an information processing system that can communicate with the image pickup means that captures the image to be imaged corresponding to the error. There,
The information processing system
An acquisition means for acquiring image data captured by the image pickup means, and
The acquisition means using the image data acquired by the acquisition means and a pattern of changes in the image data obtained by imaging the object to be imaged when the error does not occur in the pattern recognized in advance. Of the image data acquired in, changes different from the above pattern in a specific area according to the content of the error event based on the information indicating the occurrence of the abnormality received from the PLC that detects the error of the object to be imaged. Specific means to identify the location as the cause of the error, and
A control means for controlling a portion specified by the specific means to be zoomed and imaged by the image pickup means, and a control means.
In addition to controlling the display of the image data acquired by the acquisition means, the displayed image data is identifiablely displayed on the display means as a portion related to the cause of the error specified by the specific means. Further, a display control means for controlling the portion related to the cause of the error so as to display the zoom image data captured by zooming with the control means.
A program characterized by making it work.

Images