JP7427050B2 - Information processing system, information processing system control method, and program - Google Patents

Description

The present invention relates to an information processing system, an information processing system control method, and a program, and particularly relates to a mechanism that allows a user to easily identify the cause of an error when an error occurs in a monitored area.

Generally, in a production factory, a PLC (Programmable Logic Controller) is used to control various machine tools, and if a problem occurs in the production process, the PLC raises an error alert. Additionally, a network camera is separately installed to monitor the production line, and both may operate as separate systems or as one system.

Patent Document 1 discloses a system for monitoring the production process of products using a network camera. Further, Patent Document 1 discloses that when an alert is issued from a machine tool, a network camera is directed in the direction in which the alert is issued.

Japanese Patent Application Publication No. 2008-17158

By the way, as in Patent Document 1, when an alert is issued from a machine tool, there are cases where it is difficult to identify the cause of the error by simply pointing the network camera in that direction. It was possible. For example, a sensor attached to the arm of a manufacturing device that lifts products and packs them into boxes should have turned on if machine tool 101 had lifted product 301, but it did not turn on. In this case, even if the cause of the error event is due to a screw coming off of the arm of the machine tool 101, the error event or the direction in which the alert occurred can be imaged over a wide range. There was a possibility that this information could not be determined from the image data of the network camera 104.

An object of the present invention is to provide a mechanism that allows a user to easily identify the cause of an error when an error occurs in a monitored area.

The present invention provides an acquisition means for acquiring an image captured by an imaging means for capturing an image of a monitoring target area, and an image acquired by the acquisition unit and a pre-recognized pattern, wherein an error occurs in the monitoring target area. an abnormality occurrence received from an error detection means for detecting an error in the imaged object among the images acquired by the acquisition means using a change pattern of an image of the imaged object when the imaged object is not detected; identification means for identifying a location that changes differently from the pattern as a location related to the cause of the error in a specific area according to the content of an error event based on information indicating the above; and a location identified by the identification means. The invention is characterized by comprising a control means for controlling the zoomed image to be imaged and to display the zoomed and imaged image .

According to the present invention, when an error occurs in a monitored area, it is possible for a user to easily identify the cause of the error.

FIG. 1 is a system configuration diagram showing an example of the configuration of a production process monitoring system of the present invention. 2 is a diagram showing an example of the hardware configuration of an information processing device applicable to the PLC client terminal 103, relay server 105, recording server 106, and viewing client terminal 107 shown in FIG. 1. FIG. FIG. 1 is a diagram showing an example to which the production process monitoring system of the present invention is applied. The figure which shows an example of a network camera database. FIG. 2 is a diagram illustrating an example of a display screen when an image captured by a network camera 104 is displayed on a viewing client terminal 107. The figure which shows an example of the flowchart regarding the whole process in the production process monitoring system of this invention. FIG. 1 is a diagram showing an example to which the production process monitoring system of the present invention is applied. 7 is a diagram illustrating details of the process of step S605 in FIG. 6. FIG. FIG. 2 is a diagram illustrating an example of a display screen when an image captured by a network camera 104 is displayed on a viewing client terminal 107. FIG. 7 is a diagram illustrating details of the process of step S609 in FIG. 6; 7 is a diagram illustrating details of the process of step S612 in FIG. 6. FIG. FIG. 2 is a diagram illustrating an example of a display screen when an image captured by a network camera 104 is displayed on a viewing client terminal 107. 7 is a diagram illustrating details of the process of step S614 in FIG. 6. FIG.

Embodiments of the present invention will be described in detail below with reference to the drawings. Note that the embodiment described below shows an example of a case where the present invention is specifically implemented, and is one of the specific embodiments of the configuration described in the claims.

FIG. 1 shows a production process for monitoring the production process of items flowing on a factory line, including a machine tool 101, a PLC 102, a PLC client terminal 103, a network camera 104, a relay server 105, a recording server 106, and a viewing client terminal 107 according to the present invention. FIG. 1 is a system configuration diagram showing an example of the configuration of a monitoring system.

In this production process monitoring system, each device is communicably connected via a network 108 or a wireless LAN (including a mobile communication network).
The machine tool 101 performs assembly, processing, packaging, etc. of products to be produced.

PLC 102 controls machine tool 101. Further, when an error occurs in the machine tool 101, the error is detected and the processing result of the PLC 102 is transmitted to the PLC client terminal 103. Further, when an abnormality occurs in the machine tool 101, a code indicating the cause of the abnormality is transmitted to the PLC client terminal 103.

The PLC client terminal 103 receives the processing result of the PLC 102 sent from the PLC 102 and a code indicating the cause of the abnormality, and if the received information is a code indicating the cause of the abnormality of the PLC 102, An event indicating an error (also referred to as an error event) is sent to relay server 105.

Note that an error event is a production process monitoring event such as "container takeout robot grip OFF error in production process A (the sensor attached to the hand of the machine tool 101 that takes out the container as a product does not turn ON)." These are various errors that occur on the system.

When the relay server 105 receives an event indicating an error from the PLC client terminal 103, the relay server 105 adjusts the PTZ (pan left and right, tilt up and down, and telephoto or wide angle) of the network camera 104 according to the content of the event. (Zoom) to be adjusted, and sends the determined content to the recording server 106.

The recording server 106 sends a request to acquire image data (also referred to as imaged data) to the network camera 104, acquires and stores the image data captured by the network camera 104, or displays the acquired image on the viewing client terminal 107. or send data. Furthermore, the recording server analyzes the acquired image data and determines whether the PTZ of the network camera 104 needs to be adjusted.

In this embodiment, the relay server 105 and the recording server 106 are separate devices, but in other embodiments, the relay server 105 and the recording server 106 may be the same device.

One or more network cameras 104 are provided for each production process, and take images of each production process. Then, the image data captured by the network camera 104 is transmitted to the recording server 106. Further, according to instructions from the recording server 106, the PTZ of the network camera 104 is adjusted.

The viewing client terminal 107 is used by the user to check the image data when an error event occurred, and displays the image data transmitted from the recording server 106 on a display.
This concludes the explanation of FIG.

Next, using FIG. 2, a hardware configuration of an information processing apparatus applicable to the PLC client terminal 103, relay server 105, recording server 106, and viewing client terminal 107 shown in FIG. 1 will be described.

In FIG. 2, 201 is a CPU that centrally controls each device and controller connected to the system bus 204. In addition, the ROM 202 or external memory 211 contains BIOS (Basic Input/Output System), which is a control program for the CPU 201, an operating system program (hereinafter referred to as OS), and various other programs described below that are necessary to realize the functions executed by the PC. Programs etc. are stored.

203 is a RAM, which functions as the main memory, work area, etc. of the CPU 201; The CPU 201 loads programs and the like necessary for execution of processing from the ROM 202 or the external memory 211 into the RAM 203, and executes the loaded programs to realize various operations.

Further, an input controller 205 controls input from a pointing device such as a keyboard (KB) 209 or the like. A video controller 206 controls the display on a display device such as a display 210 (regardless of whether it is a liquid crystal or a cathode ray tube).

207 is a memory controller that stores boot programs, various applications, font data, user files, editing files, various data, etc. in an external storage device (hard disk (HD)), flexible disk (FD), or PCMCIA card slot. Controls access to external memory 211 such as CompactFlash (registered trademark) memory connected via an adapter.

A communication I/F controller 208 connects and communicates with external devices via a network, and executes communication control processing on the network. For example, communication using TCP/IP is possible.

Note that the CPU 201 enables display on the display 210 by, for example, executing an outline font development (rasterization) process in a display information area in the RAM 203. Further, the CPU 201 allows the user to give instructions using a mouse cursor (not shown) on the display 210.

Various programs to be described later for realizing the present invention are recorded in the external memory 211, and are executed by the CPU 201 by being loaded into the RAM 203 as necessary. Further, configuration files and the like used when executing the above program are also stored in the external memory 211, and detailed explanations thereof will be given later. This concludes the explanation of FIG. 2.

Next, using FIG. 3, it is a diagram showing an example of application of the production process monitoring system of the present invention, and is a diagram of a production line of a certain factory viewed from above.

The production line shown in FIG. 3 consists of three production processes, production process A to production process C, and one network camera 104 is installed for each production process. The network camera 104 normally captures images using a preset determined by the user at the time of installation.

Note that information on the network camera 104 installed on the production line is managed in a network camera database shown in FIG. 4.

Further, a machine tool 101 is provided for each production process, and the machine tool 101 performs assembly, processing, packaging, etc. of the product 301. A product 301 is transported to each production process by a transport platform 302. This concludes the explanation of FIG. 3.
Next, the network camera database will be explained using FIG. 4.

FIG. 4 is a diagram showing an example of a network camera database managed in the external memory 211 of the relay server 105 or the recording server 106. The network camera database includes a network camera name for uniquely identifying each network camera 104. 401, an imaging target 402 indicating the production process being photographed by the network camera 104 indicated by the network camera name 401, and a preset 403 indicating the preset of the network camera 104 indicated by the network camera name 401. This concludes the explanation of FIG. 4.

Next, an example of a display screen when an image captured by the network camera 104 is displayed on the viewing client terminal 107 will be described using FIG. 5.

FIG. 5 is a diagram showing an example of a display screen when an image captured by the network camera 104 is displayed on the display 210 of the viewing client terminal 107.

The display screen 500 includes a production line display area 501 that displays illustrated image data of a production line, a live video area 502 that displays image data (live video) currently being captured by the network camera 104, and an error message in the production process. It consists of a recorded video display area 503 that displays image data (recorded video) when an event occurs, and a list area 504 that displays a list of error events that occurred in the production process.

Image data is displayed in the production line display area 501 so that the user can recognize the position of the network camera 104 currently displaying live video.

The list area 504 includes a date and time 505 indicating the date and time of occurrence of the error event, a name 506 indicating the status of the error (if it is a newly added error event, it is displayed as "new"), a message 507 indicating the content of the error event, and an error. It consists of a source 508 indicating the production process in which the event occurred and the network camera 104 that captures the image of the production process. This concludes the explanation of FIG. 5.
Next, the overall processing in the production process monitoring system of the present invention will be explained using FIG.

FIG. 6 is a diagram showing an example of a flowchart regarding the overall processing in the production process monitoring system of the present invention.

Each step of the flowcharts shown in FIG. 6 and later-described FIGS. 8, 10, 11, and 13 is executed by the CPU of each device.

In step S601, the PLC client terminal 103 receives the processing result of the PLC 102 and a code indicating the cause of the abnormality that is transmitted from the PLC 102, and the received information is a code that indicates the cause of the abnormality of the PLC 102. If the received information is a code indicating the cause of abnormality in the PLC 102 (step S602), an error event is transmitted to the relay server 105 (step S603).

In step S604, upon receiving the error event from the PLC client terminal 103 (step S604), the relay server 105 executes the received error event analysis process (step S605). Details of the process in step S605 will be explained later using FIG. 8.

Note that it is possible to determine from which PLC 102 the error event was received from the sender's IP address. Furthermore, although the PLC 102 has different error event transmission procedures and transmission data formats depending on the manufacturer, the relay server 105 can implement manufacturer-specific reception drivers for the transmission procedures and transmission data formats of multiple PLC manufacturers. , it is possible to absorb differences between manufacturers.

Further, in the present invention, it is possible to specify the approximate location where an error occurs and the content of the error from the error event.

If there are multiple error events, the relay server 105 acquires all the error events in step S604.

Step S604 is an example of an acquisition unit in the present invention that acquires a detection result by a detection unit that detects that an error has occurred in the production process.

In step S606, the relay server 105 adjusts the PTZ (pan that moves left and right, tilt that moves up and down, and zoom that makes telephoto or wide angle) of the network camera 104 determined in the process of step S605. The error event information including the above contents is transmitted to the recording server 106, and an adjustment instruction is given to the network camera 104 via the recording server 106.

In step S606, in the present invention, the determining means causes the other imaging means to image the production process other than the imaging means that images the production process in which the error has occurred to image the production process in which the error has occurred. This is an example of a control means that controls the other imaging means to take an image of the production process in which an error has occurred when it is determined to perform such control.

In step S607, the recording server 106 transmits to the network camera 104 a request to acquire image data captured by the network camera 104.

In step S608, the recording server 106 acquires the image data transmitted from the network camera 104 in response to the image data acquisition request in step S607, and executes processing on the image data (step S609). Details of the process in step S609 will be explained later using FIG. Note that the acquired image data is buffered for a predetermined period of time.

In step S610, the recording server 106 transmits the image data acquired from the network camera 104 to the viewing client terminal 107.

In step S611, the network camera 104 receives a request from the recording server 106, and executes processing according to the request (step S612). Details of the process in step S612 will be explained later using FIG. 11.

In step S613, the viewing client terminal 107 receives the image data transmitted from the recording server 106, and executes a process of displaying the image data on the display 210 (step S614). Details of the process in step S614 will be explained later using FIG. 13.

Step S613 is an example of an acquisition unit according to the present invention that acquires image data captured by the imaging unit when an error occurs in the monitoring target area.
This concludes the explanation of FIG.
Next, details of the process in step S605 in FIG. 6 will be described using FIGS. 7 and 8.

As explained in FIG. 4, the network camera 104 that captures images for each production process is determined, so if an error event occurs in a production process, the network camera 104 that captures images of the production process (imaging target 402) (the network camera indicated by the network camera name 401) images the production process. However, depending on the content of the error, the error state may not be ascertainable only from the image data of the network camera 104.

For example, when an error event occurs in production process A in the state shown in FIG. There is a possibility that the error location cannot be imaged due to a blind spot. Further, even if an error event occurs below the conveyance platform 302 that conveys the product 301, there is a possibility that the error location cannot be imaged.

Therefore, the present invention provides a system that can image an error location even in the above-described case by executing the process shown in FIG.

Specifically, by executing the process shown in FIG. 8, if an error event occurs at a position that is a blind spot of the network camera 104 that images the production process where the error event occurred, the location where the error event occurred is Since the approximate location can be identified from the error event information sent from the PLC client terminal 103, in such a case, the error event will also occur in the network camera 104 that captures images of other production processes, as shown in FIG. decided to image the production process.

FIG. 7 is a diagram showing an example to which the production process monitoring system of the present invention is applied. In the production line shown in FIG. In this diagram, when an error event occurs, the network cameras 104 (camera 2 and camera 3) that take images of other production processes also take images of the production process in which the error event has occurred.

The network camera 104, which captures images of other production processes, also captures images of the production process in which the error event has occurred, so that the camera 1 can also capture images of blind spots.

Furthermore, at this time, the network camera 104 (in the case of FIG. 7, camera 3) that images the production process farthest from the production process in which the error event has occurred is controlled to zoom out (that is, to image a wide range). By doing so, in the present invention, even if an error event that occurs in a production process affects other production processes, the effect can be confirmed from the zoomed-out image data of the network camera 104.
From here, each step in FIG. 8 will be explained.

In step S801, the relay server 105 determines that the error event acquired in step S604 causes the network camera 104, which captures images of production processes other than the production process in which the error event has occurred, to also image the production process in which the error event has occurred. Determine whether it is an error event that should be done. In other words, it is determined whether the error event has occurred at a position that is a blind spot of the network camera 104 that captures images of the production process in which the error event has occurred. If the error event is such that the network camera 104 that images another production process should also image the production process in which the error event occurred, the process moves to step S803, and the network camera 104 that images the other production process If it is not an error event that should cause the production process in which the error event occurred to be imaged, the process moves to step S802.

In step S802, the relay server 105 determines that the network camera 104, which images the production process where the error event occurred, will image the location where the error occurred, and ends this process.

In step S803, the relay server 105 determines whether the error event has occurred in other production processes by acquiring a plurality of error events in step S604 and determining whether each error event occurs in a different production process. . If it is determined that an error event has occurred in another production process, the process moves to step S805; otherwise, the process moves to step S804.

In step S804, the relay server 105 determines that the network camera 104, which captures images of production processes other than the production process where the error occurred, should also capture images of the production process where the error occurred.

Step S804 of the present invention uses the detection result acquired by the acquisition means to select the other imaging means for imaging a production process other than the imaging means for imaging the production process in which an error has occurred. This is an example of a determining means that determines whether to control the production process that has occurred to be imaged.

In step S805, the relay server 105 determines whether an error event occurring in another production process is an error event indicating a serious error based on the content of the error event acquired in step S604. If the error event indicates a serious error, the relay server 105 moves the process to step S806, and if the error event does not indicate a serious error, the process moves to step S804.

In other words, even if an error event occurs in another production process, if it is a minor error event, the network camera 104 that captures images of other production processes other than the production process in which the error occurred will also It is decided that the image will be taken.

The user can determine whether an error event is a serious error event by setting a weight for each error event, and if the weight value exceeds a reference value, it can be determined that the error event is a serious error.

The serious error mentioned here is, for example, a case where the machine tool 101 malfunctions and becomes uncontrollable, or a case where the machine tool 101 completely stops.

In step S806, the relay server 105 determines that each network camera 104 is to image the production process that was originally the imaging target (imaging target 402).

Note that in step S801, even if an error event occurs in a position that is a blind spot of the network camera 104 that images the production process where the error event occurred, if it is a minor error event, the network camera 104 that images the other production process is It is not necessary to image the production process in which the error occurred. This concludes the explanation of FIG. 8.
Next, details of the process in step S609 in FIG. 6 will be described using FIGS. 9 and 10.

As explained above, when the relay server 105 receives an error event from the PLC client terminal 103, the network camera 104, which targets the production process in which the error event has occurred, images the production process. In some cases, it is difficult for a user to identify the cause of an error by simply visually checking the content of the error event and the image data of the network camera 104 that captures images of the production process using preset settings. exist. For example, a sensor attached to the arm of the machine tool 101 that lifts the product 301 and packs it in a box should have turned on if the machine tool 101 had lifted the product 301, but it did not turn on. Even if the cause of the error event is due to a part of the arm of the machine tool 101 being dislodged, the error event or the image capturing using a preset preset cannot be performed. There was a possibility that this information could not be determined from the image data of the network camera 104.

Therefore, in the present invention, image data when the machine tool 101 is operating correctly in the production process (normal image data), which is managed by the recording server 106 in advance, is read into the Plug-in application. , the pattern of changes in image data over a certain period of time during normal times (the pattern of changes in image data when the machine tool 101 is operating correctly) is recognized, and the relevant production process when an error event occurs is imaged. Among the changes in the image data, changes that are different from the recognized change pattern of the image data are detected. Then, the user instructs the network camera 104 to zoom in and capture an image of a position in real space corresponding to the portion where a different change has been detected. By checking the image data captured by the zoomed network camera 104, the user can check the location that caused the error event in more detail.

Furthermore, in the present invention, when detecting a change in an image, a rectangle is set in advance at a rough position in the image data that is likely to be the cause of the error, based on the contents of the error event, and the rectangle is By detecting whether the change in the image data inside is different from the recognized image data, the processing time required to identify the change can be shortened and the load on the recording server 106 can be reduced.
Note that the rectangle is set for a location where a problem is expected to occur based on the content of the error event.
FIG. 9 shows an image of the result of performing the processing up to this point.

FIG. 9 is a diagram showing an example of a display screen when image data captured by the network camera 104 is displayed on the display 210 of the viewing client terminal 107.

In FIG. 9, a rectangle 901 is set at a rough position in the image data in the recorded video display area 503 that is likely to be the cause of an error. In the example of FIG. 9, within the rectangle 901, a frame 902 is identified as a location where the change is different from the pattern of change over a certain period of time in the image data taken of the production process during normal times that has been recognized in advance. Indicates the state in which Thereafter, when the network camera 104, which has been instructed by the recording server 106 to zoom in on the area in the frame 902, performs zoom imaging, the zoomed image data (zoom image data 903) is displayed in the recorded video display area 503. be done. In addition, as another embodiment, by temporarily displaying image data of a production process captured during normal operation in the production line display area 501 or live video area 502, the production process when an error event occurs can be displayed. It is also possible to enable the user to compare the difference in change with the captured image data (image data displayed in the recorded video display area 503).

In step S1001, the recording server 106 determines whether an error event has been received. If the recording server 106 receives an error event, the process moves to step S1004, and if not, the process moves to step S1002-1.

In step S1002-1, the recording server 106 transmits the image data acquired from the network camera 104 in step S608 to the viewing client terminal 107 in order to display the image data in the live video area 502 of the viewing client terminal 107, and then If an error event is not received in step S1001 while image data is buffered for a predetermined period of time, the image data is deleted in step S1002.

In step S1003, the recording server 106 determines whether a termination instruction has been received from the user. If the termination instruction has been received, the process ends; otherwise, the process returns to step S1001.

In step S1004, the recording server 106 instructs the network camera 104 to operate from the relay server 105 (specifically, the network camera 104 that captures images of production processes other than the production process in which the error occurred in step S804 also It is determined whether an instruction to image the production process that has occurred has been received. If an operation instruction for the network camera 104 has been received from the relay server 105, the received operation instruction is transmitted to the network camera 104 (step S1005); otherwise, the process moves to step S1006.

In step S1006, the recording server 106 stores (records) image data of a predetermined time period before and after the occurrence of the error event in the external memory 211 among the buffered image data.

In step S1007, the recording server 106 identifies a pattern of changes over a certain period of time in image data captured during normal times, which has been recognized in advance, among the changes in the image data captured during the production process when the error event occurs. Identify areas that are changing differently.

Step S1007 uses the image data acquired by the acquisition means of the present invention and the image data obtained by capturing the imaged object when the error has not occurred to obtain the image data acquired by the acquisition means. This is an example of a method for identifying a location that is likely to have caused an error.

In step S1008, the recording server 106 determines whether zooming of the network camera 104 is necessary as a result of the process in step S1007. If zooming of the network camera 104 is necessary, the process moves to step S1009; otherwise, the process moves to step S1001.

In step S1009, the recording server 106 sends a request to the network camera 104 to zoom in and capture an image of a position in the production process corresponding to the location identified as undergoing a different change in step S1007. Thereafter, the network camera 104 zooms and captures image data, which is then stored (recorded) in the external memory 211.

Step S1009 is an example of a control means in the present invention that controls the imaging means to zoom and image the location specified by the identification means.
This concludes the explanation of FIG. 10.
Next, details of the process in step S612 in FIG. 6 will be described using FIG. 11.

In step S1101, the network camera 104 determines what the request received in step S611 is for. If it is a request to send image data captured by the network camera 104, the image data is sent to the recording server 106 in step S1102, and if it is an instruction to change the preset position, it is changed to the supported preset position in step S1103, and zoomed in. , or if the instruction is to zoom out, zoom in or zoom out is executed; if the instruction is any other instruction, the operation according to the content of the instruction is executed; if the instruction is to terminate, this process is ended. This concludes the explanation of FIG. 11.
Next, details of the process in step S614 in FIG. 6 will be described using FIGS. 12 and 13.

As explained in FIG. 5, when an error event occurs in the production process, the viewing client terminal 107 displays the image data (recorded video) at the time the error event occurred in the recorded video display area 503. The image data (recorded video) when the event occurred is displayed in the recorded video display area 503, and when the next error event occurs while the user is viewing the image data of the error event, the next error event is displayed in the recorded video display area 503. Since the image data at the time when the error event occurred is displayed, the user's viewing of the image data of the error event that had been viewed up to that point was interrupted.

Therefore, if the user wants to continue viewing the image data of the error event whose viewing was interrupted, the user searches for the corresponding error event from the list area 504 and displays it in the recorded video display area 503, and then goes back to the point where the viewing was interrupted. It was necessary to send image data, and the operation was very complicated.

Therefore, in the present invention, as shown in FIG. 12, if the next error event occurs while the user is viewing the image data of the error event, the image data when the next error event occurs is displayed as a recorded video. A pop-up 1201 is displayed on the display screen 500 for the user to select whether to display it in the area 503, and when the YES button 1202 is selected in accordance with the user's operation, the recorded image of the image data when the next error event occurs is displayed. area 503, and when the NO button 1203 is selected according to the user's operation, the image data when the current error event occurred is continuously displayed in the recorded video display area 503, thereby saving the user's complicated operations. Reduce.
Now, each step in FIG. 13 will be explained.

In step S1301, the viewing client terminal 107 causes the image data (live video) acquired by the recording server 106 from the network camera 104 to be displayed in the live video area 502.

In step S1302, the viewing client terminal 107 determines whether image data of an error event has been received from the recording server 106. If the image data of the error event is received from the recording server 106, the process moves to step S1304, and if the image data of the error event is not received from the recording server 106, it is determined whether a termination instruction has been received from the user ( Step S1303), if the termination instruction is received, this process is terminated.

In step S1304, the viewing client terminal 107 determines whether the image data when the error event occurred is currently being displayed in the recorded video display area 503. If the image data when the error event occurred is currently displayed in the recorded video display area 503, the process moves to step S1306, and the image data when the error event occurred is displayed in the currently recorded video display area 503. If not displayed, the received image data is displayed in the recorded video display area 503 in step S1305.

In step S1304, in the present invention, when the image data is acquired by the acquisition unit, the image data captured by the image pickup unit when another error has already occurred is controlled to be displayed by the display control unit. This is an example of a determination means for determining whether the

Further, step S1305 is an example of a display control unit in the present invention that controls to display the image data acquired by the acquisition unit.

In step S1306, the viewing client terminal 107 determines whether the error event in which the image data was received from the recording server 106 is a serious error.If it is a serious error, in step S1309, the viewing client terminal 107 forcibly transmits the received image data. To allow the user to select whether to display the image data in the recorded video display area 503 and, if not, display the image data when the next error event occurs, as shown in FIG. 12, in step S1307. A pop-up 1201 is displayed on the display screen 500.

In step S1307, in the present invention, the determination means determines that the display control means is controlling to display the image data captured by the imaging means when another error has already occurred. This is an example of a notification means for notifying the conditions.

Note that the serious error mentioned here is, for example, a case where the machine tool 101 malfunctions and becomes uncontrollable, or a case where the machine tool 101 completely stops.

In step S1308, the viewing client terminal 107 determines whether the YES button 1202 of the pop-up 1201 displayed in step S1307 has been selected according to the user's operation. When the YES button 1202 is selected according to the user's operation, the process moves to step S1309, and when the NO button 1203 is selected according to the user's operation, the process moves to step S1310, where the current error event occurs. The image data is continuously displayed in the recorded video display area 503.

In step S1311, the viewing client terminal 107 determines whether a predetermined period of time has passed since the NO button 1203 was selected in accordance with the user's operation, or when the current error event that is being continuously displayed in the recorded video display area 503 occurs. When the reproduction of the image data is completed, the process moves to step S1309.
This concludes the explanation of FIG. 13.

As described above, according to the present invention, when an error occurs in the monitored area, it is possible for the user to easily identify the cause of the error.

The present invention can be implemented as a system, an apparatus, a method, a program, a storage medium, etc., and specifically, it may be applied to a system composed of a plurality of devices. It may be applied to a device consisting of two pieces of equipment.

Note that the present invention includes a system or device in which a software program that implements the functions of the embodiments described above is supplied directly or remotely. The present invention also includes a case where the present invention is achieved by the computer of the system or device reading and executing the supplied program code.

Therefore, in order to realize the functional processing of the present invention on a computer, the program code itself installed in the computer also realizes the present invention. That is, the present invention also includes the computer program itself for realizing the functional processing of the present invention.

In that case, as long as it has the function of a program, it may be in the form of an object code, a program executed by an interpreter, script data supplied to the OS, or the like.

Examples of recording media for supplying programs include flexible disks, hard disks, optical disks, magneto-optical disks, MOs, CD-ROMs, CD-Rs, and CD-RWs. There are also magnetic tapes, nonvolatile memory cards, ROMs, DVDs (DVD-ROM, DVD-R), and the like.

Another method of supplying the program is to connect to an Internet home page using a browser on a client computer. It can also be supplied by downloading the computer program of the present invention itself or a compressed file containing an automatic installation function from the homepage to a recording medium such as a hard disk.

It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, the present invention also includes a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer.

In addition, the program of the present invention is encrypted, stored on a storage medium such as a CD-ROM, and distributed to users, and users who meet certain conditions can download the key information for decryption from a homepage via the Internet. let It is also possible to execute the encrypted program by using the downloaded key information and install it on the computer.

Further, the functions of the embodiments described above are realized by the computer executing the read program. In addition, based on instructions from the program, an OS running on the computer performs part or all of the actual processing, and the functions of the embodiments described above can also be realized by this processing.

Furthermore, the program read from the recording medium is written into a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Thereafter, based on the instructions of the program, the CPU included in the function expansion board or function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are also realized by this processing.

The embodiments described above are merely examples of implementation of the present invention, and the technical scope of the present invention should not be construed as limited by these embodiments. That is, the present invention can be implemented in various forms without departing from its technical idea or main features.

101 Machine tools 102 PLC
103 PLC client terminal 104 Network camera 105 Relay server 106 Recording server 107 Viewing client terminal

Claims (8)

an acquisition unit that acquires an image captured by an imaging unit that captures an image of a monitoring target area;
The acquisition is performed by using the image acquired by the acquisition means and a pattern recognized in advance of a change in an image of the object to be imaged when no error occurs in the monitoring target area. A change different from the pattern is detected in a specific area of the image acquired by the means according to the content of an error event based on information indicating the occurrence of an abnormality received from an error detection means for detecting an error in the imaged object. identification means for identifying the location where the error occurs as a location related to the cause of the error;
Display control means for controlling to zoom and image the location specified by the identification means and to display the zoomed and imaged image ;
An information processing system comprising: The information processing system according to claim 1, wherein the error detection means is a PLC. 3. The specifying means executes specifying processing when a signal corresponding to the information is received from a PLC client terminal that receives the information indicating the occurrence of an abnormality from the PLC. Information processing system. The identifying means uses a predetermined area of the image acquired by the acquiring means and an image of the object to be imaged when the error does not occur to determine the area in the image acquired by the acquiring means. 4. The information processing system according to claim 1, wherein the information processing system identifies a location that is likely to have caused an error. The display control means is characterized in that it performs control so that the image acquired by the acquisition means and the image taken of the imaged object when the error does not occur are displayed in parallel. The information processing system according to any one of claims 1 to 4. The display control means displays the image acquired by the acquisition means, and performs control so as to display a portion of the displayed image that is associated with the cause of the error identified by the identification means so as to be identifiable. The information processing system according to any one of claims 1 to 5, characterized by: an acquisition step of acquiring an image captured by an imaging means that captures an image of the monitoring target area;
The acquisition is performed using the image acquired in the acquisition step and a pattern of change in an image of the object to be imaged when no error occurs in the monitoring area, which is a previously recognized pattern. Among the images acquired in step, a change different from the pattern occurs in a specific area according to the content of an error event based on information indicating the occurrence of an abnormality received from an error detection means for detecting an error in the imaged object. an identification step of identifying a location related to the cause of the error as a location related to the cause of the error;
a display control step of controlling to zoom and image the location identified in the specific step and display the zoomed and imaged image ;
A method for controlling an information processing system, comprising: A program for causing at least one computer to function as each means of the information processing system according to any one of claims 1 to 6 .

Images