KR102482899B1 - Method for providing industrial internet of things based smart factory remote control service - Google Patents

Abstract

A server for providing a method and a device for providing an industrial Internet of things (IIOT)-based remote control service for a smart factory can obtain programmable logic controller (PLC) data and log data of Internet of things (IoT) devices from the IoT devices. The server can obtain physical sensing data from IoT sensors. The server can obtain operation images of the IoT devices from closed-circuit television (CCTV) terminals. The server can learn error situations that occur in the IoT devices based on the PLC data and the log data through machine learning. The server can determine that a first error situation occurs in a first IoT device among the IoT devices based on the PLC data and the log data. The server can control the first IoT device according to a first preset scenario according to the occurrence of the first error situation The server can provide a user terminal with information on the first error situation and a first image of the first IoT device recorded by the CCTV terminals during the time when the first error situation has occurred in the first IoT device. Therefore, users can obtain state information and CCTV image information of each device in real time.

Description

IIOT-based smart factory remote control service provision method {METHOD FOR PROVIDING INDUSTRIAL INTERNET OF THINGS BASED SMART FACTORY REMOTE CONTROL SERVICE}

Embodiments of the present invention relate to a method and apparatus for providing a smart factory remote control service based on industrial internet of things (IIoT).

Recently, as interest in the 4th industrial revolution has increased, the applied systems for the manufacturing field are also diverse in the types and meanings of numerous systems and targets such as 'smart manufacturing' and 'smart factory'.

A smart factory is an intelligent production plant that improves productivity, quality, and customer satisfaction by applying information and communication technology (ICT) to the production process, such as design, development, manufacturing, and distribution. Smartization of the manufacturing industry is still lacking in materialized entities, and information on smart factories and guides for construction are not accurate. In addition, smart factory construction should be carried out appropriately for the purpose and size of the company, but many problems arise as existing cases apply foreign cases centered on advanced manufacturing countries such as Germany as they are. Domestic companies' understanding of smart factories is still insufficient, and the goal of leading manufacturing innovation by building 30,000 smart factories by 2022 is approaching the site unrealistically due to the burden of initial construction costs. In addition, while small and medium-sized manufacturing companies recognize that various manufacturing data used in the production process are very closely related to the quality of products, many companies are facing today's changes due to difficulties in integrated management of manufacturing data and various inconveniences for production workers. are reluctant On the other hand, global companies that actively embrace the 4th industry build new data-driven services by building industrial data platforms and virtual environments that enable data connection and exchange under common governance rules and data structures preferred by companies. and leading a new ecosystem that creates innovative business processes. Many other companies are already collecting and storing various manufacturing data generated at the manufacturing site, and are making efforts to increase productivity and increase efficiency in various factory automation systems based on the analyzed data.

Prior to the final introduction of smart factories (unmanned automatic production factories through EQP/IIoT/Big Data/Predictive Analysis), technology to minimize, optimize, increase efficiency, and unmanned production control tasks by establishing an integrated production process control environment need. In other words, methods are being sought to optimize manpower and improve line cleanliness through unmanned production lines, and to achieve immediate response to failures, improved equipment operation rates, reduced production losses, and increased production efficiency through centralized production line integration.

Embodiments of the present invention may provide a method for providing an IIoT-based smart factory remote control service, a device providing server, and an operation method thereof.

Technical tasks to be achieved in the embodiments are not limited to those mentioned above, and other technical tasks not mentioned will be considered by those skilled in the art from various embodiments to be described below. can

According to embodiments, in the IIoT-based smart factory remote control service providing method performed by at least one server, PLC (programmable logic controller) data and logs of the IoT facilities from Internet of Things (IoT) facilities. acquiring data; Obtaining physical sensing data from IoT sensors; Obtaining operation images of the IoT facilities from closed-circuit television (CCTV) terminals; Learning error situations occurring in the IoT facilities based on the PLC data and the log data through machine learning; Determining that a first error situation occurs in a first IoT device among the IoT devices based on the PLC data and the log data; controlling the first IoT device according to a first scenario according to the occurrence of the first error situation; and providing, to a user terminal, the first error situation information and a first video of the first facility recorded by the CCTV terminal during a time when the first error situation occurred in the first IoT facility. there is.

Here, the step of mapping, by the server, a first operation of the first equipment in the first image to the first error situation; Determining that a second error situation occurs in a second IoT device among the IoT devices based on the PLC data and the log data; controlling the second IoT device according to a second scenario according to the occurrence of the second error situation; Providing, to the user terminal, the second error situation information and a second image of the second IoT device recorded by the CCTV terminal during a time when the second error situation occurred in the first IoT device; and mapping a second operation of the second IoT device in the second image to the second error situation.

Here, the step of obtaining, by the server, a third image including the second operation of the second IoT facility from the CCTV terminal; Checking the PLC data and the log data for the second IoT facility; recognizing occurrence of the second error situation based on the third image; Denying occurrence of the second error situation based on the PLC data and the log data for the second IoT device; providing the user terminal with a message about the possibility of occurrence of the second error situation based on the third image and the third image; To the user terminal, based on the PLC data and the log data and the third image for the second IoT facility, providing a message about the possibility of manipulating the PLC data and the log data for the second IoT facility step; and blocking a network connected to the second IoT facility.

Here, the step of mapping, by the server, the physical sensing data related to the second IoT facility obtained from the IoT sensors during the time when the second error situation occurred in the second IoT facility to the second error situation. can include more.

Here, the step of obtaining, by the server, a fourth image including the second operation of the second IoT facility from the CCTV terminal; recognizing occurrence of the second error situation based on the fourth image; Recognizing occurrence of the second error situation based on the physical sensing data related to the second IoT facility obtained from the IoT sensors; Denying occurrence of the second error situation based on the PLC data and the log data for the second IoT device; providing the user terminal with a message about occurrence of the second error situation based on the physical sensing data related to the fourth image and the second IoT device; providing the user terminal with a message about manipulation of the PLC data and the log data for the second IoT facility; and blocking a network connected to the second IoT facility.

According to the embodiments, the user can acquire status information and CCTV image information of each facility in real time, and can check PLC data, log data, physical sensing data, and CCTV image information when a problem occurs in the facilities. When a problem occurs in the equipment, the server can automatically solve the problem and report the occurrence of the problem to the user.

Effects obtainable from the embodiments are not limited to the effects mentioned above, and other effects not mentioned are clearly derived and understood by those skilled in the art based on the detailed description below. It can be.

BRIEF DESCRIPTION OF THE DRAWINGS Included as part of the detailed description to aid understanding of the embodiments, the accompanying drawings provide various embodiments and, together with the detailed description, describe technical features of the various embodiments.
1 is a system diagram including an IIoT-based smart factory remote control service providing method and a device providing server according to an embodiment of the present invention.
2 is a block diagram showing the configuration of a server providing a method and apparatus for providing an IIoT-based smart factory remote control service according to an embodiment of the present invention.
3 is a diagram showing the structure of a smart IIoT system according to an embodiment of the present invention.
4 is a diagram illustrating a smart IIoT system structure according to an embodiment of the present invention.
5 is a diagram illustrating an embodiment of a method of operating an IIoT-based smart factory remote control service providing server.
6 is a diagram showing the configuration of a smart IIoT system according to an embodiment of the present invention.
7 is a diagram illustrating a smart IIoT system architecture according to an embodiment of the present invention.
8 is a diagram illustrating a smart IIoT data analysis UI construction method according to an embodiment of the present invention.
9 is a diagram illustrating a smart IIoT-based AI remote integrated control system R & D method according to an embodiment of the present invention.

The following embodiments combine elements and features of the embodiments in a predetermined form. Each component or feature may be considered optional unless explicitly stated otherwise. Each component or feature may be implemented in a form not combined with other components or features. In addition, various embodiments may be configured by combining some components and/or features. The order of operations described in various embodiments may be changed. Some components or features of one embodiment may be included in another embodiment, or may be replaced with corresponding components or features of another embodiment.

In the description of the drawings, procedures or steps that may obscure the gist of various embodiments are not described, and procedures or steps that can be understood by those skilled in the art are not described. did

Throughout the specification, when a part is said to "comprising" or "including" a certain element, it means that it may further include other elements, not excluding other elements, unless otherwise stated. do. In addition, terms such as “… unit”, “… unit”, and “module” described in the specification mean a unit that processes at least one function or operation, which is hardware or software or a combination of hardware and software. can be implemented as Also, “a or an”, “one”, “the” and like terms are used herein in the context of describing various embodiments (particularly in the context of the claims below). Unless otherwise indicated or clearly contradicted by context, both the singular and the plural can be used.

Hereinafter, embodiments according to various embodiments will be described in detail with reference to the accompanying drawings. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The detailed description set forth below in conjunction with the accompanying drawings is intended to describe exemplary embodiments of various embodiments, and is not intended to represent a single embodiment.

In addition, specific terms used in various embodiments are provided to help understanding of various embodiments, and the use of these specific terms may be changed into other forms without departing from the technical spirit of various embodiments. .

Referring to FIG. 1, the IIoT-based smart factory remote control service providing method and device providing method according to an embodiment of the present invention have a storage space such as a PC (Personal Computer), can be connected to the Internet, and are not easy to carry. It may be performed on a computing device or a portable terminal such as a smart phone. At this time, the method of providing the IIoT-based smart factory remote control service and device is downloaded from the App store, etc. It can be executed after being installed on a portable terminal.

In addition, the IIoT-based smart factory remote control service providing method and device method is inserted into a computing device such as a PC in a state recorded on a recording medium such as a CD (Compact Disc) or USB (Universal Serial Bus) memory, and the computing device It may be performed through an access operation of, or may be performed through an access operation of the computing device after being stored in the storage space of the computing device from the recording medium.

On the other hand, when the computing device or portable terminal can access a server connected to the Internet, the IIoT-based smart factory remote control service providing method and device providing method can also be executed on the server according to the request of the computing device or portable terminal.

Hereinafter, the IIoT-based smart factory remote control service providing method and device provision method collectively refer to a computing device, a portable terminal, or a server on which the method is executed. It can be referred to as a method and device providing device.

The “IIoT-based smart factory remote control service providing method and device providing device” may have the same configuration as the “IIoT-based smart factory remote control service providing method and device providing device” illustrated in FIG. 2, and the “IIoT-based smart factory remote The control service providing method and device providing device may not be limited to the IIoT-based smart factory remote control service providing method and device shown in FIG. 1 .

A system according to an embodiment includes a user terminal 100, a CCTV 110, facilities 120 and 130, and an IIoT-based smart factory remote control service providing method and device providing server 200 (hereinafter, server 200) ) may be included. The network may include an internet portal site server, an SNS server, a server operating a blog, and the like.

The user terminal 100, the CCTV 110, and the facilities 120 and 130 include a smart phone, a tablet PC, a PC, a smart TV, a mobile phone, a personal digital assistant (PDA), a laptop, a media player, a micro server, and a global GPS (GPS). positioning system) device, e-reader, digital broadcast terminal, navigation, kiosk, MP3 player, digital camera, home appliance, camera-equipped device, and other mobile or non-mobile computing devices, but is not limited thereto. In addition, the user terminal 100, the CCTV 110, and the facilities 120 and 130 may be wearable devices having a communication function and a data processing function. However, it is not limited thereto.

The server 200 is a computer device or a plurality of computer devices that communicate with the user terminal 100, CCTV 110, facilities 120 and 130 through a network to provide commands, codes, files, contents, services, and the like. can be implemented as

For example, the server 200 may provide a file for installing an application to the user terminal 100, CCTV 110, and facilities 120 and 130 accessed through a network. In this case, the user terminal 100, the CCTV 110, and the facilities 120 and 130 may install applications using files provided from the server 200.

In addition, the user terminal 100, the CCTV 110, and the facilities 120 and 130 access the server 200 under the control of an operating system (OS) and at least one program (for example, a browser or an installed application). Thus, the service or content provided by the server 200 can be provided.

As another example, the server 200 may establish a communication session for data transmission and reception, and route data transmission and reception between the user terminal 100, the CCTV 110, and the facilities 120 and 130 through the established communication session.

The user terminal 100, the CCTV 110, the facilities 120 and 130, and the IIoT-based smart factory remote control service providing method and device providing server 200 may perform communication using a network. For example, the network may include a Local Area Network (LAN), a Wide Area Network (WAN), a Value Added Network (VAN), a mobile radio communication network, a satellite communication network, and any of these It is a comprehensive data communication network that includes mutual combinations and enables each network constituent entity shown in FIG. 1 to communicate smoothly with each other, and may include wired Internet, wireless Internet, and mobile wireless communication network. In addition, wireless communication, for example, wireless LAN (Wi-Fi), Bluetooth, Bluetooth low energy (Bluetooth low energy), Zigbee, WFD (Wi-Fi Direct), UWB (ultra wideband), infrared communication (IrDA, infrared Data Association), NFC (Near Field Communication), etc. may be present, but are not limited thereto.

2 is a block diagram showing the configuration of a server providing a method and apparatus for providing an IIoT-based smart factory remote control service according to an embodiment of the present invention.

Referring to FIG. 2, a server 200 (hereinafter referred to as server 200) providing a method and apparatus for providing IIoT-based smart factory remote control service includes a communication unit 210, a processor 220, and a DB 230. can do. In the server 200 of FIG. 2 , only components related to the embodiment are shown. Accordingly, those skilled in the art can understand that other general-purpose components may be further included in addition to the components shown in FIG. 2 .

The communication unit 210 may include one or more components that enable wired/wireless communication between terminals. For example, the communication unit 210 may include at least one of a short-range communication unit (not shown), a mobile communication unit (not shown), and a broadcast reception unit (not shown).

For example, a request generated according to a program code stored in a recording device such as the DB 230 may be transmitted to a terminal through a network under the control of the communication unit 210 . Conversely, control signals, commands, contents, files, etc. provided under the control of the processor of the terminal may be received by the server 200 through the communication unit 210 via the network. For example, control signals, commands, contents, and files of the server 200 received through the communication unit 210 may be transferred to the processor 220 or transferred to the DB 230 and stored therein.

The DB 230 is hardware for storing various data processed in the server 200, and may store programs for processing and controlling the processor 220.

The DB 230 includes random access memory (RAM) such as dynamic random access memory (DRAM) and static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and CD-ROM. ROM, Blu-ray or other optical disk storage, hard disk drive (HDD), solid state drive (SSD), or flash memory. The DB 230 may also be referred to as a memory.

The processor 220 controls the overall operation of the server 200. For example, the processor 220 may generally control an input unit (not shown), a display (not shown), the communication unit 210, and the DB 230 by executing programs stored in the DB 230. The processor 220 may control the operation of the external server 200 by executing programs stored in the DB 230 .

The communication unit 210 may include one or more components that allow the server 200 to communicate with other devices (not shown) and servers (not shown). The other device (not shown) may be a computing device such as the server 200 or a sensing device, but is not limited thereto. The communication unit 210 may receive a user input from another electronic device or data stored in an external device from an external device through a network.

The DB 230 may store programs for processing and controlling the processor 220 . For example, the DB 230 may store instructions for providing services. Also, the DB 230 may store data generated by the processor 220 . For example, the DB 230 may store information related to an IIoT-based smart factory remote control service providing method and device provided by the processor 220 . The DB 230 may store information input to or output from the server 200 .

The processor 220 may include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, and microcontrollers. It may be implemented using at least one of micro-controllers, microprocessors, and electrical units for performing other functions.

The DB 230 may store at least one command executed through the processor 220 . The at least one instruction obtains, from internet of things (IoT) facilities, programmable logic controller (PLC) data and log data of the IoT facilities; Obtain physical sensing data from IoT sensors; obtaining operation images of the IoT facilities from closed-circuit television (CCTV) terminals; Learning error situations occurring in the IoT facilities based on the PLC data and the log data through machine learning; determining that a first error situation occurs in a first IoT device among the IoT devices based on the PLC data and the log data; controlling the first IoT device according to a first scenario according to a preset occurrence of the first error situation; And it may be set to provide the user terminal with the first error situation information and the first video of the first facility recorded by the CCTV terminal during the time when the first error situation occurred in the first IoT facility.

Here, the at least one command maps a first operation of the first facility in the first image to the first error condition; determining that a second error situation occurs in a second IoT device among the IoT devices based on the PLC data and the log data; Control the second IoT device according to a preset second scenario according to occurrence of the second error situation; providing the user terminal with the second error situation information and a second image of the second IoT device recorded by the CCTV terminal during a time when the second error situation occurred in the second IoT device; And it may be further set to map the second operation of the second IoT device in the second image to the second error situation.

Here, the at least one command obtains, from the CCTV terminal, a third image including the second operation of the second IoT facility; check the PLC data and the log data for the second IoT facility; recognizing occurrence of the second error situation based on the third image; denying occurrence of the second error situation based on the PLC data and the log data for the second IoT device; providing the user terminal with a message about the possibility of occurrence of the second error situation based on the third image and the third image; To the user terminal, based on the PLC data and the log data and the third image for the second IoT facility, a message about the manipulation possibility of the PLC data and the log data for the second IoT facility is provided, ; And it may be further set to block the network connected to the second IoT facility.

Here, the at least one command maps the physical sensing data related to the second IoT facility acquired from the IoT sensors during the time when the second error situation occurred in the second IoT facility to the second error situation. can be further set to

Here, the at least one command obtains, from the CCTV terminal, a fourth image including the second operation of the second IoT facility; recognizing occurrence of the second error situation based on the fourth image; recognizing occurrence of the second error situation based on the physical sensing data related to the second IoT facility obtained from the IoT sensors; denying occurrence of the second error situation based on the PLC data and the log data for the second IoT device; providing the user terminal with a message about occurrence of the second error situation based on the physical sensing data related to the fourth image and the second IoT device; providing the user terminal with a message about manipulation of the PLC data and the log data for the second IoT facility; And it may be further set to block the network connected to the second IoT facility.

3 is a diagram showing the structure of a smart IIoT system according to an embodiment of the present invention.

Referring to FIG. 3, the server can configure real-time monitoring/control/DB by collecting real-time sensing/PLC/log data of the most basic equipment in the manufacturing plant. The smart IIoT system can easily and quickly standardize and collect various facility status data that are generated and lost in real time, rather than standardized data linked with upper systems such as MES/FDC/CIM.

If the facility is connected only to a physical network or a virtual network (for example, Packet Exchanger technology), each unstandardized data can be collected and processed into purified data in a certain form and interlocked with the upper system.

Because it is processed based on unstructured data, it is possible to collect and process any type of data, and it is not necessary to change the DB configuration or program to collect additional equipment data, and it is possible to collect end-of-end data only by setting the environment.

The server analyzes the collected data and provides Artificial Intelligence Control, not Human Control, so that the facility can be operated in optimal conditions, providing an integrated Smart Factory solution that supports the operation of the facility at its best performance.

4 is a diagram illustrating a smart IIoT system structure according to an embodiment of the present invention.

Referring to FIG. 4, the Smart Factory IIoT system can configure a real-time monitoring/control/data lake by collecting Sensing/PLC/Log data of the most basic facility in the manufacturing process. The Smart Factory IIoT system can support high-purity basic data collection by providing a user UI for adding, changing, and processing collected data.

PLC X Factor Value and EQP Log Data can be collected into the IoT server through TM Gate & NQ File Manager, and a system that interlocks with MES/FDC can be configured by I/F with external systems.

5 is a diagram illustrating an embodiment of a method of operating an IIoT-based smart factory remote control service providing server.

Referring to FIG. 5, an IIoT-based smart factory remote control service providing server (hereinafter, a server) may obtain PLC data and log data (S500). For example, the server may obtain programmable logic controller (PLC) data and log data of Internet of Things (IoT) facilities.

For example, the server may obtain status information of the IoT facilities based on PLC data and log data of the IoT facilities.

For example, the server can collect data by designating the PLC and collected data in the form of a tag and recognizing it as an IoT logical sensor. The server can process and control the TM Tag data collected in the IoT server through the Rule engine. TM Tag data can be used to develop a customized application in conjunction with the 3rd Part application. Server can perform PLC Data Read/Write through PLC Tag (special drawing function can be provided). In addition to PLC, the server can standardize and collect and process various unstructured data such as log files, DB (MDB, My-SQL, MS-SQL, Oracle, etc.), and communication protocols. The server can reliably transmit large amounts of data at low volume and high speed by using Event Driven Message Queue. The server can broadcast facility real-time data through Topic. Even if the collected data is added, deleted, or changed, it may not be necessary to change the DB structure or change the data processing server process.

The server may acquire physical sensing data (S510). For example, the server may obtain physical sensing data from IoT sensors.

For example, IoT sensors may be included in IoT facilities or may be separately installed near IoT facilities. IoT sensors may perform physical sensing (eg, sensing of temperature, humidity, noise, motion, light, etc.).

For example, IoT sensors may provide physical sensing data acquired through physical sensing to a server, and the server may obtain physical sensing data from IoT sensors.

The server may obtain CCTV images (S520). For example, the server may acquire operation images of the IoT facilities from closed-circuit television (CCTV) terminals.

For example, CCTV terminals may be separately installed near IoT facilities. For example, one CCTV terminal may be installed in a location capable of photographing the operation of one IoT facility.

For example, each CCTV terminal may provide the server with an operation image of each IoT facility.

The server may learn the error situation (S530). For example, the server may learn error conditions occurring in the IoT facilities based on the PLC data and the log data through machine learning.

In other words, when an error occurs in IoT facilities, the server can learn what aspects PLC data and log data of IoT facilities show when an error occurs, and by mapping the PLC data and log data aspects for each error, the PLC Based on the data and log data, it is possible to determine which error occurred.

The server may determine the occurrence of an error (S540). For example, the server may determine that a first error situation occurs in a first IoT device among the IoT devices based on the PLC data and the log data.

The server may control the IoT facility according to the scenario (S550). For example, the server may control the first IoT facility according to a preset first scenario according to occurrence of the first error situation.

For example, the server may learn what response should be accompanied for each error, and may perform response according to the first scenario when a first error situation occurs. That is, a scenario may be prepared for how to respond to each error situation.

The server may provide CCTV images (S560). For example, the server provides the first error situation information and the first video of the first IoT device recorded by the CCTV terminal during the time when the first error situation occurred in the first IoT device to the user terminal. can do.

That is, when the first error situation occurs in the first IoT facility, the server provides the user terminal with the first error situation information in the first IoT facility and the first error situation recorded by the CCTV terminal during the time when the first error situation occurred. A CCTV image (ie, a first image) of an IoT facility may be provided.

The server may map the operation of the CCTV image to an error situation (S570). For example, the server may map a first operation of the first facility in the first image to the first error condition.

For example, the server may determine that a second error situation occurs in a second IoT device among the IoT devices based on the PLC data and the log data. For example, the server may control the second IoT facility according to a preset second scenario according to the occurrence of the second error situation. For example, the server sends the second error situation information and the second video of the second IoT facility recorded by the CCTV terminal during the time when the second error situation occurred in the second IoT facility to the user terminal. can provide For example, the server may map a second operation of the second IoT device in the second image to the second error situation.

The server may recognize the occurrence of a problem situation through CCTV images (S580). For example, the server may obtain a third image including the second operation of the second IoT facility from the CCTV terminal.

For example, the server may check the PLC data and the log data for the second IoT facility. For example, the server may recognize occurrence of the second error situation based on the third image.

For example, the server may deny occurrence of the second error situation based on the PLC data and the log data for the second IoT facility.

That is, based on the PLC data and log data for the second IoT facility, it may not be determined that the second error situation has occurred in the second IoT facility. However, based on the operation video of the CCTV, it may be determined that a second error situation has occurred in the second IoT facility.

In this case, for example, the server may provide the user terminal with a message about the third image and the possibility of occurrence of the second error situation based on the third image.

For example, the server may provide the user terminal with the possibility of manipulating the PLC data and log data for the second IoT facility based on the PLC data and log data for the second IoT facility and the third image. message can be provided.

That is, PLC data and log data can be sufficiently manipulated, but since CCTV is an image based on a closed circuit, the possibility of manipulation is slim and may be difficult to manipulate. Therefore, more reliability can be given to the determination of the second error situation based on the operation video of the CCTV. Therefore, if it is determined that the second error situation has occurred based on the motion image, but it is determined that the second error situation has not occurred based on the PLC data and log data, it is possible to suspect that the PLC data and log data have been manipulated. can

For example, the server may block a network connected to the second IoT facility.

For example, the server may map the physical sensing data related to the second IoT facility acquired from the IoT sensors during the time when the second error situation occurred in the second IoT facility to the second error situation. .

For example, the server may acquire a fourth image including the second operation of the second IoT facility from the CCTV terminal. For example, the server may recognize occurrence of the second error situation based on the fourth image. For example, the server may recognize the occurrence of the second error situation based on the physical sensing data related to the second IoT facility obtained from the IoT sensors. For example, the server may deny occurrence of the second error situation based on the PLC data and the log data for the second IoT device.

For example, the server may provide the user terminal with a message regarding occurrence of the second error situation based on the physical sensing data related to the fourth image and the second IoT facility.

For example, the server may provide the user terminal with a message about manipulation of the PLC data and the log data for the second IoT facility.

That is, PLC data and log data can be sufficiently manipulated, but the possibility of manipulation of physical sensing data acquired from IoT sensors is slim and may be difficult to manipulate. Therefore, more reliability can be given to the determination of the second error situation based on the physically sensed data. Therefore, if it is determined that the second error situation has occurred based on the physical sensing data, but it is determined that the second error situation has not occurred based on the PLC data and log data, it is suspected that the PLC data and log data have been manipulated. can do.

For example, the server may block a network connected to the second IoT facility.

Some of the detailed steps shown in the example of FIG. 5 may not be essential steps and may be omitted. In addition to the steps shown in FIG. 5, other steps may be added, and the order of the steps may be changed. Some of the above steps may have their own technical meaning.

6 is a diagram showing the configuration of a smart IIoT system according to an embodiment of the present invention.

Referring to FIG. 6 , the server may perform data collection and intelligent response (Control) for predicting and analyzing the cause of problems occurring in facilities. The server can collect/manage the data of all equipment where physical sensors and S/W devices are installed.

The server can perform real-time monitoring of sensor values and control actions according to situational conditions (mashup). The server can transmit data for use in the analysis system to the Big Data Server. The server can support various IoT communication protocols (HTTP, MQTT, CoAP). The server can collect specific data in the same way as the IoT sensor by interlocking with the Pcap Driver.

The server can provide (95%) an industrial PLC connection driver used in domestic/overseas production plants. The server can collect data by specifying PLC data in the form of a tag and recognizing it as a logic sensor (S-Pin) of IoT. The server can process and control the PLC Tag data collected in the IoT server through the Rule engine. The server can use the PLC tag data in conjunction with the 3rd part application to develop a customized application (web API, DLL provided). The server can perform PLC Data Read/Write through the PLC Tag.

The server can receive data from the HSMS message of the CIM without a CIM change point. The server can collect packets on the network and extract and process specific messages according to the HSMS message specifications. The server can process and collect CIM data without changing the system. The server can perform intelligent line monitoring and control by combining with IoT solution/ROS rather than simple data collection.

For example, the server of the present specification may be a concept including the device server and IIoT server of FIG. 6 .

7 is a diagram illustrating a smart IIoT system architecture according to an embodiment of the present invention.

Referring to FIG. 7 , the TM gate can acquire RAW data from a PLC, sensor, log file, SQL database, 3rd part system, EQP, and the like. The TM gate can standardize the acquired RAW data and provide the standardized data to the server. The server of the present specification may be a concept including a TM gate.

The server can analyze data based on the acquired RAW data and IoT reference data, and perform monitoring mashup and intelligent response.

8 is a diagram illustrating a smart IIoT data analysis UI construction method according to an embodiment of the present invention.

Referring to FIG. 8 , for example, the server informs the user terminal of the tact time, operation ratio, defect trace, yield trend, and defect of the facilities. A process analysis report for the facilities including defect map information may be provided.

9 is a diagram illustrating a smart IIoT-based AI remote integrated control system R & D method according to an embodiment of the present invention.

Referring to FIG. 9, the smart IIoT-based AI remote integrated control system has changed from EQP single automation (1st generation) -> centralized remote control (2nd generation) -> AI remote control (3rd generation). Smart IIoT-based AI remote integrated control system can minimize inefficiency (manpower) and uncertainty (Human Error) of Human Remote Control. Smart IIoT-based AI remote integrated control system can increase the accuracy of error situation determination through real-time image analysis + CIM + PLC data of remote control screen using Deep Learning engine. The smart IIoT-based AI remote integrated control system can support AI Auto Control and real-time reporting of results according to facility remote control scenarios for error situations.

Operations according to the embodiments of the present specification can be implemented as computer-readable programs or codes on a computer-readable recording medium. A computer-readable recording medium includes all types of recording devices in which data readable by a computer system is stored. In addition, computer-readable recording media may be distributed to computer systems connected through a network to store and execute computer-readable programs or codes in a distributed manner.

The technical features of the present specification described above may be applied to various devices and methods. For example, the technical features of the present specification described above may be performed/supported through the device of FIGS. 1 and/or 2 . For example, the technical features of the present specification described above may be applied only to a part of FIGS. 1 and/or 2 .

When the embodiment is implemented as software, the above-described technique may be implemented as a module (process, function, etc.) that performs the above-described functions. A module can be stored in memory and executed by a processor. The memory may be internal or external to the processor, and may be coupled with the processor in a variety of well-known means.

In addition, the computer-readable recording medium may include hardware devices specially configured to store and execute program instructions, such as ROM, RAM, and flash memory. The program command may include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine code generated by a compiler.

Although some aspects of the present invention have been described in the context of an apparatus, it may also represent a description according to a corresponding method, where a block or apparatus corresponds to a method step or feature of a method step. Similarly, aspects described in the context of a method may also be represented by a corresponding block or item or a corresponding feature of a device. Some or all of the method steps may be performed by (or using) a hardware device such as, for example, a microprocessor, programmable computer, or electronic circuitry. In some embodiments, one or more of the most important method steps may be performed by such an apparatus.

In embodiments, a programmable logic device (eg, a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In embodiments, a field programmable gate array may operate in conjunction with a microprocessor to perform one of the methods described herein. Generally, methods are preferably performed by some hardware device.

Various embodiments described above may be embodied in other specific forms without departing from their technical ideas and essential characteristics. Accordingly, the above detailed description should not be construed as limiting in all respects and should be considered illustrative. The scope of the various embodiments should be determined by a reasonable interpretation of the appended claims, and all changes within the equivalent scope of the various embodiments are included in the scope of the various embodiments. In addition, claims that do not have an explicit citation relationship in the claims may be combined to form an embodiment or may be included as new claims by amendment after filing.

Claims (5)

In the method of providing smart factory remote control service based on IIoT (industrial internet of things) performed by at least one server,
Acquiring programmable logic controller (PLC) data and log data of the Internet of Things (IoT) facilities;
Obtaining physical sensing data from IoT sensors;
Obtaining operation images of the IoT facilities from closed-circuit television (CCTV) terminals;
Learning error situations occurring in the IoT facilities based on the PLC data and the log data through machine learning;
Determining that a first error situation occurs in a first IoT device among the IoT devices based on the PLC data and the log data;
controlling the first IoT device according to a first scenario according to the occurrence of the first error situation; and
Providing a user terminal with the first error situation information and a first image of the first IoT device recorded by the CCTV terminal during a time when the first error situation occurred in the first IoT device,
mapping the first operation of the first IoT device in the first image to the first error situation;
Determining that a second error situation occurs in a second IoT device among the IoT devices based on the PLC data and the log data;
controlling the second IoT device according to a second scenario according to the occurrence of the second error situation;
Providing, to the user terminal, the second error situation information and a second image of the second IoT device recorded by the CCTV terminal during a time when the second error situation occurred in the second IoT device; and
Further comprising mapping a second operation of the second IoT device in the second image to the second error situation,
obtaining a third image including the second operation of the second IoT facility from the CCTV terminal;
Checking the PLC data and the log data for the second IoT facility;
recognizing occurrence of the second error situation based on the third image;
Denying occurrence of the second error situation based on the PLC data and the log data for the second IoT device;
providing the user terminal with a message about the possibility of occurrence of the second error situation based on the third image and the third image;
To the user terminal, based on the PLC data and the log data and the third image for the second IoT facility, providing a message about the possibility of manipulating the PLC data and the log data for the second IoT facility step; and
Further comprising blocking the network connected to the second IoT facility,
How to provide IIoT-based smart factory remote control service. delete delete The method of claim 1,
the server,
Mapping the physical sensing data related to the second IoT facility obtained from the IoT sensors to the second error situation during a time when the second error situation occurred in the second IoT facility,
How to provide IIoT-based smart factory remote control service. The method of claim 4,
the server,
Obtaining a fourth image including the second operation of the second IoT facility from the CCTV terminal;
recognizing occurrence of the second error situation based on the fourth image;
Recognizing occurrence of the second error situation based on the physical sensing data related to the second IoT facility obtained from the IoT sensors;
Denying occurrence of the second error situation based on the PLC data and the log data for the second IoT device;
providing the user terminal with a message about occurrence of the second error situation based on the physical sensing data related to the fourth image and the second IoT device;
providing the user terminal with a message about manipulation of the PLC data and the log data for the second IoT facility; and
Further comprising blocking the network connected to the second IoT facility,
How to provide IIoT-based smart factory remote control service.

Images