KR20220073692A - METHODS FOR ORGANIZING PROCESSING FLOW BY STEP OF BIG DATA TECHNOLOGY FOR INDUSTRIAL IoT SENSING STREAMING - Google Patents

Abstract

A sensor data processing method, which is a type of industrial IoT sensing streaming big data technology step-by-step processing flow configuration method, is disclosed. The sensor data processing method is a step of extracting raw data by referring to a tag that is metadata of the sensor data of the source when connected to at least one source of mechanical equipment, PLC, sensor, and IoT device through industrial protocols including OPC. , where OPC uses message queue (MQ) to obtain points from raw data by supporting any one or more protocols of KAFKA, MQTT, and STOMP. Sending and storing the obtained points to a non-relational database in the data pipeline , generating an alarm according to the condition of the point, transmitting and storing the alarm to a non-relational database in the data pipeline, and storing the point in an in-memory analysis engine or CEP (Memory) equipped with an in-memory analysis engine real-time analysis by

Description

How to configure industrial IoT sensing streaming big data technology step-by-step processing flow

The present invention relates to an industrial internet of thing (IIoT) sensing streaming big data technology step-by-step process flow configuration method.

Ole for Process Control (OPC) is an industrial protocol for factory automation, and was released by the OPC Foundation.

OPC has the advantage of standardizing and integrating data transmission and reception of various programmable logic controllers (PLCs) and devices. It is proposed as an architecture to overcome the disadvantage of slow decision making.

However, the existing OPC gateway does not provide a data management function by supporting simple connection in data analysis.

Industrial IoT sensing streaming big data technology step-by-step processing for roll-up processing from sensor original data to human understandable context data to provide a method that can overcome the limitations of the prior art described above. An object of the present invention is to provide a flow configuration method.

Another object of the present invention is to provide a process flow configuration method for each step of industrial IoT sensing streaming big data technology that implements real-time transactions using a non-relational database such as NoSQL and a message queue (MQ).

Industrial IoT sensing streaming big data technology step-by-step processing flow configuration method (hereinafter, simply 'sensor data processing method') according to an aspect of the present invention for solving the above technical problem is a human-comprehensible context from sensor original data Roll-up processing with data.

In addition, the sensor data processing method implements a real-time transaction using a non-relational database and message queue (MQ).

A sensor data processing method according to another aspect of the present invention for solving the technical problem is a sensor data processing method corresponding to an industrial Internet of thing (IoT) sensing streaming big data technology step-by-step processing flow configuration method, OPC (Ole for When connected to at least one source among mechanical equipment, programmable logic controller (PLC), sensor, and IoT device through industrial protocols including extracting raw data with reference - the OPC obtains a point from the raw data by supporting any one or more protocols of KAFKA, MQTT, and STOMP using MQ (message queue); transmitting and storing the acquired points to a non-relational database in a data pipeline; generating an alarm according to the condition of the point; transmitting and storing the alarm to a non-relational database in the data pipeline; and analyzing the points in real time by an in-memory analysis engine or a CEP (Memory) in which the in-memory analysis engine is mounted.

In an embodiment, the sensor data processing method may further include aggregating processing data using the points in the CEP.

In one embodiment, the aggregated processed data includes an aggregation of data or a snapshot of the data, and the aggregated processed data may be stored in a non-relational database.

In one embodiment, the sensor data processing method refers to asset data of a facility/process/site based on the point in the CEP to refer to asset data, an asset alarm, and an asset event. acquiring at least one of an (asset event), an asset aggregation, and an asset context; And at least one of the acquired asset data (asset data), asset alarm (asset alarm), asset event (asset event), asset aggregation (asset aggregation), and asset context (asset context) to transmit to a non-relational database and store it It may include further steps.

In one embodiment, the sensor data processing method is a data pipeline (forecast), abnormal (abnormal), and artificial intelligence (AI) data learned using a Feature Ex. data pipeline) may further include the step of transmitting and storing the data to a non-relational database.

In one embodiment, the data pipeline is a device that implements transactions in real time using message queues (MQ) and non-relational databases. The MQ in the data pipeline has a function to push to an application, and the non-relational database has a function to pull to the application. And the sensor data processing method rolls-up the raw data into human-readable context data through the push function and the pull function, and performs a real-time transaction using the non-relational database and the message queue. can be done

According to the present invention, it is very effective by performing roll-up processing from sensor original data to human comprehensible context data, and implementing real-time transactions using a non-relational database such as NoSQL and message queue (MQ). It can effectively process sensing streaming big data, which is industrial IoT (internet of thing) sensor data.

1 is a schematic diagram of an overall system architecture to which an industrial IoT sensing streaming big data technology step-by-step process flow configuration method (hereinafter, simply 'sensor data processing method') according to an embodiment of the present invention can be applied.
FIG. 2 is a block diagram illustrating main components of an edge gateway implementing the sensor data processing method of FIG. 1 .
3 is a schematic diagram of an overall flow architecture for data processing implementing the sensor data processing method of FIG. 1 .
FIG. 4 is a diagram for explaining a data platform design procedure that may be employed in the flow architecture for data processing of FIG. 3 .
5 is a block diagram illustrating a main configuration of a sensor data processing apparatus implementing the sensor data processing method of FIG. 1 .

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention. In describing each figure, like reference numerals have been used for like elements.

Terms such as first, second, A, and B may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. The term “and/or” includes a combination of a plurality of related listed items or any of a plurality of related listed items.

When an element is referred to as being “connected” or “connected” to another element, it is understood that it may be directly connected or connected to the other element, but other elements may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. In describing the present invention, in order to facilitate the overall understanding, the same reference numerals are used for the same components in the drawings, and duplicate descriptions of the same components are omitted.

1 is a schematic diagram of an overall system architecture to which an industrial IoT (internet of thing) sensing streaming big data technology step-by-step process flow configuration method (hereinafter abbreviated 'sensor data processing method') according to an embodiment of the present invention can be applied.

Referring to FIG. 1 , an edge gateway 100 is a type of Ole for Process Control (OPC) edge gateway, and is a machine facility, a programmable logic controller (PLC), a sensor, and an IoT device through a wired or wireless network using an industrial protocol. and the like, and is configured to collect data from them and control them in real time.

The OPC edge gateway 100 may operate as an OPC server and client, may have an artificial intelligence (A1) framework, and may be configured to perform in-memory analysis.

In addition, the OPC edge gateway 100 may be connected to the cloud through TCP/IP of the public Internet, and may provide these to a data server, an application server, and the like of the cloud for utilization of data and metadata.

FIG. 2 is a block diagram illustrating main components of an edge gateway implementing the sensor data processing method of FIG. 1 .

Referring to FIG. 2 , the edge gateway 100 is connected to a wired or wireless network and may be connected to a machine facility, a PLC, a sensor, an IoT device, etc. through an industrial message bus supporting an industrial protocol.

Also, the edge gateway 100 may include an application programming interface (API) and a first framework, a runtime, an R-based AI engine, an in-memory analysis engine, and a second framework. The first framework may be an API-related framework, and the second framework may be an in-memory analysis engine-related framework. An R-based AI engine may refer to a specific form of an AI engine. In addition, the in-memory analysis engine may include an input adapter between the industrial message bus and an output adapter between the output API and the in-memory analysis engine.

In addition, the edge gateway 100 may provide data, metadata, etc. or analysis result data based thereon to a computing device or network terminal accessible on the public Internet or a connectable network through the output API.

3 is a schematic diagram of an overall flow architecture for data processing implementing the sensor data processing method of FIG. 1 .

Referring to FIG. 3 , the sensor data processing method may include raw data and processed data that are original data. Processed data may include aggregate data, asset data, and artificial intelligence (AI) data.

Here, raw data and aggregate data may be data that is inquired by a provided tool, and asset data and AI data may be core data used by programmers.

The raw data is extracted by referring to the tag, which is metadata, and OPC supports the KAFKA/MQTT/STOMP protocol using MQ, so that a point can be obtained from the raw raw data. The acquired Point can be transmitted and stored in NoSQL in the data pipeline.

In addition, it is possible to generate an Alarm according to the condition of the Point, transmit it to NoSQL in the data pipeline, and store it.

And, the point can be analyzed in real time by CEP (Memory).

In more detail, CEP can aggregate machining data using Point.

According to this embodiment, the aggregated processed data may be Aggregation and Snapshot. The aggregated processed data can be stored in a non-relational database such as NoSQL.

In addition, CEP can obtain Asset Data, Asset Alarm, Asset Event, Asset Aggregation and Asset Context by referencing Asset (equipment/process/site) data based on the Point, and transmit it to NoSQL and store it. In this case, the asset data may be core data used by the programmer.

Meanwhile, machining learning Forecast, Abnormal and Feature Ex. AI data learned using the model can be transmitted and saved to NoSQL in the Data Pipeline. In this case, the AI data may be core data used by the programmer.

Here, the data pipeline may be a device that implements a transaction in real time using MQ and NoSQL.

MQ in the data pipeline can be pushed to the application, and NoSQL can be pulled to the application.

Therefore, the industrial IoT sensing streaming big data technology step-by-step processing flow configuration method according to an embodiment of the present invention can roll-up processing sensor original data into human context data, and use NoSQL and MQ in real time Transactions can be implemented.

FIG. 4 is a diagram for explaining a data platform design procedure that can be employed in the flow architecture for data processing of FIG. 3 .

Referring to FIG. 4 , the edge gateway 100 is an industrial protocol such as HTTP (hypertext transfer protocol), MQTT (message queuing telemetry transport), and STOMP (streaming text oriented message protocol) or sensor data collected through a connection driver. A real-time transaction may be implemented using a message queue (MQ) or the like based on the analysis result of sensor data through complex event processing (CEP) or the analysis result may be provided to the application side through an output-side API.

In addition, it is possible to store the analysis result of the CEP in the database (DB), and provide the data stored in the DB to the cloud-side application through the API. The data stored in the DB may be directly provided to the API, but may be processed into a predetermined format or standard by a business rule engine (BRE) and provided to the application through the API.

The edge gateway 100 according to the present embodiment may be configured to further include an output-side API, further include a database (DB), further include a BRE, or further include a combination thereof in a broad sense. The API can be connected to an application or cloud on a public network through protocols such as HTTP and websocket (WS).

FIG. 5 is a block diagram illustrating a main configuration of a sensor data processing apparatus implementing the sensor data processing method of FIG. 1 .

Referring to FIG. 5 , the sensor data processing device 100 , which is a kind of a computer device or edge gateway implementing the sensor data processing method, is connected to at least one processor 110 , a memory 120 and a network to perform communication. It may include a transceiver 130 that does this. In addition, the sensor data processing apparatus 100 may further include an input interface device 140 , an output interface device 250 , a storage device 260 , and the like. Each of the components included in the sensor data processing apparatus 100 may be connected by a bus 170 to communicate with each other.

However, each of the components included in the sensor data processing apparatus 100 may be connected through an individual interface or an individual bus centering on the processor 110 instead of the common bus 170 . For example, the processor 110 may be connected to at least one of the memory 120 , the transceiver 130 , the input interface device 140 , the output interface device 150 , and the storage device 160 through a dedicated interface. have.

The processor 110 may execute a program or instructions stored in at least one of the memory 120 and the storage device 160 . The processor 110 may mean a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor on which methods according to embodiments of the present invention are performed.

Each of the memory 120 and the storage device 160 may be configured of at least one of a volatile storage medium and a non-volatile storage medium. For example, the memory 120 may be configured as at least one of a read only memory (ROM) and a random access memory (RAM).

In addition, the operation of the method according to the embodiments of the present invention can be implemented as a computer-readable program or code on a computer-readable recording medium. The computer-readable recording medium includes all types of recording devices in which data that can be read by a computer system is stored. In addition, the computer-readable recording medium may be distributed in a network-connected computer system to store and execute computer-readable programs or codes in a distributed manner.

In addition, the computer-readable recording medium may include a hardware device specially configured to store and execute program instructions, such as ROM, RAM, and flash memory. The program instructions may include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

Although some aspects of the invention have been described in the context of an apparatus, it may also represent a description according to a corresponding method, wherein a block or apparatus corresponds to a method step or feature of a method step. Similarly, aspects described in the context of a method may also represent a corresponding block or item or a corresponding device feature. Some or all of the method steps may be performed by (or using) a hardware device such as, for example, a microprocessor, programmable computer or electronic circuit. In some embodiments, one or more of the most important method steps may be performed by such an apparatus.

In embodiments, a programmable logic device (eg, a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In embodiments, the field programmable gate array may operate in conjunction with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by some hardware device.

Although described above with reference to preferred embodiments of the present invention, those skilled in the art can variously modify and change the present invention within the scope without departing from the spirit and scope of the present invention described in the claims below. You will understand that you can.

Claims (6)

As a sensor data processing method corresponding to the industrial IoT (internet of thing) sensing streaming big data technology step-by-step processing flow configuration method,
When connected to at least one source among machine equipment, programmable logic controller (PLC), sensor, and IoT device through industrial protocols including OPC (Ole for Process Control), the tag ( Tag) with reference to extracting raw data - The OPC uses MQ (message queue) to support any one or more protocols of KAFKA, MQTT, and STOMP to obtain a point from raw data -;
transmitting and storing the acquired points to a non-relational database in a data pipeline;
generating an alarm according to the condition of the point;
transmitting and storing the alarm to a non-relational database in the data pipeline;
Sensor data processing method comprising the step of analyzing the point in real time by an in-memory analysis engine or a CEP (Memory) in which the in-memory analysis engine is mounted. The method according to claim 1,
The method of processing sensor data further comprising the step of aggregating processing data using the points in the CEP. 3. The method according to claim 2,
wherein the aggregated processed data comprises an aggregation of data or a snapshot of the data, wherein the aggregated processed data is stored in a non-relational database. The method according to claim 1,
Asset data, asset alarm, asset event, and asset aggregation by referencing asset data of equipment/process/site based on the point in the CEP and acquiring at least one or more of an asset context; and
Transmitting and storing at least one of the acquired asset data, asset alarm, asset event, asset aggregation, and asset context to a non-relational database ;
Sensor data processing method further comprising. The method according to claim 1,
Artificial intelligence (AI) data learned using machining learning forecast, abnormal, and feature extraction models are transmitted to and stored in a non-relational database in the data pipeline. Sensor data processing method further comprising the step of. The method according to claim 1,
The data pipeline is a device that implements a transaction in real time using a message queue (MQ) and a non-relational database,
The MQ in the data pipeline has a function to push to an application, and the non-relational database has a function to pull to the application,
Sensor data processing, which roll-up processes the raw data into human-readable context data through the push function and the pull function, and performs a real-time transaction using the non-relational database and the message queue Way.

Images