CN112491991B

CN112491991B - Industrial big data processing method and device and computer equipment

Info

Publication number: CN112491991B
Application number: CN202011286873.5A
Authority: CN
Inventors: 刘胜
Original assignee: Shanghai Qixiang Intelligent Technology Co ltd
Current assignee: Shanghai Qixiang Intelligent Technology Co ltd
Priority date: 2020-11-17
Filing date: 2020-11-17
Publication date: 2023-01-06
Anticipated expiration: 2040-11-17
Also published as: CN112491991A

Abstract

The invention discloses an industrial big data processing method, which comprises the following steps: creating different tasks according to different data sources, and confirming the label name to be read and various parameter settings in each task; circularly executing the created different tasks; when a task is executed, carrying out data acquisition on label data uploaded by a client according to the confirmed label name and various parameter settings; when a task is executed, analyzing, calculating and processing the acquired label data; and when the task is executed, writing the label result obtained by analyzing and calculating into the client where the corresponding data source is located, so as to update the label information of the client and display the label result on the page of the client. Also discloses a device and computer equipment for realizing the industrial big data processing method. The invention can simultaneously carry out multi-task and multi-port data acquisition, calculation and updating on different data sources, thereby improving the data updating efficiency.

Description

Industrial big data processing method and device and computer equipment

Technical Field

The invention relates to the technical field of computers, in particular to an industrial big data processing method and device and computer equipment.

Background

Industrial big data refers to a general term of various data and related technologies and applications generated in the whole life cycle of products from the customer's requirement to the sales, order, plan, research and development, design, process, manufacture, purchase, supply, inventory, delivery and delivery, after-sales service, operation and maintenance, scrapping or recycling remanufacturing, and the like, around a typical intelligent manufacturing model in the industrial field. The industrial big data takes the product data as the core, the traditional industrial data range is greatly expanded, and meanwhile, the related technology and application of the industrial big data are also included. The main sources of industrial big data can be divided into the following three categories: the first type is production and management related business data, the second type is equipment internet of things data, and the third type is external data.

During the real-time acquisition and processing process of industrial big data, only the data of a single data source can be acquired in real time, and then the data are returned to the server side for writing operation after calculation. At present, data of a plurality of different data sources cannot be acquired and written into the data sources for updating at the same time, the updating efficiency is low, and the data reading and writing requirements cannot be met.

To this end, the applicant has sought, through useful research and research, a solution to the above-mentioned problems, in the context of which the technical solutions to be described below have been made.

Disclosure of Invention

One of the technical problems to be solved by the present invention is: aiming at the defects of the prior art, the industrial big data processing method for improving the read-write updating efficiency is provided.

The second technical problem to be solved by the present invention is: an industrial big data processing device for realizing the industrial big data processing method is provided.

The third technical problem to be solved by the invention is that: a computer device for realizing the industrial big data processing method is provided.

An industrial big data processing method as a first aspect of the present invention includes:

creating different tasks according to different data sources, and confirming the label name to be read and various parameter settings in each task;

circularly executing the created different tasks;

when a task is executed, carrying out data acquisition on label data uploaded by a client according to the confirmed label name and various parameter settings;

when a task is executed, analyzing, calculating and processing the acquired label data; and

and when the task is executed, writing the label result obtained by analyzing and calculating into the client where the corresponding data source is located so as to update the label information of the client and display the label result on the page of the client.

In a preferred embodiment of the present invention, the creating different tasks according to different data sources, and confirming the tag name to be read and various parameter settings in each task includes:

setting a tag name to be read, a tag name to be written and IDI server configuration;

setting task names, starting time, task periods and interval time of each task; and

different Python custom scripts are imported.

In a preferred embodiment of the present invention, the data acquisition of the tag data uploaded by the client according to the confirmed tag name and various parameter settings includes:

establishing an IDI communication protocol server channel between a server and a client based on an IDI communication protocol;

receiving label data uploaded by a client through a DataHub server or a MaxComputing server; and

and acquiring data of the label data uploaded by the client according to the confirmed label name and various parameter settings.

In a preferred embodiment of the present invention, the performing parsing calculation processing on the collected tag data includes:

establishing an IDI communication protocol channel between a server and a client based on an IDI communication protocol, and writing the IDI communication protocol channel into a disconnection reconnection mechanism;

analyzing the acquired label data; and

and calculating the label data after the analysis processing by using a Python custom script.

In a preferred embodiment of the present invention, the writing the tag result obtained by the parsing and calculating process to the client where the corresponding data source is located includes:

sending the label result obtained by analyzing and calculating to a client side where the corresponding data source is located;

the client receives the label result and writes the label result into the updating process; and

and the client displays different tag results of different tasks on a page.

In a preferred embodiment of the invention, the data source is one or more of a DataHub data source, a GE iH data source, or an InSmartHistorian data source in combination.

An industrial big data processing apparatus as a second aspect of the present invention for implementing the above-described industrial big data processing method includes:

the task creating module is used for creating different tasks according to different data sources and confirming the label name to be read and various parameter settings in each task;

the task execution module is used for circularly executing the created different tasks;

the data acquisition module is used for acquiring data of the label data uploaded by the client according to the confirmed label name and various parameter settings during task execution;

the analysis calculation processing module is used for carrying out analysis calculation processing on the acquired label data when a task is executed; and

and the tag result writing module is used for writing the tag result obtained by analyzing and calculating the task into the client where the corresponding data source is located so as to update the tag information of the client and display the tag result on the page of the client.

In a preferred embodiment of the present invention, the task creation module includes:

the first task setting unit is used for setting a tag name needing to be read, a tag name needing to be written and IDI server configuration;

the second task setting unit is used for setting task names, starting time, task periods and interval time of all tasks; and

and the importing unit is used for importing different Python custom scripts.

In a preferred embodiment of the present invention, the data acquisition module includes:

the system comprises a server channel creating unit, a client side channel creating unit and a server channel creating unit, wherein the server channel creating unit is used for creating an IDI communication protocol server channel between a server and a client based on an IDI communication protocol;

the data receiving unit is used for receiving the label data uploaded by the client through a Datahub server or a MaxCommuting server; and

and the data acquisition unit is used for acquiring data of the label data uploaded by the client according to the confirmed label name and various parameter settings.

In a preferred embodiment of the present invention, the parsing calculation processing module includes:

the system comprises a protocol channel creating unit, a service end and a client, wherein the protocol channel creating unit is used for creating an IDI communication protocol channel between the service end and the client based on an IDI communication protocol and writing the IDI communication protocol channel into a disconnection reconnection mechanism;

the data analysis unit is used for analyzing and processing the acquired label data; and

and the data calculation unit is used for calculating the label data after the analysis processing by using a Python custom script.

In a preferred embodiment of the present invention, the tag result writing module includes:

the data sending unit is used for sending the label result obtained by analyzing, calculating and processing to the client side where the corresponding data source is located;

the data writing unit is used for enabling the client to receive the label result and write and update the label result; and

and the page display unit is used for enabling the client to display different tag results of different tasks on a page.

A computer device as a third aspect of the present invention for implementing the above-mentioned industrial big data processing method includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

creating different tasks according to different data sources, and confirming the label name required to be read and various parameter settings in each task;

circularly executing the created different tasks;

Due to the adoption of the technical scheme, the invention has the beneficial effects that: the invention can simultaneously carry out multi-task and multi-port data acquisition, calculation and updating on different data sources, thereby improving the data updating efficiency. Meanwhile, the invention also supports the functions of self-definition, classification management and online editing of the Python script, and a stable disconnection reconnection mechanism, thereby ensuring stable reading and writing of data.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

FIG. 1 is a flow chart of an industrial big data processing method of the present invention.

FIG. 2 is a schematic diagram of an embodiment of an industrial big data processing device according to the present invention.

Fig. 3 is an internal structural diagram of the computer apparatus of the present invention.

Detailed Description

In order to make the technical means, the creation characteristics, the achievement purposes and the effects of the invention easy to understand, the invention is further explained by combining the specific drawings.

Referring to fig. 1, an industrial big data processing method is shown, which comprises the following steps:

and S10, creating different tasks according to different data sources, and confirming the label name and various parameter settings which need to be read in each task. In this embodiment, the data source may be one or more of a DataHub data source, a GE iH data source, or an InSmartHistorian data source in combination. Wherein, the DataHub data source can be connected with hundreds of industrial communication protocols and software.

Step S20, the created different tasks are executed in a loop.

And step S30, when executing the task, acquiring data of the label data uploaded by the client according to the confirmed label name and various parameter settings.

And step S40, analyzing and calculating the collected label data when executing the task.

And step S50, writing the label result obtained by the analysis and calculation processing into the client side where the corresponding data source is located when the task is executed, so as to update the label information of the client side and display the label result on the page of the client side.

In step S10, creating different tasks according to different data sources, and confirming the tag name and various parameter settings that need to be read in each task, including the following substeps:

s11, setting a tag name to be read, a tag name to be written and IDI server configuration;

s12, setting task names, start times, task periods and interval times of all tasks;

and step S13, importing different Python custom scripts. In this embodiment, the Python custom script is a script file edited by using a Python programming language and used for performing custom calculation on the tag data.

In step S30, data collection is performed on the tag data uploaded by the client according to the confirmed tag name and various parameter settings, including the following substeps:

step S31, an IDI communication protocol server channel is established between the server and the client based on the IDI communication protocol;

step S32, receiving the label data uploaded by the client through a DataHub server or a MaxComputing server; and

and S33, acquiring data of the label data uploaded by the client according to the confirmed label name and various parameter settings.

In step S40, the analysis calculation processing is performed on the collected tag data, and includes the following sub-steps:

step S40, an IDI communication protocol channel is established between the server and the client based on the IDI communication protocol, and is written into a disconnection reconnection mechanism;

s40, analyzing the acquired label data;

and S40, calculating the label data after the analysis processing by using a Python custom script.

In step S50, writing the tag result obtained by the analysis and calculation process into the client where the corresponding data source is located includes the following substeps:

step S51, sending the label result obtained by analyzing and calculating to the client side where the corresponding data source is located;

step S52, the client receives the label result and writes the label result into the updating process; and

and step S53, the client displays different tag results of different tasks on a page.

Referring to fig. 2, the industrial big data processing device of the present invention includes a task creating module 100, a task executing module 200, a data collecting module 300, a parsing calculation processing module 400, and a tag result writing module 500.

The task creating module 100 is configured to create different tasks according to different data sources, and confirm tag names and various parameter settings that need to be read in each task. Specifically, the task creation module 100 includes a first task setting unit 110, a second task setting unit 120, and an importing unit 130. The first task setting unit 110 is configured to set a tag name to be read, a tag name to be written, and an IDI server configuration. The second task setting unit 120 is used to set a task name, a start time, a task period, and an interval time of each task. The import unit 130 is used to import different Python custom scripts. The Python self-defining script is a script file edited by a Python programming language and used for self-defining calculation of the tag data.

The task execution module 200 is used to loop through the different tasks that are created. Specifically, the created different tasks are put into a task list according to a certain sequence and then executed one by one.

The data collection module 300 is configured to collect data of the tag data uploaded by the client according to the confirmed tag name and various parameter settings when the task is executed. Specifically, the data collection module 300 includes a server channel creation unit 310, a data reception unit 320, and a data collection unit 330. The server channel creating unit 310 is configured to create an IDI communication protocol server channel between the server and the client based on the IDI communication protocol. The data receiving unit 320 is configured to receive, through the DataHub server or the MaxComputing server, the tag data uploaded by the client. The data acquisition unit 330 is configured to perform data acquisition on the tag data uploaded by the client according to the confirmed tag name and various parameter settings.

The parsing calculation processing module 400 is configured to perform parsing calculation processing on the collected tag data when executing a task. Specifically, the parsing calculation processing module 400 includes a protocol channel creation unit 410, a data parsing unit 420, and a data calculation unit 430. The protocol channel creating unit 410 is configured to create an IDI communication protocol channel between the server and the client based on the IDI communication protocol, and write the IDI communication protocol channel into a disconnection reconnection mechanism, so as to ensure stable reading and writing of data. The data parsing unit 420 is configured to parse the collected tag data. The data calculating unit 430 is configured to perform calculation processing on the parsed tag data by using a Python custom script.

The tag result writing module 500 is configured to write the tag result obtained through the parsing and calculating process into the client where the corresponding data source is located when the task is executed, so as to update the tag information of the client and display the tag result on the page of the client. Specifically, the tag result writing module 500 includes a data sending unit 510, a data writing unit 520, and a page showing unit 530. The data sending unit 510 is configured to send the tag result obtained through the parsing and calculation processing to the client where the corresponding data source is located. The data writing unit 520 is configured to enable the client to receive the tag result, and perform write update processing on the tag result. The page display unit 530 is configured to enable the client to perform display processing on different tag results of different tasks on a page.

The modules in the industrial big data processing device can be completely or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

The invention also provides a computer device for implementing the industrial big data processing method, wherein the computer device can be a server, and the internal structure diagram of the computer device can be as shown in fig. 3. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operating system and the computer program to run on the non-volatile storage medium. The database of the computer device is used for storing user information, record information, files and other data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement an industrial big data processing method as described above.

Those skilled in the art will appreciate that the configuration shown in fig. 3 is a block diagram of only a portion of the configuration associated with the present solution and does not constitute a limitation on the computing device to which the present solution applies, and that a particular computing device may include more or less components than those shown, or combine certain components, or have a different arrangement of components.

Specifically, the computer device of the present invention comprises a memory storing a computer program and a processor implementing the following steps when the processor executes the computer program:

Step S20, the created different tasks are executed in a loop.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above may be implemented by hardware instructions of a computer program, which may be stored in a non-volatile computer-readable storage medium, and when executed, may include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), rambus (Rambus) direct RAM (RDRAM), direct Rambus Dynamic RAM (DRDRAM), and Rambus Dynamic RAM (RDRAM), among others.

The foregoing shows and describes the general principles and broad features of the present invention and advantages thereof. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are described in the specification and illustrated only to illustrate the principle of the present invention, but that various changes and modifications may be made therein without departing from the spirit and scope of the present invention, which fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.

Claims

1. An industrial big data processing method is characterized by comprising the following steps:

circularly executing the created different tasks;

when a task is executed, writing a label result obtained by analyzing and calculating processing into a client where a corresponding data source is located so as to update label information of the client and display the label result on a page of the client;

the creating of different tasks according to different data sources and the confirmation of the tag name and various parameter settings to be read in each task include:

setting task names, starting time, task periods and interval time of all tasks; and

importing different Python self-defined scripts;

according to the confirmed label name and various parameter settings, data acquisition is carried out on label data uploaded by a client, and the method comprises the following steps:

acquiring data of the label data uploaded by the client according to the confirmed label name and various parameter settings;

the analyzing, calculating and processing the collected label data includes:

an IDI communication protocol channel is established between the server and the client based on the IDI communication protocol, and is written into a disconnection reconnection mechanism;

analyzing the acquired label data; and

calculating the label data after the analysis processing by using a Python custom script;

the writing of the label result obtained by the analysis calculation processing into the client where the corresponding data source is located includes:

the client receives the tag result and writes the tag result into the update processing; and

and the client displays different tag results of different tasks on a page.

2. The industrial big data processing method according to claim 1, wherein the data source is one or more of a DataHub data source, a GE iH data source, or an InSmart Historian data source.

3. An industrial big data processing apparatus implementing the industrial big data processing method according to any one of claims 1 or 2, comprising:

the task creating module is used for creating different tasks according to different data sources and confirming the label name needing to be read and various parameter settings in each task;

and the tag result writing module is used for writing the tag result obtained by analyzing and calculating the task into the client where the corresponding data source is located when the task is executed so as to update the tag information of the client and display the tag result on the page of the client.

4. The industrial big data processing device according to claim 3, wherein the task creation module comprises:

the second task setting unit is used for setting the task name, the starting time, the task period and the interval time of each task; and

and the import unit is used for importing different Python self-defined scripts.

5. The industrial big data processing device according to claim 3, wherein the data acquisition module comprises:

the system comprises a server channel creating unit, a client side channel creating unit and a service side channel creating unit, wherein the server channel creating unit is used for creating an IDI communication protocol server channel between a server and a client based on an IDI communication protocol;

6. The industrial big data processing device according to claim 3, wherein the parsing calculation processing module comprises:

and the data computing unit is used for computing the label data after the analysis processing by using a Python custom script.

7. The industrial big data processing device according to claim 3, wherein the tag result writing module comprises:

8. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the industrial big data processing method according to any of claims 1 or 2 when executing the computer program.