CN117290668B - Big data processing method and system based on industrial Internet platform - Google Patents
Big data processing method and system based on industrial Internet platform Download PDFInfo
- Publication number
- CN117290668B CN117290668B CN202311514514.4A CN202311514514A CN117290668B CN 117290668 B CN117290668 B CN 117290668B CN 202311514514 A CN202311514514 A CN 202311514514A CN 117290668 B CN117290668 B CN 117290668B
- Authority
- CN
- China
- Prior art keywords
- data
- virtual host
- equipment
- industrial
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 13
- 238000012544 monitoring process Methods 0.000 claims abstract description 53
- 238000012545 processing Methods 0.000 claims abstract description 52
- 238000007781 pre-processing Methods 0.000 claims abstract description 21
- 238000007405 data analysis Methods 0.000 claims abstract description 16
- 230000005540 biological transmission Effects 0.000 claims abstract description 12
- 239000011159 matrix material Substances 0.000 claims description 31
- 238000000034 method Methods 0.000 claims description 14
- 238000012216 screening Methods 0.000 claims description 10
- 238000013528 artificial neural network Methods 0.000 description 20
- 230000000306 recurrent effect Effects 0.000 description 12
- 238000004891 communication Methods 0.000 description 9
- 238000013480 data collection Methods 0.000 description 8
- 238000012986 modification Methods 0.000 description 6
- 230000004048 modification Effects 0.000 description 6
- 230000006403 short-term memory Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000010801 machine learning Methods 0.000 description 4
- 238000013508 migration Methods 0.000 description 4
- 230000005012 migration Effects 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000004927 fusion Effects 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 206010063385 Intellectualisation Diseases 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 230000002457 bidirectional effect Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000006855 networking Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000000712 assembly Effects 0.000 description 1
- 238000000429 assembly Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000015654 memory Effects 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000005236 sound signal Effects 0.000 description 1
- 239000004575 stone Substances 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/10—Pre-processing; Data cleansing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5072—Grid computing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/10—Office automation; Time management
- G06Q10/103—Workflow collaboration or project management
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
- G16Y40/00—IoT characterised by the purpose of the information processing
- G16Y40/10—Detection; Monitoring
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45595—Network integration; Enabling network access in virtual machine instances
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Data Mining & Analysis (AREA)
- Strategic Management (AREA)
- Human Resources & Organizations (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Entrepreneurship & Innovation (AREA)
- Artificial Intelligence (AREA)
- General Business, Economics & Management (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Tourism & Hospitality (AREA)
- Quality & Reliability (AREA)
- Operations Research (AREA)
- Marketing (AREA)
- Economics (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention provides a big data processing method and a big data processing system based on an industrial Internet platform, which relate to the field of industrial Internet, and the big data processing system comprises: the data acquisition module comprises a plurality of equipment data acquisition units and a plurality of area data acquisition units; the data transmission module comprises a plurality of edge servers, wherein a virtual host is operated on each edge server and is used for receiving the state data of the industrial equipment, which is acquired by the corresponding at least one equipment data acquisition unit, and/or the state data of the operators in the target area, which is acquired by the corresponding at least one area data acquisition unit, carrying out data preprocessing, generating an equipment heartbeat data packet and/or an area heartbeat data packet, and uploading the equipment heartbeat data packet and/or the area heartbeat data packet to the data processing cloud platform; the data processing module comprises a data processing cloud platform and is used for carrying out data analysis and determining the states of a plurality of industrial equipment and the states of operators in a plurality of target areas, and has the advantages of improving the monitoring efficiency and quality of the industrial equipment and the operators.
Description
Technical Field
The invention relates to the field of industrial Internet, in particular to a big data processing method and system based on an industrial Internet platform.
Background
In recent years, a new technological revolution and industrial revolution rapidly develop, the Internet rapidly extends from the consumption field to the production field, the industrial economy deeply expands from digitization to networking and intellectualization, and the Internet innovation development and the new industrial revolution form a historical intersection, so that the industrial Internet is induced. The industrial Internet (Industrial Internet) is a novel infrastructure, application mode and industrial ecology for deep integration of a new generation of information communication technology and industrial economy, and a brand new manufacturing and service system which covers a full industrial chain and a full value chain is constructed through comprehensive connection of people, machines, objects, systems and the like, so that an implementation way is provided for the development of industry and even industry digitization, networking and intellectualization, and the industrial Internet is an important foundation stone of the fourth industrial revolution.
The conventional industrial monitoring system takes a PLC as a core, mainly takes an industrial control function as a main, and connects a plurality of detection modules on a PLC processing module for signal acquisition and data analysis. In the existing industrial monitoring system based on the PLC core, if the industrial internet is to be upgraded, a gateway is required to be arranged on each connecting node, and a corresponding data acquisition board is required to be additionally arranged for different data signals, so that the existing industrial monitoring system based on the PLC+the data acquisition board+the gateway commonly used in the prior art has the problems of poor compatibility and poor signal synchronization, and meanwhile, in the conventional industrial monitoring system, a multimedia detection platform such as video signals, audio signals and the like is difficult to build, the coordination among the signals is poor, and cross analysis and processing are difficult. The industrial monitoring system with the traditional architecture has the problems of high cost, large volume, complex system, limited functions, low data transmission rate and poor signal delay synchronization, and is difficult to adapt to the current increasingly intelligent industrial scene.
Therefore, it is desirable to provide a big data processing method and system based on an industrial internet platform for improving the efficiency and quality of monitoring of industrial equipment and operators.
Disclosure of Invention
One of the embodiments of the present specification provides a big data processing system based on an industrial internet platform, including: the data acquisition module comprises a plurality of equipment data acquisition units and a plurality of area data acquisition units, wherein the equipment data acquisition units are used for acquiring state data of a plurality of industrial equipment, and the area data acquisition units are used for acquiring state data of operators in a plurality of target areas; the data transmission module comprises a plurality of edge servers, at least one virtual host is operated on each edge server, each virtual host is corresponding to at least one equipment data acquisition unit and/or at least one area data acquisition unit, the virtual host is used for receiving the state data of the industrial equipment, which is acquired by the corresponding at least one equipment data acquisition unit, and/or the state data of an operator in a target area, which is acquired by the corresponding at least one area data acquisition unit, carrying out data preprocessing, generating an equipment heartbeat data packet corresponding to the industrial equipment and/or an area heartbeat data packet corresponding to the target area, and uploading the equipment heartbeat data packet and/or the area heartbeat data packet to the data processing cloud platform; and the data processing module comprises the data processing cloud platform and is used for carrying out data analysis based on the equipment heartbeat data packet and/or the regional heartbeat data packet and determining the states of the plurality of industrial equipment and the states of operators in the plurality of target regions.
In some embodiments, the data transmission module further includes a plurality of server monitoring units, one of the server monitoring units corresponds to one of the edge servers, the server monitoring unit includes a server monitoring component and at least one virtual host monitoring component, one of the virtual host monitoring components corresponds to one of the virtual hosts of the edge servers to which the server monitoring unit corresponds, wherein the server monitoring component is configured to obtain status data of the corresponding edge server, and the virtual host monitoring component is configured to obtain status data of the corresponding virtual host; the data transmission module further comprises a virtual host scheduling server, and the virtual host scheduling server is used for dynamically adjusting the corresponding relation between a plurality of virtual hosts and the plurality of equipment data acquisition units and/or the plurality of area data acquisition units based on the state data of the edge server and the state data of the virtual hosts.
In some embodiments, the status data of the industrial device includes multiple types of status information of the industrial device, and the status data of the operator of the target area includes image information of the target area; the data preprocessing of the virtual host comprises the following steps: for each corresponding equipment data acquisition unit, generating an information matrix of the industrial equipment based on the state information of the industrial equipment acquired by the equipment data acquisition unit in the current monitoring period, denoising the information matrix of the industrial equipment through a denoising model to generate a denoised information matrix of the industrial equipment, and complementing the denoised information matrix of the industrial equipment through a complementing model to generate a complemented information matrix of the industrial equipment; and screening the image information of the target area acquired by the area data acquisition units through an image screening model based on a preset condition set for each corresponding area data acquisition unit.
In some embodiments, the virtual host performing data preprocessing further comprises: determining the fault probability of the industrial equipment based on the information matrix of the completed industrial equipment, and taking the industrial equipment as target industrial equipment when the fault probability of the industrial equipment is larger than a preset fault probability threshold; determining the optimal data acquisition frequency of an equipment data acquisition unit corresponding to the target industrial equipment according to the fault probability of the target industrial equipment; predicting the future load of the virtual host through a load prediction model based on the optimal data acquisition frequency of the equipment data acquisition unit corresponding to the target industrial equipment, and uploading the predicted future load of the virtual host to the virtual host scheduling server; the virtual host scheduling server dynamically adjusts the corresponding relation between a plurality of virtual hosts and a plurality of device data acquisition units and/or a plurality of region data acquisition units based on the state data of the edge server and the state data of the virtual hosts, and the method comprises the following steps: taking the virtual host for uploading future loads as a target virtual host; and determining a virtual host to be scheduled from a plurality of virtual hosts based on the state data of each edge server, the state data of each virtual host and the future load of the target virtual host, and scheduling the virtual host to be scheduled.
In some embodiments, the virtual host scheduling server determines a virtual host to be scheduled from among the plurality of virtual hosts based on the state data of each of the edge servers, the state data of each of the virtual hosts, and the future load of the virtual host, including: determining a target edge server based on the state data of each edge server, wherein all virtual hosts running on the target edge server are used as virtual hosts to be scheduled; and judging whether the target virtual host is a virtual host to be scheduled or not based on the future load of the target virtual host and the state data of the target virtual host.
In some embodiments, the virtual host scheduling server schedules the virtual host to be scheduled, including: for each virtual host to be scheduled, determining an optimal edge server from edge servers except for the target edge server and the edge server of the virtual host to be scheduled, and migrating the virtual host to be scheduled to the optimal edge server; the migrated virtual host is further configured to issue the optimal data acquisition frequency to an equipment data acquisition unit corresponding to the target industrial equipment, where the equipment data acquisition unit is further configured to acquire status data of the target industrial equipment according to the optimal data acquisition frequency.
In some embodiments, the uploading, by the virtual host, the device heartbeat packet and/or the regional heartbeat packet to a data processing cloud platform includes: encrypting the equipment heartbeat data packet and/or the regional heartbeat data packet based on the corresponding relation between the plurality of virtual hosts and the plurality of equipment data acquisition units and the plurality of regional data acquisition units and the related information of the virtual hosts, and uploading the encrypted equipment heartbeat data packet and/or regional heartbeat data packet to a data processing cloud platform.
In some embodiments, the data processing module performs data analysis based on the device heartbeat data packet and/or the area heartbeat data packet, determines a status of the plurality of industrial devices and a status of an operator of the plurality of target areas, including: establishing an equipment relation map, wherein the equipment relation map is used for recording the association relation of the plurality of industrial equipment; for each of the industrial devices, determining an associated at least one industrial device based on the device relationship map, determining a status of the industrial device based on the device heartbeat data packet of the industrial device and the device heartbeat data packet of the associated at least one industrial device.
In some embodiments, the data processing module performs data analysis based on the device heartbeat data packet and/or the area heartbeat data packet, determines a status of the plurality of industrial devices and a status of an operator of the plurality of target areas, including: establishing a region relation map, wherein the region relation map is used for recording association relations of the plurality of target regions; for each target area, determining at least one associated target area based on the area relation map, and determining the state of operators in the target area based on the area heartbeat data packet of the target area and the area heartbeat data packet of the associated at least one target area.
One of the embodiments of the present disclosure provides a big data processing method based on an industrial internet platform, including: collecting state data of a plurality of industrial devices; collecting state data of operators in a plurality of target areas; the method comprises the steps that a virtual host receives state data of at least one corresponding industrial device and/or state data of operators in at least one target area, performs data preprocessing, generates a device heartbeat data packet corresponding to the industrial device and/or an area heartbeat data packet corresponding to the target area, and uploads the device heartbeat data packet and/or the area heartbeat data packet to a data processing cloud platform; and the data processing cloud platform performs data analysis based on the equipment heartbeat data packet and/or the regional heartbeat data packet, and determines the states of the plurality of industrial equipment and the states of operators in the plurality of target regions.
Compared with the prior art, the big data processing method and system based on the industrial Internet platform provided by the specification have the following beneficial effects:
1. The method comprises the steps of setting a plurality of equipment data acquisition units and a plurality of area data acquisition units, realizing standardized acquisition of state data of industrial equipment and state data of operators, running virtual hosts on the edge servers through the plurality of edge servers, realizing synchronous preprocessing of the state data of the plurality of industrial equipment and the state data of the plurality of operators, improving timeliness of data preprocessing, carrying out comprehensive data analysis on equipment heartbeat data packets and/or area heartbeat data packets based on a data processing cloud platform, determining the states of the plurality of industrial equipment and the states of the operators in a plurality of target areas accurately, realizing automatic acquisition, automatic processing and data fusion analysis of the data, and improving monitoring efficiency and quality of the industrial equipment and the operators;
2. The optimal data acquisition frequency of the corresponding equipment data acquisition unit is adjusted according to the fault probability of the industrial equipment, so that the real-time performance of the state data of the industrial equipment, which is acquired by the equipment data acquisition unit later, is higher, the data volume is larger, and the data processing cloud platform can judge whether the industrial equipment has faults or not more accurately and in real time;
3. the corresponding relation between a plurality of virtual hosts and the plurality of equipment data acquisition units and/or the plurality of area data acquisition units is dynamically adjusted, so that the effective proceeding and real-time performance of data preprocessing can be effectively ensured;
4. Based on the corresponding relation between the plurality of virtual hosts and the plurality of equipment data acquisition units and the plurality of area data acquisition units and the related information of the virtual hosts, the equipment heartbeat data packet and/or the area heartbeat data packet are encrypted, so that leakage of the equipment heartbeat data packet and/or the area heartbeat data packet can be effectively prevented;
5. The method comprises the steps of determining at least one associated industrial device based on a device relation map, determining the state of the industrial device based on a device heartbeat data packet of the industrial device and a device heartbeat data packet of the associated at least one industrial device, realizing fusion of the device heartbeat data packets of a plurality of industrial devices, enabling the determined state of the industrial device to be more accurate, determining the associated at least one target area based on a region relation map, determining the state of an operator of the target area based on the region heartbeat data packet of the target area and the region heartbeat data packet of the associated at least one target area, and realizing fusion of the region heartbeat data packets of the plurality of target areas, enabling the determined state of the operator of the target area to be more accurate.
Drawings
The present specification will be further elucidated by way of example embodiments, which will be described in detail by means of the accompanying drawings. The embodiments are not limiting, in which like numerals represent like structures, wherein:
FIG. 1 is a block diagram of a big data processing system based on an industrial Internet platform, according to some embodiments of the present description;
FIG. 2 is a flow diagram of a big data processing method based on an industrial Internet platform according to some embodiments of the present disclosure;
FIG. 3 is a flow chart illustrating data preprocessing by a virtual host according to some embodiments of the present disclosure.
Detailed Description
In order to more clearly illustrate the technical solutions of the embodiments of the present specification, the drawings that are required to be used in the description of the embodiments will be briefly described below. It is apparent that the drawings in the following description are only some examples or embodiments of the present specification, and it is possible for those of ordinary skill in the art to apply the present specification to other similar situations according to the drawings without inventive effort. Unless otherwise apparent from the context of the language or otherwise specified, like reference numerals in the figures refer to like structures or operations.
It will be appreciated that "system," "apparatus," "unit" and/or "module" as used herein is one method for distinguishing between different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.
As used in this specification and the claims, the terms "a," "an," "the," and/or "the" are not specific to a singular, but may include a plurality, unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" merely indicate that the steps and elements are explicitly identified, and they do not constitute an exclusive list, as other steps or elements may be included in a method or apparatus.
A flowchart is used in this specification to describe the operations performed by the system according to embodiments of the present specification. It should be appreciated that the preceding or following operations are not necessarily performed in order precisely. Rather, the steps may be processed in reverse order or simultaneously. Also, other operations may be added to or removed from these processes.
The industrial Internet platform mainly comprises three core levels of an edge, a platform (industrial PaaS) and an application.
(1) Edge: the method comprises data acquisition, storage, heterogeneous data integration and cloud platform-oriented preprocessing. The edges form the data base of the internet platform architecture.
(2) And (3) a platform: based on industrial PaaS, the open cloud operating system of the industrial platform realizes the functions of big data processing, data analysis, micro-service and the like. Integrating data management capability, application development technology helps clients build customized APP.
(3) Application: on the basis of the platform, personalized industrial SaaS and APP are designed for specific clients and specific scenes, and the final value of the industrial Internet platform is realized.
FIG. 1 is a block diagram of an industrial Internet platform-based big data processing system according to some embodiments of the present disclosure, as shown in FIG. 1, which may include a data acquisition module, a data transmission module, and a data processing module. The respective modules are described in detail in order below.
The data acquisition module can comprise a plurality of equipment data acquisition units and a plurality of area data acquisition units, wherein the equipment data acquisition units are used for acquiring state data of a plurality of industrial equipment, and the area data acquisition units are used for acquiring state data of operators in a plurality of target areas.
In particular, the device data acquisition unit may comprise a variety of means for acquiring status data of the industrial device. For example, devices for acquiring various necessary information such as sound, light, heat, electricity, mechanics, chemistry, biology, and location of industrial equipment. By way of example only, the device data acquisition unit may include at least a temperature humidity sensor, a sound acquisition device, a vibration sensing device, a current sensing device, a voltage sensing device, a gas monitoring device, a locator, and the like. I.e. the status data of the industrial device comprises various types of status information of the industrial device.
The region data acquisition unit may include an image acquisition device for acquiring image information of the target region.
The data transmission module may include a plurality of edge servers, each edge server is operated with at least one virtual host, each virtual host corresponds to at least one equipment data acquisition unit and/or at least one area data acquisition unit, and the virtual host is configured to receive status data of the industrial equipment and/or status data of an operator in a target area, which are acquired by the corresponding at least one equipment data acquisition unit, perform data preprocessing, generate an equipment heartbeat data packet corresponding to the industrial equipment and/or an area heartbeat data packet corresponding to the target area, and upload the equipment heartbeat data packet and/or the area heartbeat data packet to the data processing cloud platform.
The equipment data acquisition unit and the area data acquisition unit can transmit the acquired state data of the industrial equipment and/or the state data of operators in the target area to the corresponding virtual host in a wireless network mode.
The virtual host uses special software and hardware technology to divide a real physical server host into a plurality of logic storage units. Each logical unit has no physical entity, but each logical unit can work on the network like a real physical host, with individual IP addresses (or shared IP addresses), independent domain names, and complete Internet server (supporting WWW, FTP, E-mail, etc.) functions.
FIG. 3 is a flow diagram illustrating a data preprocessing performed by a virtual host according to some embodiments of the present disclosure, in some embodiments, the data preprocessing performed by the virtual host includes:
For each corresponding device data acquisition unit, generating an information matrix of the industrial device based on various types of state information of the industrial device acquired by the device data acquisition unit in a current monitoring period (also referred to as a first current monitoring period), denoising the information matrix of the industrial device through a denoising model to generate a denoised information matrix of the industrial device, and complementing the denoised information matrix of the industrial device through a complementing model to generate a complemented information matrix of the industrial device, wherein the denoising model can be an LSTM (Long short-term memory) model, a row vector of the information matrix of the industrial device is formed by various types of state information acquired by the device data acquisition unit at a time point in the current monitoring period, and one element of the row vector represents one type of state information;
And for each corresponding region data acquisition unit, screening the image information of the target region acquired by the region data acquisition unit based on a preset condition set through an image screening model. Specifically, the set of preset conditions may include image quality (e.g., exposure, color, sharpness, noise, etc.) requirements, target requirements, and the like. For example, when the sharpness of the image information of the collected target area is low or the human body is not contained in the image information of the collected target area, the image information may be screened out as invalid image information.
As shown in fig. 3, in some embodiments, the virtual host performing data preprocessing further includes:
Determining the fault probability of the industrial equipment based on the information matrix of the completed industrial equipment, and taking the industrial equipment as target industrial equipment when the fault probability of the industrial equipment is greater than a preset fault probability threshold value;
according to the fault probability of the target industrial equipment, determining the optimal data acquisition frequency of the equipment data acquisition unit corresponding to the target industrial equipment;
And predicting the future load of the virtual host through a load prediction model based on the optimal data acquisition frequency of the equipment data acquisition unit corresponding to the target industrial equipment, and uploading the predicted future load of the virtual host to the virtual host scheduling server.
Specifically, the virtual host can determine the failure probability of the industrial equipment based on the information matrix of the completed industrial equipment through the failure prediction model. The virtual host can determine the optimal data acquisition frequency of the device data acquisition unit corresponding to the target industrial device according to the failure probability of the target industrial device, the performance of the device data acquisition unit corresponding to the target industrial device and the maximum calculation force constraint through the frequency determination model. It can be appreciated that the greater the failure probability of the target industrial device, the higher the optimal data collection frequency of the device data collection unit corresponding to the target industrial device, the better the performance and/or the higher the maximum computational force constraint of the device data collection unit corresponding to the target industrial device, and the higher the maximum value of the optimal data collection frequency of the device data collection unit corresponding to the target industrial device. The virtual host can predict the data size of the state data of the industrial equipment, which is acquired by the equipment data acquisition unit corresponding to the target industrial equipment in one monitoring period, under the optimal data acquisition frequency based on the optimal data acquisition frequency of the equipment data acquisition unit corresponding to the target industrial equipment through the load prediction model, so that the future load of the virtual host in one monitoring period under the optimal data acquisition frequency is determined. The fault prediction model, the frequency determination model and the load prediction model may be machine learning models such as an artificial neural network (ARTIFICIAL NEURAL NETWORK, ANN) model, a recurrent neural network (Recurrent Neural Networks, RNN) model, a Long Short-Term Memory (LSTM) model, a Bidirectional Recurrent Neural Network (BRNN) model, and the like.
In some embodiments, the data transmission module further includes a plurality of server monitoring units, one server monitoring unit corresponds to each edge server, the server monitoring unit includes a server monitoring component and at least one virtual host monitoring component, one virtual host monitoring component corresponds to one virtual host of each edge server corresponding to the server monitoring unit, wherein the server monitoring component is configured to obtain state data of each corresponding edge server, and the virtual host monitoring component is configured to obtain state data of each corresponding virtual host.
In some embodiments, the data transmission module further includes a virtual host scheduling server, configured to dynamically adjust a correspondence between the plurality of virtual hosts and the plurality of device data acquisition units and/or the plurality of region data acquisition units based on the state data of the edge server and the state data of the virtual hosts.
In particular, the server monitoring component may include various means for obtaining status information of the edge server, such as a temperature humidity sensor, a sound collection device, a vibration sensing device, a current sensing device, and a voltage sensing device. The server monitoring component may transmit the collected edge server status data to the virtual host scheduling server over the wireless network. The virtual host monitoring component may obtain state data of the virtual host through a software program, such as CPU utilization, memory utilization, disk utilization, network utilization, service response time, and the like.
In some embodiments, the virtual host scheduling server may pre-store the trained image screening model and the fault prediction model, frequency determination model, and load prediction model corresponding to different industrial devices, e.g., the virtual host scheduling server may pre-store the fault prediction model, frequency determination model, and load prediction model corresponding to industrial device a, the fault prediction model, frequency determination model, and load prediction model corresponding to industrial device B, and the fault prediction model, frequency determination model, and load prediction model corresponding to industrial device C. After the corresponding relation between the virtual host and the plurality of equipment data acquisition units and the plurality of area data acquisition units is determined, the virtual host scheduling server can issue a corresponding model to the virtual host. For example, the device data acquisition unit corresponding to the industrial device a corresponding to the virtual host 1 and the region data acquisition unit corresponding to the target region 1, and the device data acquisition unit corresponding to the industrial device B corresponding to the virtual host 2 and the device data acquisition unit corresponding to the industrial device C, the virtual host scheduling server may issue the image screening model and the failure prediction model, the frequency determination model, and the load prediction model corresponding to the industrial device a to the virtual host 1, and issue the failure prediction model, the frequency determination model, and the load prediction model corresponding to the industrial device B and the failure prediction model, the frequency determination model, and the load prediction model corresponding to the industrial device C to the virtual host 2.
The virtual host scheduling server dynamically adjusts the corresponding relation between a plurality of virtual hosts and a plurality of equipment data acquisition units and/or a plurality of area data acquisition units based on the state data of the edge server and the state data of the virtual hosts, and the virtual host scheduling server comprises:
Taking the virtual host for uploading future loads as a target virtual host;
And determining the virtual host to be scheduled from the plurality of virtual hosts based on the state data of each edge server, the state data of each virtual host and the future load of the target virtual host, and scheduling the virtual host to be scheduled.
In some embodiments, the virtual host scheduling server determines a virtual host to be scheduled from a plurality of virtual hosts based on the state data of each edge server, the state data of each virtual host, and the future load of the virtual host, comprising:
determining a target edge server based on the state data of each edge server, wherein all virtual hosts running on the target edge server are used as virtual hosts to be scheduled;
and judging whether the target virtual host is the virtual host to be scheduled or not based on the future load of the target virtual host and the state data of the target virtual host.
Specifically, for each edge server, the virtual host scheduling server may generate a server state information matrix corresponding to the edge server based on the state data of the edge server of the industrial equipment collected by the corresponding server monitoring component in the current monitoring period (may also be referred to as a "second current monitoring period"), a row vector of the server state information matrix is formed by state information of multiple types of edge servers acquired by the equipment data collecting unit at a time point in the second current monitoring period, one element of the row vector represents one type of state information, and determines whether the edge server is currently faulty based on the server state information matrix through a fault prediction model, if it is determined that the edge server is not currently faulty, predicts a server state information matrix corresponding to the next second current monitoring period based on the server state information matrix corresponding to the second current monitoring period, and determines whether the edge server is likely to be faulty based on the server state information corresponding to the predicted next second current monitoring period, when it is determined that the edge server is currently faulty or the edge server is likely to be faulty, and uses the element of the row vector as a virtual host to be scheduled as a virtual host for all future edge servers (if it is considered as a virtual host to be a virtual host).
For each target virtual host, the virtual host scheduling server may predict whether the target virtual host can bear the future load based on the state data of the target virtual host and the remaining computer resources and the future load of the edge server where the target virtual host is located, and if the target virtual host is predicted to be unable to bear the future load, the target virtual host is regarded as a virtual host to be scheduled (which may also be referred to as a "second type of virtual host to be scheduled").
In some embodiments, the virtual host scheduling server schedules a virtual host to be scheduled, comprising:
For each virtual host to be scheduled, determining an optimal edge server from edge servers (also called candidate edge servers) except for a target edge server and the edge server where the virtual host to be scheduled is located in a plurality of edge servers, and migrating the virtual host to be scheduled to the optimal edge server;
The migrated virtual host is also used for transmitting the optimal data acquisition frequency to the equipment data acquisition unit corresponding to the target industrial equipment, and the equipment data acquisition unit is also used for acquiring the state data of the target industrial equipment according to the optimal data acquisition frequency.
Specifically, for the first type of virtual hosts to be scheduled, the virtual host scheduling server may determine an optimal edge server from the non-target edge servers based on load information of the first type of virtual hosts to be scheduled, remaining computer resource information of the non-target edge servers, a device data acquisition unit corresponding to the first type of virtual hosts to be scheduled, and a communication distance between the device data acquisition unit and the non-target edge servers, and newly add a virtual host to the optimal edge server, and migrate the first type of virtual hosts to be scheduled to the newly added virtual hosts in the optimal edge server in a hot migration, cold migration or storage migration manner.
For example, the virtual host scheduling server may determine the priority value of the non-target edge server based on the load information of the virtual host to be scheduled of the first type, the remaining computer resource information of the non-target edge server, the device data acquisition unit corresponding to the virtual host to be scheduled of the first type, and the communication distance between the region data acquisition unit and the non-target edge server by the following formula:
Wherein, P (i,j) is the priority value of the host machine of the ith non-target edge server as the jth first-class to-be-scheduled virtual host machine, R (residue,i) is the remaining computer resources of the ith non-target edge server after normalization, R (reality,j) is the load of the jth first-class to-be-scheduled virtual host machine after normalization, L (n,i) is the communication distance between the nth device data acquisition unit corresponding to the jth first-class to-be-scheduled virtual host machine and the ith non-target edge server, L (m,i) is the communication distance between the mth region data acquisition unit corresponding to the jth first-class to-be-scheduled virtual host machine and the ith non-target edge server, N is the total number of device data acquisition units corresponding to the jth first-class to-be-scheduled virtual host machine, M is the total number of region data acquisition units corresponding to the jth first-class to-be-scheduled virtual host machine, a 11、a12 is the preset weight, and W 1 is the preset parameter.
And taking the non-target edge server with the largest priority value as the optimal edge server.
For each virtual host to be scheduled of the second class, the virtual host scheduling server may determine a scheduling priority value of each candidate edge server based on a predicted future load of the virtual host to be scheduled of the second class, a current load of the candidate edge server, a communication distance between the device data acquisition unit and the region data acquisition unit corresponding to the virtual host to be scheduled of the second class and the candidate edge server, and a degree of association between the virtual host running on the candidate edge server and the virtual host to be scheduled of the second class. For example.
For example, the scheduling priority value of each candidate edge server may be determined based on the predicted future load of the second type of virtual host to be scheduled, the remaining computer resource information of the candidate edge server, the communication distance between the device data acquisition unit and the area data acquisition unit corresponding to the second type of virtual host to be scheduled and the candidate edge server, and the association degree between the virtual host running on the candidate edge server and the second type of virtual host to be scheduled according to the following formula:
wherein, P (q,p) is the dispatching priority value of the host machine of the q second class of virtual machines to be dispatched by the P candidate edge server, R (restdue,p) is the remaining computer resource of the normalized P candidate edge server, R (prediction,q) is the future load of the normalized predicted q second class of virtual machines to be dispatched, L (c,q) is the communication distance between the C-th equipment data acquisition unit corresponding to the q-th second-class virtual host to be scheduled and the p-th candidate edge server, L (d,q) is the communication distance between the d-th area data acquisition unit corresponding to the q-th second-class virtual host to be scheduled and the p-th candidate edge server, C is the total number of equipment data acquisition units corresponding to the q-th second-class virtual host to be scheduled, D is the total number of the region data acquisition units corresponding to the qth second-class virtual host to be scheduled, a 21、a22 and a 23 are both preset weights, W 2 is a preset parameter, C (h,,q) is the association degree between the kth virtual host running on the p candidate edge server and the qth second-class virtual host to be scheduled, C (c,e) is the association between the C-th device data acquisition unit corresponding to the q-th second-class virtual host to be scheduled and the E-th device data acquisition unit corresponding to the h-th virtual host running on the p-th candidate edge server, E is the total number of device data acquisition units corresponding to the h-th virtual host running on the p-th candidate edge server, C (d,f) is the association between the d-th region data acquisition unit corresponding to the q-th second-class virtual host to be scheduled and the f-th region data acquisition unit corresponding to the h-th virtual host running on the p-th candidate edge server, F is the total number of the region data acquisition units corresponding to the h virtual host running on the p candidate edge server, the association degree between any two device data acquisition units can be determined based on the association degree between the industrial devices corresponding to the two device data acquisition units, and the association degree between any two industrial devices can be determined in any mode, for example, based on the association coefficient between any two industrial devices. as another example, based on a manual determination. The degree of association between any two region data acquisition units may be determined based on the degree of association between the target regions to which the two region data acquisition units correspond, and the degree of association between any two target regions may be determined in any manner, for example, based on the distance between the two target regions.
In some embodiments, the virtual host scheduling server may establish a scheduling scheme generation model, where the sum of loads of virtual hosts targeted for migration of the scheduling scheme generation model is minimum, inputs of the scheduling scheme generation model include predicted future loads of each virtual host to be scheduled of the second class, state data of edge servers, and scheduling priority values of each candidate edge server relative to each virtual host to be scheduled of the second class, and outputs of the scheduling scheme generation model include an optimal virtual host scheduling scheme, where the optimal virtual host scheduling scheme includes at least an optimal edge server corresponding to each virtual host to be scheduled of the second class. The scheduling scheme generation model may be a machine learning model such as an artificial neural network (ARTIFICIAL NEURAL NETWORK, ANN) model, a recurrent neural network (Recurrent Neural Networks, RNN) model, a Long Short-Term Memory (LSTM) model, a Bidirectional Recurrent Neural Network (BRNN) model, and the like.
In some embodiments, the virtual host uploads device heartbeat data packets and/or regional heartbeat data packets to the data processing cloud platform, including:
encrypting the equipment heartbeat data packet and/or the area heartbeat data packet based on the corresponding relation between the plurality of virtual hosts and the plurality of equipment data acquisition units and the plurality of area data acquisition units and the related information of the virtual hosts, and uploading the encrypted equipment heartbeat data packet and/or area heartbeat data packet to a data processing cloud platform.
Specifically, for each device data acquisition unit, the virtual host may generate a first identifier pair based on a unique device identifier of the device data acquisition unit (e.g., a device ID of a corresponding industrial device) and a unique host identifier of the virtual host (e.g., an IP of the virtual host), encrypt a device heartbeat packet corresponding to the device data acquisition unit for the first time through the first identifier pair, generate an encrypted device heartbeat packet for the first time, and encrypt the encrypted device heartbeat packet for the second time through a private key corresponding to the virtual host, so as to generate an encrypted device heartbeat packet.
For each regional data collection unit, the virtual host may generate a second identifier pair based on the unique regional identifier of the regional data collection unit (e.g., the regional ID of the corresponding target region) and the unique host identifier of the virtual host (e.g., the IP of the virtual host), encrypt the regional heartbeat packet corresponding to the regional data collection unit for the first time through the second identifier pair, generate the first encrypted regional heartbeat packet, and encrypt the first encrypted regional heartbeat packet for the second time through the private key corresponding to the virtual host, so as to generate the encrypted regional heartbeat packet.
The data processing module can comprise a data processing cloud platform which is used for carrying out data analysis based on the equipment heartbeat data packet and/or the regional heartbeat data packet and determining the states of a plurality of industrial equipment and the states of operators in a plurality of target regions.
In some embodiments, the data processing module performs data analysis based on the device heartbeat data packet and/or the area heartbeat data packet to determine a status of the plurality of industrial devices and a status of an operator of the plurality of target areas, including:
establishing an equipment relation map, wherein the equipment relation map is used for recording the association relation of a plurality of industrial equipment;
For each industrial device, determining an associated at least one industrial device based on the device relationship map, determining a status of the industrial device based on the device heartbeat data packet of the industrial device and the device heartbeat data packet of the associated at least one industrial device.
For example, the data processing cloud platform may determine a state of the industrial device based on the device heartbeat data packet of the industrial device and the associated device heartbeat data packet of the at least one industrial device through a first state determination model, where the first state determination model may be a machine learning model such as an artificial neural network (ARTIFICIAL NEURAL NETWORK, ANN) model, a recurrent neural network (Recurrent Neural Networks, RNN) model, a Long Short-Term Memory (LSTM) model, a bi-directional recurrent neural network (BRNN) model, and the like.
In some embodiments, the data processing module performs data analysis based on the device heartbeat data packet and/or the area heartbeat data packet to determine a status of the plurality of industrial devices and a status of an operator of the plurality of target areas, including:
establishing a regional relation map, wherein the regional relation map is used for recording the association relation of a plurality of target areas;
For each target area, determining at least one associated target area based on the area relation map, and determining the state of an operator of the target area based on the area heartbeat data packet of the target area and the area heartbeat data packet of the at least one associated target area.
For example, the data processing cloud platform may determine a status of an operator of the target area based on the area heartbeat data packet of the target area and the associated area heartbeat data packet of the at least one target area through a second status determination model, where the second status determination model may be a machine learning model such as an artificial neural network (ARTIFICIAL NEURAL NETWORK, ANN) model, a recurrent neural network (Recurrent Neural Networks, RNN) model, a Long Short-Term Memory (LSTM) model, a bi-directional recurrent neural network (BRNN) model, and the like.
Fig. 2 is a flow chart of a big data processing method based on an industrial internet platform according to some embodiments of the present disclosure, and as shown in fig. 2, a big data processing method based on an industrial internet platform may include the following steps.
Step 210, collecting status data of a plurality of industrial devices;
Step 220, collecting status data of operators in a plurality of target areas;
Step 230, the virtual host receives the status data of at least one corresponding industrial device and/or the status data of the operator in at least one target area, performs data preprocessing, generates a device heartbeat data packet corresponding to the industrial device and/or an area heartbeat data packet corresponding to the target area, and uploads the device heartbeat data packet and/or the area heartbeat data packet to the data processing cloud platform;
in step 240, the data processing cloud platform performs data analysis based on the device heartbeat data packet and/or the regional heartbeat data packet, and determines the states of the plurality of industrial devices and the states of the operators in the plurality of target regions.
In some embodiments, a big data processing method based on an industrial internet platform may be performed by a big data processing system based on an industrial internet platform, and further description of a big data processing method based on an industrial internet platform may be referred to a related description of a big data processing system based on an industrial internet platform, which is not repeated herein.
While the basic concepts have been described above, it will be apparent to those skilled in the art that the foregoing detailed disclosure is by way of example only and is not intended to be limiting. Although not explicitly described herein, various modifications, improvements, and adaptations to the present disclosure may occur to one skilled in the art. Such modifications, improvements, and modifications are intended to be suggested within this specification, and therefore, such modifications, improvements, and modifications are intended to be included within the spirit and scope of the exemplary embodiments of the present invention.
Meanwhile, the specification uses specific words to describe the embodiments of the specification. Reference to "one embodiment," "an embodiment," and/or "some embodiments" means that a particular feature, structure, or characteristic is associated with at least one embodiment of the present description. Thus, it should be emphasized and should be appreciated that two or more references to "an embodiment" or "one embodiment" or "an alternative embodiment" in various positions in this specification are not necessarily referring to the same embodiment. Furthermore, certain features, structures, or characteristics of one or more embodiments of the present description may be combined as suitable.
Furthermore, the order in which the elements and sequences are processed, the use of numerical letters, or other designations in the description are not intended to limit the order in which the processes and methods of the description are performed unless explicitly recited in the claims. While certain presently useful inventive embodiments have been discussed in the foregoing disclosure, by way of various examples, it is to be understood that such details are merely illustrative and that the appended claims are not limited to the disclosed embodiments, but, on the contrary, are intended to cover all modifications and equivalent arrangements included within the spirit and scope of the embodiments of the present disclosure. For example, while the system components described above may be implemented by hardware devices, they may also be implemented solely by software solutions, such as installing the described system on an existing server or mobile device.
Likewise, it should be noted that in order to simplify the presentation disclosed in this specification and thereby aid in understanding one or more inventive embodiments, various features are sometimes grouped together in a single embodiment, figure, or description thereof. This method of disclosure does not imply that the subject matter of the present description requires more features than are set forth in the claims. Indeed, less than all of the features of a single embodiment disclosed above.
Finally, it should be understood that the embodiments described in this specification are merely illustrative of the principles of the embodiments of this specification. Other variations are possible within the scope of this description. Thus, by way of example, and not limitation, alternative configurations of embodiments of the present specification may be considered as consistent with the teachings of the present specification. Accordingly, the embodiments of the present specification are not limited to only the embodiments explicitly described and depicted in the present specification.
Claims (7)
1. A big data processing system based on an industrial internet platform, comprising:
The data acquisition module comprises a plurality of equipment data acquisition units and a plurality of area data acquisition units, wherein the equipment data acquisition units are used for acquiring state data of a plurality of industrial equipment, and the area data acquisition units are used for acquiring state data of operators in a plurality of target areas;
The data transmission module comprises a plurality of edge servers, at least one virtual host is operated on each edge server, each virtual host is corresponding to at least one equipment data acquisition unit and/or at least one area data acquisition unit, the virtual host is used for receiving the state data of the industrial equipment, which is acquired by the corresponding at least one equipment data acquisition unit, and/or the state data of an operator in a target area, which is acquired by the corresponding at least one area data acquisition unit, carrying out data preprocessing, generating an equipment heartbeat data packet corresponding to the industrial equipment and/or an area heartbeat data packet corresponding to the target area, and uploading the equipment heartbeat data packet and/or the area heartbeat data packet to the data processing cloud platform;
The data processing module comprises the data processing cloud platform and is used for carrying out data analysis based on the equipment heartbeat data packet and/or the regional heartbeat data packet and determining the states of the plurality of industrial equipment and the states of operators in the plurality of target regions;
The data transmission module further comprises a plurality of server monitoring units, one server monitoring unit corresponds to one edge server, the server monitoring unit comprises a server monitoring component and at least one virtual host monitoring component, one virtual host monitoring component corresponds to one virtual host of the edge server corresponding to the server monitoring unit, wherein the server monitoring component is used for acquiring state data of the corresponding edge server, and the virtual host monitoring component is used for acquiring state data of the corresponding virtual host;
the data transmission module further comprises a virtual host scheduling server, and the virtual host scheduling server is used for dynamically adjusting the corresponding relation between a plurality of virtual hosts and the plurality of equipment data acquisition units and/or the plurality of area data acquisition units based on the state data of the edge server and the state data of the virtual hosts;
The state data of the industrial equipment comprises various types of state information of the industrial equipment, and the state data of operators in the target area comprises image information of the target area;
The data preprocessing of the virtual host comprises the following steps:
For each corresponding equipment data acquisition unit, generating an information matrix of the industrial equipment based on the state information of the industrial equipment acquired by the equipment data acquisition unit in the current monitoring period, denoising the information matrix of the industrial equipment through a denoising model to generate a denoised information matrix of the industrial equipment, and complementing the denoised information matrix of the industrial equipment through a complementing model to generate a complemented information matrix of the industrial equipment;
For each corresponding regional data acquisition unit, screening the image information of the target region acquired by the regional data acquisition unit based on a preset condition set through an image screening model;
The data preprocessing of the virtual host further comprises:
Determining the fault probability of the industrial equipment based on the information matrix of the completed industrial equipment, and taking the industrial equipment as target industrial equipment when the fault probability of the industrial equipment is larger than a preset fault probability threshold;
determining the optimal data acquisition frequency of an equipment data acquisition unit corresponding to the target industrial equipment according to the fault probability of the target industrial equipment;
predicting the future load of the virtual host through a load prediction model based on the optimal data acquisition frequency of the equipment data acquisition unit corresponding to the target industrial equipment, and uploading the predicted future load of the virtual host to the virtual host scheduling server;
The virtual host scheduling server dynamically adjusts the corresponding relation between a plurality of virtual hosts and a plurality of device data acquisition units and/or a plurality of region data acquisition units based on the state data of the edge server and the state data of the virtual hosts, and the method comprises the following steps:
Taking the virtual host for uploading future loads as a target virtual host;
And determining a virtual host to be scheduled from a plurality of virtual hosts based on the state data of each edge server, the state data of each virtual host and the future load of the target virtual host, and scheduling the virtual host to be scheduled.
2. The industrial internet platform-based big data processing system of claim 1, wherein the virtual host scheduling server determines a virtual host to be scheduled from among the plurality of virtual hosts based on the state data of each edge server, the state data of each virtual host, and the future load of the virtual host, comprising:
determining a target edge server based on the state data of each edge server, wherein all virtual hosts running on the target edge server are used as virtual hosts to be scheduled;
and judging whether the target virtual host is a virtual host to be scheduled or not based on the future load of the target virtual host and the state data of the target virtual host.
3. The industrial internet platform-based big data processing system of claim 2, wherein the virtual host scheduling server schedules the virtual hosts to be scheduled, comprising:
For each virtual host to be scheduled, determining an optimal edge server from edge servers except for the target edge server and the edge server of the virtual host to be scheduled, and migrating the virtual host to be scheduled to the optimal edge server;
the migrated virtual host is further configured to issue the optimal data acquisition frequency to an equipment data acquisition unit corresponding to the target industrial equipment, where the equipment data acquisition unit is further configured to acquire status data of the target industrial equipment according to the optimal data acquisition frequency.
4. The industrial internet platform-based big data processing system according to claim 1 or 2, wherein the uploading of the device heartbeat data packet and/or the regional heartbeat data packet by the virtual host to the data processing cloud platform comprises:
Encrypting the equipment heartbeat data packet and/or the regional heartbeat data packet based on the corresponding relation between the plurality of virtual hosts and the plurality of equipment data acquisition units and the plurality of regional data acquisition units and the related information of the virtual hosts, and uploading the encrypted equipment heartbeat data packet and/or regional heartbeat data packet to a data processing cloud platform.
5. The industrial internet platform-based big data processing system according to claim 1 or 2, wherein the data processing module performs data analysis based on the device heartbeat data packet and/or the regional heartbeat data packet, and determines the status of the plurality of industrial devices and the status of the operators in the plurality of target regions, including:
establishing an equipment relation map, wherein the equipment relation map is used for recording the association relation of the plurality of industrial equipment;
for each of the industrial devices, determining an associated at least one industrial device based on the device relationship map, determining a status of the industrial device based on the device heartbeat data packet of the industrial device and the device heartbeat data packet of the associated at least one industrial device.
6. The industrial internet platform-based big data processing system according to claim 1 or 2, wherein the data processing module performs data analysis based on the device heartbeat data packet and/or the regional heartbeat data packet, and determines the status of the plurality of industrial devices and the status of the operators in the plurality of target regions, including:
establishing a region relation map, wherein the region relation map is used for recording association relations of the plurality of target regions;
For each target area, determining at least one associated target area based on the area relation map, and determining the state of operators in the target area based on the area heartbeat data packet of the target area and the area heartbeat data packet of the associated at least one target area.
7. The big data processing method based on the industrial Internet platform is characterized by comprising the following steps of:
Collecting state data of a plurality of industrial devices through a device data collecting unit;
acquiring state data of operators in a plurality of target areas through an area data acquisition unit;
The method comprises the steps that a virtual host receives state data of at least one corresponding industrial device and/or state data of operators in at least one target area, performs data preprocessing, generates a device heartbeat data packet corresponding to the industrial device and/or an area heartbeat data packet corresponding to the target area, and uploads the device heartbeat data packet and/or the area heartbeat data packet to a data processing cloud platform;
the data processing cloud platform performs data analysis based on the equipment heartbeat data packet and/or the regional heartbeat data packet, and determines the states of the plurality of industrial equipment and the states of operators in the plurality of target regions;
Acquiring state data of an edge server;
Acquiring state data of a virtual host;
Based on the state data of the edge server and the state data of the virtual hosts, dynamically adjusting the corresponding relation between a plurality of virtual hosts and a plurality of equipment data acquisition units and/or a plurality of area data acquisition units;
The state data of the industrial equipment comprises various types of state information of the industrial equipment, and the state data of operators in the target area comprises image information of the target area;
The data preprocessing of the virtual host comprises the following steps:
For each corresponding equipment data acquisition unit, generating an information matrix of the industrial equipment based on the state information of the industrial equipment acquired by the equipment data acquisition unit in the current monitoring period, denoising the information matrix of the industrial equipment through a denoising model to generate a denoised information matrix of the industrial equipment, and complementing the denoised information matrix of the industrial equipment through a complementing model to generate a complemented information matrix of the industrial equipment;
For each corresponding regional data acquisition unit, screening the image information of the target region acquired by the regional data acquisition unit based on a preset condition set through an image screening model;
The data preprocessing of the virtual host further comprises:
Determining the fault probability of the industrial equipment based on the information matrix of the completed industrial equipment, and taking the industrial equipment as target industrial equipment when the fault probability of the industrial equipment is larger than a preset fault probability threshold;
determining the optimal data acquisition frequency of an equipment data acquisition unit corresponding to the target industrial equipment according to the fault probability of the target industrial equipment;
predicting the future load of the virtual host through a load prediction model based on the optimal data acquisition frequency of the equipment data acquisition unit corresponding to the target industrial equipment, and uploading the predicted future load of the virtual host to the virtual host scheduling server;
The virtual host scheduling server dynamically adjusts the corresponding relation between a plurality of virtual hosts and a plurality of device data acquisition units and/or a plurality of region data acquisition units based on the state data of the edge server and the state data of the virtual hosts, and the method comprises the following steps:
Taking the virtual host for uploading future loads as a target virtual host;
And determining a virtual host to be scheduled from a plurality of virtual hosts based on the state data of each edge server, the state data of each virtual host and the future load of the target virtual host, and scheduling the virtual host to be scheduled.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311514514.4A CN117290668B (en) | 2023-11-15 | 2023-11-15 | Big data processing method and system based on industrial Internet platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311514514.4A CN117290668B (en) | 2023-11-15 | 2023-11-15 | Big data processing method and system based on industrial Internet platform |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117290668A CN117290668A (en) | 2023-12-26 |
| CN117290668B true CN117290668B (en) | 2024-06-21 |
Family
ID=89240943
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311514514.4A Active CN117290668B (en) | 2023-11-15 | 2023-11-15 | Big data processing method and system based on industrial Internet platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117290668B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117544985B (en) * | 2024-01-09 | 2024-03-19 | 成都趣点科技有限公司 | Equipment offline communication management method and system |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104166589A (en) * | 2013-05-17 | 2014-11-26 | 阿里巴巴集团控股有限公司 | Heartbeat package processing method and device |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102694381B (en) * | 2011-03-25 | 2014-07-23 | 珠海优特电力科技股份有限公司 | Multistage electrical-network self-healing control method |
| CN115313662A (en) * | 2022-09-07 | 2022-11-08 | 国网山东省电力公司临沂供电公司 | Distribution automation intelligence auxiliary tool that patrols and examines |
| CN116680752B (en) * | 2023-05-23 | 2024-03-19 | 杭州水立科技有限公司 | Hydraulic engineering safety monitoring method and system based on data processing |
-
2023
- 2023-11-15 CN CN202311514514.4A patent/CN117290668B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104166589A (en) * | 2013-05-17 | 2014-11-26 | 阿里巴巴集团控股有限公司 | Heartbeat package processing method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117290668A (en) | 2023-12-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhao et al. | Intelligent digital twin-based software-defined vehicular networks | |
| CN117290668B (en) | Big data processing method and system based on industrial Internet platform | |
| US11627517B2 (en) | Network provisioning | |
| Chemouil et al. | Special issue on artificial intelligence and machine learning for networking and communications | |
| CN109873726B (en) | Robust service quality prediction and guarantee method based on deep learning in SDN | |
| EP3644581B1 (en) | Edge-cloud collaboration system for analyzing internet of things data and operating method thereof | |
| CN112199189A (en) | Adaptation of deep learning models to resource constrained edge devices | |
| CN103034267B (en) | Apparatus and a method for controlling facility devices | |
| US20210209488A1 (en) | Inference computing apparatus, model training apparatus, inference computing system | |
| CN111935820A (en) | Positioning implementation method based on wireless network and related equipment | |
| Li | Retracted: Design and implementation of music teaching assistant platform based on Internet of Things | |
| Wang et al. | IndustEdge: A time-sensitive networking enabled edge-cloud collaborative intelligent platform for smart industry | |
| Li et al. | Research and application of AI in 5G network operation and maintenance | |
| CN115756642A (en) | Business process configuration method, device, equipment and storage medium | |
| CN117640335B (en) | Dynamic adjustment and optimization method for intelligent building comprehensive wiring | |
| CN114648223A (en) | Smart city energy consumption data mining system and method based on Internet of things | |
| Banaie et al. | Complementing IIoT services through AI: Feasibility and suitability | |
| CN112995287B (en) | Keyword detection task scheduling method facing edge calculation | |
| Maatoug et al. | A location-based fog computing optimization of energy management in smart buildings: DEVS modeling and design of connected objects | |
| CN116723136B (en) | Network data detection method applying FCM clustering algorithm | |
| Kaur | A comprehensive survey on architecture for big data processing in mobile edge computing environments | |
| CN117221295A (en) | Low-delay video transmission system based on edge calculation and network slicing | |
| Giang et al. | CityFlow: exploiting edge computing for large scale smart city applications | |
| CN116224791A (en) | Collaborative training control method for intelligent manufacturing collaborative robot edge system | |
| CN115913992A (en) | Anonymous network traffic classification method based on small sample machine learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |