CN112069779B - Offline data acquisition and report method based on hierarchical control - Google Patents
Offline data acquisition and report method based on hierarchical control Download PDFInfo
- Publication number
- CN112069779B CN112069779B CN202010798693.9A CN202010798693A CN112069779B CN 112069779 B CN112069779 B CN 112069779B CN 202010798693 A CN202010798693 A CN 202010798693A CN 112069779 B CN112069779 B CN 112069779B
- Authority
- CN
- China
- Prior art keywords
- data
- execution unit
- task
- level
- layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
- G06F40/174—Form filling; Merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
- G06F40/177—Editing, e.g. inserting or deleting of tables; using ruled lines
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- Bioethics (AREA)
- Data Mining & Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
An offline data collecting and reporting method based on hierarchical control comprises the following steps: constructing a multi-level hierarchical control execution unit, wherein each layer is provided with one or more execution units, and each execution unit is provided with a local resource library; the top-level execution unit issues a task packet for data acquisition to the first level, and the execution unit in the first level receives the data acquisition task of the top-level execution unit and creates a new data acquisition task until the data acquisition task is issued to the ith-level execution unit at the bottommost layer; and the ith-level execution unit at the bottommost layer executes the reporting task after the local resource library finishes data filling, generates a corresponding data packet, and circulates until the top-level execution unit. The invention introduces the concept of the local resource library, takes the local resource library as an intermediate layer of data transition, realizes the decoupling of the acquisition tasks, and solves the problems that the task is difficult to be decomposed, the task model cannot be customized individually, the data aggregation workload between each level is high, the difficulty is high and the like when the excel is used for acquisition.
Description
Technical Field
The invention relates to the field of computers, in particular to a method for offline data acquisition and reporting by adopting a hierarchical control execution unit.
Background
In the prior art, data collection and reporting are often required. The existing offline data acquisition and reporting method mainly adopts excel, but the excel has the following defects:
a) excel is difficult to acquire data of a complex data structure with multi-table association;
b) After the collection is finished, manual and technical means are needed to import the data in the excel into a database system, so that the calculation and the application are convenient;
c) The excel collected by a plurality of units needs complicated customized technical means or consumes a large amount of manpower, and data summarization, data duplication checking and verification are carried out;
d) Excel has difficulty in handling the collection and aggregation of associated unstructured data (files).
e) And when the data volume is too large, the excel performance is in a bottleneck.
f) The hierarchical reporting path of the data needs to be manually recorded to trace the data source.
Therefore, how to solve the problem of various types of data filling, solve the problems of decomposition, reporting and summarization of data in different task scenes, reduce the workload of data acquisition and reporting, record a data reporting path, facilitate data source tracing, and become a technical problem to be solved in the prior art.
Disclosure of Invention
The invention aims to provide a method for performing offline data acquisition and report on the basis of an execution unit of hierarchical control so as to solve the problem of flexible customization of data hierarchy acquisition and report.
In order to achieve the purpose, the invention adopts the following technical scheme:
an off-line data collecting and reporting method based on hierarchical control comprises the following steps:
the hierarchical control execution unit construction step S110: the method comprises the steps of constructing a multi-level hierarchical control execution unit, wherein the hierarchical control execution unit comprises a top-level execution unit, a first-layer execution unit, a second-layer execution unit, a third-layer execution unit, a fourth-layer execution unit and an ith-layer execution unit, wherein the first-layer execution unit, the second-layer execution unit, the third-layer execution unit, the fourth-layer execution unit and the ith-layer execution unit are sequentially arranged downwards, the execution units except the top-level execution unit comprise a middle-layer execution unit and a bottom-layer execution unit, each layer is provided with one or more execution units, each execution unit is provided with a local resource library, the top-level execution unit customizes and creates a data model according to a report collecting requirement and formulates a sending task, and the sending task comprises: task name, task requirement, task content, information of the unit to be executed and the issued data model;
a top-level task packet issuing step S120: the top-level execution unit issues a task packet for data acquisition to the first level, and generates a corresponding task packet for each execution unit in the first level, wherein the task packet includes task main body information, and the task main body information includes: task name, task requirement, task content, execution unit information and data model information;
a step S130 of decoupling and issuing the hierarchical task package: the method comprises the steps that an execution unit in a first level receives a data acquisition task of a top-level execution unit, guides a task package into a local resource library and decouples the task package, wherein the decoupling is to extract a data model in the task package to the local resource library, the execution unit in the first level can serve as a task initiator, a directly subordinate acquisition execution unit is well defined, the acquisition data model loaded to the local is selected to be expanded or an original model is used according to the self condition, a new data acquisition task is created, a new task package for data acquisition is generated, the new task package is continuously issued to the directly subordinate execution unit, namely the second level execution unit, the task is decomposed and issued at the current level, and the process is circulated until the data acquisition task is issued to an i-level execution unit at the bottommost layer;
a step S140 of collecting and reporting the layered data packets: the method comprises the steps that an i-level execution unit at the bottommost layer extracts data into a corresponding reporting task after the data filling is completed in a local resource library and marks a data source, the i-level execution unit executes the reporting task and generates a corresponding data packet, the data packet comprises task main body information, task main body information task names, task requirements, task contents, execution unit information and model information, the filling data information of a model, the i-1-level execution unit receives one or more data packets from a lower level, namely the i-level execution unit, collects and examines the data packets, synchronizes the examined data to the i-1-level execution unit local resource library, extracts a data marking data source after the original local data and the data reported by the i-level execution unit are combined and revised in the local resource library, completes the data filling, generates a new data packet, performs tasks on the i-2-level execution unit, and circulates until a top-level execution unit receives one or more data packets from the first-level execution unit, collects and reports the data packets to the i-level execution unit, and collects and then completes the data packets, and then the tasks are collected and the data are reported to the top-level execution unit.
Optionally, in step S120, the task package further includes an attachment, and the task body information of the task package further includes reference dictionary information and reference model information related to the model information, where the reference dictionary information is dictionary data referenced in the data model, and the reference model information is other table data referenced in the data model.
Optionally, in step S140, the data packet further includes an attachment.
Optionally, the task packet and the task body information of the data packet are encrypted in a BASE64 manner by taking a JSON format as a data exchange media to report the task.
Optionally, the local repository is used for managing and maintaining local data, implementing flexible extension of a model, and implementing local decoupling of tasks.
Optionally, when data is extracted from the local resource library by each layer of data packet, the data source is simultaneously tagged with the current-level tag for reporting.
Optionally, each level of execution unit is only responsible for the adjacent execution units, the split granularity is defined by itself, a task decomposition unit is determined, the data task is redefined and decomposed at each level, decoupling is achieved, and consistency of the acquisition structure is achieved through the local resource library.
The invention further discloses a storage medium for storing computer executable instructions, which is characterized in that:
the computer executable instructions, when executed by a processor, perform the above-described offline data mining method based on hierarchical control.
The invention introduces the concept of a local resource library, takes the local resource library as an intermediate layer of data transition, realizes the decoupling of acquisition tasks, can flexibly disassemble or combine task models according to the acquisition requirements of the execution units at the current level, has no association between tasks, does not need to consider the data of other execution units at the level or across levels, and does not need to guide the operation of non-directly subordinate levels by each execution unit. The problems that when the excel is used for collection, tasks are difficult to decompose, a task model cannot be customized individually, data between each level are aggregated, the difficulty is high and the like are solved.
Drawings
FIG. 1 is a flow chart of an offline data mining method based on hierarchical control according to the present invention;
fig. 2 is an example of data mining, according to a specific embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not to be construed as limiting the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
The invention mainly comprises the following steps: the method introduces the concept of a local resource library, decouples the acquisition and report tasks through the local resource library, and realizes the acquisition and report of the off-line data of the layer-by-layer data. The local resource library refers to a library table which is stored in the system at the current level and comprises a plurality of data models and data.
Specifically, the method comprises the following steps: the method comprises the steps of constructing a multi-level hierarchical control execution unit, wherein the hierarchical control execution unit comprises a top-level execution unit, a first-level execution unit and a second-level execution unit \8230, wherein \8230isprovided, each layer is provided with one or more execution units, each execution unit is provided with a local resource library, and the task of collecting and reporting is decoupled through the local resource libraries, so that the off-line data collection and reporting of layer-by-layer data is realized; initiating a top-level superior of a collecting and reporting task, and finishing initial triggering of the collecting and reporting task; the acquisition execution unit in the middle layer has double-layer roles, the acquisition task is decoupled at the current level by using the local resource library, the acquisition task is executed and issued, after the task is completed, data filling of the model is completed in the local resource library, data is extracted from the local resource library, a label of the current level is added to a data source while extraction is performed, the task issued by the upper level is completed, and the task is reported layer by layer.
Due to the fact that decoupling of tasks is achieved in the local resource library, each level of execution unit is only responsible for adjacent execution units, for example, a directly superior execution unit and/or a directly inferior execution unit, each level of execution unit only concerns decomposition of the tasks to the directly inferior execution unit, and does not need to concern how the tasks are decomposed in a lower level, and an organization mechanism and execution units of a non-directly inferior level are not needed to be known. Each level of execution unit defines the split granularity according to the condition of the execution unit, and determines a task decomposition unit, and the task is redefined and decomposed at each level to realize decoupling; and the consistency of the acquisition structure is realized through the local resource library.
Specifically, referring to fig. 1, a flowchart of an offline data mining method based on hierarchical control according to the present invention is shown, which includes the following steps:
the hierarchical control execution unit construction step S110: a multi-level hierarchical control execution unit is constructed, the hierarchical control execution unit comprises a top-level execution unit, a first-level execution unit, a second-level execution unit, a third-level execution unit, a fourth-level execution unit and a fourth-level execution unit, wherein the first-level execution unit, the second-level execution unit, the third-level execution unit, the fourth-level execution unit and the fourth-level execution unit are sequentially arranged downwards, the execution units except the top-level execution unit are middle-layer execution units, one or more execution units are arranged on each layer, a local resource library is arranged in each execution unit, the top-level execution unit can establish a data model and formulate a distribution task, and the distribution task comprises: the task name, the task requirement, the task content, the information of the unit to be executed, the issued data model, and other information such as accessories.
A top-level task packet issuing step S120: the top-level execution unit issues a task packet for data acquisition to the first level, and generates a corresponding task packet for each execution unit in the first level, wherein the task packet comprises task main body information, and the task main body information comprises: task name, task requirements, task content, execution unit information, data model information (i.e., a table desired to be submitted), reference dictionary information and reference model information related to the model information, the reference dictionary information being dictionary data referenced in the data model, the reference model information being other table data referenced in the data model.
In the step, the top-level and the upper-level of the acquisition and report task are initiated, the organization execution unit structure of the lower-level and the upper-level does not need to be concerned, only the secondary acquisition execution unit needs to be defined, the acquisition data model definition is completed in the local resource library, the acquisition task is created and issued, and the initial triggering of the acquisition and report task can be completed.
A step S130 of decoupling and issuing the hierarchical task package: the method comprises the steps that an execution unit in a first stage receives a data acquisition task of a top-stage execution unit, guides a task package into a local resource library and decouples the task package, the decoupling is to extract a data model in the task package to the local resource library, the execution unit in the first stage can serve as a task initiator, a directly subordinate acquisition execution unit is well defined, according to the self condition, a collection data model loaded to the local is selected to be expanded or an original model is used, a new data acquisition task is created, a new task package for data acquisition is generated, the new task package is continuously issued to the directly subordinate execution unit, namely an execution unit in a second stage, decomposition and issuing of the task at the current stage are achieved, the execution unit in the second stage can serve as the task initiator, the directly subordinate acquisition execution unit is well defined, according to the self condition, the collection data model loaded to the local is selected to be expanded or the original model is used, the new data acquisition task package is created, the new task package for data acquisition is generated, and the new task package is continuously issued to the directly subordinate execution unit, namely an execution unit in the third stage, and the task package is issued to the lowest stage, namely the directly subordinate acquisition execution unit, and the task package is issued as the lowest stage of the currently acquired data acquisition execution unit.
In an optional embodiment, the task package further comprises an attachment, and the task body information further comprises reference dictionary information and reference model information related to model information, wherein the reference dictionary information is dictionary data referenced in the data model, and the reference model information is other table data referenced in the data model.
The task package is transmitted in an off-line mode, the task body information can be encrypted in a BASE64 mode by taking a JSON format as a data exchange medium. Of course, the present invention is not limited thereto, and any encryption algorithm is within the scope of the present invention.
For example, referring to fig. 2, the task packet A1 corresponds to the execution unit T01, the task packet A2 corresponds to the execution unit T02, and so on. And the execution unit T01 receives and imports the task package A1, analyzes the information in the task package and extracts the data model in the task package to the local resource library. The execution unit T01 will generate two completely new task packages, where the task package B1 corresponds to the execution unit T011 and the task package B2 corresponds to the execution unit T012, and at this time, the task packages B1 and B2 are completely decoupled in service because the data model is from the previous task package A1 and the object reported by the data model is not the execution unit T0. And the task packet B1 is issued to the execution unit T011 in an off-line transmission mode. And the execution unit T011 receives the task packet B1 and imports the task packet, extracts the data model in the task packet to a local resource library, and continuously copies the task decomposition process until the data model is transmitted to the execution unit Tn at the bottommost layer. T011 is currently considered the lowest level execution unit.
In step S130, the acquisition execution unit in the middle layer has a double-layer role, and simultaneously serves as an executor and an initiator of the acquisition task, and the acquisition task is decoupled at this level by using the local repository and is issued to the directly subordinate execution unit.
The meaning of decoupling: the middle execution unit is used as an executor and receives the acquisition task of the initiator, and the acquisition data model definition in the middle execution unit is loaded to the local resource library. The middle execution unit defines a directly subordinate acquisition execution unit, and selects to expand the data acquisition model loaded to the local or use the original model according to the self condition, create and issue a new acquisition task, and realize the decomposition and the issue of the task at the current level. The task A issued by the upper level is grounded locally and is converted into a task B to be executed at the level to continue issuing, but the tasks A and B have no relation, and the association between the tasks is interrupted.
The collection content definition of the task is inherited through the model, and the source of the data is reported through a label printed on the data. Tasks, while decoupled locally, lose little if any information.
A step S140 of collecting and reporting the layered data packets: the ith-level execution unit at the bottom layer extracts the data into the corresponding reporting task after the local resource library finishes data filling, marking data source, completing data filling work, the i-th level execution unit executing reporting task and generating corresponding data packet, the data packet comprises task main body information and attachments, the task main body information comprises a task name, a task requirement, task content, execution unit information, model information and filling data information of the model, the i-1 level execution unit receives the information from the lower level, namely one or more data packets of the ith-level execution unit, collects and examines the data packets, synchronizes the examined data to a local resource library, after the original local data and the data reported by the ith-level execution unit are merged and revised by the local resource library, the data label data source is extracted to complete data filling, and generates a new data packet to report to the i-2 level execution unit, the i-2 level execution unit receives the data packet from the lower level, namely one or more data packets of the i-1 level execution unit, collects and examines the data packets, synchronizes the examined data to a local resource library, after the original local data and the data reported by the i-1 level execution unit are merged and revised by the local resource library, the data marking data source is extracted to complete data filling, generating new data packets and reporting the new data packets to the i-3 level execution unit, and repeating the steps until the top level execution unit receives one or more data packets from the first level execution unit, and summarizing and examining the data packet, and synchronizing the examined data to a local resource library so as to complete the whole acquisition work.
After the task is completed, the acquisition execution unit of the middle layer receives the data acquired and reported by each directly subordinate, the received data is loaded to the local resource library, the original local data and the data reported by the subordinate execution unit are merged and revised in the local resource library, then the data returns to the superior task to be completed by the middle layer execution unit, and the reported data packet is generated by extracting the data of the local resource library, so that the superior dating task is completed.
And when the middle layer execution unit extracts data from the local resource library, the middle layer execution unit simultaneously adds the current-level label to the data source to form a data packet for reporting.
The acquisition task packages of all levels are decoupled in a local ground mode, no relation is established among tasks, data sources are recorded only in data, the quality of acquired and reported contents is guaranteed through data models with the same upper level and lower level, the data sources are recorded, flexible expansion of acquisition in all layers is achieved, and tasks are issued, filled and reported layer by layer.
In an optional embodiment, the task body information of the data packet reports the task in JSON format as a data exchange medium.
For example, referring to fig. 2, the execution unit T011 is used as the execution unit at the bottom layer, extracts data into a corresponding reporting task after the local resource library completes data report, and marks the data source as the execution unit T011 to complete task report work.
The execution unit T011 reports tasks and generates a corresponding data packet B1, wherein the data packet comprises two parts of contents: task body information and attachments, wherein the task body information reports a task by taking a JSON format as a data exchange medium, and the specific information comprises: task name, task requirement, task content, execution unit information, model information and filling data information of the model. And reporting the task packet B1 to an upper-level execution unit T01 in an off-line transmission mode. In the same manner, the execution unit T012 at the same level reports the task packet B2 to T01. And the execution unit T01 receives the data packets B1 and B2, summarizes the two data packets, reviews the summarized data, and synchronizes the reviewed data to the local resource library. After the original local data and the data reported by the two execution units T011 and T012 are merged and revised by the local resource library, the extracted data is sent to the task A1, and meanwhile, the data source is marked as T01, and the report work of the execution unit T01 is completed.
The execution unit T01 generates a corresponding data packet A1 for reporting, and the process is consistent with the reporting process of the execution unit T011. And reporting the task packet A1 to a superior execution unit T0 in an off-line transmission mode. The peer execution units T02 and T03 also report their respective task packages to the execution unit T01 in the same manner.
The top-level execution unit T0 receives the data packages A1, A2, an \8230;, collects a plurality of data packages, examines the collected data, and finally extracts the examined data to a local resource library, so that the execution unit T0 completes the whole collection work.
In the embodiment, the local resource library is used as an intermediate layer with excessive data, so that the decoupling of the acquisition tasks is realized, the task models can be flexibly disassembled or combined according to the acquisition requirements of the execution unit at the current level, no association exists between the tasks, other horizontal or cross-level execution unit data is not required to be considered, and each execution unit is not required to guide the non-directly subordinate work. The problems that when the excel is used for collection, tasks are difficult to decompose, a task model cannot be customized individually, data aggregation workload among all levels is high, and the difficulty is high are solved.
After the task is completed, the acquisition execution unit of the middle layer receives the acquired and reported data of each directly subordinate, the received data is loaded to the local resource library, the data filling of the model is completed in the local resource library, then the higher-level task received by the execution unit of the middle layer is returned, the data is extracted from the local resource library, the data source is added with the label of the current level while the data is extracted, the task issued by the higher level is completed, and the report is performed.
The collection task packages of all levels are decoupled on the ground locally, no relation is established among tasks, data sources are only recorded in data, the quality of collected and reported contents is guaranteed through data models with the same upper level and the same lower level, the data sources are recorded, flexible expansion of collection in all levels is achieved, and tasks are distributed, filled and reported layer by layer.
The invention is carried out in an off-line mode, is suitable for a scene that data with high confidentiality requirement cannot be transmitted on line through a network and can only be transmitted off-line through an encryption packet mode.
The off-line collecting and reporting software designed under the idea of the hierarchical control execution unit has good adaptability and flexibility for the scene. In the hierarchical control execution unit, the top layer execution unit does not need to care about how tasks are decomposed and filled in each layer, does not need to guide each layer to carry out specific sampling and reporting, does not need to care about organization execution units of non-directly subordinate levels, only needs to define a data structure, is responsible for directly subordinate levels, reduces a large amount of work of a superior unit, greatly reduces the work difficulty and the work load of each layer of execution unit, and saves the communication cost among all levels of execution units.
The acquisition task packages of all levels are decoupled in a local ground mode, the tasks of all levels have no relation, only data sources are recorded in data, the quality of acquired and reported contents is guaranteed through a data model with the same upper level and lower level, the data sources are recorded, the flexible expansion of acquisition in all levels is realized, and the tasks are issued, filled and reported layer by layer and are summarized and reported. In the scheme, each acquisition is carried out in a task packet mode, so that the multiplexing of the model is realized, the repeated construction is avoided, and the resources are saved. When the tasks are issued, each lower-level execution unit has a task package, and the tasks are correspondingly imported one by one when being reported, so that the completeness of task execution can be ensured, and omission can be avoided. And in the issuing and executing process of each task, the executing state of the monitoring task is tracked in the whole process, and the smooth completion of the task is ensured.
In the invention, the local resource library functions can manage and maintain local data, realize flexible extension of the model and realize local decoupling of tasks.
The local storage of the self-received task and the self-issued task information can be realized by using the task collection and report; and automatically recording a data reporting path to realize the tracing of data sources.
The invention also discloses a storage medium for storing computer executable instructions, which is characterized in that:
the computer-executable instructions, when executed by a processor, perform the above-described hierarchical control-based offline data mining method.
The invention has the following application scenes:
1. based on the service requirement scene of the task, each acquisition is carried out in a task mode;
2. the method has the advantages that the requirement of layered and multi-level data collection exists, and all levels of execution units of tree-shaped organization can only use offline packets to send and gather data layer by layer due to different networks or confidentiality requirements and the like;
3. complex data with multiple associations, including the relationship among data tables, the relationship between data and unstructured files, and the like, need to be collected;
4. the content and the report collecting organization mode which need to be collected and reported can be flexibly customized according to the requirements;
5. source tracing of the collected and reported content is required.
The technical problem that the invention needs and can solve is solved.
1. The problem that data with complex relationships are difficult to fill and process during collecting and reporting is solved.
2. The problem of decomposition, reporting and summarization of the task layer by layer in a multi-level data acquisition task scene is solved.
3. The problem that the collected and reported content and the collected and reported organization mode cannot be flexibly customized is solved, and the model can be freely disassembled, combined and customized at each level and distributed to different lower-level execution units.
4. The problem that the workload is gradually increased upwards during collection is solved. In a traditional acquisition mode, a superior unit needs to guide to pay attention to the report filling condition of each layer, so that the workload is larger the more upward.
5. The data reporting path can be automatically recorded by designing layer-by-layer decomposition reporting of the recording task, and the data source can be conveniently traced.
It will be apparent to those skilled in the art that the various elements or steps of the invention described above may be implemented using a general purpose computing device, they may be centralized on a single computing device, or alternatively, they may be implemented using program code that is executable by a computing device, such that they may be stored in a memory device and executed by a computing device, or they may be separately fabricated into various integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
While the invention has been described in further detail with reference to specific preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (8)
1. An off-line data collecting and reporting method based on hierarchical control comprises the following steps:
the hierarchical control execution unit construction step S110: a multilevel hierarchical control execution unit is constructed, the hierarchical control execution unit comprises a top-level execution unit, a first-layer execution unit, a second-layer execution unit, a third-layer execution unit, a fourth-layer execution unit, an ith-layer execution unit and a third-layer execution unit, wherein the first-layer execution unit, the second-layer execution unit, the third-layer execution unit, the fourth-layer execution unit and the fourth-layer execution unit are sequentially arranged downwards, the execution units except the top-level execution unit comprise a middle-layer execution unit and a bottommost execution unit, one or more execution units are arranged on each layer, a local resource library is arranged in each execution unit, the top-level execution unit customizes and creates a data model according to a collecting and reporting requirement, and formulates a sending task, and the sending task comprises: task name, task requirement, task content, information of the unit to be executed and the issued data model;
a top level task package issuing step S120: the top-level execution unit issues a task packet for data acquisition to the first level, and generates a corresponding task packet for each execution unit in the first level, wherein the task packet comprises task main body information, and the task main body information comprises: task name, task requirement, task content, execution unit information and data model information;
a step S130 of decoupling and issuing the hierarchical task package: the method comprises the steps that an execution unit in a first stage receives a data acquisition task of a top-stage execution unit, guides a task packet into a local resource library and decouples the task packet, wherein the decoupling is to extract a data model in the task packet to the local resource library, the execution unit in the first stage can serve as a task initiator, a directly subordinate acquisition execution unit is well defined, according to the self condition, the acquisition data model loaded to the local is selected to be expanded or an original model is used to create a new data acquisition task, a new task packet for data acquisition is generated, the new task packet is continuously issued to the directly subordinate execution unit, namely an execution unit in a second stage, the decomposition and issuing of the task in the current stage are realized, and the steps are repeated until the data acquisition task is issued to an ith-stage execution unit in the lowest stage;
a step S140 of collecting and reporting the layered data packets: the method comprises the steps that an i-level execution unit at the bottommost layer extracts data into a corresponding reporting task after the data filling is completed in a local resource library and marks a data source, the i-level execution unit executes the reporting task and generates a corresponding data packet, the data packet comprises task main body information, task main body information task names, task requirements, task contents, execution unit information and model information, the filling data information of a model, the i-1-level execution unit receives one or more data packets from a lower level, namely the i-level execution unit, collects and examines the data packets, synchronizes the examined data to the i-1-level execution unit local resource library, extracts a data marking data source after the original local data and the data reported by the i-level execution unit are combined and revised in the local resource library, completes the data filling, generates a new data packet, performs tasks on the i-2-level execution unit, and circulates until a top-level execution unit receives one or more data packets from the first-level execution unit, collects and reports the data packets to the i-level execution unit, and collects and then completes the data packets, and then the tasks are collected and the data are reported to the top-level execution unit.
2. The offline data mining method based on hierarchical control according to claim 1, characterized in that:
in step S120, the task package further includes an attachment, and the task body information of the task package further includes reference dictionary information and reference model information related to model information, where the reference dictionary information is dictionary data referenced in the data model, and the reference model information is other table data referenced in the data model.
3. The offline data mining method based on hierarchical control according to claim 1, characterized in that:
in step S140, the data packet further includes an attachment.
4. The offline data mining method based on hierarchical control according to claim 2 or 3, characterized in that:
and the task packet and the task body information of the data packet are used as data exchange media report tasks in a JSON format and are encrypted in a BASE64 mode.
5. The offline data mining method based on hierarchical control according to claim 4, wherein:
the local resource library is used for managing and maintaining local data, realizing flexible extension of a model and realizing local decoupling of tasks.
6. The offline data mining method based on hierarchical control according to claim 4, wherein:
when each layer of data packet extracts data from the local resource library, the data source is added with the label of the current level at the same time, and the data is reported.
7. The offline data mining method based on hierarchical control according to claim 1, characterized in that:
each level of execution unit is only responsible for the adjacent execution units, the split granularity is defined by self, the task decomposition unit is determined, the data task is redefined and decomposed at each level, decoupling is realized, and the consistency of the acquisition structure is realized through the local resource library.
8. A storage medium for storing computer-executable instructions, characterized in that:
the computer-executable instructions, when executed by a processor, perform the offline data mining method based on hierarchical control of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010798693.9A CN112069779B (en) | 2020-08-11 | 2020-08-11 | Offline data acquisition and report method based on hierarchical control |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010798693.9A CN112069779B (en) | 2020-08-11 | 2020-08-11 | Offline data acquisition and report method based on hierarchical control |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112069779A CN112069779A (en) | 2020-12-11 |
| CN112069779B true CN112069779B (en) | 2023-01-17 |
Family
ID=73662625
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010798693.9A Active CN112069779B (en) | 2020-08-11 | 2020-08-11 | Offline data acquisition and report method based on hierarchical control |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112069779B (en) |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101719934B (en) * | 2009-12-11 | 2014-02-12 | 杭州华三通信技术有限公司 | Method, system and device for displaying uniform summary report on distributed data |
| CN107705098A (en) * | 2017-10-31 | 2018-02-16 | 远光软件股份有限公司 | Method of summary is worked out in financing plans |
| CN110968629A (en) * | 2019-11-27 | 2020-04-07 | 开普云信息科技股份有限公司 | Cross-hierarchy heterogeneous data aggregation-based unified information resource management method and system |
-
2020
- 2020-08-11 CN CN202010798693.9A patent/CN112069779B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN112069779A (en) | 2020-12-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102341781B (en) | Software test bed generation | |
| US7716253B2 (en) | Centralized KPI framework systems and methods | |
| CN107103448A (en) | Data integrated system based on workflow | |
| US20060041539A1 (en) | Method and apparatus for organizing, visualizing and using measured or modeled system statistics | |
| US20070220451A1 (en) | Method for modeling and documenting a network | |
| CN102279848A (en) | Method and device for generating report template | |
| CN108462885A (en) | A kind of video storage method, device and medium based on block chain | |
| US10666731B2 (en) | IRC-infoid data standardization for use in a plurality of mobile applications | |
| CN114218218A (en) | Data processing method, device and equipment based on data warehouse and storage medium | |
| CN104951623A (en) | Avionics system interface management system based on models | |
| CN112540975B (en) | Multi-source heterogeneous data quality detection method and system based on petri net | |
| CN109639791A (en) | Cloud workflow schedule method and system under a kind of container environment | |
| CN108241724A (en) | A kind of metadata management method and device | |
| CN105446952B (en) | For handling the method and system of semantic segment | |
| CN113642299A (en) | One-key generation method based on power grid statistical form | |
| CN108388623A (en) | ER relationships generation method, device, computer equipment and storage medium | |
| CN115617776A (en) | Data management system and method | |
| CN111538720B (en) | Method and system for cleaning basic data of power industry | |
| CN114661832A (en) | Multi-mode heterogeneous data storage method and system based on data quality | |
| CN105279138A (en) | Automatic generation system of message research report | |
| CN109951370B (en) | Hierarchical interconnection method and device for big data centers | |
| CN112069779B (en) | Offline data acquisition and report method based on hierarchical control | |
| CN112035466B (en) | External index development framework for block chain query | |
| CN113177088A (en) | Multi-scale simulation big data management system for material irradiation damage | |
| WO2016165317A1 (en) | Method and apparatus for establishing high-speed train demand data instances |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |