CN112069779A - Offline data acquisition and report method based on hierarchical control - Google Patents
Offline data acquisition and report method based on hierarchical control Download PDFInfo
- Publication number
- CN112069779A CN112069779A CN202010798693.9A CN202010798693A CN112069779A CN 112069779 A CN112069779 A CN 112069779A CN 202010798693 A CN202010798693 A CN 202010798693A CN 112069779 A CN112069779 A CN 112069779A
- Authority
- CN
- China
- Prior art keywords
- data
- task
- execution unit
- level
- resource library
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
- G06F40/174—Form filling; Merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
- G06F40/177—Editing, e.g. inserting or deleting of tables; using ruled lines
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Artificial Intelligence (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Bioethics (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
An offline data collecting and reporting method based on hierarchical control comprises the following steps: constructing a multi-level hierarchical control execution unit, wherein each layer is provided with one or more execution units, and each execution unit is provided with a local resource library; the top-level execution unit issues a task packet for data acquisition to the first level, and the execution unit in the first level receives the data acquisition task of the top-level execution unit and creates a new data acquisition task until the data acquisition task is issued to the ith-level execution unit at the bottommost layer; and the ith-level execution unit at the bottommost layer executes the reporting task after the local resource library finishes data filling, generates a corresponding data packet, and circulates until the top-level execution unit. The invention introduces the concept of the local resource library, takes the local resource library as an intermediate layer of data transition, realizes the decoupling of the acquisition task, and solves the problems that the task is difficult to be decomposed, the task model can not be customized individually, the data aggregation workload between each level is high, the difficulty is high and the like when the excel is used for acquisition.
Description
Technical Field
The invention relates to the field of computers, in particular to a method for offline data acquisition and reporting by adopting a hierarchical control execution unit.
Background
In the prior art, data collection and reporting are often required. The existing offline data acquisition and reporting method mainly adopts excel, but the excel has the following defects:
a) excel is difficult to acquire data of a complex data structure with multi-table association;
b) after the collection is finished, manual and technical means are needed to lead the data in the excel into a database system, so that calculation and application are convenient;
c) the excel collected by a plurality of units needs complicated customized technical means or consumes a large amount of manpower, and data summarization, data duplication checking and verification are carried out;
d) excel has difficulty in handling the collection and aggregation of associated unstructured data (files).
e) When the data volume is too large, the excel performance is a bottleneck.
f) The hierarchical reporting path of the data needs to be manually recorded, and the data source is traced.
Therefore, how to solve the problem of various types of data filling, solve the problems of decomposition, reporting and summarization of data in different task scenes, reduce the workload of data acquisition and reporting, record a data reporting path, facilitate data source tracing, and become a technical problem to be solved in the prior art.
Disclosure of Invention
The invention aims to provide a method for performing offline data acquisition and report on the basis of an execution unit of hierarchical control so as to solve the problem of flexible customization of data hierarchy acquisition and report.
In order to achieve the purpose, the invention adopts the following technical scheme:
an off-line data collecting and reporting method based on hierarchical control comprises the following steps:
the hierarchical control execution unit construction step S110: the method comprises the steps of constructing a multi-level hierarchical control execution unit, wherein the hierarchical control execution unit comprises a top-level execution unit, a first-layer execution unit and a second-layer execution unit … …, an ith-layer execution unit is sequentially arranged downwards, the execution units except the top-level execution unit comprise a middle-layer execution unit and a bottommost execution unit, one or more execution units are arranged on each layer, a local resource library is arranged in each execution unit, the top-level execution unit customizes and creates a data model according to a report collecting requirement and formulates a issued task, and the issued task comprises the following steps: task name, task requirement, task content, information of the unit to be executed and the issued data model;
a top-level task packet issuing step S120: the top-level execution unit issues a task packet for data acquisition to the first level, and generates a corresponding task packet for each execution unit in the first level, wherein the task packet comprises task main body information, and the task main body information comprises: task name, task requirement, task content, execution unit information and data model information;
a step S130 of decoupling and issuing the hierarchical task package: the method comprises the steps that an execution unit in a first level receives a data acquisition task of a top-level execution unit, guides a task package into a local resource library and decouples the task package, wherein the decoupling is to extract a data model in the task package to the local resource library, the execution unit in the first level can serve as a task initiator, a directly subordinate acquisition execution unit is well defined, the acquisition data model loaded to the local is selected to be expanded or an original model is used according to the self condition, a new data acquisition task is created, a new task package for data acquisition is generated, the new task package is continuously issued to the directly subordinate execution unit, namely the second level execution unit, the task is decomposed and issued at the current level, and the process is circulated until the data acquisition task is issued to an i-level execution unit at the bottommost layer;
a step S140 of collecting and reporting the layered data packets: the ith level execution unit at the bottommost layer extracts data into a corresponding reporting task after the data filling is finished in a local resource library and marks a data source, the ith level execution unit executes the reporting task and generates a corresponding data packet, the data packet comprises task main body information, the task main body information comprises a task name, a task requirement, task content, execution unit information, model information and filling data information of a model, the ith-1 level execution unit receives one or more data packets from a lower level, namely the ith level execution unit, summarizes and examines the data packets, synchronizes the examined data to the ith-1 level execution unit local resource library, and extracts the data marking data source after the original local data and the data reported by the ith level execution unit are merged and revised in the local resource library to finish the data filling, and generating a new data packet and reporting the new data packet to the i-2 level execution unit, and repeating the steps until the top level execution unit receives one or more data packets from the first level execution unit, summarizing and examining the data packets, and synchronizing the examined data to a local resource library so as to complete the whole collection work.
Optionally, in step S120, the task package further includes an attachment, and the task body information of the task package further includes reference dictionary information and reference model information related to the model information, where the reference dictionary information is dictionary data referenced in the data model, and the reference model information is other table data referenced in the data model.
Optionally, in step S140, the data packet further includes an attachment.
Optionally, the task body information of the task packet and the data packet is encrypted in a mode of BASE64 by taking a JSON format as a data exchange media report task.
Optionally, the local repository is used for managing and maintaining local data, implementing flexible extension of a model, and implementing local decoupling of tasks.
Optionally, when data is extracted from the local resource library by each layer of data packet, the data source is simultaneously tagged with the current level of tag, and reporting is performed.
Optionally, each level of execution unit is only responsible for the adjacent execution units, the split granularity is defined by itself, the task decomposition unit is determined, the data task is redefined and decomposed at each level, decoupling is achieved, and the consistency of the acquisition structure is achieved through the local resource library.
The invention further discloses a storage medium for storing computer executable instructions, which is characterized in that:
the computer-executable instructions, when executed by a processor, perform the above-described hierarchical control-based offline data mining method.
The invention introduces the concept of a local resource library, takes the local resource library as an intermediate layer of data transition, realizes the decoupling of acquisition tasks, can flexibly disassemble or combine task models according to the acquisition requirements of the execution units at the current level, has no association between tasks, does not need to consider the data of other execution units at the level or across levels, and does not need to guide the operation of non-directly subordinate levels by each execution unit. The problems that when the excel is used for collection, tasks are difficult to decompose, a task model cannot be customized individually, data between each level are aggregated, the difficulty is high and the like are solved.
Drawings
FIG. 1 is a flow chart of an offline data mining method based on hierarchical control according to the present invention;
fig. 2 is an example of data mining, according to a specific embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
The invention mainly comprises the following steps: the method introduces the concept of a local resource library, decouples the acquisition and report tasks through the local resource library, and realizes the offline data acquisition and report of layer-by-layer data. The local resource library refers to a library table which is stored in the system at the current level and comprises a plurality of data models and data.
Specifically, the method comprises the following steps: constructing a multi-level hierarchical control execution unit which comprises a top-level execution unit, a first-level execution unit and a second-level execution unit … …, wherein each layer is provided with one or more execution units, each execution unit is provided with a local resource library, and off-line data mining and reporting of layer-by-layer data is realized by decoupling a mining and reporting task through the local resource libraries; initiating a top-level superior of a collecting and reporting task, and finishing initial triggering of the collecting and reporting task; the acquisition execution unit of the middle layer has double-layer roles, the acquisition task is decoupled at the current level by utilizing the local resource library, the acquisition task is executed and issued, after the task is completed, data filling of the model is completed in the local resource library, data is extracted from the local resource library, a label of the current level is added to a data source while extraction is performed, the task issued by the upper level is completed, and the task is reported layer by layer.
Due to the fact that task decoupling is achieved in the local resource library, each level of execution unit is only responsible for adjacent execution units, for example, a directly superior execution unit and/or a directly inferior execution unit, each level of execution unit only cares about decomposing the task to the directly inferior execution unit, and does not need to care about how the task is decomposed in a lower level, and does not need to know about the organization mechanism and the execution unit of a non-directly inferior level. Each level of execution unit defines the split granularity according to the condition of the execution unit, determines a task decomposition unit, and redefines and decomposes the task at each level to realize decoupling; and the consistency of the acquisition structure is realized through the local resource library.
Specifically, referring to fig. 1, a flowchart of an offline data mining method based on hierarchical control according to the present invention is shown, which includes the following steps:
the hierarchical control execution unit construction step S110: the method comprises the steps of constructing a multi-level hierarchical control execution unit, wherein the hierarchical control execution unit comprises a top-level execution unit, an ith layer execution unit of a first layer execution unit and a second layer execution unit … … are sequentially arranged downwards, the execution units except the top-level execution unit are middle layer execution units, one or more execution units are arranged on each layer, a local resource library is arranged in each execution unit, the top-level execution unit can establish a data model and issue tasks, and the issue tasks comprise: the task name, task requirements, task content, information of the unit to be executed, the issued data model, and other information such as accessories can be included.
A top-level task packet issuing step S120: the top-level execution unit issues a task packet for data acquisition to the first level, and generates a corresponding task packet for each execution unit in the first level, wherein the task packet comprises task main body information, and the task main body information comprises: task name, task requirements, task content, execution unit information, data model information (i.e., a table desired to be submitted), reference dictionary information and reference model information related to the model information, the reference dictionary information being dictionary data referenced in the data model, the reference model information being other table data referenced in the data model.
In the step, the top-level and the upper-level of the acquisition and report task are initiated, the organization execution unit structure of the lower-level and the upper-level does not need to be concerned, only the secondary acquisition execution unit needs to be defined, the acquisition data model definition is completed in the local resource library, the acquisition task is created and issued, and the initial triggering of the acquisition and report task can be completed.
A step S130 of decoupling and issuing the hierarchical task package: the execution unit in the first stage receives the data acquisition task of the top-stage execution unit, guides the task package into a local resource library and decouples the task package, wherein the decoupling is to extract a data model in the task package to the local resource library, the execution unit in the first stage can serve as a task initiator, a directly subordinate acquisition execution unit is defined, the acquisition data model loaded to the local is selected to be expanded or an original model is used according to the self condition, a new data acquisition task is created, a new task package for data acquisition is generated, the new task package is continuously issued to the directly subordinate execution unit, namely the execution unit in the second stage, the decomposition and the issuing of the task at the current stage are realized, the execution unit in the second stage can serve as the task initiator, the directly subordinate acquisition execution unit is defined, and the acquisition data model loaded to the local is selected to be expanded or the original model is used according to the self condition, and creating a new data acquisition task, generating a new task packet for data acquisition, continuously issuing the new task packet to a directly subordinate execution unit, namely a third-level execution unit, and repeating the steps until the data acquisition task is issued to the ith-level execution unit at the bottommost layer.
In an optional embodiment, the task package further comprises an attachment, and the task body information further comprises reference dictionary information and reference model information related to model information, wherein the reference dictionary information is dictionary data referenced in the data model, and the reference model information is other table data referenced in the data model.
The task package is transmitted in an off-line mode, and the task body information can be encrypted in a BASE64 mode by taking a JSON format as a data exchange medium. Of course, the present invention is not limited thereto, and any encryption algorithm is within the scope of the present invention.
Illustratively, referring to FIG. 2, task Package A1 corresponds to EU T01, task Package A2 corresponds to EU T02, and so on. The execution unit T01 receives and imports the task package A1, analyzes the information in the task package and extracts the data model in the task package to the local repository. The executive unit T01 will generate two completely new task packages, where the task package B1 corresponds to the executive unit T011 and the task package B2 corresponds to the executive unit T012, and at this time, the task packages B1 and B2 are completely decoupled in service because the data model is derived from the previous task package a1 and the reported object is not the executive unit T0. The task packet B1 is sent to the execution unit T011 through offline transmission. And the execution unit T011 receives and imports the task package B1, extracts the data model in the task package to a local resource library, and continuously copies the task decomposition process until the data model is transmitted to the execution unit Tn at the bottom layer. T011 is currently considered the lowest level execution unit.
In step S130, the acquisition execution unit in the middle layer has a double-layer role, and simultaneously serves as an executor and an initiator of the acquisition task, and the acquisition task is decoupled at this level by using the local repository and is issued to the directly subordinate execution unit.
The meaning of decoupling: the middle execution unit is used as an executor and receives the acquisition task of the initiator, and the acquisition data model definition in the middle execution unit is loaded to the local resource library. The middle execution unit defines a directly subordinate acquisition execution unit, and selects to expand the data acquisition model loaded to the local or use the original model according to the self condition, create and issue a new acquisition task, and realize the decomposition and the issue of the task at the current level. The task A issued by the upper level is landed locally, and is converted into a task B to be executed at the current level to be issued continuously, but A, B tasks have no relation, and the association between the tasks is broken.
The collection content definition of the task is inherited through the model, and the source of the data is reported through a label printed on the data. Tasks, while decoupled locally, lose little if any information.
A step S140 of collecting and reporting the layered data packets: the ith level execution unit at the bottommost layer extracts data into a corresponding reporting task after the data filling is finished in a local resource library, marks a data source and finishes the data filling work, the ith level execution unit executes the reporting task and generates a corresponding data packet, the data packet comprises task main body information and accessories, the task main body information task name, task requirements, task content, execution unit information, model information and the filling data information of the model, the ith-1 level execution unit receives one or more data packets from a subordinate level, namely the ith level execution unit, summarizes and examines the data packets, synchronizes the examined data to the local resource library, and extracts the data marked data source after the local resource library merges and reports the originally existing local data and the data reported by the ith level execution unit, completing data filling, generating a new data packet and reporting the new data packet to the i-2 level execution unit, wherein the i-2 level execution unit receives the data from the lower level, namely one or more data packets of the i-1 level execution unit, collects and examines the data packets, synchronizes the examined data to a local resource library, after the original local data and the data reported by the i-1 level execution unit are merged and revised by the local resource library, the data marking data source is extracted to complete data filling, generating new data packets and reporting the new data packets to the i-3 th level execution unit, and repeating the steps until the top level execution unit receives one or more data packets from the first level execution unit, and summarizing and examining the data packet, and synchronizing the examined data to a local resource library so as to complete the whole acquisition work.
After the task is completed, the acquisition execution unit of the middle layer receives the data acquired and reported by each directly subordinate, the received data is loaded to the local resource library, the original local data and the data reported by the subordinate execution unit are merged and revised in the local resource library, then the data returns to the superior task to be completed by the middle layer execution unit, and the reported data packet is generated by extracting the data of the local resource library, so that the superior dating task is completed.
And when the middle layer execution unit extracts data from the local resource library, the middle layer execution unit simultaneously adds the current-level label to the data source to form a data packet for reporting.
The collection task packages of all levels are decoupled on the ground locally, no relation is established among tasks, data sources are only recorded in data, the quality of collected and reported contents is guaranteed through data models with the same upper level and the same lower level, the data sources are recorded, flexible expansion of collection in all levels is achieved, and tasks are distributed, filled and reported layer by layer.
In an optional embodiment, the task body information of the data packet reports the task in JSON format as a data exchange medium.
For example, referring to fig. 2, the execution unit T011 is used as the execution unit at the bottom layer, extracts data into a corresponding reporting task after the local resource library completes data report, and marks the data source as the execution unit T011 to complete task report work.
The execution unit T011 performs a reporting task, and generates a corresponding data packet B1, where the data packet includes two parts: task body information and attachments, wherein the task body information reports a task by taking a JSON format as a data exchange medium, and the specific information comprises: task name, task requirement, task content, execution unit information, model information and filling data information of the model. The task packet B1 is reported to the upper execution unit T01 by an offline transmission mode. In the same manner, the execution unit T012 at the same level reports the task package B2 to T01. The execution unit T01 receives the data packets B1 and B2, aggregates the two data packets, reviews the aggregated data, and synchronizes the reviewed data to the local repository. After the local resource library merges and revises the originally existing local data and the data reported by the two execution units T011 and T012, the extracted data is sent to task A1, and meanwhile, the data source is marked as T01, and the reporting work of the execution unit T01 is completed.
The execution unit T01 generates the corresponding data packet a1, and reports the data packet a1, in a process consistent with the reporting process of the execution unit T011. The task package a1 is reported to the upper execution unit T0 by an offline transmission mode. In the same manner, the peer execution units T02 and T03 report their respective task packages to the execution unit T01.
The top-level execution unit T0 receives the data packets a1, a2, An … …, aggregates a plurality of data packets, examines the aggregated data, and extracts the examined data to the local repository, so that the execution unit T0 completes the whole collection work.
In the embodiment, the local resource library is used as an intermediate layer with excessive data, so that the decoupling of the acquisition tasks is realized, the task models can be flexibly disassembled or combined according to the acquisition requirements of the execution units at the current level, no association exists between the tasks, other horizontal or cross-level execution unit data is not required to be considered, and each execution unit does not need to guide the non-directly subordinate work. The problems that when the excel is used for collection, tasks are difficult to decompose, a task model cannot be customized individually, data between each level are aggregated, the difficulty is high and the like are solved.
After the task is completed, the acquisition execution unit of the middle layer receives data acquired and reported by each directly subordinate, loads the received data to the local resource library, completes data filling of the model in the local resource library, returns to the superior task received by the middle layer execution unit, extracts the data from the local resource library, adds a label of the current level to a data source during extraction, completes a task issued by the superior level, and reports the task.
The collection task packages of all levels are decoupled on the ground locally, no relation is established among tasks, data sources are only recorded in data, the quality of collected and reported contents is guaranteed through data models with the same upper level and the same lower level, the data sources are recorded, flexible expansion of collection in all levels is achieved, and tasks are distributed, filled and reported layer by layer.
The invention is carried out in an off-line mode, is suitable for the scenes that data with high confidentiality requirements cannot be transmitted on line through a network and can only be transmitted off line in an encryption packet mode.
The off-line reporting software designed under the idea of the hierarchical control execution unit has good adaptability and flexibility for the scene. In the hierarchical control execution unit, the top layer execution unit does not need to care about how tasks are decomposed and filled in each layer, does not need to guide each layer to carry out specific sampling and reporting, does not need to care about organization execution units of non-directly subordinate levels, only needs to define a data structure, is responsible for directly subordinate levels, reduces a large amount of work of a superior unit, greatly reduces the work difficulty and the work load of each layer of execution unit, and saves the communication cost among all levels of execution units.
The collection task packages of all levels are decoupled on the ground locally, the tasks of all levels have no relation, only data sources are recorded in data, the quality of collected and reported contents is ensured through data models with the same upper level and lower level, the data sources are recorded, the collection is flexibly expanded on all levels, and the tasks are distributed layer by layer, filled and reported and summarized and reported. In the scheme, each acquisition is carried out in a task package mode, the reuse of the model is realized, the repeated construction is avoided, and the resources are saved. When the tasks are issued, each lower-level execution unit has a task package, and the tasks are correspondingly imported one by one when being reported, so that the completeness of task execution can be ensured, and omission can be avoided. And in the issuing and executing process of each task, the executing state of the monitoring task is tracked in the whole process, and the smooth completion of the task is ensured.
In the invention, the local resource library functions can manage and maintain local data, realize flexible extension of the model and realize local decoupling of tasks.
The task collection and report can realize the local storage of the self-received task and the self-issued task information; and automatically recording a data reporting path to realize the tracing of data sources.
The invention also discloses a storage medium for storing computer executable instructions, which is characterized in that:
the computer-executable instructions, when executed by a processor, perform the above-described hierarchical control-based offline data mining method.
The invention has the following application scenes:
1. based on the service requirement scene of the task, each acquisition is carried out in a task mode;
2. the method has the advantages that the requirements of layered and multi-level data collection exist, and due to different networks or confidentiality requirements and the like, all levels of execution units of tree-shaped organization can only use off-line packets to send and gather data layer by layer;
3. complex data with multiple associations, including the relationship among data tables, the relationship between data and unstructured files, and the like, need to be collected;
4. the content and the report collecting organization mode which need to be collected and reported can be flexibly customized according to the requirements;
5. source tracing of the collected content is required.
The technical problem that the invention needs and can solve is solved.
1. The problem that data with complex relationships are difficult to fill and process during collecting and reporting is solved.
2. The problem of decomposition, reporting and summarization of the task layer by layer in a multi-level data acquisition task scene is solved.
3. The problem that the collected report content and the collected report organization mode cannot be flexibly customized is solved, and the model can be freely disassembled, combined and customized at each level and distributed to different subordinate execution units.
4. The problem that the workload is gradually increased upwards during collection is solved. In a traditional acquisition mode, a superior unit needs to guide to pay attention to the report filling condition of each layer, so that the workload is larger the more upward.
5. The data reporting path can be automatically recorded by designing layer-by-layer decomposition reporting of the recording task, and the data source can be conveniently traced.
It will be apparent to those skilled in the art that the various elements or steps of the invention described above may be implemented using a general purpose computing device, they may be centralized on a single computing device, or alternatively, they may be implemented using program code that is executable by a computing device, such that they may be stored in a memory device and executed by a computing device, or they may be separately fabricated into various integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
While the invention has been described in further detail with reference to specific preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (8)
1. An off-line data collecting and reporting method based on hierarchical control comprises the following steps:
the hierarchical control execution unit construction step S110: the method comprises the steps of constructing a multi-level hierarchical control execution unit, wherein the hierarchical control execution unit comprises a top-level execution unit, a first-layer execution unit and a second-layer execution unit … …, an ith-layer execution unit is sequentially arranged downwards, the execution units except the top-level execution unit comprise a middle-layer execution unit and a bottommost execution unit, one or more execution units are arranged on each layer, a local resource library is arranged in each execution unit, the top-level execution unit customizes and creates a data model according to a report collecting requirement and formulates a issued task, and the issued task comprises the following steps: task name, task requirement, task content, information of the unit to be executed and the issued data model;
a top-level task packet issuing step S120: the top-level execution unit issues a task packet for data acquisition to the first level, and generates a corresponding task packet for each execution unit in the first level, wherein the task packet comprises task main body information, and the task main body information comprises: task name, task requirement, task content, execution unit information and data model information;
a step S130 of decoupling and issuing the hierarchical task package: the method comprises the steps that an execution unit in a first level receives a data acquisition task of a top-level execution unit, guides a task package into a local resource library and decouples the task package, wherein the decoupling is to extract a data model in the task package to the local resource library, the execution unit in the first level can serve as a task initiator, a directly subordinate acquisition execution unit is well defined, the acquisition data model loaded to the local is selected to be expanded or an original model is used according to the self condition, a new data acquisition task is created, a new task package for data acquisition is generated, the new task package is continuously issued to the directly subordinate execution unit, namely the second level execution unit, the task is decomposed and issued at the current level, and the process is circulated until the data acquisition task is issued to an i-level execution unit at the bottommost layer;
a step S140 of collecting and reporting the layered data packets: the ith level execution unit at the bottommost layer extracts data into a corresponding reporting task after the data filling is finished in a local resource library and marks a data source, the ith level execution unit executes the reporting task and generates a corresponding data packet, the data packet comprises task main body information, the task main body information comprises a task name, a task requirement, task content, execution unit information, model information and filling data information of a model, the ith-1 level execution unit receives one or more data packets from a lower level, namely the ith level execution unit, summarizes and examines the data packets, synchronizes the examined data to the ith-1 level execution unit local resource library, and extracts the data marking data source after the original local data and the data reported by the ith level execution unit are merged and revised in the local resource library to finish the data filling, and generating a new data packet and reporting the new data packet to the i-2 level execution unit, and repeating the steps until the top level execution unit receives one or more data packets from the first level execution unit, summarizing and examining the data packets, and synchronizing the examined data to a local resource library so as to complete the whole collection work.
2. The offline data mining method based on hierarchical control according to claim 1, characterized in that:
in step S120, the task package further includes an attachment, and the task body information of the task package further includes reference dictionary information and reference model information related to model information, where the reference dictionary information is dictionary data referenced in the data model, and the reference model information is other table data referenced in the data model.
3. The offline data mining method based on hierarchical control according to claim 1, characterized in that:
in step S140, the data packet further includes an attachment.
4. The offline data mining method based on hierarchical control according to claim 2 or 3, characterized in that:
the task package and the task body information of the data package serve as data exchange media report tasks in a JSON format and are encrypted in a BASE64 mode.
5. The offline data mining method based on hierarchical control according to claim 4, wherein:
the local resource library is used for managing and maintaining local data, realizing flexible extension of a model and realizing local decoupling of tasks.
6. The offline data mining method based on hierarchical control according to claim 4, wherein:
when each layer of data packet extracts data from the local resource library, the data source is added with the label of the current level at the same time, and the data is reported.
7. The offline data mining method based on hierarchical control according to claim 1, characterized in that:
each level of execution unit is only responsible for the adjacent execution units, the split granularity is defined by self, the task decomposition unit is determined, the data task is redefined and decomposed at each level, decoupling is realized, and the consistency of the acquisition structure is realized through the local resource library.
8. A storage medium for storing computer-executable instructions, characterized in that:
the computer-executable instructions, when executed by a processor, perform the offline data mining method based on hierarchical control of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010798693.9A CN112069779B (en) | 2020-08-11 | 2020-08-11 | Offline data acquisition and report method based on hierarchical control |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010798693.9A CN112069779B (en) | 2020-08-11 | 2020-08-11 | Offline data acquisition and report method based on hierarchical control |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112069779A true CN112069779A (en) | 2020-12-11 |
| CN112069779B CN112069779B (en) | 2023-01-17 |
Family
ID=73662625
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010798693.9A Active CN112069779B (en) | 2020-08-11 | 2020-08-11 | Offline data acquisition and report method based on hierarchical control |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112069779B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101719934A (en) * | 2009-12-11 | 2010-06-02 | 杭州华三通信技术有限公司 | Method, system and device for displaying uniform summary report on distributed data |
| CN107705098A (en) * | 2017-10-31 | 2018-02-16 | 远光软件股份有限公司 | Method of summary is worked out in financing plans |
| CN110968629A (en) * | 2019-11-27 | 2020-04-07 | 开普云信息科技股份有限公司 | Cross-hierarchy heterogeneous data aggregation-based unified information resource management method and system |
-
2020
- 2020-08-11 CN CN202010798693.9A patent/CN112069779B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101719934A (en) * | 2009-12-11 | 2010-06-02 | 杭州华三通信技术有限公司 | Method, system and device for displaying uniform summary report on distributed data |
| CN107705098A (en) * | 2017-10-31 | 2018-02-16 | 远光软件股份有限公司 | Method of summary is worked out in financing plans |
| CN110968629A (en) * | 2019-11-27 | 2020-04-07 | 开普云信息科技股份有限公司 | Cross-hierarchy heterogeneous data aggregation-based unified information resource management method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112069779B (en) | 2023-01-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102341781B (en) | Software test bed generation | |
| US7716253B2 (en) | Centralized KPI framework systems and methods | |
| US20180225346A1 (en) | Data processing method, device and system | |
| CN107103448A (en) | Data integrated system based on workflow | |
| US20070220451A1 (en) | Method for modeling and documenting a network | |
| Trojer et al. | Living modeling of IT architectures: challenges and solutions | |
| CN101344941A (en) | Intelligent auditing decision tree generation method of 4A management platform | |
| CN105141441A (en) | IP network graphical configuration method | |
| CN114218218A (en) | Data processing method, device and equipment based on data warehouse and storage medium | |
| CN109639791A (en) | Cloud workflow schedule method and system under a kind of container environment | |
| CN111176613A (en) | Collaborative task automatic decomposition system based on architecture model | |
| CN111538720B (en) | Method and system for cleaning basic data of power industry | |
| CN113642299A (en) | One-key generation method based on power grid statistical form | |
| CN105279138A (en) | Automatic generation system of message research report | |
| CN112069779B (en) | Offline data acquisition and report method based on hierarchical control | |
| WO2016165317A1 (en) | Method and apparatus for establishing high-speed train demand data instances | |
| CN104898933A (en) | High-speed train demand data processing method and high-speed train demand data processing device | |
| CN115730022A (en) | Data processing construction method and platform system adopting event triggering and process arrangement | |
| CN115796758A (en) | Factory rule management platform | |
| CN105373996A (en) | Modeling system based on approval data | |
| CN108470087B (en) | Data bus of ramjet design simulation platform | |
| CN113805850A (en) | Artificial intelligence management system based on multiple deep learning and machine learning frameworks | |
| Gernhardt et al. | A semantic representation for process-oriented knowledge management based on functionblock domain models supporting distributed and collaborative production planning | |
| CN113342874A (en) | Wind power big data analysis system and process based on cloud computing | |
| CN112053134A (en) | B/S architecture-based power equipment information management system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |