CN114499624B - Multi-source data fusion processing method and system in heaven-earth integrated information network - Google Patents
Multi-source data fusion processing method and system in heaven-earth integrated information network Download PDFInfo
- Publication number
- CN114499624B CN114499624B CN202111495136.0A CN202111495136A CN114499624B CN 114499624 B CN114499624 B CN 114499624B CN 202111495136 A CN202111495136 A CN 202111495136A CN 114499624 B CN114499624 B CN 114499624B
- Authority
- CN
- China
- Prior art keywords
- task
- network
- module
- graph
- execution
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B7/00—Radio transmission systems, i.e. using radiation field
- H04B7/14—Relay systems
- H04B7/15—Active relay systems
- H04B7/185—Space-based or airborne stations; Stations for satellite systems
- H04B7/1851—Systems using a satellite or space-based relay
- H04B7/18519—Operations control, administration or maintenance
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B7/00—Radio transmission systems, i.e. using radiation field
- H04B7/14—Relay systems
- H04B7/15—Active relay systems
- H04B7/185—Space-based or airborne stations; Stations for satellite systems
- H04B7/1851—Systems using a satellite or space-based relay
- H04B7/18513—Transmission in a satellite or space-based system
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/30—Definitions, standards or architectural aspects of layered protocol stacks
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Astronomy & Astrophysics (AREA)
- Aviation & Aerospace Engineering (AREA)
- General Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a multi-source data fusion processing method and a system in a world-wide integrated information network, which comprises the following steps: the heaven and earth integrated information network carries out real-time management and control on network states and resources according to the logic management level; the ground control center divides a plurality of tasks acquired within a certain time and fuses the tasks according to the relationship among the tasks to form a multi-task execution graph; grouping and abstracting subtasks for the multitask execution graph, and forming a full-order abstract task graph after serialization; the ground control center carries out processing path planning according to the network view of the managed and controlled network domain and the serialized abstract task graph, and carries out minimum cost deployment planning on the abstract task; the controller in the control layer splits the received abstract tasks and carries out processing path planning and task deployment planning; after receiving the subtasks, the execution layer calculates and transmits the tasks according to the calculation strategy and the flow table issued by the control node; and returns the result of the calculation to the user.
Description
Technical Field
The invention relates to the field of task processing and data transmission, in particular to a multi-source data fusion processing method and system in a space-ground integrated information network, and more particularly to a multi-source data fusion processing method and system in a software-defined space-ground integrated network.
Background
The heaven-earth integrated information network is a huge and complex network architecture integrating a heaven-earth base network and a foundation network, wherein the heaven-earth base network mainly comprises various satellites (communication satellites, remote sensing satellites, navigation satellites and the like) operating in different orbits and various near-earth aircrafts, and the foundation network mainly comprises a ground internet and a mobile communication network. The main objective of the world integration is to provide a low-delay and high-bandwidth communication service for global users, realize interconnection and intercommunication of global cross-space-time networks, and optimize the use of network resources.
The software defined network technology is a new network management method, which can realize dynamic and high-efficiency network configuration and improve the performance of the whole network. The static architecture of traditional networks is decentralized and complex, while for all networks more flexibility and easier control methods are needed, SDN concentrates network intelligence in one network component by separating the forwarding process of network packets (data plane) from the routing decision process (control plane). The heaven-earth network adopts the architecture of the software defined network and a heaven-earth cooperative transmission scheme to realize the high integration of heaven-earth network resources, thereby optimizing the sharing of the whole network resources and providing a high-quality network service which is across space-time and across platforms for users with huge scale and wide dispersion through inter-satellite, satellite-earth and ground network links.
In the heaven and earth integrated network, the capabilities of nodes are uneven, various types of tasks are too expensive in the network, and the conventional heaven and earth integrated information network architecture lacks a distributed dynamic processing technology for the tasks. Specifically, the existing skynet transmission-computation architecture collects data required by tasks and transmits the data back to a data center for unified processing. Such a transmission-computation model ignores the computation power of the nodes in the network, resulting in a higher loss of transmission resources, while also placing considerable demands on the processing power of the data center.
Patent document CN110247699B (application number: 201910550797.5) discloses a method for realizing inter-space resource cooperative scheduling in a space-ground integrated network, and relates to the technical field of cloud computing; the method comprises the following steps: step one, establishing a resource coordination scheduling system; the method comprises the steps of comprising a resource scheduling node and N space-based resource nodes; step two, the resource scheduling node receives an external new application resource deployment command; acquiring resource occupation information of new application resources; thirdly, the resource scheduling node acquires the residual resource information of each space-based resource node; step four, comparing the residual resource information of each space-based resource node Ni at the time t with the resource occupation information of the new application resource, and screening out available space-based resource nodes; calculating an optimal space-based resource node from the screened space-based resource nodes, and deploying new application resources in the space-based resource node; according to the method, a resource scheduling system is established, externally deployed applications are received, resource occupation information of the externally deployed applications is acquired, meanwhile, the optimal resource nodes in the space-based resource nodes are screened, and finally the applications are deployed in the space-based resource nodes. The method solves the problem of resource imbalance in the world network.
A routing algorithm based on an intersatellite link is adopted in order to reduce queue delay and propagation delay caused by congestion, and the routing algorithm based on the intersatellite link is adopted in a heaven-earth integrated network based on a congestion state 37154inkstone et al weapon equipment engineering newspaper 2021,42 (06). The algorithm finds out the link between the source node and the destination node through a simulated annealing algorithm based on local network information. However, the scheme does not consider the global information of the network in the path selection process, and lacks of uniform regulation and control.
An inter-satellite routing algorithm for ensuring low orbit satellites by high-performance QoS based on an SDN (software defined network), wangQuyu and the like, computer engineering, 2021-10-18, researches state information interaction of inter-satellite links and routing calculation of low orbit satellites. The routing algorithm obtains the optimal path through weighting calculation of the weight function, the stability function and the load function provided by the routing algorithm, and is suitable for scenes with limited satellite communication network resources. But the scheme does not consider the possibility of reducing the network load through task calculation in the data routing process.
In addition, other transmission and calculation schemes in the integrated network based on software definition exist, but the research of the technical proposal does not consider the hierarchy of node management in the integrated network, the complexity of path planning and task deployment can be greatly reduced according to the hierarchy, and in addition, the research of the technical proposal does not consider the possible relevance of tasks in the integrated network, but carries out independent processing on each task, thereby increasing the load of a ground control center and further reducing the network resource loss.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a multi-source data fusion processing method and system in a heaven-earth integrated information network.
The invention provides a multisource data fusion processing method in a world-integrated information network, which comprises the following steps:
step S1: the heaven and earth integrated information network carries out real-time management and control on network states and resources according to the logic management level;
step S2: the ground control center divides a plurality of tasks acquired within a certain time and fuses the tasks according to the relationship among the tasks to form a multi-task execution graph;
and step S3: grouping and abstracting subtasks for the multitask execution graph, and forming a full-order abstract task graph after serialization;
and step S4: the ground control center carries out processing path planning according to the network view of the managed and controlled network domain and the serialized abstract task graph, and carries out minimum cost deployment planning on the abstract task;
step S5: the controller in the control layer splits the received abstract tasks and carries out processing path planning and task deployment planning;
step S6: after receiving the task, the execution layer calculates and transmits the task according to the calculation strategy and the flow table issued by the control node; and returning the transmission result to the user;
the logic management hierarchy of the heaven and earth integrated information network is divided into a ground control center, a control layer and an execution layer from top to bottom.
Preferably, the step S1 employs: the SDN architecture-based multi-layer heaven-earth integrated network is divided into an execution layer, a control layer and a ground control layer from bottom to top according to a logic management layer;
the ground control layer is a ground management center and is responsible for controlling a plurality of controllers of the control layer to acquire all control domain information;
the controller is deployed in a satellite network and a ground network and is divided into a plurality of areas according to the control range of the controller, each area is controlled by one controller, and the controllers control execution layer equipment;
the execution layer consists of transmission and calculation nodes in a world-wide integration network;
the heaven and earth integrated network carries out layered management and control on network states and resources from bottom to top, the execution nodes periodically upload network state information, and the controller integrates the network state information of the nodes in the controlled area and sends the information to the ground control center.
Preferably, the step S2 employs: the ground control center divides a plurality of tasks acquired within a certain time, and then performs task graph fusion according to task relevance and time delay sensitivity to form a multi-task execution graph.
Preferably, the step S2 employs:
step S2.1: the ground control center acquires corresponding position information, task source information and task type information aiming at each task uploaded by a user;
step S2.2: the ground control center determines a task delay sensitive value and a task waiting value according to the task source information and the task type information;
step S2.3: comparing the task waiting value with a delay sensitivity threshold, and triggering the step S3 when the task waiting value is greater than the delay sensitivity threshold; when the task waiting value is less than or equal to the delay sensitivity threshold, triggering step S2.4;
step S2.4: set T = { T) formed by ground control center for N tasks 1 ,t 2 ,…,t B Grouping according to the position information and the task source information;
step S2.5: performing task segmentation on all tasks in the grouped group to form a plurality of task graphs, wherein the task graphs are directed acyclic graphs formed by executable codes;
step S2.6: performing task graph fusion according to whether the pre-subtasks are the same or not; and if the unfused subtasks occur, increasing the task waiting value of the task to a preset value, and repeatedly triggering the step S2.3 to the step S2.6 until all the subtasks are fused.
Preferably, the step S3 employs:
step S3.1: performing full-order grouping according to the subtask execution dependence in the task graph;
step S3.2: abstracting data compression ratio and resource demand for each group of subtasks after grouping;
step S3.3: serializing the non-full-order task graph according to the data compression rate;
the non-full-order task graph is formed by the fact that input and output of subtasks in the task graph have no context.
Preferably, the step S4 employs:
step S4.1: determining a source node and a target node of a transmission path of a control layer;
step S4.2: planning a path by taking the path minimum cost as a target, and performing iterative computation through a heuristic algorithm, wherein each iteration adds a control domain to a routing path to obtain a minimum cost path;
step S4.3: and determining the deployment position of the abstract task by an abstract task deployment method with the aim of minimum deployment cost.
Preferably, the step S5 employs:
step S5.1: splitting the abstract task, and serializing a plurality of subtasks;
step S5.2: determining an execution layer transmission path source node and a target node;
step S5.3: iteratively calculating through a heuristic algorithm, adding an execution node to the routing path in each iteration, and acquiring a minimum cost path;
step S5.4: and determining the deployment position of the subtask by using a subtask deployment method with the minimum deployment cost as a target.
The invention provides a multisource data fusion processing system in a world integration information network, which comprises:
a module M1: the heaven and earth integrated information network carries out real-time management and control on network states and resources according to the logic management level;
a module M2: the ground control center divides a plurality of tasks acquired within a certain time and fuses the tasks according to the relationship among the tasks to form a multi-task execution graph;
a module M3: grouping and abstracting subtasks for the multitask execution graph, and forming a full-order abstract task graph after serialization;
a module M4: the ground control center carries out processing path planning according to the network view of the managed and controlled network domain and the serialized abstract task graph, and carries out minimum cost deployment planning on the abstract task;
a module M5: the controller in the control layer splits the received abstract tasks and carries out processing path planning and task deployment planning;
a module M6: after receiving the task, the execution layer calculates and transmits the task according to the calculation strategy and the flow table issued by the control node; and returning the transmission result to the user;
the logic management hierarchy of the heaven and earth integrated information network is divided into a ground control center, a control layer and an execution layer from top to bottom.
Preferably, the module M1 employs: the SDN architecture-based multi-layer heaven-earth integrated network is divided into an execution layer, a control layer and a ground control layer from bottom to top according to a logic management level;
the ground control layer is a ground management center and is responsible for controlling a plurality of controllers of the control layer to acquire all control domain information;
the controller is deployed in a satellite network and a ground network, and is divided into a plurality of areas according to the control range of the controller, each area is controlled by one controller, and the controller controls and controls the execution layer equipment;
the execution layer consists of transmission and calculation nodes in a world-wide integration network;
the heaven and earth integrated network carries out layered management and control on network states and resources from bottom to top, the execution nodes periodically upload network state information, and the controller integrates the network state information of the nodes in the controlled area and sends the information to the ground control center;
the module M2 employs: the ground control center divides a plurality of tasks acquired within a certain time, and then performs task graph fusion according to task correlation and time delay sensitivity to form a multi-task execution graph;
the module M2 employs:
module M2.1: the ground control center acquires corresponding position information, task source information and task type information aiming at each task uploaded by a user;
module M2.2: the ground control center determines a task delay sensitive value and a task waiting value according to the task source information and the task type information;
module M2.3: comparing the task waiting value with a delay sensitivity threshold, and triggering a module M3 when the task waiting value is greater than the delay sensitivity threshold; when the task waiting value is less than or equal to the delay sensitive threshold value, triggering a module M2.4;
module M2.4: ground control center sets T = { T) composed of N tasks 1 ,t 2 ,…,t N Performing grouping according to the position information and the task source information;
module M2.5: performing task segmentation on all tasks in the grouped group to form a plurality of task graphs, wherein the task graphs are directed acyclic graphs formed by executable codes;
module M2.6: performing task graph fusion according to whether the pre-subtasks are the same or not; if the unfused subtasks appear, the task waiting value of the task is increased to a preset value, and the module M2.3 to the module M2.6 are repeatedly triggered until all subtasks are fused.
Preferably, the module M3 employs:
module M3.1: performing full-order grouping according to the subtask execution dependencies in the task graph;
module M3.2: abstracting data compression ratio and resource demand for each group of subtasks after grouping;
module M3.3: serializing the non-full-order task graph according to a data compression rate;
the non-full-order task graph is characterized in that input and output of subtasks in the task graph have no front-back relation;
the module M4 employs:
module M4.1: determining a source node and a target node of a transmission path of a control layer;
module M4.2: planning a path by taking the path minimum cost as a target, and performing iterative computation through a heuristic algorithm, wherein each iteration adds a control domain to a routing path to obtain a minimum cost path;
module M4.3: determining an abstract task deployment position by an abstract task deployment method with the minimum deployment cost as a target;
the module M5 employs:
module M5.1: splitting the abstract task, and serializing a plurality of subtasks;
module M5.2: determining a source node and a target node of a transmission path of an execution layer;
module M5.3: iteratively calculating through a heuristic algorithm, adding an execution node to the routing path in each iteration, and acquiring a minimum cost path;
module M5.4: and determining the deployment position of the subtask by using a subtask deployment method with the minimum deployment cost as a target.
Compared with the prior art, the invention has the following beneficial effects:
1. through task fusion, certain task execution processes with the same processing process but different data sources can be combined, so that the loss of network transmission resources is reduced, and the task deployment time is reduced;
2. the task graph abstraction aims at that under the multi-data source task scene, data sources are distributed more dispersedly, and if the task graph serialization is directly carried out, an additional transmission link is added.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:
fig. 1 is a timing chart of a multisource data fusion processing method in a software-defined space-ground integrated information network used in the embodiment of the present invention.
Fig. 2 is a schematic diagram of a system module for multi-source data fusion processing in a software-defined space-ground integrated information network according to an embodiment of the present invention.
FIG. 3 is a diagram illustrating multi-task fusion according to an embodiment of the present invention.
FIG. 4 is a diagram illustrating task grouping and abstraction according to an embodiment of the present invention.
FIG. 5 is a diagram illustrating task serialization according to an embodiment of the present invention.
Detailed Description
The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that variations and modifications can be made by persons skilled in the art without departing from the concept of the invention. All falling within the scope of the present invention.
Example 1
The invention provides a multi-source data fusion processing method in a heaven and earth integrated information network, which comprises the following steps:
step S1: the heaven and earth integrated information network carries out real-time management and control on network states and resources according to the logic management level, describes network topology according to the level and integrates and quantizes available resources in the network topology;
step S2: the ground control center divides a plurality of tasks acquired within a certain time and fuses the tasks according to the relationship among the tasks to form a multi-task execution graph;
and step S3: grouping and abstracting subtasks on the multitask execution graph to form an abstract task graph, wherein the abstract task graph is formed by connecting abstract tasks, and the abstract tasks are the connection of a plurality of subtasks; forming a full-order abstract task graph after serialization;
and step S4: the ground control center carries out processing path planning according to the network view of the managed and controlled network domain and the serialized abstract task graph, and carries out minimum cost deployment planning on the abstract task;
step S5: after the controller in the control layer receives the abstract tasks issued by the ground control center, the abstract tasks are split, transmission path planning is carried out according to the controlled control domain network view, and each subtask is deployed;
step S6: after receiving the subtasks, the execution layer transmission computing node performs task computation and transmission according to a computation strategy and a flow table issued by the control node; and returning the transmission result to the user;
the logic management hierarchy of the heaven and earth integrated information network is divided into a ground control center, a control layer and an execution layer from top to bottom.
Specifically, the step S1 employs: the SDN architecture-based multi-layer heaven-earth integrated network is divided into an execution layer, a control layer and a ground control layer from bottom to top according to a logic management level;
the ground control layer is a ground management center and is responsible for controlling a plurality of controllers of the control layer to acquire all control domain information;
the controller is deployed in a satellite network and a ground network, and is divided into a plurality of areas according to the control range of the controller, each area is controlled by one controller, and the controller controls and controls the execution layer equipment; the controller integrates the network states of the nodes in the controlled area and provides the information to a ground control center;
the execution layer consists of transmission and calculation nodes in a world-wide integrated network;
the heaven and earth integrated network carries out layered management and control on network states and resources from bottom to top, the execution nodes periodically upload network state information, and the controller integrates the network state information of the nodes in the controlled area and sends the information to the ground control center.
Specifically, the step S2 employs: the ground control center divides a plurality of tasks acquired within a certain time, the time delay sensitivity of the tasks can be quantitatively expressed, and the task graph is fused according to the task correlation and the time delay sensitivity degree to form a multi-task execution graph. The subtasks are obtained by splitting each task into a plurality of execution blocks with dependency relationship. The task fusion has the advantages that in the face of extremely numerous tasks in the world network, some tasks with extremely high processing process correlation are merged, and network load and task deployment time can be greatly reduced.
Specifically, the step S2 employs:
step S2.1: the ground control center acquires corresponding position information, task source information and task type information aiming at each task uploaded by a user;
step S2.2: the ground control center determines a task delay sensitive value and a task waiting value according to the task source information and the task type information;
step S2.3: comparing the task waiting value with a delay sensitivity threshold, and triggering the step S3 when the task waiting value is greater than the delay sensitivity threshold; when the task waiting value is less than or equal to the delay sensitive threshold, triggering the step S2.4;
step S2.4: set T = { T) formed by ground control center for N tasks 1 ,t 2 ,…,t N Dividing the tasks in advance in order to increase the similarity of a plurality of tasks, and grouping the tasks according to the position information, the task source information and a plurality of factors of software;
step S2.5: performing task segmentation on all tasks in the grouped group to form a plurality of task graphs, wherein the task graphs are directed acyclic graphs formed by executable codes; a task can be split at multiple compiling stages, such as source code and intermediate code levels, and many methods exist to solve the problem of splitting the task, such as according to the dependency relationship between multiple code segments. Second, after the task is sliced, a number of execution blocks of code are formed, which may be logically viewed as being made up of input, output, and code segments. Each subtask is a block of code that is executable. The code execution blocks can form a task execution graph (task graph) according to the input and output dependency relationship.
Step S2.6: performing task graph fusion according to whether the pre-subtasks are the same or not; and if the unfused subtasks occur, increasing the task waiting value of the task to a preset value, and repeatedly triggering the step S2.3 to the step S2.6 until all the subtasks are fused. The pre-subtask means that after a task is divided into a task execution graph, the multiple execution graphs of multiple tasks have the same execution block (the same subtask), and the execution block can determine whether the multiple execution graphs are the same according to the code segment, the input and the output. The pre-task block refers to a plurality of sub-task blocks of the plurality of task graphs starting from a source point (a source point of the directed acyclic graph).
Specifically, the step S3 employs: and grouping according to the task graph after the multi-task fusion, wherein the grouping process is according to the input and output correlation among the subtasks, and after the grouping, a plurality of subtasks can be used as one task to be deployed and can also be divided into a plurality of subtasks again.
In particular, the amount of the solvent to be used,
step S3.1: performing full-order grouping according to the subtask execution dependencies in the task graph;
step S3.2: abstracting data compression rate and resource demand for each group of subtasks after grouping;
step S3.3: serializing the non-full-order task graph according to the data compression rate;
the non-full-order task graph is formed by the fact that input and output of subtasks in the task graph have no front-back relation.
Specifically, the step S4 employs: and processing the abstract task, planning a path according to the network state uploaded by the control domain controlled by the abstract task, planning the path of the data source and the target point of the abstract task, and searching for a path with the minimum cost by adopting a heuristic method. Task deployment refers to deciding at which control domain to place an abstract task in detail.
Step S4.1: determining a source node and a target node of a transmission path of a control layer;
step S4.2: planning a path by taking the path minimum cost as a target, and performing iterative computation through a heuristic algorithm, wherein each iteration adds a control domain to a routing path to obtain a minimum cost path;
step S4.3: and determining the deployment position of the abstract task by an abstract task deployment method with the aim of minimum deployment cost.
The abstract task deployment method is to deploy the abstract task on a control plane (namely, a layer from a ground controller to a controller) according to a subtask deployment method.
Specifically, the step S5 employs: and splitting the abstract task consisting of the plurality of subtasks so as to deploy each subtask respectively. And planning the processing path according to the task data source, the target point and the network state in the control domain. And the task deployment is completed by adopting a heuristic algorithm on the basis of the completion of the path planning.
Step S5.1: splitting the abstract task and serializing a plurality of subtasks;
step S5.2: determining an execution layer transmission path source node and a target node;
step S5.3: iteratively calculating through a heuristic algorithm, adding an execution node to the routing path in each iteration, and acquiring a minimum cost path;
step S5.4: and determining the deployment position of the subtask by using a subtask deployment method with the minimum deployment cost as a target.
The invention provides a multisource data fusion processing system in a world integration information network, which comprises:
a module M1: the heaven and earth integrated information network carries out real-time management and control on network states and resources according to the logic management level, describes network topology according to the level and integrates and quantizes available resources in the network topology;
a module M2: the ground control center divides a plurality of tasks acquired within a certain time and fuses the tasks according to the relationship among the tasks to form a multi-task execution graph;
a module M3: grouping and abstracting subtasks on the multitask execution graph to form an abstract task graph, wherein the abstract task graph is formed by connecting abstract tasks, and the abstract tasks are the connection of a plurality of subtasks; forming a full-order abstract task graph after serialization;
a module M4: the ground control center carries out processing path planning according to the network view of the controlled network domain and the serialized abstract task graph, and carries out minimum cost deployment planning on the abstract task;
a module M5: after receiving the abstract tasks issued by the ground control center, the controller in the control layer splits the abstract tasks, performs transmission path planning according to the controlled control domain network view, and deploys each subtask;
a module M6: after receiving the subtasks, the execution layer transmission computing node computes and transmits the tasks according to the computation strategy and the flow table issued by the control node; and returning the transmission result to the user;
the logic management hierarchy of the heaven and earth integrated information network is divided into a ground control center, a control layer and an execution layer from top to bottom.
Specifically, the module M1 employs: the SDN architecture-based multi-layer heaven-earth integrated network is divided into an execution layer, a control layer and a ground control layer from bottom to top according to a logic management level;
the ground control layer is a ground management center and is responsible for controlling a plurality of controllers of the control layer to acquire all control domain information;
the controller is deployed in a satellite network and a ground network and is divided into a plurality of areas according to the control range of the controller, each area is controlled by one controller, and the controllers control execution layer equipment; the controller integrates the network states of the nodes in the controlled area and provides the information to a ground control center;
the execution layer consists of transmission and calculation nodes in a world-wide integrated network;
the heaven and earth integrated network carries out layered management and control on network states and resources from bottom to top, the execution nodes periodically upload network state information, and the controller integrates the network state information of the nodes in the managed and controlled area and sends the information to the ground control center.
Specifically, the module M2 employs: the ground control center divides a plurality of tasks acquired within a certain time, the time delay sensitivity of the tasks can be quantitatively expressed, and task graphs are fused according to the task relevance and the time delay sensitivity degree to form a multi-task execution graph. The subtasks are obtained by splitting each task into a plurality of execution blocks with dependency relationship. The task fusion has the advantages that in the face of extremely numerous tasks in the world network, some tasks with extremely high processing process correlation are merged, and network load and task deployment time can be greatly reduced.
Specifically, the module M2 employs:
module M2.1: the ground control center acquires corresponding position information, task source information and task type information aiming at each task uploaded by a user;
module M2.2: the ground control center determines a task delay sensitive value and a task waiting value according to the task source information and the task type information;
module M2.3: comparing the task waiting value with a delay sensitivity threshold, and triggering a module M3 when the task waiting value is greater than the delay sensitivity threshold; when the task waiting value is less than or equal to the delay sensitivity threshold, triggering a module M2.4;
module M2.4: set T = { T) formed by ground control center for N tasks 1 ,t 2 ,…,t N Dividing the tasks in advance in order to increase the similarity of a plurality of tasks, and grouping the tasks according to the position information, the task source information and a plurality of factors of software;
module M2.5: performing task segmentation on all tasks in the grouped group to form a plurality of task graphs, wherein the task graphs are directed acyclic graphs formed by executable codes; a task can be split in multiple compiling stages, such as source code and intermediate code levels, and a plurality of methods can solve the problem of splitting the task, such as dependency relationship among a plurality of code segments. Second, after the task is sliced, a number of execution blocks of code are formed, which may be logically viewed as being made up of input, output, and code segments. Each subtask is an executable code block. The code execution blocks can form a task execution graph (task graph) according to the input and output dependency relationship.
Module M2.6: performing task graph fusion according to whether the pre-subtasks are the same or not; if the unfused subtasks appear, the task waiting value of the task is increased to a preset value, and the module M2.3 to the module M2.6 are repeatedly triggered until all subtasks are fused. The pre-subtask means that after a task is divided into a task execution graph, the multiple execution graphs of multiple tasks have the same execution block (the same subtask), and the execution block can determine whether the multiple execution graphs are the same according to the code segment, the input and the output. The pre-task block refers to a plurality of sub-task blocks of the plurality of task graphs starting from a source point (a source point of the directed acyclic graph).
Specifically, the module M3 employs: and grouping according to the task graph after the multi-task fusion, wherein the grouping process is according to the input and output correlation among the subtasks, and after the grouping, a plurality of subtasks can be used as one task to be deployed and can also be divided into a plurality of subtasks again.
In particular, the amount of the solvent to be used,
module M3.1: performing full-order grouping according to the subtask execution dependence in the task graph;
module M3.2: abstracting data compression rate and resource demand for each group of subtasks after grouping;
module M3.3: serializing the non-full-order task graph according to the data compression rate;
the non-full-order task graph is formed by the fact that input and output of subtasks in the task graph have no context.
Specifically, the module M4 employs: and processing the abstract task, planning a path according to the network state uploaded by the control domain controlled by the abstract task, planning the path of the data source and the target point of the abstract task, and searching for a path with the minimum cost by adopting a heuristic method. Task deployment refers to deciding at which control domain to place an abstract task in detail.
Module M4.1: determining a source node and a target node of a transmission path of a control layer;
module M4.2: planning a path by taking the path minimum cost as a target, and performing iterative computation through a heuristic algorithm, wherein each iteration adds a control domain to a routing path to obtain a minimum cost path;
module M4.3: and determining the deployment position of the abstract task by an abstract task deployment method with the minimum deployment cost as a target.
The abstract task deployment method is to deploy the abstract task on a control plane (namely, a layer from a ground controller to a controller) according to a subtask deployment method.
Specifically, the module M5 employs: and splitting the abstract task consisting of the plurality of subtasks so as to deploy each subtask respectively. And planning the processing path according to the task data source, the target point and the network state in the control domain. And the task deployment is completed by adopting a heuristic algorithm on the basis of the completion of the path planning.
Module M5.1: splitting the abstract task and serializing a plurality of subtasks;
module M5.2: determining an execution layer transmission path source node and a target node;
module M5.3: iteratively calculating through a heuristic algorithm, adding an execution node to the routing path in each iteration, and acquiring a minimum cost path;
module M5.4: and determining the deployment position of the subtask by using a subtask deployment method with the minimum deployment cost as a target.
Example 2
Example 2 is a preferred example of example 1
The invention provides a multi-source data fusion processing method in a software-defined space-ground integrated information network, which comprises the following steps as shown in figure 1:
step s1: through the characteristic property of an SDN framework, in a multi-layer control heaven-earth integrated network environment, a network layer is divided into an execution layer, a control layer and a ground control layer from bottom to top. The ground control layer is a ground management center and is responsible for controlling a plurality of controllers of the control layer. The controller is deployed in a satellite network and a ground network and controls the execution layer equipment. The network carries out hierarchical management and control on network states and resources from bottom to top. The ground data center grasps all control domain information. The execution nodes periodically upload the network state information, and each layer of control nodes dynamically update the network state in each domain to the upper layer according to the network state information;
step s2: the user uploads the task to the ground data center in the network, and the data center can obtain the following information, namely the position information loc aiming at each task t t Task source information app t Task type information tp t (ii) a As shown in fig. 3, the data center performs fusion of a plurality of task graphs according to task correlations and delay sensitive values of a plurality of tasks;
step s2.1: the ground data center app according to task source t Task type tp t Judging, and determining the time delay sensitivity threshold rho of a task t t At the same time toThe task assigns a task wait value wait t ；
Step s2.2: comparing the task waiting value with the time delay sensitive threshold value, if wait t ＞ρ t If the waiting time of the task exceeds the tolerance level, step s3 is directly executed, otherwise step s2.3 is executed, wherein rho t Representing a delay sensitive threshold;
step s2.3: set T = { T) composed of ground data center for N tasks 1 ，t 2 ，…，t N }, according to the location information loc of the task requester t And task source information app t Classifying various factors; the method aims to integrate tasks which are possible to be successfully fused in advance before data fusion. In particular, the amount of the solvent to be used,
step s2.3.1: if task t i And task t j The task source information of (1) is app i And app j Wherein i, j belongs to N and has app i ＝app j Then the task t is processed i And task t j Dividing into a group;
step s2.3.2: for ungrouped tasks, if task t i And task t j The location source information of (2) is loc i And loc j Where i, j ∈ N, has | loc i -loc j If | < ε, will task t i And task t j Grouping into a group, wherein epsilon is a task position correlation measurement and can be manually adjusted;
step s2.4: and performing task segmentation on all tasks in the grouped group, wherein the group of tasks can be represented as a set containing S tasks ST = { ST = { (ST) 1 ，st 2 ，…，st S }。
Step s2.4.1: with a single task st therein i For example, i e [1, S ]]In the task segmentation mode, the intermediate code level is used for segmenting the task, and an execution block (subtask) set BLK = { BLK } is obtained 1 ，blk 2 ，…，blk M A task execution block blk i A triplet (body, din, dout) can be used to abstract an expression, i ∈ [1, M]Where body is the execution body of the execution block, din is the input data, and dout is the output numberAccording to, e.g. a subtask blk i May be denoted as blk i (body)。
Step s2.4.2: for task st i After the set of task execution blocks BLK is obtained, a directed acyclic graph TG (TV, TE) is formed according to the relationship between the input data and the output data of the task execution blocks. TG (TV, TE) is formed in the following manner: TV = BLK, i.e. the set of points for TG is the set of task execution blocks, TV = { TV = { TV = } 1 ，tv 2 ，…，tv M }; slave node tv i To node tv j Is directed to edge te (tv) i ，tv j ) TE if and only if
Step s2.4.3: for the code block TV, it has a certain quantifiable data compression capacity comp (TV), which represents the change of the flow rate of the data stream after passing through the subtask TV, and the subtask TV has the requirement of computation and storage, and is quantified as nd (TV). After this step has been performed, a plurality of task graphs TG will be formed in a class 1 ，TG 2 ，…，TG S 。
Step s2.5: for multiple task graph sets { TG 1 ，TG 2 ，…，TG S And performing task graph fusion, and performing fusion according to whether the preposed execution blocks between the task graphs are the same or not. The fusion mode is as follows: from TG 1 ，TG 2 ，…，TG S If for two task graphs TG there is no directed edge to the starting node of the task graph (the starting node of the task graph is defined as the point where there is no directed edge in the task graph) starting traversal i And TG j ，i，j∈[1，S]，TG i Is executed block blk i And TG j Is executed block blk j Presence of blk i (body)＝blk j (body), then merge the graphs TG i And TG j TG' was obtained. The combination mode is that TG' point set is TG i And TG j Union set of point sets; TG' edge set of TG i And TG j And combining the edge sets, and simultaneously combining the edges connected by the same code block, wherein a schematic diagram is shown in fig. 3. The fused graph TG ' has a set of points TV ' and a set of edges TE '.
Step s2.6: after step 2.5 execution is complete, for unfused tasks t i Increase its task waiting value
And returns to step 2.2.
And step s3: grouping and abstracting the fused task graph, and quantifying the data compression rate and the resources required by calculation of each abstract subtask;
step s3.1: firstly, performing full-order grouping according to the subtask execution dependency in the task graph, for example, the process from the task graph before abstraction to the task graph after abstraction in fig. 4, the specific method is as follows: if the set of certain subtasks in the task graph TV' contains a start node (as defined above), and two subtasks TV i ，tv j e.TV' has at most one edge connected. The fully-ordered task group is a subgraph in the original task graph TG ', and can be represented as OG (OV, OE) containing a partial point set OV and an edge set OE of the original task graph, namely OV belongs to TV ' and OE belongs to TE '. The meaning of full-order fusion is that if a group of subtasks is full-order, the group of subtasks is closely related in general and is not influenced by other subtasks, and the group of subtasks is abstracted into an abstract task and is deployed in a control domain to accelerate processing.
Step s3.2: abstracting each full-order task group, wherein two indexes are provided, one index is a data compression ratio comp, the other index is a resource demand nd, and the abstracting method for the full-order task group OG (OV, OE) is as follows:
comp(OG)＝Π tv∈OV comp(tv)，
nd(OG)＝max{nd(tv)，tv∈OV}，
after grouping and abstraction, the original task graph TG '(TV', TE ') can be abstracted to a new abstract task graph AG (AV, AE), where AV represents the set of abstract subtasks and AE ∈ TE'.
And S4, serializing the abstracted task graph. The task graph is directed acyclic, as shown in fig. 5, if the input of one node av is a plurality of node outputs, the nodes are used as input sets and are sorted according to comp, and are connected according to the sequence from small to large, a new directed edge is added into AE, and the original edge is deleted. After ordering AG (AV, AE) becomes a new ordered abstract task graph SG (SV, SE), where SV represents the set of abstract subtasks and SV = AV, SE is the new set of directed edges.
And step s5: the ground control center grasps the network views of the controlled multiple control domains, and G is used for DC (V DC ，E DC ) Representing a network composed of a plurality of control domains, G DC Is an undirected graph and comprises a control domain node set V DC And control domain direct link set E DC Calculating path planning and task deployment by combining an ordered abstract task map SG (SV, SE);
step s5.1: in the abstract task processing path planning stage, an algorithm is adopted, a source control domain is taken as a starting point, a target control domain is taken as an end point, a path is planned with the path minimum cost as a target, the path cost is described as follows, and a control domain u and a control domain V are respectively equal to V DC And it is directly connected, the path u to v costs
w u，v ＝weight(load u，v +spec u，v )，
Where weight is a monotonically non-decreasing function, the argument load u， v and spec u，v An increase in (c) would result in an increase in the cost of the link. load u，v Is the current transmission load, spce, of the link u，v This value is an importance measure for the current link and is used to give different transmission costs to links of different importance. Each time, circularly searching a control domain adding path, continuously accumulating the control domains in the path until the calculation and storage resources in the path are larger than the required resources of the ordered abstract task graph SG, namely larger than pi sv∈SV nd(sv)。
Taking the ith loop as an example, from the found execution node v i Beginning, v i ∈V DC Using the minimum path cost as the measurement to find the next execution node v j Simultaneously, a path between two nodes is added to the routing path. If enough nodes have been found for the computation, the iteration is stopped. After step s5.1 is completed, a path from the source point to the destination point is obtainedMinimized cost PATH DC (PV DC ，PE DC )，PATH DC Is a directed acyclic graph and is a network graph G DC (V DC ，E DC ) A subset of (a). And the resources on the path are sufficient to handle the multiple abstract subtasks to be deployed.
Step s5.2: and in the abstract task deployment stage, after the routing path is determined in the first stage, each abstract task is greedily deployed in sequence by taking the minimization of the deployment cost of each step as a target. Existing full-order abstract task graphs SG (SV, SE), the abstract subtasks have a total Size (SV). In PATH DC (PV DC ，PE DC ) Node v of i When the jth task in the full-order abstract task graph is deployed, the slave node v i To v i+1 The cost of the link is:
wherein, the first and the second end of the pipe are connected with each other,
means that the task j is executed and then is executed at the node v i To v i+1 The new load change on the link. If the input flow rate of the full-order abstract task set is tr, sv k ∈SV，k∈[1，size(SV)]Then there are:
taking the deployment of the jth task as an example, sequentially taking nodes in the PATH after the deployment position of the jth-1 task as deployment nodes of the jth task, calculating new costs, taking the minimum cost deployment position as an optimal position, and putting the new costs into an ordered set location, wherein elements in the location correspond to nodes in SG (SV, SE) in sequence.
Step s5.3: and issuing the subtasks or the subtask groups to an override domain according to the obtained deployment position location and the corresponding full-sequence task graph SG (SV, SE), and simultaneously issuing a flow table to control the flow direction of data of each control domain.
And step s6: after the controller receives the abstract tasks, performing transmission path planning on each task group according to the controlled override domain network view;
step s6.1: disassembling abstract tasks distributed from a control center and reducing the abstract tasks into a full-sequence task group OG (OV, OE);
step s6.2: in the task processing path planning stage, the controller knows the network topology and uses G C (V C ，E C ) Representation, including executing a set of nodes V C Performing node Link set E C And meanwhile, determining a transmission path source point and a transmission path target node in the control domain according to a flow table issued by the data center. Planning a path by taking the minimum path cost as a target, describing the path cost as follows, and executing nodes u, V and V E to V C And directly connected, the path cost of the node u to v is weight (load) u，v +spec u，v ) Where weight is a monotonically non-decreasing function. load u，v Is the current transmission load of the link, spec u，v This value is an importance measure for the current link and is used to give different transmission costs to links of different importance. Each time, circularly searching an execution node to add a path, and continuously accumulating the paths until the calculation and storage resources in the paths are larger than the resources required by all subtasks in the full-order task group OG (OV, OE), namely larger than pi ov∈OV nd (ov). Finally obtaining a PATH with minimized cost from the PATH source point to the target point C (PV C ，PE C )，PATH C Is a directed acyclic graph and is a control domain network graph G C (V C ，E C ) A subset of (a). And the resources on the path are sufficient to handle the multiple subtasks to be deployed.
Step s6.3: and in the task deployment stage, after the routing path is determined in the step s6.2, each task is greedily deployed in sequence by taking the minimization of the deployment cost of each step as a target. The existing full-order task graph OG (OV, OE) has a total size (OV) of subtasks. In PATH C (PV C ，PE C ) Node v of i When the jth task in the full-order task graph is deployed (j < size (OV)), the slave node v i To v i+1 The cost of the link is:
wherein the content of the first and second substances,
refers to that the task j is executed and then is executed at the node v i To v i+1 New load changes on the link, given the incoming traffic rate of the task group is tr c ，ov k ∈OV，k∈[1，size(OV)]Then there are:
taking the deployment of the jth task as an example, sequentially taking the nodes in the PATH after the deployment position of the jth-1 task as the next deployment node, calculating new costs, and putting the deployment position of the minimum costs as the optimal position into the ordered set location c Location of c The middle element corresponds to the node sequence in OG (OV, OE).
And step s7: and after receiving the subtasks, the execution node calculates and transmits the tasks according to the calculation strategy and the flow table issued by the control node.
The multi-source data fusion processing system in the software-defined world-integrated information network implemented according to the technology of the invention, as shown in fig. 2, comprises the following modules:
the network real-time management and control module: integrating network states and resources of all layers, wherein each area is controlled by an upper node in an abstract node mode;
a task fusion module: performing task graph fusion according to the relevance of multiple tasks and the time delay sensitivity;
a task processing module: grouping the fused task graphs, then serializing each group of tasks, and finally abstracting related data of the subtask groups;
a path calculation module: the control nodes of each layer continue routing path planning according to the resource requirements of the subtask groups, the task source nodes and the task target nodes;
a task deployment module: the control node of each layer takes the path calculated by the module 4 as a task placement path according to the data compression rate of the subtask or the subtask group, and places the subtask with the minimized link cost as a target;
a transfer execution module: and after receiving the transmission and calculation signaling sent by the superior controller, the transmission and calculation node positioned in the transmission and calculation execution layer executes corresponding subtask calculation and data transmission.
It is known to those skilled in the art that, in addition to implementing the system, apparatus and its various modules provided by the present invention in pure computer readable program code, the system, apparatus and its various modules provided by the present invention can be implemented in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like by completely programming the method steps. Therefore, the system, the device and the modules thereof provided by the present invention can be considered as a hardware component, and the modules included in the system, the device and the modules thereof for implementing various programs can also be considered as structures in the hardware component; modules for performing various functions may also be considered to be both software programs for performing the methods and structures within hardware components.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention. The embodiments and features of the embodiments of the present application may be combined with each other arbitrarily without conflict.
Claims (10)
1. A multi-source data fusion processing method in a world-wide integrated information network is characterized by comprising the following steps:
step S1: the heaven and earth integrated information network carries out real-time management and control on network states and resources according to the logic management level;
step S2: the ground control center divides a plurality of tasks acquired within a certain time and fuses the tasks according to the relationship among the tasks to form a multi-task execution graph;
and step S3: grouping and abstracting subtasks for the multitask execution graph, and forming a full-order abstract task graph after serialization;
and step S4: the ground control center carries out processing path planning according to the network view of the managed and controlled network domain and the serialized abstract task graph, and carries out minimum cost deployment planning on the abstract task;
step S5: the controller in the control layer splits the received abstract tasks and carries out processing path planning and task deployment planning;
step S6: after receiving the task, the execution layer calculates and transmits the task according to the calculation strategy and the flow table issued by the control node; and returning the transmission result to the user;
the logic management hierarchy of the heaven and earth integrated information network is divided into a ground control center, a control layer and an execution layer from top to bottom.
2. The method for fusion processing of multi-source data in the integrated information network of heaven and earth according to claim 1, wherein the step S1 adopts: the SDN architecture-based multi-layer heaven-earth integrated network is divided into an execution layer, a control layer and a ground control center from bottom to top according to a logic management level;
the ground control center is a ground management center and is responsible for controlling a plurality of controllers of the control layer to acquire all control domain information;
the controller is deployed in a satellite network and a ground network and is divided into a plurality of areas according to the control range of the controller, each area is controlled by one controller, and the controllers control execution layer equipment;
the execution layer consists of transmission and calculation nodes in a world-wide integration network;
the heaven and earth integrated network carries out layered management and control on network states and resources from bottom to top, the execution nodes periodically upload network state information, and the controller integrates the network state information of the nodes in the controlled area and sends the information to the ground control center.
3. The method for fusion processing of multi-source data in the integrated information network of heaven and earth according to claim 1, wherein the step S2 adopts: the ground control center divides a plurality of tasks acquired within a certain time, and then performs task graph fusion according to task relevance and time delay sensitivity to form a multi-task execution graph.
4. The method for fusion processing of multi-source data in the integrated information network of heaven and earth according to claim 3, wherein the step S2 adopts:
step S2.1: the ground control center acquires corresponding position information, task source information and task type information aiming at each task uploaded by a user;
step S2.2: the ground control center determines a task delay sensitive value and a task waiting value according to the task source information and the task type information;
step S2.3: comparing the task waiting value with a delay sensitivity threshold, and triggering the step S3 when the task waiting value is greater than the delay sensitivity threshold; when the task waiting value is less than or equal to the delay sensitive threshold, triggering the step S2.4;
step S2.4: ground control center sets T = { T) composed of N tasks 1 ,t 2 ,···,t N Grouping according to the position information and the task source information;
step S2.5: performing task segmentation on all tasks in the grouped group to form a plurality of task graphs, wherein the task graphs are directed acyclic graphs formed by executable codes;
step S2.6: performing task graph fusion according to whether the pre-subtasks are the same or not; and if the unfused subtasks appear, increasing the task waiting value of the task to a preset value, and repeatedly triggering the step S2.3 to the step S2.6 until all the subtasks are fused.
5. The method for fusion processing of multi-source data in the integrated information network of heaven and earth according to claim 1, wherein the step S3 adopts:
step S3.1: performing full-order grouping according to the subtask execution dependence in the task graph;
step S3.2: abstracting data compression ratio and resource demand for each group of subtasks after grouping;
step S3.3: serializing the non-full-order task graph according to a data compression rate;
the non-full-order task graph is formed by the fact that input and output of subtasks in the task graph have no context.
6. The method for multi-source data fusion processing in the integrated information network of heaven and earth according to claim 1, wherein the step S4 comprises:
step S4.1: determining a source node and a target node of a transmission path of a control layer;
step S4.2: planning a path by taking the path minimum cost as a target, and performing iterative computation through a heuristic algorithm, wherein each iteration adds a control domain to a routing path to obtain a minimum cost path;
step S4.3: and determining the deployment position of the abstract task by an abstract task deployment method with the minimum deployment cost as a target.
7. The method for multi-source data fusion processing in the integrated information network of heaven and earth according to claim 1, wherein the step S5 comprises:
step S5.1: splitting the abstract task and serializing a plurality of subtasks;
step S5.2: determining a source node and a target node of a transmission path of an execution layer;
step S5.3: iteratively calculating through a heuristic algorithm, adding an execution node to the routing path in each iteration, and acquiring a minimum cost path;
step S5.4: and determining the deployment position of the subtask by using a subtask deployment method with the minimum deployment cost as a target.
8. A multisource data fusion processing system in a world-wide integrated information network is characterized by comprising:
a module M1: the heaven and earth integrated information network carries out real-time management and control on network states and resources according to the logic management level;
a module M2: the ground control center divides a plurality of tasks acquired within a certain time and fuses the tasks according to the relationship among the tasks to form a multi-task execution graph;
a module M3: grouping and abstracting subtasks for the multitask execution graph, and forming a full-order abstract task graph after serialization;
a module M4: the ground control center carries out processing path planning according to the network view of the controlled network domain and the serialized abstract task graph, and carries out minimum cost deployment planning on the abstract task;
a module M5: the controller in the control layer splits the received abstract tasks and carries out processing path planning and task deployment planning;
a module M6: after receiving the task, the execution layer calculates and transmits the task according to the calculation strategy and the flow table issued by the control node; and returning the transmission result to the user;
the logic management hierarchy of the heaven and earth integrated information network is divided into a ground control center, a control layer and an execution layer from top to bottom.
9. The system for multi-source data fusion processing in the integrated information network in heaven and earth according to claim 8, wherein the module M1 adopts: the SDN architecture-based multi-layer heaven-earth integrated network is divided into an execution layer, a control layer and a ground control center from bottom to top according to a logic management layer;
the ground control center is a ground management center and is responsible for controlling a plurality of controllers of the control layer to acquire all control domain information;
the controller is deployed in a satellite network and a ground network, and is divided into a plurality of areas according to the control range of the controller, each area is controlled by one controller, and the controller controls and controls the execution layer equipment;
the execution layer consists of transmission and calculation nodes in a world-wide integration network;
the heaven and earth integrated network carries out layered management and control on network states and resources from bottom to top, the execution nodes periodically upload network state information, and the controller integrates the network state information of the nodes in the controlled area and sends the information to the ground control center;
the module M2 employs: the ground control center divides a plurality of tasks acquired within a certain time, and then performs task graph fusion according to task correlation and time delay sensitivity to form a multi-task execution graph;
the module M2 employs:
module M2.1: the ground control center acquires corresponding position information, task source information and task type information aiming at each task uploaded by a user;
module M2.2: the ground control center determines a task delay sensitive value and a task waiting value according to the task source information and the task type information;
module M2.3: comparing the task waiting value with a delay sensitivity threshold, and triggering a module M3 when the task waiting value is greater than the delay sensitivity threshold; when the task waiting value is less than or equal to the delay sensitive threshold value, triggering a module M2.4;
module M2.4: ground control center sets T = { T) composed of N tasks 1 ,t 2 ,···,t N Grouping according to the position information and the task source information;
module M2.5: performing task segmentation on all tasks in the grouped group to form a plurality of task graphs, wherein the task graphs are directed acyclic graphs formed by executable codes;
module M2.6: performing task graph fusion according to whether the pre-subtasks are the same or not; if the unfused subtasks appear, the task waiting value of the task is increased to a preset value, and the module M2.3 to the module M2.6 are repeatedly triggered until all the subtasks are fused.
10. The system for multi-source data fusion processing in the integrated information network of heaven and earth of claim 8, wherein the module M3 adopts:
module M3.1: performing full-order grouping according to the subtask execution dependence in the task graph;
module M3.2: abstracting data compression ratio and resource demand for each group of subtasks after grouping;
module M3.3: serializing the non-full-order task graph according to a data compression rate;
the non-full-order task graph is characterized in that input and output of subtasks in the task graph have no front-back relation;
the module M4 employs:
module M4.1: determining a source node and a target node of a transmission path of a control layer;
module M4.2: planning a path by taking the path minimum cost as a target, and performing iterative computation through a heuristic algorithm, wherein each iteration adds a control domain to a routing path to obtain a minimum cost path;
module M4.3: determining an abstract task deployment position by an abstract task deployment method with the minimum deployment cost as a target;
the module M5 employs:
module M5.1: splitting the abstract task and serializing a plurality of subtasks;
module M5.2: determining an execution layer transmission path source node and a target node;
module M5.3: iteratively calculating through a heuristic algorithm, adding an execution node to the routing path in each iteration, and acquiring a minimum cost path;
module M5.4: and determining the deployment position of the subtask by using a subtask deployment method with the minimum deployment cost as a target.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111495136.0A CN114499624B (en) | 2021-12-08 | 2021-12-08 | Multi-source data fusion processing method and system in heaven-earth integrated information network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111495136.0A CN114499624B (en) | 2021-12-08 | 2021-12-08 | Multi-source data fusion processing method and system in heaven-earth integrated information network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114499624A CN114499624A (en) | 2022-05-13 |
| CN114499624B true CN114499624B (en) | 2022-12-13 |
Family
ID=81491976
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111495136.0A Active CN114499624B (en) | 2021-12-08 | 2021-12-08 | Multi-source data fusion processing method and system in heaven-earth integrated information network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114499624B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106685511A (en) * | 2016-11-02 | 2017-05-17 | 北京邮电大学 | Spatial information network architecture |
| CN107180309A (en) * | 2017-05-31 | 2017-09-19 | 中南大学 | The collaborative planning method of resource is observed in a kind of empty world |
| CN111342993A (en) * | 2020-02-03 | 2020-06-26 | 西安交通大学 | SDN-based air-space-ground controller deployment architecture and control method |
| CN113346938A (en) * | 2021-05-20 | 2021-09-03 | 天地信息网络有限公司 | Edge computing resource fusion management method for air-space-ground integrated network |
| WO2021221736A2 (en) * | 2020-05-01 | 2021-11-04 | Intel Corporation | Edge computing in satellite connectivity environments |
-
2021
- 2021-12-08 CN CN202111495136.0A patent/CN114499624B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106685511A (en) * | 2016-11-02 | 2017-05-17 | 北京邮电大学 | Spatial information network architecture |
| CN107180309A (en) * | 2017-05-31 | 2017-09-19 | 中南大学 | The collaborative planning method of resource is observed in a kind of empty world |
| CN111342993A (en) * | 2020-02-03 | 2020-06-26 | 西安交通大学 | SDN-based air-space-ground controller deployment architecture and control method |
| WO2021221736A2 (en) * | 2020-05-01 | 2021-11-04 | Intel Corporation | Edge computing in satellite connectivity environments |
| CN113346938A (en) * | 2021-05-20 | 2021-09-03 | 天地信息网络有限公司 | Edge computing resource fusion management method for air-space-ground integrated network |
Non-Patent Citations (1)
| Title |
|---|
| "天基信息港的多源信息融合任务调度研究";王之等;《中国空间科学技术》;20180625;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114499624A (en) | 2022-05-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108880663A (en) | Incorporate network resource allocation method based on improved adaptive GA-IAGA | |
| CN104636204B (en) | A kind of method for scheduling task and device | |
| CN108985709A (en) | Workflow management method towards more satellite data centers collaboration Remote Sensing Products production | |
| CN112272381B (en) | Satellite network task deployment method and system | |
| CN113794494B (en) | Edge computing system and computing unloading optimization method for low-orbit satellite network | |
| CN110113761B (en) | Deployment method and device of stream data processing application in edge computing network | |
| CN108307435A (en) | A kind of multitask route selection method based on SDSIN | |
| CN110119399A (en) | Work Flow Optimizing method based on machine learning | |
| Hu et al. | Throughput optimized scheduler for dispersed computing systems | |
| CN114915630A (en) | Task allocation method based on Internet of things equipment, network training method and device | |
| CN103116525A (en) | Map reduce computing method under internet environment | |
| CN112087384A (en) | SDN environment-based data transmission method and system | |
| Tang et al. | Layer dependency-aware learning scheduling algorithms for containers in mobile edge computing | |
| CN114499624B (en) | Multi-source data fusion processing method and system in heaven-earth integrated information network | |
| CN107360235A (en) | A kind of task immigration method based on reliability classification | |
| CN111950874A (en) | Satellite autonomous task planning satellite-borne knowledge base design method and system | |
| Long et al. | A novel fault-tolerant approach to web service composition upon the edge computing environment | |
| CN113010296A (en) | Task analysis and resource allocation method and system based on formalized model | |
| Liu et al. | On distributed computation rate optimization for deploying cloud computing programming frameworks | |
| CN110119268A (en) | Workflow optimization method based on artificial intelligence | |
| CN114024894A (en) | Dynamic transmission and calculation method and system in software-defined space-ground integrated network | |
| Qiao et al. | A service function chain deployment scheme of the software defined satellite network | |
| CN110135747A (en) | Process customizing method neural network based | |
| US20030177166A1 (en) | Scalable scheduling in parallel processors | |
| CN111813525A (en) | Heterogeneous system workflow scheduling method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |