CN111444007B - Remote sensing big data automatic processing method based on cloud computing - Google Patents
Remote sensing big data automatic processing method based on cloud computing Download PDFInfo
- Publication number
- CN111444007B CN111444007B CN202010545489.6A CN202010545489A CN111444007B CN 111444007 B CN111444007 B CN 111444007B CN 202010545489 A CN202010545489 A CN 202010545489A CN 111444007 B CN111444007 B CN 111444007B
- Authority
- CN
- China
- Prior art keywords
- remote sensing
- task
- big data
- computing
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
Abstract
The invention discloses a remote sensing big data automatic processing method based on cloud computing.A remote sensing big data automatic processing engine identifies an input remote sensing big data processing request and obtains a storage address of remote sensing data to be processed, a remote sensing data processing flow, a resource-time mapping table, a storage address of a data processing task related to the flow and a storage address of a task scheduling algorithm; acquiring an algorithm from a storage unit according to the acquired algorithm storage address, taking a flow and a resource-time mapping table as the input of the algorithm, and executing the algorithm to obtain a task scheduling result; deploying a computing environment in a computing unit according to the obtained scheduling result; and respectively acquiring the remote sensing data and the task according to the acquired storage addresses of the remote sensing data and the task, and executing the task in the acquired computing environment according to the acquired task scheduling result to obtain a processing result of the remote sensing data. According to the remote sensing method and the remote sensing system, the efficiency of processing remote sensing big data by the cloud computing platform can be effectively improved.
Description
Technical Field
The invention belongs to the field of remote sensing big data processing, and particularly relates to a cloud computing-based remote sensing big data automatic processing method.
Background
Remote sensing image processing, i.e., a technology for performing a series of optical processing and digital image processing on a remote sensing image to achieve an expected purpose, has been widely applied to the fields of navigation, weather, agriculture, resources and the like. With the continuous development of remote sensing technology, the resolution of remote sensing images is gradually increased, the data volume of the remote sensing images is increased explosively, the problem that single machine processing of remote sensing big data is difficult is generated, and in order to solve the problem, more and more students combine the remote sensing big data processing with cloud computing.
Cloud computing is a business computing model, and is based on cloud resource management software and virtualization tools, a CPU, a memory and a GPU are virtualized into a resource pool convenient for management and application, and a user can obtain storage and computing services according to requirements. At present, after the network bandwidth is rapidly developed, especially after the 5G business is officially commercialized, a user can utilize personal portable equipment to control a server at the cloud end to complete processing and calculation of mass data, so that the management cost and the maintenance cost of the equipment are reduced. The cloud computing framework can realize mass data storage and efficient parallel computing on the basis of the cloud platform, and can distribute a large number of complex computing tasks to a plurality of nodes when executing the computing tasks, so that the task processing time is reduced, and the processing efficiency of mass data is effectively improved.
However, when the remote sensing data is processed based on the existing cloud computing platform, a plurality of tasks in the processing flow are often deployed by adopting a manual method, which is time-consuming, labor-consuming and easy to make mistakes; meanwhile, a scheduling algorithm is not used for scheduling and optimizing a plurality of tasks related to the process, and the efficiency is low.
Disclosure of Invention
The invention aims to provide a remote sensing big data automatic processing method based on cloud computing, which describes a task of remote sensing big data processing as a flow by utilizing an autonomously designed standardized interface, and simultaneously performs scheduling optimization on the task related to the flow by using a task scheduling algorithm, thereby not only realizing the cloud computing automatic processing of a remote sensing image, but also effectively improving the data processing efficiency.
The technical scheme for realizing the purpose of the invention is as follows: a remote sensing big data automatic processing method based on cloud computing is realized through a processing system, the processing system comprises a remote sensing big data automatic processing engine, a storage unit and a computing unit, and the method comprises the following steps:
step 1, recognizing an input remote sensing big data processing request by a remote sensing big data automatic processing engine, and acquiring a storage address of remote sensing data to be processed, a remote sensing data processing flow, a resource-time mapping table, a storage address of a data processing task related to the remote sensing data processing flow and a storage address of a data processing task scheduling algorithm;
step 2, the remote sensing big data automatic processing engine acquires a data processing task scheduling algorithm from a storage unit according to the data processing task scheduling algorithm storage address acquired in the step 1, and executes the data processing task scheduling algorithm by taking a remote sensing data processing flow and a resource-time mapping table as the input of the data processing task scheduling algorithm to obtain a data processing task scheduling result;
3, deploying a computing environment in a computing unit by the remote sensing big data automatic processing engine according to the scheduling result obtained in the step 2;
and 4, respectively acquiring the remote sensing data and the data processing task by the remote sensing big data automatic processing engine according to the storage addresses of the remote sensing data and the data processing task in the step 1, and executing the data processing task in the computing environment obtained in the step 3 according to the task scheduling result obtained in the step 2 to obtain a processing result of the remote sensing data.
Further, step 1 is that the client submits a remote sensing big data processing request, and the engine recognizes the request, obtains the storage address of the remote sensing data to be processed, the remote sensing data processing flow, the resource-time mapping table RDMT, the storage address of the task related to the remote sensing data processing flow and the storage address of the algorithm, and writes the storage addresses into the standardized interface file.
Furthermore, the standardized interface file describes the input and output of each module of the method in a standardized manner, and is realized by a markup language.
Further, the remote sensing data processing flow in the step 1 is formed by combining tasks with partial order relation in remote sensing big data algorithm processing.
Further, the resource-time mapping table in step 1 describes the execution time of the task under different computing slices and different data volumes.
Further, the remote sensing big data automatic processing engine in the step 2 acquires the algorithm from the storage unit according to the algorithm storage address in the standardized interface file, takes the remote sensing data processing flow and the resource-time mapping table as the input of the algorithm, analyzes the parallel relation between tasks and in the tasks in the flow by utilizing the optimization mechanism of the algorithm, calculates the scheduling result, and writes the scheduling result into the standardized interface file.
Further, the scheduling result includes: the number of the computing nodes required by each subtask, the serial number of the computing nodes, the estimated task starting time and the task finishing time.
Further, the remote sensing big data automatic processing engine in step 3 deploys a computing environment in the computing unit according to the scheduling result obtained in step 2, specifically: and the remote sensing big data automatic processing engine applies for corresponding computing resources to the computing unit according to the scheduling result in the standardized interface file, and deploys the computing environment through the computing environment automatic deployment script.
Further, the automated deployment script is implemented by a programming language; the deployment process of the computing environment is as follows: preparing a server system with complete functions, starting a server cluster, configuring a computing frame and a storage frame file, and starting a computing environment.
Further, the remote sensing big data automatic processing engine in the step 4 respectively obtains the remote sensing data and the data processing task according to the remote sensing data in the standardized interface file and the storage address of the task, and executes the task in the computing environment obtained in the step 3 according to the task scheduling result obtained in the step 2 to obtain a final data processing result.
Compared with the prior art, the invention has the following remarkable advantages: (1) compared with other remote sensing big data processing methods, the method of the invention describes a plurality of tasks of remote sensing big data processing as a flow, and the flow is processed by an automatic processing engine, so that the automation of remote sensing big data processing is realized; (2) the invention uses the task scheduling algorithm, and improves the execution efficiency of the process.
(1) Compared with other remote sensing big data processing methods, the method and the system have the advantages that the automatic deployment engine of the computing environment is designed, and the deployment efficiency of the computing environment is improved. (2) The invention pre-evaluates the resource allocation and scheduling in the calculation process by utilizing the optimizing mechanism of the task scheduling optimization algorithm, thereby improving the resource utilization rate and the calculation efficiency. (3) The invention uses the standardized interface to link the modules together, thereby improving the data processing efficiency and realizing the automatic processing of the remote sensing big data.
Drawings
Fig. 1 is a schematic diagram of a remote sensing big data automatic processing system based on cloud computing.
Fig. 2 is a flow chart of a remote sensing big data automatic processing method based on cloud computing.
FIG. 3 is a flow chart of a remote sensing image change detection task.
FIG. 4 is a flow diagram of automated deployment of a computing environment.
Fig. 5 is a diagram of the scheduling results of three scheduling algorithms when the number of computing resources is 8.
Fig. 6(a) and 6(b) are comparison graphs of the remote sensing image change detection results, wherein fig. 6(a) is a graph of the actual situation of the earth surface, and fig. 6(b) is a graph of the detection results.
Detailed Description
In order to realize the automatic processing of the remote sensing big data and improve the efficiency of the cloud computing platform in processing the remote sensing big data, the invention provides the automatic processing method of the remote sensing big data based on the cloud computing, which not only realizes the automatic processing of the cloud computing of the remote sensing image, but also effectively improves the data processing efficiency by using a task scheduling algorithm on the basis of ensuring the processing flow effect of the remote sensing big data.
A remote sensing big data automatic processing method based on cloud computing comprises the following steps:
and (A) identifying an input remote sensing big data processing request by an engine, and obtaining a storage address of remote sensing data to be processed, a process, a resource-time mapping table (RDMT), a storage address of a task related to the process and a storage address of an algorithm. The method comprises the following specific steps: the client submits a remote sensing big data processing request, and the engine recognizes the request, obtains the storage address, the flow, the resource-time mapping table RDMT, the storage address of the task related to the flow and the storage address of the algorithm of the remote sensing data to be processed, and writes the remote sensing big data processing request into a standardized interface file.
Wherein, the standardized interface file describes the input and output of each module of the method in a standardized way and is realized by XML markup language; the storage address is a link address of various information stored in the storage unit and is similar to 'hdfs:// 10.10.10.100: 8020/hadoop/SA/cs.xml', and the engine can acquire required data or algorithm from the storage unit according to the address.
The remote sensing data processing flow is formed by combining tasks with partial order relation in remote sensing big data processing. The invention describes the flow by using a directed acyclic graph, wherein nodes in the graph represent tasks, and edges in the graph represent partial order relations of the tasks. In the standardized interface file, a TASK is represented by a TASK field, and includes a Name (Name), a number (ID), a parallel Flag (Flag), a remark (Description), an Input (Input), an Output (Output), and a resource-duration mapping table (abbreviated as RDMT), where the RDMT describes execution time of the TASK under different computation slices and different data amounts. In the standardized interface file, EDGEs indicating the partial order relationship of tasks are indicated by fields EDGE, including a preamble (PreTask) and a successor (SucTask).
And (B) the engine acquires the algorithm from the storage unit according to the storage address in the step (A), and obtains a scheduling result by taking the process and the RDMT as the input of the algorithm. The method comprises the following specific steps: the engine acquires the algorithm from the storage unit according to the algorithm storage address in the standardized interface file, takes the flow and the RDMT as the input of the algorithm, analyzes the parallel relation between tasks and inside the tasks in the flow by utilizing the optimization mechanism of the algorithm, calculates the scheduling result, and writes the scheduling result into the standardized interface file. The scheduling result comprises the following contents: the number of compute nodes required for each subtask (ComQuantity), the compute node number (ComNum), the estimated task start time (StartTime), and the task completion time (CompleteTime).
And (C) deploying the computing environment in the computing unit by the engine according to the number of the computing nodes in the scheduling result obtained in the step (B). The method comprises the following specific steps: the engine applies for corresponding computing resources from the computing unit according to the scheduling result in the standardized interface file, and deploys the computing environment through the automated deployment script of the computing environment. The automatic deployment script is designed and realized by Shell script language; the specific deployment process comprises the following steps: preparing a server system with complete functions, starting a server cluster, configuring a computing frame and a storage frame file, and starting a computing environment.
And (D) respectively acquiring the remote sensing data and the task by the engine according to the storage addresses of the remote sensing data and the task in the step (A), and executing the task in the computing environment obtained in the step (C) according to the task scheduling result obtained in the step (B) to obtain a processing result of the remote sensing data. The method comprises the following specific steps: and (C) respectively acquiring the remote sensing data and the task by the engine according to the remote sensing data written in the step (A) and the storage address of the task in the standardized interface file, and executing the task in the computing environment obtained in the step (C) according to the task scheduling result obtained in the step (B) to obtain a final data processing result.
The technical solution of the present invention will be described in detail by examples.
Examples
As shown in fig. 1 and 2, the specific embodiment of the cloud computing-based remote sensing big data automatic processing method of the present invention takes a remote sensing image change detection request, Cuckoo Search task scheduling algorithm (CS), a CentOS system, a Spark calculation framework, a calculation resource number of 8, and a maximum processing data size of 683MB as an example, and includes the following steps:
and (A) identifying the input remote sensing image change detection request by the engine, acquiring a multi-temporal remote sensing data storage address, a remote sensing image change detection algorithm flow (hereinafter referred to as a change detection flow), a resource-time mapping table RDMT, a storage address of a data processing task (hereinafter referred to as a change detection task) involved in the remote sensing image change detection flow and a storage address of a CS algorithm, and writing the storage addresses into a standardized interface file. The file comprises five fields of a data storage address, a change detection process, an RDMT, a change detection task storage address and a CS algorithm storage address.
And (B) the engine acquires the CS algorithm from the storage unit according to a CS algorithm storage address 'hdfs:// 10.10.10.100: 8020/hadoop/SA/cs.xml' in the standardized interface file, takes the change detection flow and the RDMT as the input of the CS algorithm, analyzes the parallel relation between the change detection tasks and inside the change detection tasks in the change detection flow by using an optimization mechanism of the CS algorithm, calculates a scheduling result, and writes the scheduling result into the standardized interface file.
The CS algorithm inputs RDMT for the change detection process and all change detection tasks shown in fig. 3, and the algorithm parameters are shown in table 1:
TABLE 1 scheduling Algorithm parameter Table
Without loss of generality, the multi-objective optimization task scheduling modeling is carried out by taking the shortest data processing time and the lowest server power consumption as targets, and the obtained model is shown as the following formula 1 and formula 2:
the above formulas are respectively a data processing time model and a server power consumption model. Wherein the content of the first and second substances,
is shown askThe completion time after each compute node has processed all tasks,
which represents the total length of time,
representing the total energy consumption (joules/J) that the server generates to calculate the entire task flow,
representing the dynamic power of the server per unit time (watts/W),
indicating the utilization of the server CPU,
representing the static power (watts/W) of the server per unit time,
represents the idle time (seconds/s) of a single virtual machine of the server;Runtimerepresenting the duration (seconds) that a server's single virtual machine processes a task.
In the multi-objective task scheduling, the result is generally a Pareto solution set containing a plurality of Pareto solutions, and each Pareto solution describes the number of computing nodes, the serial number of the computing nodes, the estimated change detection task start time and the change detection task completion time required by each change detection task in detail. The digital model can be represented by equation 3:
（3）
wherein the content of the first and second substances,
is shown asjPlatform computing node computing tasksiThe start time of (c) is,
representing tasksiThe processing time of (1).
The scheduling result of remote sensing image change detection when the number of computing resources is 8 is shown in fig. 5, wherein a circle represents the scheduling result of CS, a pentagram and a cross represent the scheduling results of a particle swarm algorithm and a genetic algorithm, respectively, and a Pareto solution of one CS algorithm is shown in table 2:
TABLE 2 remote sensing image change detection distributed parallel algorithm cuckoo search scheduling Pareto solution
And (C) the engine applies for corresponding computing resources from the computing unit according to the scheduling result in the standardized interface file, and then prepares and deploys the computing environment through the automated deployment script of the computing environment. As shown in fig. 4, the specific implementation process of the automated deployment script is as follows:
1) and acquiring the IP of the computing node. The control node scans the started computing nodes in the subnet through a network scanning sniffing packet NMap and screens out the control node and the debugging nodes;
2) the IP mapping file is initialized. Editing a system IP mapping file hosts through a Linux sed instruction, and adding a computer virtual machine cluster IP into the file;
3) the control node calculates a framework configuration. Editing Spark and Hadoop configuration files of the control nodes by using an AWK and Linux sed command;
4) the configuration file is broadcast. Broadcasting a configuration file to each computing node by combining a Shell scp command with an Expect automatic interaction toolkit;
5) SSH mutual trust is configured. The computing cluster needs that each node and other nodes communicate without secret, the Center node firstly performs SSH mutual communication with the child nodes, and then controls each child node to perform SSH public key and private key broadcasting by using an SSH remote command;
6) the HDFS is formatted. Executing a command "% HADOOP _ HOME/bin/HADOOP name-format" to perform some clearing and preparation work before the HDFS is used;
7) the computing framework is started. And respectively executing start-all.sh starting calculation clusters of the Hadoop and Spark.
And (D) respectively acquiring the remote sensing data and the task by the engine according to the remote sensing data written in the standardized interface file and the storage address of the task, and executing the task in the computing environment obtained in the step (C) according to the CS algorithm scheduling result obtained in the step (B) to obtain a final data processing result.
Taking the K-Means algorithm as an example, the description information in the standardized interface file is shown in table 3:
TABLE 3K-Means algorithmic description Table
The specific treatment results are shown in fig. 6(b), table 4 and table 5. Compared with the ground surface actual condition diagram of fig. 6(a), it can be found that the result of the parallel algorithm obtained by the remote sensing big data automatic processing method has high similarity with the 'actual condition diagram', which indicates that the parallel algorithm maintains higher precision. Comparing table 4 and table 5, it can be seen that the parallel algorithm achieves a higher acceleration ratio. This shows that the parallel algorithm can obtain a considerable speed-up ratio on the premise of keeping higher precision.
TABLE 4 results of change detection without parallelism
| Slicing | In series | 2 | 4 | 8 | 16 | 32 |
| Overall accuracy | 0.95748 | 0.9394 | 0.9381 | 0.9376 | 0.9372 | 0.9367 |
TABLE 5 Experimental parallel processing time to acceleration ratio
Wherein, Image1, Image2 and Image3 are obtained by copying and expanding an original Image, and the sizes are respectively as follows: 138MB, 341MB, 683 MB.
The foregoing illustrates and describes the principles, general features, and advantages of the present invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are described in the specification and illustrated only to illustrate the principle of the present invention, but that various changes and modifications may be made therein without departing from the spirit and scope of the present invention, which fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (8)
1. The remote sensing big data automatic processing method based on cloud computing is characterized in that the method is realized through a processing system, the system comprises a remote sensing big data automatic processing engine, a storage unit and a computing unit, and the method comprises the following steps:
step 1, a remote sensing big data automatic processing engine identifies an input remote sensing big data processing request, acquires a storage address of remote sensing data to be processed, a remote sensing data processing flow, a resource-time mapping table, a storage address of a data processing task related to the remote sensing data processing flow and a storage address of a data processing multi-target task scheduling algorithm, and writes a standardized interface file, wherein the multiple targets are minimum data processing time and minimum server total power consumption, the data processing multi-target task scheduling algorithm is a cuckoo algorithm, and the cuckoo algorithm comprises a parameter of bird nest probability abandoning;
step 2, the remote sensing big data automatic processing engine acquires the cuckoo algorithm from a storage unit according to a cuckoo algorithm storage address in the standardized interface file, the remote sensing data processing flow and a resource-time mapping table are used as the input of the cuckoo algorithm, the parallel relation between tasks and the parallel relation inside the tasks in the flow are analyzed by utilizing an optimization mechanism of the cuckoo algorithm, the shortest data processing time and the lowest server power consumption are taken as targets, multi-objective optimization task scheduling modeling is carried out, and a scheduling result is written into the standardized interface file;
3, deploying a computing environment in a computing unit by the remote sensing big data automatic processing engine according to the scheduling result obtained in the step 2;
and 4, respectively acquiring the remote sensing data and the data processing task by the remote sensing big data automatic processing engine according to the storage addresses of the remote sensing data and the data processing task in the step 1, and executing the data processing task in the computing environment obtained in the step 3 according to the task scheduling result obtained in the step 2 to obtain a processing result of the remote sensing data.
2. The automatic processing method for remote sensing big data based on cloud computing according to claim 1, wherein the standardized interface file describes input and output of each module of the method in a standardized manner and is implemented by a markup language.
3. The automatic processing method for remote sensing big data based on cloud computing according to claim 1, wherein the remote sensing data processing flow in step 1 is formed by combining tasks with partial order relation in remote sensing big data algorithm processing.
4. The cloud computing-based remote sensing big data automatic processing method according to claim 1, wherein the resource-time mapping table in step 1 describes execution times of tasks under different computing slices and different data volumes.
5. The automatic processing method for remote sensing big data based on cloud computing according to claim 1, wherein the scheduling result comprises: the number of the computing nodes required by each subtask, the serial number of the computing nodes, the estimated task starting time and the task finishing time.
6. The cloud computing-based remote sensing big data automatic processing method according to claim 1, wherein the remote sensing big data automatic processing engine in step 3 deploys a computing environment in a computing unit according to the scheduling result obtained in step 2, and specifically comprises: and the remote sensing big data automatic processing engine applies for corresponding computing resources to the computing unit according to the scheduling result in the standardized interface file, and deploys the computing environment through the computing environment automatic deployment script.
7. The cloud computing-based remote sensing big data automatic processing method according to claim 6, wherein the automatic deployment script is implemented by a programming language; the deployment process of the computing environment comprises the following steps: preparing a server system with complete functions, starting a server cluster, configuring a computing frame and a storage frame file, and starting a computing environment.
8. The remote sensing big data automatic processing method based on cloud computing according to claim 1, wherein the remote sensing big data automatic processing engine in step 4 respectively obtains remote sensing data and a data processing task according to a storage address of the remote sensing data and a storage address of the task in a standardized interface file, and executes the task in the computing environment obtained in step 3 according to the task scheduling result obtained in step 2 to obtain a final data processing result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010545489.6A CN111444007B (en) | 2020-06-16 | 2020-06-16 | Remote sensing big data automatic processing method based on cloud computing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010545489.6A CN111444007B (en) | 2020-06-16 | 2020-06-16 | Remote sensing big data automatic processing method based on cloud computing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111444007A CN111444007A (en) | 2020-07-24 |
| CN111444007B true CN111444007B (en) | 2020-11-17 |
Family
ID=71655381
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010545489.6A Active CN111444007B (en) | 2020-06-16 | 2020-06-16 | Remote sensing big data automatic processing method based on cloud computing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111444007B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112217556A (en) * | 2020-09-30 | 2021-01-12 | 广州中海电信有限公司 | System and method for processing various satellite data based on cloud computing platform |
| CN115033343B (en) * | 2022-05-27 | 2024-03-01 | 河南大学 | Method for establishing remote sensing data flow scheduling model in cloud environment |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109325580A (en) * | 2018-09-05 | 2019-02-12 | 南京邮电大学 | A kind of adaptive cuckoo searching method for Services Composition global optimization |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106775632B (en) * | 2016-11-21 | 2020-06-12 | 中国科学院遥感与数字地球研究所 | High-performance geographic information processing method and system with flexibly-expandable business process |
| CN108335012A (en) * | 2017-12-26 | 2018-07-27 | 佛山科学技术学院 | A kind of intelligence remote sensing satellite stratification distributed freedom cotasking planning system |
| CN109684088B (en) * | 2018-12-17 | 2023-04-07 | 南京理工大学 | Remote sensing big data rapid processing task scheduling method based on cloud platform resource constraint |
-
2020
- 2020-06-16 CN CN202010545489.6A patent/CN111444007B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109325580A (en) * | 2018-09-05 | 2019-02-12 | 南京邮电大学 | A kind of adaptive cuckoo searching method for Services Composition global optimization |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111444007A (en) | 2020-07-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8521782B2 (en) | Methods and systems for processing large graphs using density-based processes using map-reduce | |
| Eriksson et al. | A consensus-based framework for distributed bundle adjustment | |
| US8645429B1 (en) | Resolving conflicting graph mutations | |
| CN111258744A (en) | Task processing method based on heterogeneous computation and software and hardware framework system | |
| US9053067B2 (en) | Distributed data scalable adaptive map-reduce framework | |
| JP2012022558A (en) | Distributed computation system | |
| CN111444007B (en) | Remote sensing big data automatic processing method based on cloud computing | |
| CN114787824A (en) | Combined hybrid model | |
| CN102663005A (en) | Mass video file storage system based on cloud computation, analysis method and system thereof | |
| CN105512083A (en) | YARN based resource management method, device and system | |
| CN107807983B (en) | Design method of parallel processing framework supporting large-scale dynamic graph data query | |
| CN112463290A (en) | Method, system, apparatus and storage medium for dynamically adjusting the number of computing containers | |
| Dichev et al. | Optimization of collective communication for heterogeneous hpc platforms | |
| Han et al. | Parallel DBSCAN algorithm using a data partitioning strategy with spark implementation | |
| CN104778235A (en) | Tree traversal searching method based on MapReduce cloud calculation model | |
| CN110622146B (en) | Programmable synthesis mechanism for device factor graph | |
| Dai et al. | Research and implementation of big data preprocessing system based on Hadoop | |
| CN108319604B (en) | Optimization method for association of large and small tables in hive | |
| Salama | A swarm intelligence based model for mobile cloud computing | |
| CN114691873A (en) | Semantic processing method and device for automatic driving log data and storage medium | |
| CN104090895A (en) | Method, device, server and system for obtaining cardinal number | |
| Vinothina | Scheduling scientific workflow tasks in cloud using swarm intelligence | |
| CN115730663A (en) | Quantum computing task mapping method and quantum computer operating system | |
| Searles et al. | Creating a portable, high-level graph analytics paradigm for compute and data-intensive applications | |
| Chaari et al. | On the robot path planning using cloud computing for large grid maps |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |