CN114625794A - Satellite Internet of things Spark data processing method, system, terminal and storage medium - Google Patents
Satellite Internet of things Spark data processing method, system, terminal and storage medium Download PDFInfo
- Publication number
- CN114625794A CN114625794A CN202210238071.XA CN202210238071A CN114625794A CN 114625794 A CN114625794 A CN 114625794A CN 202210238071 A CN202210238071 A CN 202210238071A CN 114625794 A CN114625794 A CN 114625794A
- Authority
- CN
- China
- Prior art keywords
- data
- spark
- things
- data processing
- ground
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 25
- 238000012545 processing Methods 0.000 claims abstract description 40
- 238000004364 calculation method Methods 0.000 claims abstract description 21
- 238000000034 method Methods 0.000 claims description 17
- 230000008569 process Effects 0.000 claims description 12
- 238000001514 detection method Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 238000004891 communication Methods 0.000 claims description 4
- 230000006870 function Effects 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 239000008186 active pharmaceutical agent Substances 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
Abstract
The application relates to a satellite Internet of things Spark data processing method, a system, a terminal and a storage medium, which belong to the field of data processing, wherein the satellite Internet of things Spark data processing method comprises the steps of acquiring ground acquisition data and storing the ground acquisition data into a database; calling a Spark calculation engine to perform parallel processing on ground acquisition data stored in a database; and generating an RDD model, and distributing the host resources to the Spark calculation engine by using the RDD model. The data processing method and device have the effect of improving data processing efficiency.
Description
Technical Field
The application relates to the field of data processing, in particular to a satellite Internet of things Spark data processing method, a satellite Internet of things Spark data processing system, a satellite Internet of things Spark data processing terminal and a storage medium.
Background
The internet of things means that any object or process needing monitoring, connection and interaction is collected in real time through various information sensing devices or technologies, and some information needing to be collected is collected; moreover, through the access of the network, the connection between objects and people can be realized, and the intelligent perception, identification and management of the objects are realized; the key of the technology of the Internet of things is data communication transmission, and a large amount of data is gathered in the information transmission process of objects; at present, the processing methods for these data are generally performed in sequence, and in the process of processing data, the inventors found that when the data are processed by using the conventional data processing method, the efficiency of data processing is low.
Disclosure of Invention
The application provides a satellite Internet of things Spark data processing method, a satellite Internet of things Spark data processing system, a satellite Internet of things Spark data processing terminal and a storage medium, and has the characteristic of improving data processing efficiency.
The application aims to provide a satellite Internet of things Spark data processing method.
The above object of the present application is achieved by the following technical solutions:
a satellite Internet of things Spark data processing method comprises the following steps:
acquiring ground acquisition data and storing the ground acquisition data into a database;
calling a Spark calculation engine to perform parallel processing on ground acquisition data stored in a database;
and generating an RDD model, and distributing the host resources to the Spark calculation engine by using the RDD model.
The present application may be further configured in a preferred example, where the step of acquiring the ground collected data includes collecting various types of data by using a ground information collecting terminal, and summarizing the data to form the ground collected data.
In a preferred example, the method may further include performing quality detection on the ground collected data before storing the ground collected data in the database, and if the quality detection is successful, calling the mongoDB database and storing the ground collected data in the MongoDB database.
In a preferred example, the invoking Spark calculation engine to perform parallel processing on the ground collected data stored in the database may further include:
starting a driving module and creating an execution object, wherein the execution object comprises task information;
according to the communication between the execution object and a plurality of working modules, selecting the working module in an idle state to receive a task;
starting an execution module corresponding to the working module in the idle state;
and decomposing the execution object after reversely registering the execution module and the driving module.
In a preferred example, the invoking Spark calculation engine to perform parallel processing on the ground collected data stored in the database may further include:
applying for host resources and distributing the host resources to the working module;
constructing a DAG graph, and decomposing the DAG graph into stages;
when the Job is touched, urging Job to give birth; each Job contains at least one Stage.
In a preferred example, the generating an RDD model and allocating the host resource to the Spark calculation engine using the RDD model may further include rooted the dependency relationship of the RDD and constructed as a DAG graph.
In a preferred example, the step of generating an RDD model and allocating the host resource to the spare computing engine by using the RDD model may further include dividing the constructed DAG graph into a complete stage, tracing back forward according to a last RDD in the stage, and determining the dependency relationship in the process of tracing back.
The second purpose of the application is to provide a satellite Internet of things Spark data processing system.
The second application object of the present application is achieved by the following technical scheme:
a satellite Internet of things Spark data processing system comprises:
the acquisition module is used for acquiring ground acquisition data and storing the ground acquisition data into a database;
the calling module is used for calling a Spark calculation engine to perform parallel processing on the ground acquisition data stored in the database;
and the generating module is used for generating the RDD model and distributing the host resources to the Spark calculation engine by using the RDD model.
The third purpose of the application is to provide an intelligent terminal.
The third objective of the present application is achieved by the following technical solutions:
an intelligent terminal comprises a memory and a processor, wherein the memory stores computer program instructions of the satellite internet of things Spark data processing method, and the computer program instructions can be loaded and executed by the processor.
It is a fourth object of the present application to provide a computer medium capable of storing a corresponding program.
The fourth application purpose of the present application is achieved by the following technical solutions:
a computer readable storage medium storing a computer program capable of being loaded by a processor and executing any one of the above satellite internet of things Spark data processing methods.
Drawings
Fig. 1 is a schematic flow chart of a satellite internet of things Spark data processing method in an embodiment of the present application.
Fig. 2 is a schematic structural diagram of a satellite internet of things Spark data processing system in the embodiment of the present application.
Description of reference numerals: 1. an acquisition module; 2. calling a module; 3. and generating a module.
Detailed Description
The present embodiment is only for explaining the present application and is not limited to the present application, and those skilled in the art can make modifications without inventive contribution to the present embodiment as needed after reading the present specification, but all of them are protected by patent law within the scope of the claims of the present application.
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The embodiments of the present application will be described in further detail below with reference to the drawings.
The application provides a satellite Internet of things Spark data processing method, and the main flow of the method is described as follows.
As shown in fig. 1:
step S101: and acquiring ground acquisition data and storing the ground acquisition data into a database.
Step S102: and calling a Spark calculation engine to perform parallel processing on the ground acquisition data stored in the database.
Step S103: and generating an RDD model, and distributing the host resources to the Spark calculation engine by using the RDD model.
It can be understood that Spark data is adopted in the embodiment of the present application, and compared with a conventional data processing manner, Spark has the advantage that the data processing performance is high; by utilizing the principle of distributed data processing, data can be quickly converted and iterated and cached for subsequent frequent access requirements; under the condition that all data are loaded into the memory, Spark can be faster than Hadoop by 100 times and faster than the traditional data processing by multiple times; the system has good compatibility, can be compatible with Java, Scala, Python and SQL standard APIs so as to be conveniently used in various industries of the Internet of things, also contains a machine learning library used immediately, is ecologically compatible with the existing Hadoop v1 (SIMR) and 2.x (YARN), and can carry out seamless migration on data; various problems of the data processing flow can be continuously improved under the support of an excellent big data processing architecture system.
In the embodiment of the application, the overall process of satellite data transmission and processing is as follows, firstly, data acquired on the ground are uploaded, then the quality detection is carried out on the uploaded data, if the data quality detection is successful, the data are stored in the database, and then the data in the database are correspondingly processed by utilizing the computing engine.
Firstly, a ground terminal collects data information of various industries such as ground temperature and humidity and the like to form preliminary data summarization; in the embodiment of the application, the terminal acquires the sensor data through the gateway, specifically, each industry sets a plurality of sensors according to data acquisition requirements to acquire related data, then sends the related data acquired by the sensors to the gateway, sends the data to the terminal through the gateway, and then the terminal uploads the data.
In the embodiment of the application, a terminal acquires sensor acquisition information sent by a gateway, wherein the sensor acquisition information comprises gateway identification information; sorting the sensor acquisition information according to corresponding priorities according to the gateway identification information and a pre-stored gateway priority rule; the gateway priority rule comprises gateway identification information, gateway priority information and corresponding relation information between the gateway identification and the gateway priority; and the terminal uploads the sorted sensor acquisition information in sequence.
Specifically, the step of establishing the gateway priority rule includes acquiring gateway identification information and gateway priority information; the step of acquiring the gateway priority information comprises the steps of acquiring network coverage information of a gateway; obtaining the terminal quantity information in the coverage range of the gateway according to the network coverage range information of the gateway; setting gateway priority information of the gateway according to the terminal number information, wherein the terminal number is the gateway priority; after the gateway identification information and the gateway priority information are bound, generating a gateway priority rule according to the gateway identification information, the gateway priority information and the corresponding relation information between the gateway identification and the gateway priority; the gateway priority rules are then stored in the respective terminals.
In the embodiment of the application, when the gateway sends data to the terminals, the gateway sends the stored sensor data to all the terminals in the coverage range of the gateway, the terminals receive the sensor data and simultaneously receive the gateway identification information corresponding to the sensor data, then the prestored gateway priority rule is called, then the priority of the sensor data is obtained by utilizing the gateway priority rule and the gateway identification information, then the sensor data are sequenced according to the priority, and the sensor data are uploaded in sequence.
To describe the above process in detail, the following description is given by way of example, for example, three terminals are arranged in the network coverage area of the gateway a, five terminals are arranged in the network coverage area of the gateway B, and one terminal is arranged in the network coverage area of the gateway C; then the gateway priority information of each gateway can be obtained according to the number of terminals in the network coverage range; the gateway priority of the gateway A is 3, the gateway priority of the gateway B is 5, the gateway priority of the gateway C is 1, then the sensor data are sent to the terminal at the gateways A, B and C, the sensor data are attached with gateway identification, and the terminal processes the gateway identification to obtain the priority of the data; it can be understood that, data 1 and data 2 are stored in the terminal 1, the priority of the data 1 is 3, and the priority of the data 2 is 4, then the data 1 is uploaded first; then, data 2 and data 5 are stored in the terminal 2, the priority of the data 2 is 4, the priority of the data 5 is 6, and then the data 2 is uploaded first; therefore, in an actual situation, although there are three terminals in the range covered by the gateway a, the priority of the data sent by the gateway a is 3, but for different terminals, the priority 3 may be sent first or sent later, so that by using this way, the integrity of data uploading is ensured, the possibility of data loss in the data uploading process is reduced, the stability and the security of data uploading are improved, and the timeliness of data uploading is also ensured.
Then, uploading ground acquisition data, receiving the data when the satellite load passes through ground acquisition equipment, and detecting the data quality; if the data quality detection is successful, the Spark regularly schedules the Shell script to read the configuration file, and the configuration file has the information of interface specification, XDR data acquisition frequency, XDR storage position, MongoDB database and the like; then calling a MongoDB database; and then carrying out warehousing management and configuration, finally calling a Spark data task execution engine, and starting to distribute and execute data tasks on the collected large-scale data warehoused.
In the process of processing data, firstly, a Driver is started to drive to start a Spark data processing flow, a Submit request of the data is started, the Submit starts to make a request to a Master, an execution object Spark context is created in the process of making the request, the Spark context actually contains task content, and then a task message is sent to the Master; then, after the Master receives the task information, resource scheduling is started, communication is carried out with all the Workers, idle Workers need to be found, the idle Workers are informed to receive the tasks, and corresponding executors, namely task execution, are started; when the Executor is started, reverse registration with the Driver is started, then the Driver starts to send the task to the corresponding Executor, and the Executor starts to calculate the task; after decomposing the Task, establishing an execution thread pool, and finally completing Task.
The sparkContext is connected to the Master, registers the Master and applies for resources; the Master determines which Worker the resource is allocated on according to the resource application requirement of the SparkContext and the information reported in the Worker heartbeat period, then obtains the resource on the Worker, and then starts the StandaloneExecutionBackend; then using the StandaloneExecutionBackend to register to SparkContext, and the SparkContext sending the Applicaton code to the StandaloneExecutionBackend; and the SparkContext analyzes the Applicationton code, constructs a DAG graph, submits the DAG graph to a DAG Scheduler to be decomposed into stages (when an Action operation is met, a Job is promoted, each Job contains 1 or more stages, and the stages are generally generated before external data and a shuffle are obtained), and then submits the stages (or called as a TaskSet) to the Task Scheduler, and the Task Scheduler is responsible for distributing tasks to corresponding workers and finally submits the tasks to a StandaleExecutionBackend for execution; the StandaloneExecutionBackend establishes an Executor thread pool, starts to execute the Task and reports to the SparkContext until the Task is finished; and finally, after all tasks are completed, the SparkContext logs out to the Master, and resources are released.
It can be understood that, in the embodiment of the present application, when a user submits a job, an RDD is generated through a series of operations, such as join, groupByKey, filter, and the like, and then a DAG schedule builds a DAG graph by rooted the dependency relationship of the RDD, but does not execute the job, and only when an action operation of the RDD is encountered, all tasks before execution is triggered. The DAGScheduler divides the task into stages according to the dependency relationship and submits the TaskSet to the TaskScheduler, and the TaskScheduler applies for resources to be distributed to the Wroeker node according to the task manager, so that the tasks are submitted to the Wroeker node to be executed. The DAG scheduler firstly divides a constructed DAG graph into a complete stage, then backtracks forward according to the last RDD in the stage, continuously judges the dependency relationship of the RDD in the backtracking process, continuously backtracks if the dependency relationship is narrow, and divides a new stage if the dependency relationship is wide, so that the whole stage is divided into a plurality of new stages. The DAG graph is thus divided into multiple stages, each of which is made up of multiple tasks.
The application also provides a satellite internet of things Spark data processing system, as shown in fig. 2, the satellite internet of things Spark data processing system comprises an acquisition module 1, a storage module and a data processing module, wherein the acquisition module is used for acquiring ground acquisition data and storing the ground acquisition data into a database; the calling module 2 is used for calling a Spark calculation engine to perform parallel processing on the ground collected data stored in the database; and the generating module 3 is configured to generate an RDD model, and allocate the host resource to the Spark calculation engine by using the RDD model.
In order to better execute the program of the method, the application also provides an intelligent terminal which comprises a memory and a processor.
Wherein the memory is operable to store an instruction, a program, code, a set of codes, or a set of instructions. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the above satellite internet of things Spark data processing method, and the like; the data storage area can store data and the like related to the satellite Internet of things Spark data processing method.
A processor may include one or more processing cores. The processor executes or executes the instructions, programs, code sets, or instruction sets stored in the memory, calls data stored in the memory, performs various functions of the present application, and processes the data. The processor may be at least one of an application specific integrated circuit, a digital signal processor, a digital signal processing device, a programmable logic device, a field programmable gate array, a central processing unit, a controller, a microcontroller, and a microprocessor. It is understood that the electronic devices for implementing the above processor functions may be other devices, and the embodiments of the present application are not limited in particular.
The present application also provides a computer-readable storage medium, for example, comprising: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk. The computer readable storage medium stores a computer program which can be loaded by a processor and executes the satellite internet of things Spark data processing method.
The foregoing description is only exemplary of the preferred embodiments of the invention and is provided for the purpose of illustrating the general principles of the technology. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the disclosure. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.
Claims (10)
1. A satellite Internet of things Spark data processing method is characterized by comprising the following steps:
acquiring ground acquisition data and storing the ground acquisition data into a database;
calling a Spark calculation engine to perform parallel processing on the ground acquisition data stored in the database;
and generating an RDD model, and distributing the host resources to the Spark calculation engine by using the RDD model.
2. The satellite internet of things Spark data processing method according to claim 1, wherein the step of acquiring the ground collected data includes collecting various types of data by using a ground information collection terminal, and summarizing the data to form the ground collected data.
3. The satellite internet of things Spark data processing method as claimed in claim 1, wherein before the ground collected data is stored in the database, quality detection needs to be performed on the ground collected data, and if the quality detection is successful, the mongoDB database is called and the ground collected data is stored in the MongoDB database.
4. The satellite internet of things Spark data processing method according to claim 1, wherein the step of calling a Spark calculation engine to perform parallel processing on the ground collected data stored in the database comprises the following steps:
starting a driving module and creating an execution object, wherein the execution object comprises task information;
according to the communication between the execution object and a plurality of working modules, selecting the working module in an idle state to receive a task;
starting an execution module corresponding to the working module in the idle state;
and decomposing the execution object after reversely registering the execution module and the driving module.
5. The satellite internet of things Spark data processing method according to claim 4, wherein the step of calling a Spark calculation engine to perform parallel processing on the ground collected data stored in the database comprises the following steps:
applying for host resources and distributing the host resources to the working module;
constructing a DAG graph, and decomposing the DAG graph into stages;
when the Job is touched, urging Job to give birth; each Job contains at least one Stage.
6. The satellite internet of things Spark data processing method according to claim 1, wherein the step of generating an RDD model and allocating host resources to Spark calculation engines by using the RDD model includes the step of eradicating dependency relationships of RDD and constructing a DAG graph.
7. The satellite Internet of things Spark data processing method according to claim 1, wherein the step of generating an RDD model and allocating host resources to a Spark calculation engine by using the RDD model includes dividing a constructed DAG into a complete stage, tracing back forward according to the last RDD in the stage, and judging the dependency relationship in the process of tracing back.
8. A satellite Internet of things Spark data processing system is characterized by comprising:
the acquisition module (1) is used for acquiring ground acquisition data and storing the ground acquisition data into a database;
the calling module (2) is used for calling a Spark calculation engine to perform parallel processing on the ground acquisition data stored in the database;
and the generating module (3) is used for generating the RDD model and allocating the host resources to the Spark calculation engine by using the RDD model.
9. An intelligent terminal, comprising a memory and a processor, the memory having stored thereon computer program instructions capable of being loaded by the processor and performing the method of any of claims 1-7.
10. A computer-readable storage medium, in which a computer program is stored which can be loaded by a processor and which executes the method according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210238071.XA CN114625794A (en) | 2022-03-10 | 2022-03-10 | Satellite Internet of things Spark data processing method, system, terminal and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210238071.XA CN114625794A (en) | 2022-03-10 | 2022-03-10 | Satellite Internet of things Spark data processing method, system, terminal and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114625794A true CN114625794A (en) | 2022-06-14 |
Family
ID=81901419
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210238071.XA Pending CN114625794A (en) | 2022-03-10 | 2022-03-10 | Satellite Internet of things Spark data processing method, system, terminal and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114625794A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109446395A (en) * | 2018-09-29 | 2019-03-08 | 上海派博软件有限公司 | A kind of method and system of the raising based on Hadoop big data comprehensive inquiry engine efficiency |
| CN109690517A (en) * | 2016-09-15 | 2019-04-26 | 甲骨文国际公司 | Snapshot and state are managed using micro- batch processing |
| CN111143097A (en) * | 2018-11-03 | 2020-05-12 | 千寻位置网络有限公司 | GNSS positioning service-oriented fault management system and method |
| CN112527945A (en) * | 2021-02-10 | 2021-03-19 | 中关村科学城城市大脑股份有限公司 | Method and device for processing geographic space big data |
| CN114090537A (en) * | 2022-01-20 | 2022-02-25 | 北京航天驭星科技有限公司 | Real-time analysis method, device, system, equipment and medium for satellite in-orbit state |
-
2022
- 2022-03-10 CN CN202210238071.XA patent/CN114625794A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109690517A (en) * | 2016-09-15 | 2019-04-26 | 甲骨文国际公司 | Snapshot and state are managed using micro- batch processing |
| CN109446395A (en) * | 2018-09-29 | 2019-03-08 | 上海派博软件有限公司 | A kind of method and system of the raising based on Hadoop big data comprehensive inquiry engine efficiency |
| CN111143097A (en) * | 2018-11-03 | 2020-05-12 | 千寻位置网络有限公司 | GNSS positioning service-oriented fault management system and method |
| CN112527945A (en) * | 2021-02-10 | 2021-03-19 | 中关村科学城城市大脑股份有限公司 | Method and device for processing geographic space big data |
| CN114090537A (en) * | 2022-01-20 | 2022-02-25 | 北京航天驭星科技有限公司 | Real-time analysis method, device, system, equipment and medium for satellite in-orbit state |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106406987B (en) | Task execution method and device in cluster | |
| CN111176852A (en) | Resource allocation method, device, chip and computer readable storage medium | |
| CN110781007A (en) | Task processing method, device, server, client, system and storage medium | |
| CN111768006A (en) | Artificial intelligence model training method, device, equipment and storage medium | |
| AU2019201544B2 (en) | Distributed multiple tier multi-node serverless framework for complex analytics task execution | |
| CN109189572B (en) | Resource estimation method and system, electronic equipment and storage medium | |
| CN112699040B (en) | Pressure testing method, device, equipment and computer readable storage medium | |
| CN105022668B (en) | Job scheduling method and system | |
| CN111400288A (en) | Data quality inspection method and system | |
| CN110780870A (en) | Service execution method, device, equipment and storage medium | |
| CN111932257A (en) | Block chain parallelization processing method and device | |
| CN110781180A (en) | Data screening method and data screening device | |
| CN111680085A (en) | Data processing task analysis method and device, electronic equipment and readable storage medium | |
| CN112037061A (en) | Processing method and device for transactions in block chain, electronic equipment and storage medium | |
| CN107870921B (en) | Log data processing method and device | |
| CN114625794A (en) | Satellite Internet of things Spark data processing method, system, terminal and storage medium | |
| CN111753011A (en) | Intelligent contract execution node selection method, equipment and medium based on block chain | |
| CN111831408A (en) | Asynchronous task processing method and device, electronic equipment and medium | |
| CN114039878B (en) | Network request processing method and device, electronic equipment and storage medium | |
| CN115794382A (en) | Data processing method, device and nonvolatile storage medium | |
| CN112685158B (en) | Task scheduling method and device, electronic equipment and storage medium | |
| JP2010072876A (en) | Rule creation program, rule creation method, and rule creation device | |
| CN110968420A (en) | Scheduling method and device for multi-crawler platform, storage medium and processor | |
| CN115168509A (en) | Processing method and device of wind control data, storage medium and computer equipment | |
| CN112948092A (en) | Batch job scheduling method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220614 |