CN114564547A - Data processing method, device, equipment and storage medium - Google Patents
Data processing method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN114564547A CN114564547A CN202210124413.5A CN202210124413A CN114564547A CN 114564547 A CN114564547 A CN 114564547A CN 202210124413 A CN202210124413 A CN 202210124413A CN 114564547 A CN114564547 A CN 114564547A
- Authority
- CN
- China
- Prior art keywords
- operator
- node
- task
- nodes
- operator node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 25
- 238000000034 method Methods 0.000 claims abstract description 61
- 238000012545 processing Methods 0.000 claims abstract description 32
- 230000008569 process Effects 0.000 claims description 23
- 238000004590 computer program Methods 0.000 claims description 18
- 238000004422 calculation algorithm Methods 0.000 claims description 12
- 238000006243 chemical reaction Methods 0.000 claims description 11
- 230000010354 integration Effects 0.000 claims description 5
- 230000003993 interaction Effects 0.000 claims description 3
- 238000002156 mixing Methods 0.000 claims description 2
- 230000006870 function Effects 0.000 description 24
- 238000004364 calculation method Methods 0.000 description 18
- 238000010586 diagram Methods 0.000 description 12
- 230000009471 action Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000007689 inspection Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 238000012732 spatial analysis Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 125000002015 acyclic group Chemical group 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000002344 surface layer Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/29—Geographical information databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24568—Data stream processing; Continuous queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Remote Sensing (AREA)
- Computational Linguistics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application discloses a data processing method, a data processing device, data processing equipment and a storage medium. The method comprises the steps of obtaining a task flow to be executed, wherein the task flow comprises task characteristics; determining scheduling information of at least two operator nodes for executing a logic operation instruction of a task flow and executing the logic operation instruction according to task characteristics, wherein the at least two operator nodes comprise a first operator node and a second operator node, and the operator types of the at least two operator nodes are different; and calling the first operator node and the second operator node to execute the logic instruction through the preset mixed-encoding operator node according to the scheduling information to obtain a target task result of the task flow. According to the data processing method provided by the embodiment of the application, the geographic information service can be effectively provided for the user through the mixed arrangement of the nodes with different sources and different operator types in a complex service scene.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data processing method, apparatus, device, and storage medium.
Background
With the continuous development of Geographic Information Systems (GIS) and big data technologies, the service requirements of collecting, storing, managing, operating, analyzing, displaying and describing relevant Geographic distribution data in the whole or part of the space of the earth surface layer (including the atmosphere) can be realized through the computing functions provided by the GIS and the GIS.
Currently, geographic information services can be provided to users through a single geographic information system and the computing functionality that it provides. However, as the complexity of service scenarios increases, in a complex service scenario, a plurality of geographic information systems and computing functions provided by the geographic information systems are required to provide geographic information services for users. However, the above-mentioned manner of providing geographic information services to users is not suitable for complex service scenarios because of the inability of interworking between multiple geographic information systems and the computing functions provided thereby.
Disclosure of Invention
The embodiment of the application provides a data processing method, a data processing device, data processing equipment and a storage medium, and solves the problems that data of a plurality of geographic information systems and a computing function provided by the geographic information systems cannot be communicated at present and geographic information service cannot be provided for a user in a complex service scene.
According to a first aspect of embodiments of the present application, there is provided a data processing method, including:
acquiring a task flow to be executed, wherein the task flow comprises task characteristics;
determining scheduling information of at least two operator nodes for executing a logic operation instruction of a task flow and executing the logic operation instruction according to task characteristics, wherein the at least two operator nodes comprise a first operator node and a second operator node, and the operator types of the at least two operator nodes are different;
And calling the first operator node and the second operator node to execute the logic instruction through the preset mixed-coding operator node according to the scheduling information to obtain a target task result of the task flow.
According to a second aspect of embodiments of the present application, there is provided an operator node generation method, including:
receiving access requests of at least two operator nodes, wherein the operator types of the at least two operator nodes are different;
acquiring node characteristics of each operator node in at least two operator nodes;
constructing target hybrid-coded operator nodes according to the node characteristics of each operator node and preset integration conditions; wherein,
the target mixed operator node is used for converting task data and result data of the task flow according to the node characteristics of each target operator node in at least two target operator nodes executing the task flow in the task flow executing process, so that the task data and the result data are transmitted between the at least two target operator nodes, and the result data is data obtained after each target operator node processes the task data.
According to a third aspect of embodiments of the present application, there is provided a data processing apparatus comprising:
the acquisition module is used for acquiring a task flow to be executed, and the task flow comprises task characteristics;
The determining module is used for determining scheduling information of at least two operator nodes for executing a logical operation instruction of a task stream and executing the logical operation instruction according to task characteristics, wherein the at least two operator nodes comprise a first operator node and a second operator node, and the operator types of the at least two operator nodes are different;
and the processing module is used for calling the first operator node and the second operator node to execute the logic instruction through the preset mixed coding operator node according to the scheduling information to obtain a target task result of the task flow.
According to a fourth aspect of embodiments of the present application, there is provided an operator node generation apparatus, including:
the receiving module is used for receiving access requests of at least two operator nodes, and the operator types of the at least two operator nodes are different;
the acquisition module is used for acquiring the node characteristics of each operator node in at least two operator nodes;
the construction module is used for constructing target hybrid-coded operator nodes according to the node characteristics of each operator node and preset integration conditions; wherein,
the target mixed operator node is used for converting task data and result data of the task flow according to the node characteristics of each target operator node in at least two target operator nodes executing the task flow in the task flow executing process, so that the task data and the result data are transmitted between the at least two target operator nodes, and the result data is data obtained after each target operator node processes the task data.
According to a fifth aspect of embodiments of the present application, there is provided a computer apparatus, comprising: a memory and a processor;
a memory for storing a computer program;
a processor for executing a computer program stored in the memory, the computer program when executed causing the processor to perform the steps of the data processing method as shown in the first aspect or to perform the steps of the operator node generation method as shown in the second aspect.
According to a sixth aspect of embodiments of the present application, there is provided a computer-readable storage medium on which a program or instructions are stored, which, when executed by a computer device, causes the computer device to perform the steps of the data processing method as shown in the first aspect or to perform the steps of the operator node generation method as shown in the second aspect.
According to a seventh aspect of embodiments of the present application, there is provided a computer program product comprising a computer program which, in a case where the computer program is executed by a computer device, causes the computer device to execute the steps of the data processing method as shown in the first aspect or the steps of the operator node generation method as shown in the second aspect.
According to the data processing method, device, equipment and storage medium in the embodiment of the application, through the task flow to be executed, various geographic information systems and calculation functions for executing the task flow are abstracted into at least two operator nodes, at least two operator nodes with different sources and different types are expanded and integrated on the basis, then, a preset mixed-encoding operator node capable of realizing data transmission between the at least two operator nodes is defined, so that the data between the various operator nodes can be mutually transmitted, then, according to the logic of the executed task flow, the scheduling information of the at least two operator nodes according to the instruction and the execution operation instruction is quickly carried out, the various operator nodes are called through the preset mixed-encoding operator node to complete the task flow to be executed, therefore, the compatible calculation of the various operator nodes is met, and the arrangement, the distribution and the distribution of cross-platform, cross-language and cross-type operator nodes are realized, Scheduling and execution can effectively provide geographic information service for users through mixed arrangement of nodes with different sources and different operator types in a complex service scene.
Drawings
The present application may be better understood from the following description of specific embodiments of the application taken in conjunction with the accompanying drawings, in which like or similar reference numerals identify like or similar features.
FIG. 1 is a diagram illustrating a data processing architecture according to one embodiment;
FIG. 2 is a schematic diagram illustrating the structure of an operator editor, according to one embodiment;
FIG. 3 is a flow diagram illustrating a data processing method according to one embodiment;
FIG. 4 is a flow diagram illustrating a method of operator node generation according to one embodiment;
FIG. 5 is a flow diagram illustrating a determination of a target task result according to one embodiment;
FIG. 6 is a schematic diagram showing a configuration of a data processing apparatus according to an embodiment;
FIG. 7 is a schematic diagram showing the structure of an operator node generation apparatus according to one embodiment;
fig. 8 is a diagram showing a hardware configuration of a computer apparatus according to an embodiment.
Detailed Description
Features and exemplary embodiments of various aspects of the present application will be described in detail below, and in order to make objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail below with reference to the accompanying drawings and the embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application. It will be apparent to one skilled in the art that the present application may be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present application by illustrating examples thereof.
It should be noted that, in this document, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.
With the continuous development of GIS and big data technologies, the sources and types of computing functions provided by GIS and GIS are more and more abundant, for example, GIS includes ArcGIS platform and SuperMap platform, and the computing functions provided by GIS may include spatial operators provided by various GIS, spatial computing functions in spatial databases, algorithm applications, and the like. The spatial operator involved in the embodiment of the application is a function model for performing two-dimensional and three-dimensional calculation and analysis on data with geographic characteristics based on a mapping, geography and mathematical formula in a computer programming mode, and the spatial operator includes but is not limited to function models of geocoding, spatial analysis, three-dimensional calculation, projection transformation, topology inspection and the like.
In the related art, a user may be provided with a geographic information service through a single GIS and a computing function provided by the GIS. For example, the ArcGIS platform can provide spatial operators such as geocoding, spatial analysis, three-dimensional computation, projective transformation, topology inspection and the like, and provide functions such as data synchronization, data exploration, data quality inspection, data service and the like for users.
However, with the increase of complexity of a service scenario, in a complex service scenario, because a plurality of geographic information systems and the computing functions provided by the geographic information systems cannot be intercommunicated, a single GIS platform and the functions of a single spatial operator provided by the platform cannot meet complex service logic, so that geographic information services cannot be provided for users in the complex service scenario. For example, the ArcGIS platform and the SuperMap platform support only the arrangement and execution of spatial operators built in the ArcGIS platform, and do not support mixed arrangement with other types of spatial operators.
Therefore, how to support data intercommunication between a plurality of geographic information systems and the computing functions provided by the geographic information systems at present becomes a problem which needs to pay attention to in order to provide geographic information services for users in a complex service scene.
Based on this, the embodiment of the present application provides a data processing method supporting spatial operator hybrid computation, which abstracts various geographic information systems and computation functions for executing a task flow into at least two operator nodes through the task flow to be executed, and expands and integrates at least two operator nodes of any types such as different languages, different sources, different standards, and different uses (space/non-space) on the basis of the at least two operator nodes, and then defines a preset hybrid operator node capable of implementing data transmission between the at least two operator nodes, so that data between the various operator nodes can be mutually transmitted, and then calls the various operator nodes to complete the task flow to be executed according to the logic for executing the task flow and scheduling information of the at least two operator nodes for executing an operation instruction.
Therefore, compatible calculation of various operator nodes is met, arrangement, scheduling and execution of cross-platform, cross-language and cross-type operator nodes are achieved, description and analysis of complex service scenes can be effectively achieved through mixed arrangement of nodes from different sources and in different operator types in the complex service scenes, and geographic information services are provided for users. Based on this, the data processing method provided by the embodiment of the present application is described in detail below with reference to the data processing architecture in the embodiment of the present application shown in fig. 1.
In one or more possible embodiments, as shown in fig. 1, the data processing architecture provided in the embodiment of the present application may include a data resource platform, where the data resource platform may be a data resource platform in at least one of the following clouds: private cloud, public cloud, hybrid cloud.
In order to meet hot plug and expandability of operator nodes in a data resource platform, an operator editor 10 is arranged in the data resource platform, and scheduling and execution of the operator nodes in the operator editor adopt a layered design mode, that is, the operator editor comprises a console (console)101 and a service desk (server) 102.
The console 101 is used to provide a running framework for the overall task flow in order to schedule the plurality of operator nodes provided by the data resource platform 10. In one example, as shown in fig. 2, the console 101 may include a Unit Service class module (Unit Service)1011, a Data Source Service class module (Data Source Service)1012, a task flow Service module (Work flow Service)1013, and an interface Service module (Api Service) 1014.
Specifically, the Unit service class module 1011 is configured to read operator information of at least two operator nodes through a configuration file (Unit Mapper). The data source service module 1012 is configured to manage a database structure and field information of Structured Query Language (SQL) for executing logical operation instructions and scheduling information, where the field information is used to perform data required by the data processing method provided in the embodiment of the present application. The task flow service module 1013 is configured to schedule a task flow and schedule a preset hybrid operator Node (Node), further, the task flow scheduling may include determining scheduling information of at least two operator nodes that execute a logical operation instruction on the task flow and execute the logical operation instruction when the task flow to be executed is obtained, and the scheduling Node may include an operator information of at least two operator nodes that access the data resource platform 10 and is read by the unit service class module 1011 to construct a Node, or construct a Node according to at least two operator nodes that execute the logical operation instruction. The interface service module 1014 is configured to communicate with the service desk 102, that is, send the logic operation instruction, the scheduling information, the operator information of at least two operator nodes, and the operator information of the Node output by the task flow service module 1013 to the service desk 102, so as to implement invoking of a specific operator instance method in this embodiment. It should be noted that, in order to enable data among various operator nodes to be transmitted mutually, in an example, the operator nodes may be packaged in an Application Programming Interface (API) manner through the Interface service module 1014, so as to implement cross-platform access and call of various operator nodes in an API manner, and may effectively provide geographic information services for users through mixed arrangement of nodes from different sources and with different operator types in a complex service scenario.
The service desk 102 is used to invoke instances of at least two operator nodes of a task flow, executing the task flow (or workflow). In one example, still referring to fig. 2, the Service desk 102 may include a task flow execution Service class module (Work flow export Service)1021 for communicating with the interface Service module 1014, and in one example, the communicated content may include a logical operation instruction to execute the task flow, scheduling information of at least two operator nodes to execute the logical operation instruction, operator information of the at least two operator nodes, and operator information of the Node.
Based on this, the task flow execution service class module 1021 may be further configured to determine, according to the operator information of the at least two operator nodes, a first operator Node and a second operator Node that execute the logical operation instruction, and determine the Node according to the operator information of the Node. And then, according to the scheduling information, calling the first operator Node and the second operator Node through the Node to execute a logic instruction to obtain a target task result of the task flow.
Further, the task flow execution service class module 1021 may be specifically configured to, according to the scheduling information, invoke the first operator Node to execute the logic instruction, obtain a first task result, convert the first task result by the Node, obtain a second task result, and use the second task result as input information of the next operator Node, that is, based on the second task result, invoke the second operator Node to execute the logic instruction, and obtain a target task result.
Therefore, the embodiment of the application defines the general preset mixed editing operator node by abstracting the characteristics of various GIS platform operators, meets the compatible calculation of various operators, provides an operator editor with a console-service desk architecture in order to enable data between different geographic information systems and the provided calculation functions to be mutually transmitted, realizes the arrangement, scheduling and execution of Directed Acyclic Graphs (DAG) of cross-platform, cross-language and cross-type operators, and solves the problem of the requirement of mixed arrangement and execution of different defined operator nodes from different sources in a complex service scene.
The operator editor abstracts and integrates various types of geographic information systems and computing functions into operator nodes, and abstracts preset mixed operator nodes of the operator nodes with different languages, different sources, different standards and different purposes (space/non-space).
Based on the method, when the task flow to be executed is obtained, at least two operator nodes for executing the task flow are arranged and mobilized through the preset mixed-encoding operator nodes, and therefore, the preset mixed-encoding operator nodes are used for carrying out mixed-encoding calculation on the operator nodes, data among various operator nodes can be transmitted mutually, cross-platform access and calling of various operator nodes are achieved, geographic information services can be effectively provided for users through mixed arrangement of nodes with different sources and different computation subtypes in a complex service scene, and flexibility and expansibility of the operator nodes are supported in a loose coupling mode.
It should be noted that the hybrid computing in the embodiment of the present application may be a computing manner for describing and analyzing a complex scene by hybrid arrangement of any type of operator nodes of different languages, different sources, different standards, different purposes (space/non-space), and the like.
According to the above architecture and application scenario, the data processing method provided by the embodiment of the present application is described in detail below with reference to fig. 3 to 5, respectively.
FIG. 3 is a flow diagram illustrating a data processing method according to one embodiment.
As shown in fig. 3, the data processing method may be applied to fig. 1 or fig. 2, and specifically may include:
The above steps are described in detail below, specifically as follows.
The embodiment of the present application provides the following two ways to trigger execution of the data processing method provided in the embodiment of the present application.
The first mode is as follows: firstly, receiving a request that a plurality of geographic information systems and computing functions provided by the geographic information systems access to a data resource platform; then, according to the requests, abstracting a plurality of geographic information systems with different languages, different sources, different standards and different purposes (space/non-space) and the provided computing functions thereof as operator nodes through a data resource platform; and then, constructing a target mixed-encoding operator node of the data resource platform based on the operator nodes, so that data among a plurality of abstracted operator nodes can be mutually transmitted through the target mixed-encoding operator node, cross-platform access and calling of various operator nodes are realized, and geographic information service can be effectively provided for users through mixed arrangement of nodes with different sources and different operator types in a complex service scene.
The second mode is as follows: the data resource platform can acquire a task flow to be executed, and a geographic information system and a computing function for executing the task flow are abstractly used as operator nodes through the data resource platform according to the task flow, wherein the geographic information system and the computing function for executing the task flow can belong to different languages, different sources, different standards and different purposes (space/non-space); then, based on the operator node executing the task flow, a special preset mixed editing operator node for executing the operator node of the task flow is constructed, so that data between at least two abstracted operator nodes executing the task flow can be mutually transmitted through the preset mixed editing operator node, cross-platform access and calling of various operator nodes are realized, and geographic information service can be effectively provided for a user through mixed editing of nodes with different sources and different operator types in a complex service scene.
Based on the above two manners, the manners of determining at least two operator nodes and determining the preset hybrid encoding operator node provided in the embodiment of the present application are different, which are specifically shown as follows.
Based on the first way, as shown in fig. 4, an embodiment of the present application provides a flowchart of an operator node generation method, where the method may include: step 410 to step 430.
And step 420, acquiring the node characteristics of each operator node in at least two operator nodes.
And step 430, constructing a target hybrid-coded operator node according to the node characteristics of each operator node and preset integration conditions.
The target mixed-encoding operator node is used for converting task data and result data of the task flow according to the node characteristics of each target operator node in at least two target operator nodes of the execution task flow in the task flow execution process, so that the task data and the result data are transmitted between the at least two target operator nodes, and the result data are data obtained after each target operator node processes the task data.
Specifically, referring to step 430, the embodiment of the present application provides the following four ways to integrate at least two operator nodes to obtain a target hybrid operator node.
In the first mode, an operator application mirror image package of each operator node in at least two operator nodes is constructed according to node characteristics of the at least two operator nodes with different sources and different operator types accessed to the data resource platform, wherein the operator application mirror image package comprises operator operation system information and editing environment information.
And then, generating a target mixed-encoding operator node based on the operator application mirror package of each operator node.
Illustratively, the at least two operator nodes include a first operator node and a second operator node. If the first operator node is the SuperMap operator, the operator operation system information of the SuperMap operator is a centOS system, and the editing environment information can be a Java compiling environment. If the second operator node is a QGIS operator, the operator operating system information of the QGIS operator is an Ubuntu system, and the editing environment information may be a Python compiling environment.
And then, generating a target mixed-encoding operator node based on the operator operation system information and the editing environment information of each operator node, wherein the target mixed-encoding operator node records the operator operation system information and the editing environment information of each operator node, so that input data and output data of each operator node can be converted through the target mixed-encoding operator node, data among various operator nodes can be mutually transmitted, cross-platform access and calling of various operator nodes are realized, and geographic information service can be effectively provided for a user through mixed arrangement of nodes with different sources and different operator types in a complex service scene.
And abstracting the general attribute of each operator node in the at least two operator nodes according to the node characteristics of the at least two operator nodes with different sources and different operator types accessed to the data resource platform, wherein the general attribute can comprise an operator input parameter attribute, a calculation parameter attribute and an operator output parameter attribute of the operator node.
And then, generating a target hybrid-coded operator node based on the operator input parameter attribute, the calculation parameter attribute and the operator output parameter attribute of each operator node.
Further, the operator input parameter attribute may specifically include two parameters, namely, an input Name (input Name) and an input Type (input Type), for characterizing the input parameters of the operator node; calculating parameter attributes for representing the execution logic of the operator nodes; the operator output parameter attribute is used for setting the output mode of the operator node, such as a binary stream or a database.
Based on this, the target co-woven operator node records the general attribute of each operator node, so that data to be transmitted to the operator node can be converted through the target co-woven operator node according to the operator input parameter and the calculation parameter attribute of the operator node, so that the operator node can identify and process the converted data, and then the operator node can output parameters according to the set operator, so that the target co-woven operator node can identify the data output by the operator node and transmit the data to other operator nodes, so that the data among various operator nodes can be mutually transmitted, the cross-platform access and call of various operator nodes are realized, and the geographic information service can be effectively provided for users in a complex service scene through the co-woven arrangement of the nodes with different sources and different operator types.
And analyzing the operator types of at least two operator nodes accessing different sources and different operator types of the data resource platform through a preset parameter analysis algorithm to obtain the operator data format of each operator node.
And generating a target mixed-encoding operator node based on the operator data format of each operator node.
Illustratively, the preset parameter parsing algorithm may be Data Set Convert To wkb or Data Set Convert To WKT, and the at least two operator nodes may be ArcGIS, SuperMap and QGIS, so as To obtain an operator Data format of each operator node.
And then, the operator data format of each operator node is stored in the target hybrid-coded operator node and converted into the open-source GIS data format, so that the target hybrid-coded operator node converts the data output by each operator node into a uniform data format, the data among various operator nodes can be mutually transmitted, the cross-platform access and call of various operator nodes are realized, and the geographic information service can be effectively provided for users by mixing and arranging the nodes with different sources and different operator types in a complex service scene.
And fourthly, constructing a target hybrid-encoding operator node according to at least two operator nodes with different sources and different operator types, which are accessed to the data resource platform, wherein the target hybrid-encoding operator node provides an open source API (open API), the open API is used for converting task data and result data of the task flow in the task flow execution process, so that the task data and the result data are transmitted between the at least two target operator nodes, and the result data are data obtained after each target operator node processes the task data. Alternatively, a directed acyclic graph DAG is constructed and executed based on a task flow.
Based on this, the open API may include a Save model for saving operator parameters of each operator node in the DAG, a Run model for accessing and calling logical operation instructions of each operator node to execute the task flow, and a get Status model for listening to the execution results, for example, when the get Status returns a preset value such as 200, indicating that a target task result of the task flow is obtained, where, in some embodiments, the execution result of the task flow may be queried from the database.
Therefore, DAG is constructed through the open API, the execution result of DAG execution task flow is monitored, data among various operator nodes can be mutually transmitted, cross-platform access and call of the various operator nodes are achieved, and geographic information service can be effectively provided for users through mixed arrangement of nodes with different sources and different operator types in a complex service scene.
In a second way, referring to step 320, the embodiment of the present application provides at least two ways of determining at least two operator nodes and scheduling information for executing a logical operation instruction, which are specifically shown below.
In a possible embodiment, the step 320 may specifically include:
Identifying whether the task characteristics comprise information of at least two appointed operator nodes of a user appointed execution task flow;
and under the condition that the task characteristics are identified to comprise information of at least two designated operator nodes of the task flow executed by the user, determining a logic operation instruction for executing the task flow according to the task flow, and determining the at least two designated operator nodes as the at least two operator nodes according to the information of the designated operator nodes.
In another possible embodiment, the step 320 may specifically include: according to task characteristics, constructing a directed acyclic graph of a task flow through preset mixed-encoding operator nodes, wherein the directed acyclic graph comprises a path and at least two spatial operators on the path;
and determining at least two spatial operators as at least two operator nodes, and generating scheduling information of the at least two operator nodes according to the path.
Based on the preset hybrid operator node and the target hybrid operator node determined in the first manner and the second manner, the step 330 may be executed, and it should be noted that the preset hybrid operator node in the step 330 is equivalent to the target hybrid operator node.
Referring to step 330, in one or more possible embodiments, as shown in fig. 5, step 330 may specifically include steps 3301 to 3303, specifically as shown below.
And 3301, according to the scheduling information, calling the first operator node to execute the logic instruction, and obtaining a first task result.
And 3302, converting the first task result through the preset hybrid programming sub-node to obtain a second task result.
Further, the step 3302 may specifically include:
and converting the first task result through the preset mixed-encoding sub-nodes according to the node attribute of the second sub-nodes to obtain a second task result.
And 3303, based on the second task result, calling the second operator node to execute the logic instruction, so as to obtain a target task result.
The following describes steps 3302 and 3303 in detail with reference to different scenarios.
In one example, referring to step 3302, at least two node attributes are provided in the embodiments of the present application, which are described below, respectively for this step 3302.
(1) And under the condition that the node attribute comprises an operator application mirror image package which comprises operator operation system information and editing environment information, converting the first task result through a preset mixed editing operator node according to the operator operation system information and the editing environment information of the second operator node to obtain a second task result.
(2) And under the condition that the node attribute comprises an operator input parameter attribute, converting the first task result through a preset hybrid operator node according to the input parameter attribute to obtain a second task result.
Based on (2) possibilities, step 3303 in this embodiment may specifically include:
calling a second operator node to execute a logic instruction based on a second task result according to the calculation parameter attribute to obtain a stage task result;
and converting the stage task result according to the operator output parameter attribute to obtain a target task result.
In another example, before the step 3302, the method may further include:
determining a preset parameter conversion algorithm of each operator node according to the operator type of each operator node in at least two operator nodes;
and converting the operator parameters in the operator nodes corresponding to the preset parameter conversion algorithm through a preset parameter conversion algorithm to obtain an open source data format of data interaction between at least two operator nodes.
Based on this, the step 3302 may specifically include:
and converting the first task result through a preset mixed coding sub-node according to the open source data format to obtain a second task result.
In yet another example, prior to step 330, the method may further comprise:
the method comprises the steps of obtaining a plurality of operator nodes, and constructing an open source programming interface of each operator node through a preset mixed-coded operator node based on each operator node in the operator nodes.
Based on this, referring to step 330, in the case that the first operator node corresponds to the first open source programming interface, and the second operator node corresponds to the second open source programming interface, the step 330 may specifically include:
calling a first open source programming interface to execute a logic instruction through a preset mixed-programming sub-node according to the scheduling information to obtain a third task result;
converting the third task result through a preset hybrid coding sub-node to obtain a fourth task result;
and calling a second open-source programming interface to execute the logic instruction based on the fourth task result to obtain a target task result.
Therefore, the embodiment of the application provides a data processing method supporting operator node mixed editing, which defines a universal preset mixed editing operator node by abstracting the characteristics of various operator nodes, expands and integrates operator nodes from various sources on the basis, realizes the arrangement, scheduling and execution of cross-platform, cross-language and cross-type operator nodes, and solves the problem of the requirement of mixed arrangement and execution of operator nodes from different sources and different definitions in a complex service scene.
It should be understood that the present application is not limited to the particular configurations and processes described in the above embodiments and illustrated in the drawings. For convenience and simplicity of description, detailed description of a known method is omitted here, and for the specific working processes of the system, the module and the unit described above, reference may be made to corresponding processes in the foregoing method embodiments, which are not described again here.
Based on the same inventive concept, the present application provides a data processing apparatus corresponding to the above-described data processing method. This is explained in detail with reference to fig. 6.
Fig. 6 is a schematic configuration diagram showing a data processing apparatus according to an embodiment.
As shown in fig. 6, the data processing apparatus 60 is applied to the operator editor 10 shown in fig. 1 or fig. 2, and the data processing apparatus 60 may specifically include:
an obtaining module 601, configured to obtain a task stream to be executed, where the task stream includes task features;
a determining module 602, configured to determine, according to task characteristics, scheduling information of at least two operator nodes that execute a logical operation instruction on a task stream and execute the logical operation instruction, where the at least two operator nodes include a first operator node and a second operator node, and the operator types of the at least two operator nodes are different;
the processing module 603 is configured to call the first operator node and the second operator node to execute the logic instruction through the preset hybrid-coded operator node according to the scheduling information, so as to obtain a target task result of the task stream.
Based on this, the following describes the data processing device 60 provided in the embodiment of the present application in detail:
in one or more possible embodiments, the processing module 603 may be specifically configured to, according to the scheduling information, invoke the first operator node to execute the logic instruction, so as to obtain a first task result;
Converting the first task result through a preset mixed-encoding sub-node to obtain a second task result;
and calling a second operator node to execute the logic instruction based on the second task result to obtain a target task result.
In another or multiple possible embodiments, the processing module 603 may be specifically configured to transform the first task result through the preset hybrid-encoded sub-node according to the node attribute of the second sub-node, so as to obtain the second task result.
In still another or more possible embodiments, the processing module 603 may be specifically configured to, when the node attribute includes an operator application mirror package, and the operator application mirror package includes operator operation system information and editing environment information, convert the first task result through the preset co-encoding operator node according to the operator operation system information and the editing environment information of the second operator node, to obtain a second task result.
In still another or more possible embodiments, the processing module 603 may be specifically configured to, when the node attribute includes an operator input parameter attribute, transform the first task result through a preset hybrid-programmed operator node according to the input parameter attribute, and obtain a second task result.
In one or more possible embodiments, the processing module 603 may be specifically configured to, in a case that the node attribute further includes an operator output parameter attribute and a calculation parameter attribute, call, according to the calculation parameter attribute and based on the second task result, the second operator node to execute the logic instruction, so as to obtain a stage task result;
and converting the stage task result according to the operator output parameter attribute to obtain a target task result.
In one or more possible embodiments, the data processing apparatus 60 provided in the embodiment of the present application may further include a conversion module; wherein,
the determining module 602 may be further configured to determine a preset parameter conversion algorithm of each operator node according to an operator type of each operator node of the at least two operator nodes;
and the conversion module is used for converting the operator parameters in the operator nodes corresponding to the preset parameter conversion algorithm through a preset parameter conversion algorithm to obtain an open source data format of data interaction between at least two operator nodes.
In still another possible embodiment or multiple possible embodiments, the processing module 603 may be specifically configured to obtain the second task result by converting the first task result through the preset hybrid-encoded sub-node according to the open-source data format.
In still another or multiple possible embodiments, the data processing apparatus 60 provided in this embodiment may further include a building module, configured to obtain multiple operator nodes, and build, based on each of the multiple operator nodes, an open source programming interface of each operator node through a preset hybrid programming operator node.
In still another or multiple possible embodiments, the processing module 603 may be specifically configured to, under the condition that the first operator node corresponds to the first open source programming interface and the second operator node corresponds to the second open source programming interface, call, according to the scheduling information, the first open source programming interface to execute the logic instruction through the preset mixed programming operator node, and obtain a third task result;
converting the third task result through a preset hybrid coding sub-node to obtain a fourth task result;
and calling a second open-source programming interface to execute the logic instruction based on the fourth task result to obtain a target task result.
In still another or more possible embodiments, the determining module 602 may be specifically configured to construct, according to the task characteristics, a directed acyclic graph of the task stream through preset hybrid-encoding sub-nodes, where the directed acyclic graph includes a path and at least two spatial operators on the path;
And determining at least two spatial operators as at least two operator nodes, and generating scheduling information of the at least two operator nodes according to the path.
Therefore, through the task flow to be executed, various geographic information systems for executing the task flow are abstracted, the calculation function is at least two operator nodes, at least two operator nodes with different sources and different types are expanded and integrated on the basis, then, preset mixed operator nodes capable of realizing data transmission between the at least two operator nodes are defined, the data between the various operator nodes can be mutually transmitted, then, the scheduling information of the at least two operator nodes for executing the operation instruction and the instruction can be quickly and quickly transmitted according to the logic of the executed task flow, the various operator nodes are called through the preset mixed operator nodes to finish the task flow to be executed, and therefore, the compatible calculation of the various operator nodes is met, the arrangement, scheduling and execution of the cross-platform, cross-language and cross-type operator nodes are realized, and the method can effectively realize the arrangement, scheduling and execution of the operator nodes with different sources, cross-type and the like in a complex service scene, And the mixed arrangement of the nodes of different operator types provides geographic information service for users.
Based on the same inventive concept, the application provides an operator node generation device corresponding to the above-mentioned operator node generation method. This is explained in detail with reference to fig. 7.
Fig. 7 is a schematic diagram showing a structure of an operator node generation apparatus according to an embodiment.
As shown in fig. 7, the operator node generating apparatus 70 is applied to the operator editor 10 shown in fig. 1 or fig. 2, and the operator node generating apparatus 70 may specifically include:
a receiving module 701, configured to receive access requests of at least two operator nodes, where the operator types of the at least two operator nodes are different;
an obtaining module 702, configured to obtain a node characteristic of each operator node in at least two operator nodes;
a constructing module 703, configured to construct a target hybrid operator node according to the node characteristics of each operator node and according to a preset integration condition; wherein,
the target mixed operator node is used for converting task data and result data of the task flow according to the node characteristics of each target operator node in at least two target operator nodes executing the task flow in the task flow executing process, so that the task data and the result data are transmitted between the at least two target operator nodes, and the result data is data obtained after each target operator node processes the task data.
Therefore, the data output by the operator node is identified through the constructed target mixed editing operator node and is forwarded to other operator nodes, so that the data among various operator nodes can be mutually transmitted, the cross-platform access and calling of various operator nodes are realized, and the geographic information service can be effectively provided for users through the mixed editing of the nodes with different sources and different operator types in a complex service scene.
Fig. 8 is a diagram showing a hardware configuration of a computer apparatus according to an embodiment.
As shown in fig. 8, computer device 800 includes input device 801, input interface 802, processor 803, memory 804, output interface 805, and output device 806.
The input interface 802, the processor 803, the memory 804, and the output interface 805 are connected to each other via a bus 810, and the input device 801 and the output device 806 are connected to the bus 810 via the input interface 802 and the output interface 805, respectively, and further connected to other components of the computer apparatus 800. Specifically, the input device 801 receives input information from the outside and transmits the input information to the processor 803 through the input interface 802; the processor 803 processes input information based on computer-executable instructions stored in the memory 804 to generate output information, stores the output information in the memory 804 temporarily or permanently, and then transmits the output information to the output device 806 via the output interface 805; output device 806 outputs output information to the exterior of computer device 800 for use by a user.
In one embodiment, the computer device 800 shown in fig. 8 may be implemented as a data processing device that may include: a memory configured to store a program; a processor configured to execute the program stored in the memory to perform the data processing method described in the above embodiments. Alternatively, the computer device 800 shown in fig. 8 may be implemented as an operator node generation device, which may include: a memory configured to store a program; a processor configured to execute the program stored in the memory to perform the operator node generation method described in the above embodiments.
In one embodiment, the memory may be further configured to store a target task result of the task stream and a calculation result of each step in the data processing and operator node generation process described in conjunction with fig. 1 to 5 above. As an example, the calculation result includes at least: executing a logical operation instruction on the task stream, and executing scheduling information of at least two operator nodes of the logical operation instruction.
According to an embodiment of the present application, the process described above with reference to the flowchart may be implemented as a computer-readable storage medium. For example, embodiments of the present application include a computer-readable storage medium comprising a program or instructions stored thereon, which, if executed by a computer device, cause the computer device to perform the steps of the above-described method.
According to an embodiment of the application, the process described above with reference to the flow chart may be implemented as a computer software program. For example, embodiments of the present application include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network, and/or installed from a removable storage medium.
In the above embodiments, all or part of the implementation may be realized by software, hardware, firmware, or any combination thereof. When implemented in software, it may be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions which, when run on a computer, cause the computer to perform the methods described in the various embodiments above. The procedures or functions according to the embodiments of the present application are all or partially generated when the computer program instructions are loaded and executed on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wirelessly (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk), among others.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and these modifications or substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.
Claims (12)
1. A method of data processing, comprising:
acquiring a task flow to be executed, wherein the task flow comprises task characteristics;
Determining scheduling information of at least two operator nodes for executing a logic operation instruction of the task flow and executing the logic operation instruction according to the task characteristics, wherein the at least two operator nodes comprise a first operator node and a second operator node, and the operator types of the at least two operator nodes are different;
and calling the first operator node and the second operator node to execute the logic instruction through a preset mixed coding operator node according to the scheduling information to obtain a target task result of the task flow.
2. The method of claim 1, wherein the obtaining the target task result of the task stream by calling the first operator node and the second operator node through a preset hybrid-coded operator node to execute the logic instruction according to the scheduling information comprises:
calling the first operator node to execute the logic instruction according to the scheduling information to obtain a first task result;
converting the first task result through the preset mixed editing sub-node to obtain a second task result;
and calling the second operator node to execute the logic instruction based on the second task result to obtain the target task result.
3. The method as claimed in claim 2, wherein said converting said first task result by said predetermined hash operator node to obtain a second task result comprises:
and converting the first task result through the preset mixed-encoding sub-nodes according to the node attribute of the second sub-node to obtain a second task result.
4. The method of claim 3, wherein the node attributes comprise an operator application mirror package comprising operator run system information and editing environment information;
the converting the first task result through the preset hybrid-coded sub-node according to the node attribute of the second sub-node to obtain a second task result, including:
and converting the first task result through the preset hybrid-coded operator node according to the operator operation system information and the editing environment information of the second operator node to obtain a second task result.
5. The method according to claim 2, wherein before the step of calling the first operator node and the second operator node through a preset hybrid programming operator node to execute the logic instruction according to the scheduling information to obtain a target task result of the task, the method further comprises:
Determining a preset parameter conversion algorithm of each operator node according to the operator type of each operator node in the at least two operator nodes;
converting operator parameters in operator nodes corresponding to the preset parameter conversion algorithm through the preset parameter conversion algorithm to obtain an open source data format of data interaction between the at least two operator nodes;
and converting the first task result through the preset mixed-encoding sub-node according to the open source data format to obtain a second task result.
6. The method according to claim 1, wherein before the step of calling the first operator node and the second operator node through a preset blending operator node to execute the logic instruction according to the scheduling information to obtain a target task result of the task flow, the method further comprises:
and acquiring a plurality of operator nodes, and constructing an open source programming interface of each operator node through the preset mixed-programming operator node based on each operator node in the operator nodes.
7. The method of claim 1, wherein said determining scheduling information for at least two operator nodes executing a logical operation instruction on the task stream and executing the logical operation instruction according to the task characteristics comprises:
According to the task characteristics, a directed acyclic graph of the task flow is constructed through the preset mixed-encoding operator nodes, and the directed acyclic graph comprises a path and at least two spatial operators on the path;
and determining the at least two spatial operators as the at least two operator nodes, and generating scheduling information of the at least two operator nodes according to the path.
8. An operator node generation method, comprising:
receiving access requests of at least two operator nodes, wherein the operator types of the at least two operator nodes are different;
acquiring node characteristics of each operator node in the at least two operator nodes;
constructing target mixed-woven operator nodes according to the node characteristics of each operator node and preset integration conditions; wherein,
the target mixed-encoding operator node is used for converting task data and result data of the task flow according to the node characteristics of each target operator node of at least two target operator nodes executing the task flow in the task flow executing process, so that the task data and the result data are transmitted between the at least two target operator nodes, and the result data is obtained after each target operator node processes the task data.
9. A data processing apparatus comprising:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a task flow to be executed, and the task flow comprises task characteristics;
a determining module, configured to determine, according to the task characteristics, scheduling information of at least two operator nodes that execute a logical operation instruction on the task stream and execute the logical operation instruction, where the at least two operator nodes include a first operator node and a second operator node, and the operator types of the at least two operator nodes are different;
and the processing module is used for calling the first operator node and the second operator node to execute the logic instruction through a preset mixed-encoding operator node according to the scheduling information to obtain a target task result of the task flow.
10. A computer device, comprising: a memory and a processor, wherein the processor is configured to,
the memory is used for storing a computer program;
the processor for executing a computer program stored in the memory, the computer program when executed causing the processor to perform the steps of the data processing method of any one of claims 1 to 7 or the steps of the operator node generation method of claim 8.
11. A computer-readable storage medium, on which a program or instructions are stored, which, if executed by a computer device, cause the computer device to carry out the steps of the data processing method according to any one of claims 1 to 7, or to carry out the steps of the operator node generation method according to claim 8.
12. A computer program product comprising a computer program which, if executed by a computer device, causes the computer device to carry out the steps of the data processing method of any one of claims 1 to 7 or the steps of the operator node generation method of claim 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210124413.5A CN114564547A (en) | 2022-02-10 | 2022-02-10 | Data processing method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210124413.5A CN114564547A (en) | 2022-02-10 | 2022-02-10 | Data processing method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114564547A true CN114564547A (en) | 2022-05-31 |
Family
ID=81714206
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210124413.5A Pending CN114564547A (en) | 2022-02-10 | 2022-02-10 | Data processing method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114564547A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114942792A (en) * | 2022-06-02 | 2022-08-26 | 广联达科技股份有限公司 | Equipment access method and device and electronic equipment |
| CN115145965A (en) * | 2022-09-01 | 2022-10-04 | 浙江大华技术股份有限公司 | Data stream generation method, electronic device and computer-readable storage medium |
| CN115202765A (en) * | 2022-06-14 | 2022-10-18 | 浙江智臾科技有限公司 | Stream data processing method and stream data processing system |
| CN117033027A (en) * | 2023-08-18 | 2023-11-10 | 百度(中国)有限公司 | Data processing method, device, electronic equipment and medium |
| CN117472553A (en) * | 2023-12-28 | 2024-01-30 | 中移(苏州)软件技术有限公司 | Workflow processing method, device, processing equipment and readable storage medium |
| WO2024087702A1 (en) * | 2022-10-27 | 2024-05-02 | 华为技术有限公司 | Operator compiling method and apparatus |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112685154A (en) * | 2020-12-25 | 2021-04-20 | 北京有竹居网络技术有限公司 | Data processing method of task flow engine, device and medium |
| CN113449841A (en) * | 2020-03-27 | 2021-09-28 | 华为技术有限公司 | Method and device for inserting conversion operator |
| CN113656369A (en) * | 2021-08-13 | 2021-11-16 | 辽宁华盾安全技术有限责任公司 | Log distributed streaming acquisition and calculation method in big data scene |
-
2022
- 2022-02-10 CN CN202210124413.5A patent/CN114564547A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113449841A (en) * | 2020-03-27 | 2021-09-28 | 华为技术有限公司 | Method and device for inserting conversion operator |
| CN112685154A (en) * | 2020-12-25 | 2021-04-20 | 北京有竹居网络技术有限公司 | Data processing method of task flow engine, device and medium |
| CN113656369A (en) * | 2021-08-13 | 2021-11-16 | 辽宁华盾安全技术有限责任公司 | Log distributed streaming acquisition and calculation method in big data scene |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114942792A (en) * | 2022-06-02 | 2022-08-26 | 广联达科技股份有限公司 | Equipment access method and device and electronic equipment |
| CN115202765A (en) * | 2022-06-14 | 2022-10-18 | 浙江智臾科技有限公司 | Stream data processing method and stream data processing system |
| CN115202765B (en) * | 2022-06-14 | 2023-03-28 | 浙江智臾科技有限公司 | Stream data processing method and stream data processing system |
| CN115145965A (en) * | 2022-09-01 | 2022-10-04 | 浙江大华技术股份有限公司 | Data stream generation method, electronic device and computer-readable storage medium |
| WO2024087702A1 (en) * | 2022-10-27 | 2024-05-02 | 华为技术有限公司 | Operator compiling method and apparatus |
| CN117033027A (en) * | 2023-08-18 | 2023-11-10 | 百度(中国)有限公司 | Data processing method, device, electronic equipment and medium |
| CN117472553A (en) * | 2023-12-28 | 2024-01-30 | 中移(苏州)软件技术有限公司 | Workflow processing method, device, processing equipment and readable storage medium |
| CN117472553B (en) * | 2023-12-28 | 2024-05-03 | 中移(苏州)软件技术有限公司 | Workflow processing method, device, processing equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114564547A (en) | Data processing method, device, equipment and storage medium | |
| Kovacs et al. | Standards-based worldwide semantic interoperability for IoT | |
| CN108696381B (en) | Protocol configuration method and device | |
| CN109213519B (en) | Heterogeneous API adaptation processing method and system | |
| US10635408B2 (en) | Method and apparatus for enabling agile development of services in cloud computing and traditional environments | |
| JP7003867B2 (en) | API adapter creation device, API adapter creation method and API adapter creation program | |
| MX2007015887A (en) | Data centric workflows. | |
| US20070033582A1 (en) | Transforming a Flow Graph Model to a Structured Flow Language Model | |
| CN110928548B (en) | Data processing method and device | |
| CN112906206B (en) | Digital twin model construction method and device | |
| CN112698921A (en) | Logic code operation method and device, computer equipment and storage medium | |
| CN112948110B (en) | Topology and arrangement system and method of cloud application, storage medium and electronic equipment | |
| KR20220031522A (en) | Process to combine semantic descriptions of digital twins with knowledge graphs | |
| CN111142849B (en) | Flow processing method, device, storage medium and processor | |
| El Kaed et al. | A semantic based multi-platform IoT integration approach from sensors to chatbots | |
| US20100251207A1 (en) | Framework for variation oriented analysis for service-oriented architecture | |
| CN113822557A (en) | Data fusion management system, data fusion management device, electronic device, and medium | |
| CN111143408B (en) | Event processing method and device based on business rule | |
| JP2016004359A (en) | Opc ua server creation method | |
| CN115809292A (en) | Data processing method, device, equipment and medium | |
| CN111045928A (en) | Interface data testing method, device, terminal and storage medium | |
| WO2022220982A1 (en) | Database query execution on multiple databases | |
| US11216294B2 (en) | User interface by way of unknown data structure | |
| CN111126883B (en) | Block chain-based research and development project information management method and device and electronic equipment | |
| CN115357240B (en) | Micro-service model construction method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |