CN115309562A - Operator calling system, operator generating method and electronic equipment - Google Patents
Operator calling system, operator generating method and electronic equipment Download PDFInfo
- Publication number
- CN115309562A CN115309562A CN202110497298.1A CN202110497298A CN115309562A CN 115309562 A CN115309562 A CN 115309562A CN 202110497298 A CN202110497298 A CN 202110497298A CN 115309562 A CN115309562 A CN 115309562A
- Authority
- CN
- China
- Prior art keywords
- operator
- calling
- software package
- function
- configuration information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/547—Remote procedure calls [RPC]; Web services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/544—Remote
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/547—Messaging middleware
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/548—Queue
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Stored Programmes (AREA)
Abstract
The application discloses an operator calling system, belongs to the technical field of computers, and is used for solving the problem of platform compatibility of operators. The operator calling system disclosed in the embodiment of the present application obtains configuration information of an operator through a configuration module, and generates an operator execution engine according to the configuration information of the selected operator, where the configuration information includes: operator attribute information, dependence functions and parameters of operators, and function realization of the dependence functions; then, generating a first operator calling software package according to a preset remote calling interface and the operator execution engine through a first operator deployment module, and sending the first operator calling software package to the operator server; and after the operator server stores the first operator calling software package, responding to the calling of the preset remote calling interface, executing the operator execution engine and outputting a calculation result, so that the platform compatibility of operator calling is improved, and the development and maintenance cost of operators is reduced.
Description
Technical Field
The embodiment of the application relates to the technical field of computers, in particular to an operator calling system, an operator generating method, electronic equipment and a computer readable storage medium.
Background
Under the scenes of various system platforms such as a decision system, machine learning, model prediction, risk scheduling operation and the like, a large amount of characteristic index calculation exists. Usually, the logic and the caliber of the characteristic index calculation are expressed in different forms by different roles such as a service RD or an algorithm RD. However, the use of the feature indicators may be distributed under the online, near-line and offline full-scene links, and the computing apertures of the shared feature indicators must ensure consistency, and the computing logic of the feature indicators needs to support sharing or multiplexing under each system platform.
In the prior art, for the situation that feature indexes expressed in different forms are shared or multiplexed in different platforms or in different use modes, it is often necessary to convert calculation operators of the feature indexes for the different platforms, and then copy the converted operators to the corresponding platforms for use.
Therefore, in the prior art, the operator calling system needs to perform secondary development and independent maintenance on different platforms, the platform compatibility is poor, and the development and maintenance costs are high.
Disclosure of Invention
The embodiment of the application provides an operator calling system, which is beneficial to improving platform compatibility of operator calling and reducing operator development and maintenance cost.
In a first aspect, an embodiment of the present application provides an operator invoking system, including:
the configuration module is configured to obtain configuration information of an operator, and generate an operator execution engine according to the configuration information of the selected operator, where the configuration information includes: operator attribute information, dependence functions and parameters of operators, and function realization of the dependence functions;
the first operator deployment module is used for generating a first operator calling software package according to a preset remote calling interface and the operator execution engine and sending the first operator calling software package to the operator server;
and the operator server is used for storing the first operator calling software package and responding to the calling of the preset remote calling interface to execute the operator execution engine.
In a second aspect, an embodiment of the present application provides an operator generation method, including:
obtaining configuration information of an operator, wherein the configuration information comprises: operator attribute information, dependence functions and parameters of operators, and function realization of the dependence functions;
responding to the generation operation of the first operator calling software package, generating an operator code according to the configuration information of an operator, and generating an operator execution engine according to the operator code, so that after the first operator calling software package which encapsulates a preset remote calling interface and the operator execution engine is stored in an operator service end, the operator service end responds to the calling of the preset remote calling interface to execute the operator execution engine.
In a third aspect, an embodiment of the present application further discloses an electronic device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the operator generation method according to the embodiment of the present application when executing the computer program.
In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor, and the program includes the steps of the operator generation method disclosed in the present application.
The operator calling system disclosed in the embodiment of the present application obtains configuration information of an operator through a configuration module, and generates an operator execution engine according to the configuration information of the selected operator, where the configuration information includes: operator attribute information, dependence functions and parameters of operators, and function realization of the dependence functions; then, generating a first operator calling software package according to a preset remote calling interface and the operator execution engine through a first operator deployment module, and sending the first operator calling software package to the operator server; and after the operator server stores the first operator calling software package, responding to the calling of the preset remote calling interface, executing the operator execution engine and outputting a calculation result, so that the platform compatibility of operator calling is improved, and the development and maintenance cost of operators is reduced.
The foregoing description is only an overview of the technical solutions of the present application, and the present application can be implemented according to the content of the description in order to make the technical means of the present application more clearly understood, and the following detailed description of the present application is given in order to make the above and other objects, features, and advantages of the present application more clearly understandable.
Drawings
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
FIG. 1 is a diagram of a structure of an operator calling system according to a first embodiment of the present application;
FIG. 2 is a schematic diagram of an application scenario for operator invocation in the first embodiment of the present application;
FIG. 3 is a second schematic diagram of an operator invocation system according to a first embodiment of the present application;
FIG. 4 is a schematic flow chart of an operator generation method according to a second embodiment of the present application;
FIG. 5 is a second flowchart illustrating a method for generating an operator according to a second embodiment of the present invention;
FIG. 6 schematically shows a block diagram of an electronic device for performing a method according to the present application; and
fig. 7 schematically shows a storage unit for holding or carrying program code implementing a method according to the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Example one
As shown in fig. 1, an operator invoking system disclosed in an embodiment of the present application includes: a configuration module 110, a first operator deployment module 120, and an operator server 130; wherein the content of the first and second substances,
the configuration module 110 is configured to obtain configuration information of an operator, and generate an operator execution engine according to the configuration information of the selected operator, where the configuration information includes: operator attribute information, dependence functions and parameters of operators, and function realization of the dependence functions;
the first operator deployment module 120 is configured to generate a first operator calling software package according to a preset remote calling interface and the operator execution engine, and send the first operator calling software package to the operator server 130;
the operator service end 130 is configured to store the first operator call software package, and execute the operator execution engine in response to the call to the preset remote call interface.
The operator is obtained by packaging calculation logic of preset data characteristics and/or indexes.
The operator described in the embodiments of the present application refers to mapping between a function space and a function space. Each operator can only be realized by one function, and one function can be used as a dependent function of a plurality of operators, so that the online efficiency of the operators can be improved through the multiplexing of the functions. The operators are implemented by functions based on specified parameters. The functions described in the embodiments of the present application include, but are not limited to, one or more of the following: aggregation functions, scalar functions, table functions. The parameters described in the examples of the present application include, but are not limited to, one or more of the following: constants, variables, operators.
The following describes, by way of example, a specific implementation of each component in the operator calling system disclosed in the present application with reference to an operator calling scenario diagram shown in fig. 2.
The operator calling system in the embodiment of the application can be applied to calling modes such as online, near-line or offline and can be applied to any one of the following scenes: decision-making system, machine learning, model prediction, task scheduling, monitoring and early warning and the like. In some application scenarios, the calculation logics of the index set and the feature set which need to be calculated are packaged into operators. For example, for the calculation logic for calculating the similarity of two groups of features, the calculation logic is packaged into a similarity calculation operator; then, mapping the concrete realization of each operator into a bottom function realization through a configuration module, abstracting and packaging a uniform bottom function interface; and then, defining a Python service interface and packaging the Python service interface into a Python component, or packaging a large data platform interface through an adapter mode and then embedding a dynamic script engine (such as Groovy, aviator and the like) into the Python service interface, so that an operator can be executed on a plurality of platforms, one-time development and multi-scene sharing of the operator are finally realized, and the problems of cross-language conversion and inconsistent caliber of index calculation logic are solved.
The configuration module 110 in this embodiment of the present application includes a configuration front end and a background configuration processing center, where the configuration front end acquires configuration information of an operator and a function by displaying an operator configuration interface or a function configuration interface, and sends the configuration information to the background configuration processing center in response to a confirmation operation of a user on the configuration information, and the background configuration processing center caches the configuration information and generates an operator or a function according to the configuration information.
In some embodiments of the application, the background configuration processing center generates an implementation script of an operator according to the dependency relationship between the operator and a function in the configuration information.
In some embodiments of the present application, the configuration information of the operator includes, but is not limited to, any one or more of the following: operator names, labels, description information, dependence functions and other information; the configuration information of the function includes, but is not limited to, any one or more of the following: function name, function type, description information, state, parameter-entering information (such as parameter-entering name, type, necessary or not, parameter description, etc.), and parameter-exiting information (such as parameter-entering name, type, necessary or not, parameter description, etc.). Because the operator calculation bottom layer is realized by depending on function classes, the operator development is easy to expand, namely the function development is easy to expand, when an operator needs to be added, the operator development can be realized only by multiplexing the existing or added function realization classes and then configuring the dependency relationship between the operator and the corresponding function.
In some embodiments of the application, the template mode is used for function development, so that the development of function implementation classes is facilitated, developers can focus on the development of function functions, the perception of a computation framework layer is reduced, the complexity of function development is reduced, and the development efficiency is improved. For example, by setting basic function class templates of different types of functions, and then by setting interface calls of function implementation codes in the function implementation classes, the functions are enabled to implement specified functions.
The development of the functions may be based on a script engine and different programming languages. The Aviator engine and Python language can support non-developers (such as algorithm personnel, service personnel and the like) to develop corresponding functions, so that cross-language code conversion is avoided, cross-team communication cost is saved, code error rate is reduced, and operator online efficiency and accuracy are improved. The functions comprise two types, one is a local realization function, and the other is a remote realization function. In some embodiments of the present application, the functions include functions developed based on a scripting language; wherein the scripting language includes: the Python language is developed based on the Python language, the function is deployed at a remote Python server side, and the operator execution engine comprises a mapping relation between a preconfigured operator and the function deployed at the remote Python server side. Taking a function realized by a Python language as an example, the Python function only supports a basic Python code, and if a three-way library such as machine learning, scientific calculation and the like needs to be used, the Python code needs to be deployed at a remote Python server, and then the mapping relation between an operator and the Python function is configured. When the function of the operator is generated to be realized, the realization code of the operator is generated according to the mapping relation between the operator and the Python function.
After the user completes the configuration of the operator and the function through the configuration interface of the configuration front end, the configuration front end can be triggered to send the configuration information of the operator to the background configuration processing center through a storage or refresh button set on the configuration interface of the front end, and the first operator deployment module 120 of the background configuration processing center generates the implementation code of the operator according to the received operator configuration information. The user can select operators through configuring the front-end interface, and the background configuration processing center generates an operator execution engine according to the selected operators.
As can be seen from the above description, the operator implementation code and the function implementation code generated in the above manner do not define an adaptation platform, and the implementation code of the operator may not cover all platforms, that is, the operator execution engine generated in the above step cannot run on all target platforms. Next, the problem of operator cross-platform needs to be solved.
In some embodiments of the present application, the operator calling system supports both a remote mode and a local mode. For the Remote calling mode, operators run at an operator server, and other platforms access interface addresses externally provided by the operator server through an HTTP (Hypertext Transfer Protocol) or RPC (Remote Procedure Call) mode, so that the operators can be remotely accessed by a client conveniently. In the remote mode, because the operator execution engine runs at an operator server, operator calls of various platforms are supported.
In some embodiments of the present application, in order to adapt to remote invocation, when a first operator invocation software package running on an operator server is generated according to an operator execution engine, a preset remote invocation interface and the operator execution engine need to be packaged together into the first operator invocation software package. For example, remote calling of operators in an operator execution engine based on different calling forms is realized by setting Restful Api to perform calling interface conversion on remote operator calls based on HTTP service and setting Thrift Api to perform calling interface conversion on remote operator calls based on RPC service.
After generating the first operator calling software package, the first operator deployment module 120 sends the first operator calling software package to the operator server 130 for storage. Then, the operator server 130 may respond to remote invocation of an operator in the first operator invocation software package by other platforms or applications, and execute the invoked operator at the operator server. For example, when the application client accesses an HTTP service interface or an RPC service interface of the operator server, after receiving a remote call, a preset remote call interface in the first operator call software package converts the remote call into a call to a corresponding operator in the operator execution engine.
In some embodiments of the present application, as shown in fig. 3, the operator invoking system further includes: a second operator deployment module 140.
In some embodiments of the present application, the configuration module 110 is further configured to obtain an open interface and an operator adaptation platform of the operator execution engine. For example, the operator execution engine can be exported in the form of SDK (software development kit) by the user through the front-end interface operation of the configuration module 110, and then downloaded to the client of the different platform to run locally. In such application scenarios, the operator execution engine needs to run on different platforms, and therefore, the platform adaptation problem needs to be solved. In some embodiments of the present application, the second operator deployment module 140 is configured to generate, according to the open interface, an adaptation interface matched with an operator adaptation platform, and the operator execution engine, a second operator calling software package, where the second operator calling software package is configured to be loaded by an application client, so that the application client executes the operator execution engine in response to a call to the open interface. For example, when the application client calls a preset open interface, after the corresponding open interface in the second operator calling software package receives the local call of the application client, the corresponding platform adaptation interface in the software package is called by the second operator, and the corresponding operator in the operator execution engine is called.
Firstly, after detecting an operation of generating an SDK by a user, the second operator deployment module 140 further obtains configuration information and open interface information of the SDK adaptation platform to be generated through a configuration module, and then, the second operator deployment module 140 obtains a platform adapter according to the configuration information of the SDK adaptation platform, and packages an implementation code of an open interface, the platform adapter, and the operator execution engine into a JAR package (JAVA archive file) as a second operator calling software package. The open interface information is a general interface for calling an operator engine by a client; the platform adapter is used for carrying out platform adaptation on the calling of the operator by the corresponding platform, so that the calling of the operator can be executed locally at the client. And for second operator calling software packages which are adaptive to different platforms, the application scenes of the second operator calling software packages to operators are different, and platform adapters in the second operator calling software packages are different from each other.
Taking a near-end operator calling scene as an example, such as index analysis, feature calculation and the like, the operator can be directly called through a client interface to carry out index or feature calculation; in an operator calling scene suitable for a flink sql big data platform, a flink (a data stream processing framework) existing adapter automatically packages a flink udf (flink self-defined scalar function), and operators can be directly used in sql calling, for example, the hash in a 'select hash (id) from table' is an operator name; in an operator application scene applicable to hive sql (a data platform tool), the hive (a database architecture) has an automatic adapter encapsulation (hive custom scalar function), and in sql call, operators can be directly used, for example, the hash in 'select hash (id) from table' is an operator name.
According to the operator generation and deployment scheme, different modes (such as HTTP, RPC and SDK) of operator calling share the same set of operator execution engine, and no matter a client side calls are real-time online systems or offline systems such as online systems and offline systems, the same operator calling implementation logic is the same, and ambiguity of operator implementation and difference of cross-platform do not exist.
In some embodiments of the present application, as shown in fig. 3, the operator invoking system further includes: an operator trial computation module 150 and an operator refresh module 160, wherein,
the operator trial computation module 150 is configured to pre-compile an operator and/or a function configured by the configuration module, and execute a pre-compiled operator script and/or function script;
the operator refreshing module 160 is configured to update the configuration information of the operator and/or the function to be refreshed to the operator server.
In some embodiments of the present application, the trial calculation of the selected operator or function may be initiated by setting a trigger button on the front-end configuration interface. Taking the trial calculation of an operator as an example, at this time, the operator trial calculation module 150 may obtain configuration information and dependent function information of the selected operator, determine an implementation code of the operator according to the implementation script of the function, and then call a local script engine to compile and execute the implementation code of the operator to test an execution result of the operator. The operator trial computation module 150 in the embodiment of the present application is disposed in the background configuration processing center.
Those skilled in the art will appreciate that dynamic scripting functions include: the Groovy, aviator and Python scripts can all realize real-time updating and deployment to be effective, so that the real-time hot deployment of operators can be realized in the operator calling system disclosed by the embodiment of the application.
In some embodiments of the present application, hot deployment of selected or functional operators may be initiated by setting a trigger button on the front-end configuration interface. Taking operator updating as an example, at this time, the operator refreshing module 160 updates the configuration information of the operator to be refreshed, which is selected through the front-end configuration interface, to the operator server 130. And the operator server 130 updates the implementation code of the corresponding operator according to the received configuration information. The operator update module 150 in this embodiment of the present application may include two parts, namely a front-end interface and a background update application, where the background update application may be disposed at the operator server 130.
In some embodiments of the present application, the operator server 130 is further configured to push configuration information of an operator and/or a function to be refreshed to an application server through a network address registered in advance by the application server that loads a second operator calling software package, so that the application server updates the second operator calling software package through a configuration update module preset in the loaded second operator calling software package; or, the operator server is further configured to output full-update configuration information or output configuration information of the operator and/or the function to be refreshed, so that the second operator can call a configuration update module in the software package to pull.
For the second operator locally downloaded by the client, the operator in the software package is called, the operator can be updated in a push or pull mode, and in the operator updating process, the full operator updating or partial operator updating can be performed. The following respectively introduces the technical schemes of updating the operator by adopting a push mode and updating the operator by adopting a pull mode.
In the scheme of updating the second operator calling software package in a push mode, the second operator calling software package encapsulates a configuration updating module. And the configuration updating module registers information such as network addresses and the like in the operator server through a callback function when the client loads the second operator calling software package, and the operator server stores the monitoring address of the client according to the registration information of the client. And after the operator server receives the configuration information updated by the operator, pushing the operator update information to the client through the registered monitoring address. And then, the second operator loaded by the client calls the configuration updating module in the software package to cache the operator updating information pushed by the operator server, and updates the corresponding operator locally loaded by the client according to a preset strategy.
In the scheme of updating the second operator calling software package in a pulling mode, the second operator calling software package encapsulates a configuration updating module. The configuration updating module actively downloads operator updating information from the operator server according to a preset strategy (such as Monday or every second operator calls a software package to be loaded), caches the downloaded operator updating information, and then updates the corresponding operator locally loaded at the client according to the preset strategy.
In some embodiments of the present application, the configuration updating module may update the second operator calling software package by using an incremental update or a full update. When the configuration updating module updates the second operator calling software package in an incremental updating mode, the configuration updating module detects the operator updating information of the operator server at regular time, and performs incremental updating on the second operator calling software package loaded locally at the client according to the downloaded operator updating information. And the operator server side sets the operator updating database to store the operator increment updating information.
For example, in some embodiments of the present application, the operator server is further provided with an operator incremental update monitoring module, configured to monitor operator update information received by the operator server. In some embodiments of the present application, a middleware manner may be adopted to monitor operator information change of an operator server. The operator server side can monitor the change of a first operator calling software package which is locally stored by setting an operator configuration change monitoring module, or monitor the configuration information of an operator to be refreshed which is sent by the operator refreshing module, generate incremental configuration information after monitoring the information, and synchronize the generated incremental configuration information to a preset middleware through a message system. The preset middleware continuously updates the messages sent by the monitoring module through the operator increment through a producer mode and a consumer mode of the message queue, and stores the messages into a preset database through the consumer mode for being pulled by the configuration updating module of the client.
The operator calling system disclosed in the embodiment of the present application obtains configuration information of an operator through a configuration module, and generates an operator execution engine according to the configuration information of the selected operator, where the configuration information includes: operator attribute information, dependence functions and parameters of operators, and function realization of the dependence functions; then, generating a first operator calling software package according to a preset remote calling interface and the operator execution engine through a first operator deployment module, and sending the first operator calling software package to the operator server; and after the operator server stores the first operator calling software package, responding to the calling of the preset remote calling interface, executing the operator execution engine and outputting a calculation result, so that the platform compatibility of operator calling is improved, and the development and maintenance cost of operators is reduced.
According to the operator calling system disclosed in the embodiment of the application, the calculation rules of the characteristics or indexes in the application scene are mapped into the operators, the dependency relationship between the operators and the functions is configured through the front-end configuration interface, the dependency functions of the operators are realized through the script language, and the operators obtained by executing one-time operator development (such as operator configuration and function development) can be shared in multiple scenes through the embedded dynamic script engine, so that the problem of inconsistent index calibers is solved, and the research, development, operation and maintenance efficiency is improved.
On the other hand, a configuration module can configure that a plurality of operators depend on a bottom layer function and are distinguished through different parameters; meanwhile, functions are packaged according to scalar quantity, aggregation and table abstraction, so that the functions can be easily expanded and used in different scenes, and the research and development efficiency is improved.
Because the function of the operator supports development language implementation interfaces such as Groovy, aviator, python and the like, the operator server can update the operator and the function in real time and take effect, and the operation and maintenance efficiency of the operator calling system is improved.
Example two
An operator generation method disclosed in the embodiment of the present application is applied to an operator calling system described in the first embodiment, and as shown in fig. 4, the method includes: step 410 and step 420.
The operator described in the embodiments of the present application is obtained by encapsulating the computation logic of the preset data features and/or indicators. The operator is realized based on a formulated function, and one function can be used for realizing a plurality of operators by configuring different parameters. The dependency function of the operator is a low-level functional implementation of the operator, i.e. the dependency function of the operator is an implementation of low-level computational logic.
In some embodiments of the present application, the function includes at least one or more of: aggregation functions, table functions, scalar functions; the function is developed based on a function template. The parameters described in the examples of the present application include, but are not limited to, one or more of the following: constants, variables, operators.
In some embodiments of the present application, the function comprises a function developed based on a scripting language and/or an executable language; wherein the scripting language includes: the Python language is developed based on the Python language, the function is deployed at a remote Python server side, and the operator execution engine comprises a mapping relation between a preconfigured operator and the function deployed at the remote Python server side.
In some embodiments of the present application, the attribute information of the operator, the function that the operator depends on, the function parameter that implements the operator, and other information may be configured through the front-end configuration interface. For a specific implementation of configuring the operator and the function, reference is made to the relevant description in the first embodiment, and details are not described in this embodiment again.
For a specific implementation of generating the first operator invoking software package, refer to the first embodiment, which is not described in detail in this embodiment.
The first operator calls the software package to be sent to the operator server side for storage, and the software package runs on the operator server side.
For the calling scheme of the first operator for calling the operator in the software package, reference is made to the relevant description in the first embodiment, which is not described in detail in this embodiment.
In some embodiments of the present application, as shown in fig. 5, after the step 410, the method further includes:
For a specific implementation of generating the second operator invocation software package, reference is made to the first embodiment, which is not described in detail in this embodiment.
In some embodiments of the present application, the application client may download the second operator calling software package to the local storage, and call the second operator to call the operator in the software package.
The calling scheme of the second operator for calling the operator in the software package refers to the relevant description in the first embodiment, and is not described in detail in this embodiment.
In some embodiments of the present application, after obtaining the configuration information of the operator, the method further includes:
responding to the triggering of an operator trial calculation operation, determining an operator realization script of an operator to be tried according to the configuration information, and pre-compiling the operator realization script;
and executing the precompiled operator realization script and outputting an execution result of the operator realization script.
The trial calculation scheme of the operator refers to the related description in the first embodiment, and is not described in detail in this embodiment.
In some embodiments of the present application, the method further comprises:
and updating the configuration information of the operator to be refreshed to the operator server in response to the operator refreshing operation being triggered.
The refresh scheme of the operator refers to the related description in the first embodiment, and is not described in detail in this embodiment.
In some embodiments of the present application, the method further comprises:
the operator server side pushes configuration information of an operator and/or a function to be refreshed to the application server through a network address registered in advance by the application server loading a second operator calling software package, so that the application server updates the second operator calling software package through a configuration updating module preset in the loaded second operator calling software package.
In some embodiments of the present application, the method further comprises:
and the operator server outputs full update configuration information or configuration information of the operator and/or the function to be refreshed, so that the second operator can call a configuration update module in the software package to pull conveniently.
For a technical solution for performing a full update or a partial update on a second operator calling software package at an application client, refer to the related description in the first embodiment, which is not described in detail in this embodiment.
The operator generation method disclosed in the embodiment of the present application is applied to the operator calling system described in the first embodiment of the present application, and specific implementation manners of each step of the method refer to specific implementation manners of corresponding modules, which are not described in detail in this embodiment.
The operator generation method disclosed in the embodiment of the present application obtains configuration information of an operator, where the configuration information includes: operator attribute information, dependence functions and parameters of operators, and function realization of the dependence functions; and then, responding to a first operator calling software package generation operation, generating an operator code according to the configuration information of the operator, and generating an operator execution engine according to the operator code, so that a first operator calling software package which encapsulates a preset remote calling interface and the operator execution engine is stored in an operator service end, and the operator service end responds to the calling of the preset remote calling interface, executes the operator execution engine, outputs a calculation result, improves the platform compatibility of operator calling, and reduces the operator development and maintenance cost.
According to the operator generation method disclosed in the embodiment of the application, the calculation rules of the characteristics or indexes in the application scene are mapped into the operators, the dependency relationship between the operators and the functions is configured through the front-end configuration interface, the dependency functions of the operators are realized through the scripting language, and the operators obtained by executing one-time operator development (such as operator configuration and function development) can be shared in multiple scenes through the embedded dynamic script engine, so that the problem of inconsistent index calibers is solved, and the research, development, operation and maintenance efficiency is improved.
On the other hand, in the operator generation method disclosed in the embodiment of the application, a plurality of operators depend on one bottom layer function and are distinguished through different parameters; meanwhile, functions are packaged according to scalar quantity, aggregation and table abstraction, so that the functions can be easily expanded and used in different scenes, and the research and development efficiency is improved.
Because the function of the operator supports development language implementation interfaces such as Groovy, aviator and Python, the operator server can update the operator and the function in real time and take effect, and the operation and maintenance efficiency of the operator calling system is improved.
The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.
The operator calling system and the operator generating method provided by the application are introduced in detail, a specific example is applied in the description to explain the principle and the implementation of the application, and the description of the embodiment is only used for helping to understand the method and a core idea of the application; meanwhile, for a person skilled in the art, according to the idea of the present application, the specific implementation manner and the application scope may be changed, and in summary, the content of the present specification should not be construed as a limitation to the present application.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
The various component embodiments of the present application may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functionality of some or all of the components in an electronic device according to embodiments of the present application. The present application may also be embodied as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present application may be stored on a computer readable medium or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.
For example, fig. 6 illustrates an electronic device that may implement a method in accordance with the present application. The electronic device can be a PC, a mobile terminal, a personal digital assistant, a tablet computer and the like. The electronic device conventionally comprises a processor 610 and a memory 620 and program code 630 stored on said memory 620 and executable on the processor 610, said processor 610 implementing the method described in the above embodiments when executing said program code 630. The memory 620 may be a computer program product or a computer readable medium. The memory 620 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read only memory), an EPROM, a hard disk, or a ROM. The memory 620 has a storage space 6201 for program code 630 of a computer program for performing any of the method steps described above. For example, the storage space 6201 for the program code 630 may include respective computer programs for implementing the various steps in the above method, respectively. The program code 630 is computer readable code. The computer programs may be read from or written to one or more computer program products. These computer program products comprise a program code carrier such as a hard disk, a Compact Disc (CD), a memory card or a floppy disk. The computer program comprises computer readable code which, when run on an electronic device, causes the electronic device to perform the method according to the above embodiments.
The embodiment of the present application further discloses a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the operator generation method according to the second embodiment of the present application.
Such a computer program product may be a computer-readable storage medium that may have memory segments, memory spaces, etc. arranged similarly to the memory 620 in the electronic device shown in fig. 6. The program code may be stored in a computer readable storage medium, for example, compressed in a suitable form. The computer readable storage medium is typically a portable or fixed storage unit as described with reference to fig. 7. Typically, the storage unit comprises computer readable code 630', said computer readable code 630' being code read by a processor, which when executed by the processor implements the steps of the method described above.
Reference herein to "one embodiment," "an embodiment," or "one or more embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Furthermore, it is noted that instances of the word "in one embodiment" are not necessarily all referring to the same embodiment.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The application may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.
Claims (11)
1. An operator invocation system, comprising: the system comprises a configuration module, a first operator deployment module and an operator server; wherein the content of the first and second substances,
the configuration module is used for acquiring configuration information of an operator and generating an operator execution engine according to the selected configuration information of the operator; wherein the configuration information comprises: operator attribute information, dependence functions and parameters of operators, and function realization of the dependence functions;
the first operator deployment module is used for generating a first operator calling software package according to a preset remote calling interface and the operator execution engine and sending the first operator calling software package to the operator server;
and the operator server is used for storing the first operator calling software package and responding to the calling of the preset remote calling interface to execute the operator execution engine.
2. The system of claim 1, further comprising: a second operator deployment module;
the configuration module is further used for acquiring an open interface and an operator adaptation platform of the operator execution engine;
the second operator deployment module is configured to generate a second operator calling software package according to the open interface, an adaptation interface matched with an operator adaptation platform, and the operator execution engine, where the second operator calling software package is used for being loaded by an application client, so that the application client executes the operator execution engine in response to a call to the open interface.
3. The system of claim 2, further comprising: an operator trial calculation module and an operator refreshing module,
the operator trial calculation module is used for pre-compiling the operator and/or the function configured by the configuration module and executing the pre-compiled operator script and/or function script;
and the operator refreshing module is used for updating the configuration information of the operator and/or the function to be refreshed to the operator server side.
4. The system of claim 3,
the operator server is further configured to push configuration information of an operator and/or a function to be refreshed to the application server through a network address registered in advance by the application server which loads the second operator calling software package, so that the application server updates the second operator calling software package through a configuration updating module preset in the loaded second operator calling software package; alternatively, the first and second electrodes may be,
and the operator server is also used for outputting full-scale updating configuration information or outputting configuration information of the operator and/or the function to be refreshed, so that the second operator can call a configuration updating module in the software package to pull conveniently.
5. The system of any one of claims 1 to 4, wherein the function includes at least one or more of: aggregation functions, table functions, scalar functions; the function is developed based on a function template.
6. The system of claim 5, wherein the function comprises a function developed based on a scripting language and/or an executable language; wherein the scripting language includes: the Python language is developed based on the Python language, the function is deployed at a remote Python server side, and the operator execution engine comprises a mapping relation between a preconfigured operator and the function deployed at the remote Python server side.
7. The system according to any of claims 1 to 4, wherein the operators are encapsulated in the computation logic of the pre-set data features and/or indices.
8. An operator generation method, comprising:
acquiring configuration information of an operator, wherein the configuration information comprises: operator attribute information, dependence functions and parameters of operators, and function realization of the dependence functions;
responding to the generation operation of the first operator calling software package, generating an operator code according to the configuration information of an operator, and generating an operator execution engine according to the operator code, so that after the first operator calling software package which encapsulates a preset remote calling interface and the operator execution engine is stored in an operator service end, the operator service end responds to the calling of the preset remote calling interface to execute the operator execution engine.
9. The method of claim 8, wherein the step of obtaining the configuration information of the operator is followed by:
responding to a second operator calling software package generation operation, generating an operator code according to the configuration information of an operator, generating an operator execution engine according to the operator code, packaging a pre-configured open interface and an adaptation interface matched with an operator adaptation platform, and responding to the calling of the open interface by an application client to execute the operator execution engine after the second operator calling software package generated by the operator execution engine is loaded by the application client.
10. An electronic device comprising a memory, a processor and program code stored on said memory and executable on said processor, wherein said processor implements the operator generation methods of claims 8 and 9 when executing said program code.
11. A computer-readable storage medium on which a program code is stored, characterized in that the program code, when being executed by a processor, realizes the steps of the operator generation method of claims 8 and 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110497298.1A CN115309562A (en) | 2021-05-07 | 2021-05-07 | Operator calling system, operator generating method and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110497298.1A CN115309562A (en) | 2021-05-07 | 2021-05-07 | Operator calling system, operator generating method and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115309562A true CN115309562A (en) | 2022-11-08 |
Family
ID=83854182
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110497298.1A Pending CN115309562A (en) | 2021-05-07 | 2021-05-07 | Operator calling system, operator generating method and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115309562A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116225669A (en) * | 2023-05-08 | 2023-06-06 | 之江实验室 | Task execution method and device, storage medium and electronic equipment |
| CN116932092A (en) * | 2023-09-18 | 2023-10-24 | 之江实验室 | Method, device, medium and equipment for automatically generating operator calling code |
| CN117118962A (en) * | 2023-07-25 | 2023-11-24 | 领悦数字信息技术有限公司 | Method and system for deploying user methods across multiple cloud platforms |
-
2021
- 2021-05-07 CN CN202110497298.1A patent/CN115309562A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116225669A (en) * | 2023-05-08 | 2023-06-06 | 之江实验室 | Task execution method and device, storage medium and electronic equipment |
| CN116225669B (en) * | 2023-05-08 | 2024-01-09 | 之江实验室 | Task execution method and device, storage medium and electronic equipment |
| CN117118962A (en) * | 2023-07-25 | 2023-11-24 | 领悦数字信息技术有限公司 | Method and system for deploying user methods across multiple cloud platforms |
| CN117118962B (en) * | 2023-07-25 | 2024-03-22 | 领悦数字信息技术有限公司 | Method and system for deploying user methods across multiple cloud platforms |
| CN116932092A (en) * | 2023-09-18 | 2023-10-24 | 之江实验室 | Method, device, medium and equipment for automatically generating operator calling code |
| CN116932092B (en) * | 2023-09-18 | 2024-01-09 | 之江实验室 | Method, device, medium and equipment for automatically generating operator calling code |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108037961B (en) | Application program configuration method, device, server and storage medium | |
| CN115309562A (en) | Operator calling system, operator generating method and electronic equipment | |
| US11042387B2 (en) | Deploying cross-platform applications on mobile devices with native and web components | |
| CN112256321A (en) | Static library packaging method and device, computer equipment and storage medium | |
| CN111221572B (en) | Method, device, medium and equipment for automatically adapting to running environment | |
| CN110555550B (en) | Online prediction service deployment method, device and equipment | |
| CN112306586A (en) | Data processing method, device, equipment and computer storage medium | |
| CN108965375B (en) | Service call proxy control system, method, server and readable storage medium | |
| US20140280431A1 (en) | System and method for interoperability between flex applications and .net applications | |
| CN112015519A (en) | Model online deployment method and device | |
| CN103595770A (en) | Method and device for achieving file downloading through SDK | |
| CN110851211A (en) | Method, apparatus, electronic device, and medium for displaying application information | |
| CN111651169A (en) | Block chain intelligent contract operation method and system based on web container | |
| CN111683005B (en) | Internet of things intelligent gateway equipment and construction method thereof | |
| WO2019014101A1 (en) | Model driven methods for ontology based application development | |
| US11307839B2 (en) | Updating of container-based applications | |
| CN108804088B (en) | Protocol processing method and device | |
| CN116166457A (en) | Data processing method and related equipment | |
| CN115344275A (en) | Method and device for generating image file of operating system and computer equipment | |
| CN115328526A (en) | Dependency package updating method, device, equipment and storage medium | |
| CN109062714A (en) | The method, apparatus and electronic equipment of long-range control Android device | |
| CN114237644A (en) | System, method and device for running mirror image and method and device for constructing mirror image | |
| CN117056317B (en) | Data processing method, device, equipment and computer readable storage medium | |
| CN117170738B (en) | Method, system, equipment and storage medium for interaction of Python and Fortran | |
| CN118057306A (en) | Method, device, server, medium and program product for generating installation package |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |