CN114741372A - Method for realizing in-library artificial intelligence and database system - Google Patents
Method for realizing in-library artificial intelligence and database system Download PDFInfo
- Publication number
- CN114741372A CN114741372A CN202210292699.8A CN202210292699A CN114741372A CN 114741372 A CN114741372 A CN 114741372A CN 202210292699 A CN202210292699 A CN 202210292699A CN 114741372 A CN114741372 A CN 114741372A
- Authority
- CN
- China
- Prior art keywords
- function
- target
- artificial intelligence
- user
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/211—Schema design and management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
- G06F16/2448—Query languages for particular applications; for extensibility, e.g. user defined types
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention discloses a method for realizing in-library artificial intelligence and a database system, wherein the method comprises the following steps: responding to the definition operation of a user on the self-defined function to generate a target self-defined function corresponding to the target artificial intelligence algorithm; responding to the deployment operation of a user on the target self-defining function, and deploying the target self-defining function at a specified position in the database system; when a structured query statement which is sent by a user and corresponds to a target custom function is received, calling the target custom function from a specified position to execute data processing related to a target artificial intelligence algorithm, and returning a data processing result to the user; the target self-defined function comprises a training function or a prediction function, the training and prediction of the artificial intelligence algorithm model are completed by defining and deploying the user self-defined function and calling the realized self-defined function based on the structured query statement, so that the artificial intelligence in the library is realized more simply and the expansibility is enhanced.
Description
Technical Field
The present application relates to the field of database technologies, and in particular, to a method for implementing in-library artificial intelligence and a database system.
Background
The artificial intelligence in the database refers to artificial intelligence model training in the database and data prediction in the database by using a trained artificial intelligence algorithm model. Because the algorithm model is in the database, the data and the algorithm are in the same memory space, data transfer is not needed, performance loss caused by data transmission can be effectively reduced, and the efficiency of artificial intelligence algorithm model training and artificial intelligence algorithm model prediction is improved by reducing the performance loss of data transmission.
In the prior art, the corresponding artificial intelligence algorithm is realized in the database engine, and the in-library artificial intelligence is completed based on the enhanced structured query language, so that the distributed artificial intelligence algorithm needs to be realized in the database engine again, the realization difficulty is usually too high, the existing widely popular artificial intelligence algorithm library cannot be used, and the number of support models is too small.
Therefore, how to more simply realize artificial intelligence in the library and enhance expansibility is a technical problem to be solved at present.
Disclosure of Invention
The invention discloses a method for realizing artificial intelligence in a library, which is used for solving the technical problems of complex realization mode and poor expansibility of the artificial intelligence in the library in the prior art, and is applied to a database system supporting a user-defined function, and comprises the following steps:
responding to the definition operation of a user on the self-defined function to generate a target self-defined function corresponding to the target artificial intelligence algorithm;
responding to the deployment operation of the user on the target custom function to deploy the target custom function at a specified position in the database system;
when a structured query statement which is sent by a user and corresponds to the target custom function is received, calling the target custom function from the specified position to execute data processing related to the target artificial intelligence algorithm, and returning a data processing result to the user;
the target self-defined function comprises a training function or a prediction function, the data processing comprises model training corresponding to the training function or model prediction corresponding to the prediction function, the training function represents the name of the target artificial intelligence algorithm, training data to be transmitted, a storage path of a model file of the target artificial intelligence algorithm and algorithm parameters of the target artificial intelligence algorithm, and the prediction function represents the name, test data to be transmitted and the storage path.
Correspondingly, the invention also provides a database system, which supports the function of user-defined, and the system comprises:
the definition module is used for responding to the definition operation of a user on the self-defined function to generate a target self-defined function corresponding to the target artificial intelligence algorithm;
the deployment module is used for responding to the deployment operation of the user on the target custom function to deploy the target custom function at a specified position in the database system;
the execution engine is used for calling the target self-defined function from the specified position to execute data processing related to the target artificial intelligence algorithm when receiving a structured query statement which is sent by a user and corresponds to the target self-defined function, and returning a data processing result to the user;
the storage module is used for storing the model file of the target artificial intelligence algorithm and the model prediction result of the target artificial intelligence algorithm;
the target custom function comprises a training function or a prediction function, the data processing comprises model training corresponding to the training function or model prediction corresponding to the prediction function, the training function represents the name of the target artificial intelligence algorithm, the training data to be transmitted, the storage path of the model file of the target artificial intelligence algorithm and the algorithm parameters of the target artificial intelligence algorithm, and the prediction function represents the name, the test data to be transmitted and the storage path.
By applying the technical scheme, in a database system supporting the user-defined function, a target user-defined function corresponding to a target artificial intelligence algorithm is generated in response to the definition operation of a user on the user-defined function; responding to the deployment operation of a user on the target self-defining function, and deploying the target self-defining function at a specified position in the database system; when a structured query statement which is sent by a user and corresponds to a target custom function is received, calling the target custom function from a specified position to execute data processing related to a target artificial intelligence algorithm, and returning a data processing result to the user; the target self-defined function comprises a training function or a prediction function, the training and the prediction of the artificial intelligence algorithm model are completed by defining and deploying the user self-defined function and calling the realized self-defined function based on the structured query statement, so that the artificial intelligence in the library is realized more simply and the expansibility is enhanced.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a flow chart illustrating a method for implementing intra-library artificial intelligence according to an embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating an implementation of artificial intelligence within a library in an embodiment of the invention;
fig. 3 shows a schematic structural diagram of a database system according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The database user-defined function means that the database system provides a mode which can enable a user to define any service logic function and can use the user-defined function in the database structured query.
Generally, a database system needs to deploy a custom function developed by a user to a specified location or path of the database system to ensure that the database system can load the custom function. The database system does not care about the internal logic of the user-defined function, and only transmits the data specified by the user in the database into the user-defined function in the calling process. The invention has the premise that the database system supports the user-defined function, and if the database system does not support the user-defined function, the support of the user-defined function needs to be provided for the database system.
The embodiment of the application provides a method for realizing the artificial intelligence in the library, which is applied to a database system supporting the user-defined function, the support of the artificial intelligence in the library is realized simply by supporting the user-defined function, in addition, the use of the existing artificial intelligence algorithm can be completed in the user-defined function, and rich algorithm model support can be provided.
As shown in fig. 1, the method comprises the steps of:
and S101, responding to the definition operation of the user on the custom function to generate a target custom function corresponding to the target artificial intelligence algorithm.
In this embodiment, the user performs the defining operation on the custom function according to the database system specification and the function attribute of the target custom function, where the function attribute includes a training function or a prediction function. In response to the defining operation, a training function or a prediction function corresponding to the target artificial intelligence algorithm may be generated, the training function representing a name of the target artificial intelligence algorithm, training data to be transmitted (i.e., training data of the target artificial intelligence algorithm), a storage path of a model file of the target artificial intelligence algorithm, and algorithm parameters of the target artificial intelligence algorithm, and the prediction function representing the name, test data to be transmitted (i.e., test data of the target artificial intelligence algorithm), and the storage path.
The target artificial intelligence algorithm is selected by the user as desired, and may include, for example, but not limited to, any one of multiple linear regression, gradient descent, Lasso regression, Ridge regression, polynomial regression, logistic regression, Softmax regression, SVM support vector machine, SMO optimization, K-means clustering, PCA dimension reduction, decision tree, random forest, Adaboost, GBDT, XGBoost, bayesian classification, hidden markov, maximum entropy, conditional random fields, deep learning neural network, and the like.
Optionally, the form of the training function is xxxModelFunction (trainingData, modelPath, modelParams), and the form of the prediction function is xxxPredictFunction (testData, modelPath), where xxx represents a name of the target artificial intelligence algorithm, the trainingData represents training data to be transmitted, the testData represents test data to be transmitted, the modelPath represents the storage path, and the modelParams represents the algorithm parameters.
In some embodiments of the present application, in order to accurately generate the target custom function, the specific process of the defining operation includes:
creating a project in any integrated development environment and naming the project according to the name of the target artificial intelligence algorithm;
creating a project package in the project, and creating a class xxxModel in the project package;
if the target custom function is a training function, creating a public xxxModelFunction (trainingData, modelPath, modelParams) method in the xxxModel class, and packaging the item into a Jar package;
if the target self-defined function is a prediction function, a public xxxPredictFunction (testData, modelPath) method is created in an xxxModel class, and the items are packaged into a Jar packet;
the xxx represents the name of the target artificial intelligence algorithm, the trainingData represents training data to be transmitted, the testData represents test data to be transmitted, the modelPath represents the storage path, and the modelParams represents the algorithm parameters.
In this embodiment, the name of the item is related to the name of the target artificial intelligence algorithm, and a common xxxModel function (modelPath, modelParams) method created in the xxxModel class can call a model of the target artificial intelligence algorithm, transmit the trainingData and the modelParams to the model for model training, and store the trained model file in the modelPath.
A common xxxPredictFunction (testData, modelPath) method created in the xxxModel class can load a trained model file from the modelPath, and transmits the testData to the model file for prediction to obtain a prediction result.
And S102, responding to the deployment operation of the target self-defining function by the user, and deploying the target self-defining function at a specified position in the database system.
In this embodiment, a user deploys a target custom function according to a requirement of a database system, and deploys the target custom function at a specified position in the database system in response to the deployment operation.
In order to reliably deploy the target custom function, in some embodiments of the present application, if the database system is a rapidsd db, a specific process of the deployment operation includes:
the Jar package is placed under the current/lib directory of the rapiddsdb installation directory and the rapiddsdb is restarted.
It will be appreciated that where the database system is of a type other than rapidsd db, the deployment operation is performed according to the requirements or specifications of other types of database systems.
And step S103, when receiving a structured query statement corresponding to the target custom function sent by a user, calling the target custom function from the specified position to execute data processing related to the target artificial intelligence algorithm, and returning a data processing result to the user.
In this embodiment, the data processing includes model training corresponding to a training function or model prediction corresponding to a prediction function.
When a user needs to realize the in-library artificial intelligence by using the target custom function, sending a structured query statement corresponding to the target custom function, calling the target custom function from a specified position based on the structured query statement to execute data processing related to the target artificial intelligence algorithm, and returning a data processing result to the user.
In a specific application scenario of the application, if a target self-defined function is a training function, a structured query statement corresponding to the target self-defined function is select xxxModelFunction (, modelPath, modelParams) from table trainingData; if the target self-defined function is a prediction function, the structured query statement corresponding to the target self-defined function is select xxxPredictFunction (, modelPath) from table _ data.
In order to reliably perform model training, in some embodiments of the present application, if the target custom function is a training function, the target custom function is called from the specified location to perform data processing related to the target artificial intelligence algorithm, and a data processing result is returned to a user, specifically:
calling the target self-defining function from the specified position, and transmitting training data of the target artificial intelligence algorithm to the target self-defining function for model training;
storing the trained model file in the storage path based on the target custom function;
and returning the model training result to the user.
In this embodiment, as shown in fig. 2, if the target custom function is a training function, the target custom function is called from the specified location, so that an execution engine of the database system transmits training data of a target artificial intelligence algorithm for model training, and then stores the trained model file in the storage path through the storage module, and returns a result of the model training, such as prompt information such as completion of the training, to the client of the user.
In a specific application scenario of the present application, if the target custom function is a training function, the target custom function called from the specified location is xxxModelFunction (trainingData, modelPath, modelParams), so that the execution engine transfers the data of the trainingData to xxxModelFunction () for model training, and stores the trained model file in the modelPath file.
In order to reliably perform model prediction, in some embodiments of the present application, if the target custom function is a prediction function, the target custom function is called from the specified location to perform data processing related to the target artificial intelligence algorithm, and a data processing result is returned to a user, which specifically includes:
calling the target self-defining function from the designated position, and transmitting test data of the target artificial intelligence algorithm based on the target self-defining function;
loading a trained model file from the storage path based on the target custom function and carrying out model prediction on the test data;
and returning the result of model prediction to the user.
In this embodiment, as shown in fig. 2, if the target custom function is a prediction function, the target custom function is called from a specified location, so that an execution engine of the database system transmits test data of the target artificial intelligence algorithm, then loads a trained model file from a storage path based on a storage module to perform model prediction on the test data, and returns a result of the model prediction to a client of a user.
In a specific application scenario of the present application, if the target custom function is a prediction function, the target custom function called from the specified location is xxxPredictFunction (testData, modelPath), so that the execution engine transfers the data of test _ data to xxxPredictFunction () for model prediction, and returns the prediction result to the user.
By applying the technical scheme, in a database system supporting the user-defined function, a target user-defined function corresponding to a target artificial intelligence algorithm is generated in response to the user defining operation on the user-defined function; responding to the deployment operation of a user on the target custom function, and deploying the target custom function at a specified position in the database system; when a structured query statement which is sent by a user and corresponds to a target custom function is received, calling the target custom function from a specified position to execute data processing related to a target artificial intelligence algorithm, and returning a data processing result to the user; the target self-defined function comprises a training function or a prediction function, the data processing comprises model training corresponding to the training function or model prediction corresponding to the prediction function, the training function represents the name of the target artificial intelligence algorithm, training data to be transmitted into the training function, the storage path of a model file of the target artificial intelligence algorithm and algorithm parameters of the target artificial intelligence algorithm, the prediction function represents the name, test data to be transmitted into the prediction function and the storage path, the training and prediction of the artificial intelligence algorithm model are completed by defining and deploying the user-defined function and calling the realized self-defined function based on the structured query statement, and therefore the in-library artificial intelligence is realized more simply and the expansibility is enhanced.
In order to further illustrate the technical idea of the present invention, the technical solution of the present invention will now be described with reference to specific application scenarios.
The embodiment of the application provides a method for realizing artificial intelligence in a library, taking a K mean value algorithm as an example, and simply realizing the support of the artificial intelligence in the library by a mode of supporting a user-defined function.
The specific implementation of artificial intelligence learning in the database is as follows:
1. a user-defined artificial intelligence algorithm model training function kMeanModelfunction (modelParams) is established, the function clusters data through a K-means algorithm, the function receives the data internally and calls the K-means algorithm to realize the function, and the K-means algorithm model trained based on the calling of the K-means algorithm is appointed to be stored in a storage unit with a path being the modelPath in the function.
The specific steps of defining the function attribute as the target self-defining function of the training function by the user are as follows:
a. creating a project kmeans in the IDEA;
b. creating a com. rapiddsdata. extensions package under the item, and creating a KMeanModel class in the package;
c. a common KMeanModel function (data, modelPath) method is created in the KMeanModel class, the open source kmeans algorithm is imported, and Model = kmeans.dokmeans (2, data, param) is called at KMeanModel function to generate a Model, and the Model is stored in the modelPath.
d. The java item is packaged into a Jar package named kmeans.
2. And (3) putting the Jar package generated in the step (1) into a current/lib directory under an installation directory of the RapidsDB, and restarting the RapidsDB to complete deployment.
3. The method comprises the steps of calling a deployed target custom function through a structured query statement selection kMeanModelfunction ("/root/home/robin/kMeanModel", 3) from table Iris _ train, transmitting data of the Iris _ train to the kMeanModelfunction to perform modeling to generate a model with 3 classifications by an execution engine, and storing the established model to a model Path/root/home/robin/kMeanModel for later use after the execution is finished.
Secondly, for the specific implementation mode of artificial intelligence prediction in the database:
1. establishing a custom function kMeanPredictFunction (testData, modelPath); the function is used for loading a model from the modelPath and predicting the incoming data, wherein the model prediction data of the trained K-means is loaded.
The specific steps of defining the function attribute as the target self-defining function of the prediction function by the user are as follows:
a. creating a project kmeans in the IDEA;
b. creating a com.rapidsdata.extensions package under the project, and creating a KMeanModel class in the package;
c. creating a public kMeanePredictFunction (testData, Model Path) method in the KMeanModel class, importing an open-source kMeans algorithm, calling a Model import method Model = kMeans. int [ ] result = model. Finally returning result;
d. the java item is packaged into a Jar package named kmeans.
2. And (3) putting the Jar package generated in the step (1) into a current/lib directory under an installation directory of the RapidsDB, and restarting the RapidsDB to complete deployment.
3. Calling a deployed target self-defining function, wherein an execution engine calls the kMeanepredictfunction, loads a trained K-means model from/root/home/kme/kMeanemodel, and predicts data in the Iris _ test by using the model, and returns a result which is a classification of test data, wherein the data of Iris is divided into 3, and the result returned for each piece of data is one of 0, 1 and 2, and is a prediction result which is returned as a result of the structured query statement.
An embodiment of the present application further provides a database system, where the database system supports a user-defined function, as shown in fig. 3, the system includes:
the defining module 10 is used for responding to the defining operation of a user on the self-defining function to generate a target self-defining function corresponding to the target artificial intelligence algorithm;
the deployment module 20 is used for responding to the deployment operation of the user on the target custom function to deploy the target custom function at a specified position in the database system;
the execution engine 30 is configured to, when receiving a structured query statement sent by a user and corresponding to the target custom function, invoke the target custom function from the specified location to execute data processing related to the target artificial intelligence algorithm, and return a data processing result to the user;
the storage module 40 is used for storing the model file of the target artificial intelligence algorithm and the model prediction result of the target artificial intelligence algorithm;
the target self-defined function comprises a training function or a prediction function, the data processing comprises model training corresponding to the training function or model prediction corresponding to the prediction function, the training function represents the name of the target artificial intelligence algorithm, training data to be transmitted, a storage path of a model file of the target artificial intelligence algorithm and algorithm parameters of the target artificial intelligence algorithm, and the prediction function represents the name, test data to be transmitted and the storage path.
In a specific application scenario of the present application, the execution engine 30 is specifically configured to:
calling the target self-defining function from the designated position, and transmitting training data of the target artificial intelligence algorithm to perform model training based on the target self-defining function;
storing the trained model file in the storage path based on the target custom function;
and returning the model training result to the user.
In a specific application scenario of the present application, the execution engine 30 is further specifically configured to:
calling the target self-defining function from the designated position, and transmitting test data of the target artificial intelligence algorithm based on the target self-defining function;
loading a trained model file from the storage path based on the target custom function and carrying out model prediction on the test data;
and returning the result of model prediction to the user.
In a specific application scenario of the present application, the specific process of the defining operation includes:
creating a project in any integrated development environment and naming the project according to the name of the target artificial intelligence algorithm;
creating a project package in the project, and creating a class xxxModel in the project package;
if the target custom function is a training function, creating a public xxxModelFunction (trainingData, modelPath, modelParams) method in the xxxModel class, and packaging the item into a Jar package;
if the target self-defined function is a prediction function, a public xxxPredictFunction (testData, modelPath) method is created in an xxxModel class, and the items are packaged into a Jar packet;
the xxx represents the name of the target artificial intelligence algorithm, the trainingData represents training data to be transmitted, the testData represents test data to be transmitted, the modelPath represents the storage path, and the modelParams represents the algorithm parameters.
In a specific application scenario of the present application, if the system is rapidsd db, the specific process of the deployment operation includes:
the Jar package is placed under the current/lib directory of the rapiddsdb installation directory and the rapiddsdb is restarted.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not necessarily depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.
Claims (8)
1. A method for realizing in-library artificial intelligence is applied to a database system supporting user-defined function functions, and comprises the following steps:
responding to the definition operation of a user on the self-defined function to generate a target self-defined function corresponding to the target artificial intelligence algorithm;
responding to the deployment operation of the user on the target custom function, and deploying the target custom function at a specified position in the database system;
when a structured query statement which is sent by a user and corresponds to the target custom function is received, calling the target custom function from the specified position to execute data processing related to the target artificial intelligence algorithm, and returning a data processing result to the user;
the target self-defined function comprises a training function or a prediction function, the data processing comprises model training corresponding to the training function or model prediction corresponding to the prediction function, the training function represents the name of the target artificial intelligence algorithm, training data to be transmitted, a storage path of a model file of the target artificial intelligence algorithm and algorithm parameters of the target artificial intelligence algorithm, and the prediction function represents the name, test data to be transmitted and the storage path.
2. The implementation method of claim 1, wherein if the target custom function is a training function, invoking the target custom function from the specified location to perform data processing related to the target artificial intelligence algorithm, and returning a data processing result to a user, specifically:
calling the target self-defining function from the designated position, and transmitting training data of the target artificial intelligence algorithm to perform model training based on the target self-defining function;
storing the trained model file in the storage path based on the target custom function;
and returning the model training result to the user.
3. The implementation method of claim 1, wherein if the target custom function is a prediction function, invoking the target custom function from the specified location to perform data processing related to the target artificial intelligence algorithm, and returning a data processing result to a user, specifically:
calling the target self-defining function from the designated position, and transmitting test data of the target artificial intelligence algorithm based on the target self-defining function;
loading a trained model file from the storage path based on the target custom function and carrying out model prediction on the test data;
and returning the result of model prediction to the user.
4. The implementation method of claim 1, wherein the specific process of defining the operation comprises:
creating a project in any integrated development environment and naming the project according to the name of the target artificial intelligence algorithm;
creating a project package in the project, and creating a class xxxModel in the project package;
if the target custom function is a training function, creating a public xxxModelFunction (trainingData, modelPath, modelParams) method in the xxxModel class, and packaging the item into a Jar package;
if the target custom function is a prediction function, creating a public xxxPredictFunction (testData, modelPath) method in the xxxModel class, and packaging the item into a Jar packet;
the xxx represents the name of the target artificial intelligence algorithm, the trainingData represents training data to be transmitted, the testData represents test data to be transmitted, the modelPath represents the storage path, and the modelParams represents the algorithm parameters.
5. A database system, wherein the database system supports user-defined function functions, the system comprising:
the definition module is used for responding to the definition operation of a user on the self-defined function to generate a target self-defined function corresponding to the target artificial intelligence algorithm;
the deployment module is used for responding to the deployment operation of the user on the target custom function to deploy the target custom function at a specified position in the database system;
the execution engine is used for calling the target self-defined function from the specified position to execute data processing related to the target artificial intelligence algorithm when receiving a structured query statement which is sent by a user and corresponds to the target self-defined function, and returning a data processing result to the user;
the storage module is used for storing the model file of the target artificial intelligence algorithm and the model prediction result of the target artificial intelligence algorithm;
the target self-defined function comprises a training function or a prediction function, the data processing comprises model training corresponding to the training function or model prediction corresponding to the prediction function, the training function represents the name of the target artificial intelligence algorithm, training data to be transmitted, a storage path of a model file of the target artificial intelligence algorithm and algorithm parameters of the target artificial intelligence algorithm, and the prediction function represents the name, test data to be transmitted and the storage path.
6. The system of claim 5, wherein the execution engine is specifically configured to:
calling the target self-defining function from the designated position, and transmitting training data of the target artificial intelligence algorithm to perform model training based on the target self-defining function;
storing the trained model file in the storage path based on the target custom function;
and returning the model training result to the user.
7. The system of claim 5, wherein the execution engine is further specifically configured to:
calling the target self-defining function from the designated position, and transmitting test data of the target artificial intelligence algorithm based on the target self-defining function;
loading a trained model file from the storage path based on the target custom function and carrying out model prediction on the test data;
and returning the result of model prediction to the user.
8. The system of claim 5, wherein the specific process of defining operations comprises:
creating a project in any integrated development environment and naming the project according to the name of the target artificial intelligence algorithm;
creating a project package in the project, and creating a class xxxModel in the project package;
if the target self-defined function is a training function, creating a public xxxModelFunction (trainingData, modelPath, modelParams) method in the xxxModel class, and packaging the items into a Jar packet;
if the target custom function is a prediction function, creating a public xxxPredictFunction (testData, modelPath) method in the xxxModel class, and packaging the item into a Jar packet;
the xxx represents the name of the target artificial intelligence algorithm, the trainingData represents training data to be transmitted, the testData represents test data to be transmitted, the modelPath represents the storage path, and the modelParams represents the algorithm parameters.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210292699.8A CN114741372B (en) | 2022-03-24 | 2022-03-24 | Method for realizing in-library artificial intelligence and database system |
| PCT/CN2022/122445 WO2023178977A1 (en) | 2022-03-24 | 2022-09-29 | Method for implementing in-database artificial intelligence, and database system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210292699.8A CN114741372B (en) | 2022-03-24 | 2022-03-24 | Method for realizing in-library artificial intelligence and database system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114741372A true CN114741372A (en) | 2022-07-12 |
| CN114741372B CN114741372B (en) | 2022-11-15 |
Family
ID=82277648
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210292699.8A Active CN114741372B (en) | 2022-03-24 | 2022-03-24 | Method for realizing in-library artificial intelligence and database system |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN114741372B (en) |
| WO (1) | WO2023178977A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023178977A1 (en) * | 2022-03-24 | 2023-09-28 | 北京柏睿数据技术股份有限公司 | Method for implementing in-database artificial intelligence, and database system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8505005B1 (en) * | 2007-07-24 | 2013-08-06 | United Services Automobile Association (Usaa) | Multi-platform network-based software distribution |
| CN109657803A (en) * | 2018-03-23 | 2019-04-19 | 新华三大数据技术有限公司 | The building of machine learning model |
| CN111260305A (en) * | 2018-11-30 | 2020-06-09 | 塔塔顾问服务有限公司 | Generating an extensible and customizable location-independent agile delivery model |
| CN111897525A (en) * | 2020-07-15 | 2020-11-06 | 深圳市腾讯网域计算机网络有限公司 | Big data processing method and system |
| CN112884066A (en) * | 2021-03-15 | 2021-06-01 | 网易（杭州）网络有限公司 | Data processing method and device |
| WO2022026984A1 (en) * | 2020-07-31 | 2022-02-03 | Splunk Inc. | Data field extraction model training for a data intake and query system |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11023460B2 (en) * | 2017-12-22 | 2021-06-01 | Teradata Us, Inc. | Transparent user-defined function (UDF) optimization |
| CN112783921A (en) * | 2021-01-26 | 2021-05-11 | 中国银联股份有限公司 | Database operation method and device |
| CN113986889B (en) * | 2021-12-28 | 2022-04-05 | 天津南大通用数据技术股份有限公司 | Method and system for realizing intelligent expansion of database function |
| CN114741372B (en) * | 2022-03-24 | 2022-11-15 | 北京柏睿数据技术股份有限公司 | Method for realizing in-library artificial intelligence and database system |
-
2022
- 2022-03-24 CN CN202210292699.8A patent/CN114741372B/en active Active
- 2022-09-29 WO PCT/CN2022/122445 patent/WO2023178977A1/en unknown
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8505005B1 (en) * | 2007-07-24 | 2013-08-06 | United Services Automobile Association (Usaa) | Multi-platform network-based software distribution |
| CN109657803A (en) * | 2018-03-23 | 2019-04-19 | 新华三大数据技术有限公司 | The building of machine learning model |
| CN111260305A (en) * | 2018-11-30 | 2020-06-09 | 塔塔顾问服务有限公司 | Generating an extensible and customizable location-independent agile delivery model |
| CN111897525A (en) * | 2020-07-15 | 2020-11-06 | 深圳市腾讯网域计算机网络有限公司 | Big data processing method and system |
| WO2022026984A1 (en) * | 2020-07-31 | 2022-02-03 | Splunk Inc. | Data field extraction model training for a data intake and query system |
| CN112884066A (en) * | 2021-03-15 | 2021-06-01 | 网易（杭州）网络有限公司 | Data processing method and device |
Non-Patent Citations (3)
| Title |
|---|
| BO WEI: ""Applying Hibernate to persist Java3D virtual scene in Oracle"", 《 2010 2ND INTERNATIONAL CONFERENCE ON ADVANCED COMPUTER CONTROL》 * |
| 刘婧妍等: ""一种复杂事件处理语言的自定义函数功能扩展"", 《计算机与现代化》 * |
| 魏房忠等: ""SQL CLR聚合函数在环境质量业务数据计算中的应用"", 《环境监控与预警》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023178977A1 (en) * | 2022-03-24 | 2023-09-28 | 北京柏睿数据技术股份有限公司 | Method for implementing in-database artificial intelligence, and database system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114741372B (en) | 2022-11-15 |
| WO2023178977A1 (en) | 2023-09-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20200272338A1 (en) | Optimizing serverless computing using a distributed computing framework | |
| CN103186466B (en) | Information interactive testing device and method based on association automatic generating test case | |
| CN109858607A (en) | A kind of application method, management method, system and server | |
| CN108449410A (en) | Information management method, system and relevant apparatus in a kind of cloud platform | |
| CN109492017A (en) | Business information inquiry processing method, system, computer equipment and storage medium | |
| Nalepa et al. | Algorithms for rule inference in modularized rule bases | |
| CN114741372B (en) | Method for realizing in-library artificial intelligence and database system | |
| US11640323B2 (en) | Method and machine learning agent for executing machine learning in an edge cloud | |
| CN108492005A (en) | Project data processing method, device, computer equipment and storage medium | |
| CN110276074A (en) | Distributed training method, device, equipment and the storage medium of natural language processing | |
| CN109460250A (en) | A kind of generation method and equipment of application program | |
| CN110516000B (en) | Workflow management system supporting complex workflow structure | |
| CN108897587A (en) | Plug type machine learning algorithm operation method, device and readable storage medium storing program for executing | |
| CN109002434A (en) | Customer service question and answer matching process, server and storage medium | |
| CN114064062A (en) | Kubernetes platform and load balancing component-based default gray level issuing method and device | |
| CN113849300A (en) | Edge configuration system and method | |
| CN110515712A (en) | Task management method and computer storage medium | |
| CN111813529B (en) | Data processing method, device, electronic equipment and storage medium | |
| CN114493493A (en) | Decision engine and decision engine implementation method | |
| CN113961179A (en) | Service access method, system, electronic device and storage medium of SOAR platform | |
| CN114461358A (en) | Service processing method and system | |
| US20200242702A1 (en) | Capital chain information traceability method, system, server and readable storage medium | |
| CN114115967A (en) | Service data processing method and device, computer equipment and storage medium | |
| CN114116051B (en) | Processing method, device, equipment and storage medium based on neural network model | |
| CN111400399A (en) | Account book synchronization method and device of block chain system and hardware equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |