CN111415014B - Model prediction result data management system and method - Google Patents
Model prediction result data management system and method Download PDFInfo
- Publication number
- CN111415014B CN111415014B CN202010201917.3A CN202010201917A CN111415014B CN 111415014 B CN111415014 B CN 111415014B CN 202010201917 A CN202010201917 A CN 202010201917A CN 111415014 B CN111415014 B CN 111415014B
- Authority
- CN
- China
- Prior art keywords
- data
- result data
- user
- result
- management
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Storage Device Security (AREA)
Abstract
The invention discloses a result data management system and method for model prediction, wherein the system comprises: the result data storage module is used for acquiring result data generated in the model prediction process and storing the result data under a preset path; the result data management module is used for receiving the control instruction input in the result data management page, processing the result data, wherein the control instruction comprises instructions of data storage, data preview, data maintenance, data deletion and data export; wherein, according to the instruction of data storage, store the result data in the result dataset; previewing, maintaining or deleting the result data according to the data previewing, data maintaining or data deleting instructions; the result data export module is used for exporting corresponding result data according to the data export instruction and transmitting the result data to a data user in a file transmission mode; and the result data set is used for storing result data generated in the model prediction process.
Description
Technical Field
The invention relates to the technical field of model training, in particular to a result data management system and method for model prediction.
Background
In the artificial intelligence modeling process, after a model is trained, a user needs to take the model away, and then a prediction task is performed independently, so that data required by the user is obtained, and then business adjustment is performed according to the data. If the effect of the model is found to be bad after the prediction task is executed, the predicted result data is not the effect intended by the user, then the model needs to be retrained, and then the model is taken away again to execute the prediction task, so that the development period and the cost are high.
In summary, in the prior art, the following disadvantages exist in the process of taking away the model through modeling training, then executing a prediction task, and evaluating result set data until the process of meeting the service requirement: due to the lack of unified management of result set data, a modeler cannot directly execute a prediction task and store the result data set and the evaluation result data set, so that the development period is longer, the development cost is higher, and the access right cannot be controlled.
Accordingly, there is a need for a result data management scheme that overcomes the above-described problems, enables unified management of the result data sets, and enables control of access rights.
Disclosure of Invention
In order to overcome the problems, the invention provides a result data management system and a method for model prediction, which enable modeling staff to directly execute a prediction task and store a result data set and an evaluation result set, store, evaluate and export the result data after model training and execution of the prediction task for service departments, shorten development period, save cost, realize unified management (including view management) of the result data set, provide access authority control and ensure the safety of data set access.
In one embodiment of the present invention, a result data management system for model prediction is provided, the system comprising:
the result data storage module is used for acquiring result data generated in the model prediction process and storing the result data under a preset path;
the result data management module is used for receiving a control instruction input into a result data management page and processing the result data, wherein the control instruction comprises instructions including data storage, data preview, data maintenance, data deletion and data export; wherein, according to the data storage instruction, the result data is stored in a result data set; previewing, maintaining or deleting the result data according to the data previewing, data maintaining or data deleting instructions;
the result data export module is used for exporting corresponding result data according to the data export instruction and transmitting the result data to a data user in a file transmission mode;
and the result data set is used for storing result data generated in the model prediction process.
In another embodiment of the present invention, a method for managing result data of model prediction is also provided, the method comprising:
obtaining result data generated in the model prediction process and storing the result data under a preset path;
receiving a control instruction input in a result data set management page, and processing the result data, wherein the control instruction comprises instructions including data storage, data preview, data maintenance, data deletion and data derivation; wherein, the liquid crystal display device comprises a liquid crystal display device,
storing the result data in a result data set according to the data storage instruction;
previewing, maintaining or deleting the result data according to the data previewing, data maintaining or data deleting instructions;
and according to the instruction for data export, exporting corresponding result data, and transmitting the result data to a data user in a file transmission mode.
In another embodiment of the present invention, a computer device is also presented, including a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing a result data management method of model prediction when executing the computer program.
In another embodiment of the present invention, a computer-readable storage medium storing a computer program that, when executed by a processor, implements a result data management method of model prediction is also presented.
The result data management system and method for model prediction can provide a unified SDK access mode aiming at the data storage address of the heterogeneous result set, and modeling staff does not need to pay attention to data sources or singly adapt; the system and the method also provide a result set export function, a modeler can export the result set after confirming that the result set meets the service requirement, repeated operation in the development process is avoided, the development period is shortened, the cost is saved, and unified result data set management is also provided, and the modeler can inquire and manage authorized result data through a visual interface.
Drawings
FIG. 1 is a schematic diagram of a model predictive outcome data management system architecture according to an embodiment of the invention.
Fig. 2 is a schematic diagram of an architecture design of a core device according to an embodiment of the invention.
FIG. 3 is a flow chart of a method for managing result data of model prediction according to an embodiment of the invention.
FIG. 4 is a detailed flow chart of the processing of result data according to an embodiment of the present invention.
FIG. 5 is a flow chart of a method for managing result data of model prediction according to another embodiment of the present invention.
FIG. 6 is a detailed flow chart of a method for managing result data of model prediction according to an embodiment of the present invention.
FIG. 7 is a schematic diagram of a computer device according to an embodiment of the invention.
Detailed Description
The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are presented merely to enable those skilled in the art to better understand and practice the invention and are not intended to limit the scope of the invention in any way. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
Those skilled in the art will appreciate that embodiments of the invention may be implemented as a system, apparatus, device, method, or computer program product. Accordingly, the present disclosure may be embodied in the following forms, namely: complete hardware, complete software (including firmware, resident software, micro-code, etc.), or a combination of hardware and software.
According to the embodiment of the invention, a result data management system and method for model prediction are provided.
Note that terms involved in this embodiment are as follows:
artificial intelligence: english is abbreviated as AI (Artificial Intelligence). It is a new technical science for researching, developing theory, method, technology and application system for simulating, extending and expanding human intelligence.
AI model: according to the data structure summarized by the training data, new data can be predicted based on the structure, and the target classification or value of the new data can be judged.
AI algorithm: summarizing the AI model methods, there are machine learning algorithms (e.g., logistic regression LR, gradient boost decision tree GBDT, etc.) and deep learning algorithms (DNN/CNN/RNN, etc.).
(AI/artificial intelligence) modeling: and (3) constructing an AI model.
AI development task: items to which AI modeling belongs.
AI modeling engineering: models that AI development tasks need to develop.
Modeling scheme/code: in order to construct the code written by the AI model, the code generally includes a series of steps such as data introduction, algorithm calling, parameter setting, model evaluation, etc., which are aimed at producing an available AI model.
(modeling) result data: the modeling personnel trains the model, and then executes a prediction process on the prediction data to generate data. Such data is data obtained by modeling personnel through prediction data and models, and is used by business departments or related departments.
The principles and spirit of the present invention are explained in detail below with reference to several representative embodiments thereof.
FIG. 1 is a schematic diagram of a model predictive outcome data management system architecture according to an embodiment of the invention. As shown in fig. 1, the system includes:
a result data storage module 110, configured to obtain result data generated in the model prediction process;
the result data management module 120 is configured to receive a manipulation instruction input into a result data set management page, and process the result data, where the manipulation instruction includes an instruction including data storage, data preview, data maintenance, data deletion and data export; wherein, the liquid crystal display device comprises a liquid crystal display device,
storing the result data in a result data set according to the data storage instruction;
previewing, maintaining or deleting the result data according to the data previewing, data maintaining or data deleting instructions;
in a specific embodiment, the modeling personnel can perform corresponding operations on the result data set management page, for example, preview the result data generated by the prediction task, so as to ensure that the result data meets the service requirement; the modeler may also perform a delete operation on unsatisfactory result data or export satisfactory result data.
The result data export module 130 is configured to export corresponding result data according to the data export instruction, and transmit the result data to a data user through a file transmission manner;
the result data set 200 is used to store the result data generated during the model prediction process.
Further, as shown in fig. 1, the system further includes: a user management module 140, an SDK 300, a visual graphical interface 400; wherein, the liquid crystal display device comprises a liquid crystal display device,
a user management module 140, configured to verify the identity of a user when the user initiates a request;
correspondingly, the result data management module 120 is further configured to receive a manipulation instruction input by the user on the result data set management page after the verification is passed, and process the result data.
An SDK 300 (software development kit), wherein a result data set storage interface (result data set storage API) and a result data set export interface (result data set export API) are provided in the SDK 300;
the result data storage module 110 is specifically configured to obtain, through the result data set storage interface, result data generated in the model prediction process, and store the result data in a preset path of the result data set 200;
the result data export module 130 is specifically configured to export, through a result data set export interface, result data corresponding to the data export instruction.
The SDK is used as a mode of storing the result data set, so that access difference of heterogeneous data storage addresses in the AI prediction process can be shielded for modeling staff (users), a unified access right control mechanism is provided, and the modeling staff does not need to pay attention to data sources or singly adapt, so that the management of the result data is more convenient.
The visual graphical interface 400 is used to provide relevant functions for user login, result dataset preview, maintenance, export, storage, and deletion, etc., on which a user can operate.
In one embodiment, as shown in fig. 1, the result data storage module 110, the result data management module 120, the result data export module 130, and the user management module 140 form the core device 100.
Fig. 2 is a schematic diagram illustrating the architecture design of a core device 100 according to an embodiment of the invention. The core device 100 specifically includes:
the RESTful interface 101 serves as a unified portal for external requests. When a new user requests, the user name and password are input first, and the subsequent data set related operation can be performed after the user name and password are successful. Of these, RESTful (representational state transfer, representational state transition) is an architectural style of web services.
The session cache 102 is used for storing the session information of the current logged-in user through a distributed cache.
Further, through the user management module 140, when a user initiates a request, the input user name and password are verified; after passing the verification, inquiring whether the user number of the user is stored in a stored user login information table; if not, generating a user number of the user and recording the user number in the user login information table; the user login information table stores user numbers corresponding to each user, result data with management rights and result data with access rights.
The user management part can be used for checking the current user login, or can be integrated by docking with a third party single sign-on, and when a new third party user passes the check and logs in, the unique number of the user in the device can be automatically created and recorded in the user table. After the login is successful, the system stores the login information of the current user, and the current user is checked to be logged in every subsequent request operation aiming at the data set.
The metadata base 103 stores metadata mainly including result dataset management information, dataset storage configuration information, dataset right information, user information, and the like.
After the user logs in successfully, the result data management module 120 processes the result data of the result data set 200 according to the control instruction input by the user on the result data set management page;
specifically, the result data management module 120 includes:
a data storage unit 121, configured to store, according to an instruction for storing data, result data in a preset path to a result data set 200, for a modeler to view and set a management authority of the modeler with the result data;
after training the model, the modeler may directly perform the predictive task and store the resulting data to facilitate an assessment of whether the data meets business requirements.
The data preview unit 122 is configured to display, according to an instruction of data preview, result data having access rights of a user through a visual interface;
specifically, the data set name, the description information, the data content and the generation time in the result data with the access right of the user can be displayed through a visual interface; the data content may be detailed data information, such as the content of each column or each row or the content of a picture, to ensure that the data meets business requirements.
The preview result data supports preview of text data, structured data and picture data.
The data maintenance unit 123 is configured to maintain management information (data set description, cleaning policy, etc.) and authority information of the result data according to an instruction of data maintenance.
Where access rights may be a complete disclosure or a specified project disclosure (accessible to users within the project and inaccessible to other users), the particular rights are managed and verified by the data rights management module 160.
And the data deleting unit 124 is configured to delete the result data with the management authority according to the instruction of data deletion.
The user can delete the result data which has authority to access, such as some result data which does not meet the service requirement, and can delete the result data immediately after the preview evaluation.
The data export unit 125 is configured to control the result data export module 130 to export the result data with access rights of the user according to the instruction of data export.
After the modeling personnel evaluate that the result data meets the service requirement, the result data can be exported to a designated area, and then the data is transmitted to a place which can be accessed by the own service department in a file transmission mode.
After determining the derived result data, the result data deriving module 130 derives the result data corresponding to the instruction derived from the data to a designated area through the result data set deriving interface, so that the user can transmit the data to a place accessible to the service department thereof through a file transmission mode.
The core device 100 is further provided with a data cleaning module 150 and a data authority management module 160; wherein, the liquid crystal display device comprises a liquid crystal display device,
the data cleansing module 150 is configured to cleanse the result data set exceeding the retention time or the retention memory space according to a given period according to a cleansing policy of the set retention time or the retention memory space. The function of the module is mainly to clean the result data set satisfied by the user regularly, and the data unsatisfied by the user can be deleted by using the data deleting function.
The data authority management module 160 is connected to the data maintenance unit 123, and is configured to verify whether the current user has access authority to a certain data set according to the policy according to the data set access authority formulated by the data set owner. The access rights mainly comprise 2 types: 1. designating project disclosure, which is accessible to users within the project, and not accessible to other users; 2. fully disclosed, all can be accessed.
The usual rights can be divided into management rights and access rights, and after modeling personnel build a model, execute a prediction task and obtain result data, the modeling personnel can have the management rights (such as storage, deletion, derivation, maintenance, preview and the like) of the result data, and can set the access rights, such as complete disclosure or appointed project disclosure and the like; the user with access authority can preview the result data, and if the result data is satisfied, the result data can be exported to the corresponding business department for use.
By utilizing the system, a modeling person can directly execute a prediction task, store a result data set and an evaluation result set, and then export the result data set for a business department to use, so that the development period is shortened, and the development cost is saved.
In one embodiment, the important metadata designs in the metadata database 103 are shown in tables 1-4 below.
Table 1 results data set management
Table 2 data set internal storage means
Table 3 dataset rights assignment overt person mapping
TABLE 4 user verification information
The information of the result data set management mainly comprises a result data set name, a result data set description, an AI research and development task, an AI modeling project, a modeling scheme, a creator, creation time, a management mode, a result data set storage days, a result data set storage size and a disclosure mode by combining with the table 1;
with reference to table 2, the internal storage mode of the data set mainly comprises data set storage configuration;
in combination with table 3, the dataset rights assignment public person map mainly includes assignment public person numbers;
in combination with table 4, the user verification information mainly includes a user number, a user login name, a user password verification and a default specified public number list of the result data set.
In one embodiment, the SDK 300 is configured to conveniently store result data sets in the modeling code, with two types of important APIs (application program interfaces) being result data set storage and result data set export, respectively.
The interface is as follows:
# results dataset storage interface
defsaveResultSet(sys.argv,resSet_name,res_path):ResultSave
# results dataset export interface
defexportResultSet(sys.argv,resSet_name):ResultSetExport
The above interface eventually invokes the RESTful interface 101 of the core device 100.
In the process of storing and deriving result data sets through artificial intelligence modeling training, the data names of the result data generated by the prediction tasks of different modeling schemes can be repeated, but the result data names under the same modeling scheme cannot be repeated. Therefore, by utilizing the system, the result data generated in the prediction process can be stored under a canonical path, the path can be specific to the modeling scheme, and the result data of each modeling scheme is ensured to be stored separately; in this way, the modeler can preview the resulting data generated to ensure that it is data that meets business requirements. The modeler can delete the data which does not meet the service requirement immediately. After the resulting data is generated, the modeler needs to transmit the data to a location accessible to the business segment for use by the business segment.
The model prediction result data management system provided by the invention can provide a unified SDK access mode aiming at the heterogeneous result set data storage address, and modeling staff does not need to pay attention to data sources and does not need to be independently adapted. The system also provides a result set export function, a modeler can export the result set after confirming that the result set meets the service requirement, repeated operation in the development process is avoided, the development period is shortened, the cost is saved, and unified result data set management (including view) is also provided, and the modeler can inquire and manage authorized result data through a visual interface.
It should be noted that while several modules of the result data management system of model prediction are mentioned in the above detailed description, such partitioning is merely exemplary and not mandatory. Indeed, the features and functions of two or more modules described above may be embodied in one module in accordance with embodiments of the present invention. Conversely, the features and functions of one module described above may be further divided into a plurality of modules to be embodied.
Having described the system of the exemplary embodiment of the present invention, next, a result data management method of model prediction of the exemplary embodiment of the present invention will be described with reference to fig. 3 to 6.
Based on the same inventive concept, the invention also provides a result data management method of model prediction, as shown in fig. 3, the method comprises the following steps:
step S310, obtaining result data generated in the model prediction process and storing the result data in a preset path;
step S320, receiving a control instruction input in a result data set management page, and processing the result data, wherein the control instruction comprises instructions including data storage, data preview, data maintenance, data deletion and data export; wherein, the liquid crystal display device comprises a liquid crystal display device,
step S321, according to the data storage instruction, storing the result data in a result data set;
step S322, previewing, maintaining or deleting the result data according to the data previewing, data maintaining or data deleting instruction;
step S323, according to the instruction for data export, export the corresponding result data, and transmit to the data user by means of file transmission.
In one embodiment, in conjunction with fig. 4, the detailed process of processing the result data includes:
step S321, according to the data storage instruction, storing the result data in the preset path into a result data set for a modeling person to check and set the management authority of the modeling person with the result data;
the detailed process of step S322 is:
step S3221, according to the instruction of data preview, the result data with access right of the user is displayed through a visual interface; specifically, the displayed information can be the name of the data set, the description information, the data content and the generation time; the preview result data supports preview of text data, structured data and picture data. The data content may be detailed data information, such as the content of each column or each row or the content of a picture, to ensure that the data meets business requirements.
Step S3222, according to the data maintenance instruction, maintaining the management information and the authority information of the result data; wherein the access rights are fully open or specified item open.
Step S3223, according to the data deleting instruction, deleting the result data with the management authority of the user.
Step S323, according to the instruction of data export, controls the result data export module to export the result data with access rights of the user.
In one embodiment, in conjunction with fig. 5, the result data management method of model prediction further includes:
step S300, setting a result data set storage interface and a result data set export interface by utilizing the SDK. Based on the two interfaces, the corresponding data storage and data export steps are as follows:
step S310, obtaining result data generated in the model prediction process through the result data set storage interface, and storing the result data in a preset path;
the process of step S321 and step S322 may refer to the foregoing embodiment of fig. 3.
Step S323, export the result data corresponding to the instruction of the data export through the result data set export interface.
Step S330, according to the set retention time or the cleaning policy of the reserved storage space, the result data exceeding the retention time or the reserved storage space in the result data set is cleaned according to the designated period.
In one embodiment, in connection with fig. 6, before performing step S320, the result data management method of model prediction further includes a process of verifying the identity of the user:
step S315, when a user initiates a request, verifying the identity of the user;
the detailed process is as follows:
step S3151, when a user initiates a request, verifying the input user name and password;
step S3152, after the verification is passed, inquiring whether the user number of the user is stored in a stored user login information table; if not, generating a user number of the user and recording the user number in the user login information table; the user login information table stores user numbers corresponding to each user, result data with management rights and result data with access rights.
After the verification is passed, step S320 is performed, in which a manipulation instruction input by a modeler (user) in a result data set management page may be received, and the result data may be processed.
It should be noted that although the operations of the method of the present invention are described in a particular order in the above embodiments and the accompanying drawings, this does not require or imply that the operations must be performed in the particular order or that all of the illustrated operations be performed in order to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform.
Based on the foregoing inventive concept, as shown in fig. 7, the present invention further proposes a computer device 700, including a memory 710, a processor 720, and a computer program 730 stored on the memory 710 and executable on the processor 720, where the processor 720 implements the result data management method of the foregoing model prediction when executing the computer program 730.
In another embodiment of the present invention, a computer readable storage medium is also presented, the computer readable storage medium storing a computer program which, when executed by a processor, implements the result data management method of model prediction described above.
The result data management system and method for model prediction can provide a unified SDK access mode aiming at the data storage address of the heterogeneous result set, and modeling staff does not need to pay attention to data sources or singly adapt; the system and the method also provide a result set export function, a modeler can export the result set after confirming that the result set meets the service requirement, repetitive operation in the development process is avoided, the development period is shortened, the cost is saved, and unified result data set management (including view) is also provided, and the modeler can inquire and manage authorized result data through a visual interface.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Finally, it should be noted that: the above examples are only specific embodiments of the present invention, and are not intended to limit the scope of the present invention, but it should be understood by those skilled in the art that the present invention is not limited thereto, and that the present invention is described in detail with reference to the foregoing examples: any person skilled in the art may modify or easily conceive of the technical solution described in the foregoing embodiments, or perform equivalent substitution of some of the technical features, while remaining within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (9)
1. A model predictive outcome data management system, the system comprising:
the result data storage module is used for acquiring result data generated in the model prediction process and storing the result data under a preset path;
the result data management module is used for receiving a control instruction input into a result data management page and processing the result data, wherein the control instruction comprises instructions including data storage, data preview, data maintenance, data deletion and data export; wherein, according to the data storage instruction, the result data is stored in a result data set; previewing, maintaining or deleting the result data according to the data previewing, data maintaining or data deleting instructions;
the result data export module is used for exporting corresponding result data according to the data export instruction and transmitting the result data to a data user in a file transmission mode;
the result data set is used for storing result data generated in the model prediction process;
the SDK is provided with a result data set storage interface and a result data set export interface; the SDK is used as a mode of storing a result data set, so that access difference of heterogeneous data storage addresses in the AI prediction process is shielded for modeling personnel, and an access right control mechanism is provided;
the user management module is used for verifying the input user name and password when a user initiates a request; after passing the verification, inquiring whether the user number of the user is stored in a stored user login information table; if not, generating a user number of the user and recording the user number in the user login information table; the user login information table stores user numbers corresponding to each user, result data with management rights and result data with access rights; the user management module is used for checking the current user login or integrating the third party single sign-on, and automatically creating a unique number corresponding to the user and recording the unique number in a user table when a new third party user passes the check and logs in; after successful login, storing the login information of the current user, and checking that the current user is logged in every subsequent request operation for the data set;
the data authority management module is used for managing and checking the authorities, the authorities are divided into management authorities and access authorities, and after modeling personnel build a model and execute a prediction task and obtain result data, the modeling personnel have the management authorities of the result data and set the access authorities; wherein, the access authority comprises: designating project disclosure, which is accessible to users within the project, and not accessible to other users; fully disclosed, all can be accessed;
the metadata base is used for storing result data set management data, data set internal storage modes, data set authority assignment publicity personnel mapping and user verification information; wherein the information for result dataset management includes: the method comprises the steps of a result data set name, a result data set description, an AI research and development task, an AI modeling project, a modeling scheme, a creator, creation time, a management mode, a result data set storage days, a result data set storage size and a disclosure mode; the internal storage mode of the data set comprises the following steps: a data set storage configuration; the dataset rights assignment disclosure person mapping includes: designating a public number; the user verification information includes: the user number, user login name, user password verification and result data set default specified public numbering list.
2. The model predictive outcome data management system of claim 1, further comprising:
the result data storage module is specifically configured to obtain, through the result data set storage interface, result data generated in a model prediction process, and store the result data in a preset path;
the result data export module is specifically configured to export, through a result data set export interface, result data corresponding to the data export instruction.
3. The model predictive outcome data management system of claim 1, further comprising: and the result data management module is also used for receiving the control instruction input in the result data set management page after the verification is passed and processing the result data.
4. The model predicted outcome data management system of claim 3 wherein said outcome data management module comprises:
the data storage unit is used for storing result data under a preset path into a result data set according to the data storage instruction, so that a modeling person can check and set the management authority of the modeling person with the result data;
the data preview unit is used for displaying the result data with the access right of the user through the visual interface according to the instruction of data preview;
the data maintenance unit is used for maintaining management information and authority information of the result data according to the data maintenance instruction; wherein the access right is completely disclosed or the project is disclosed;
the data deleting unit is used for deleting the result data with the management authority of the user according to the data deleting instruction;
and the data export unit is used for controlling the result data export module to export the result data with the access right of the user according to the data export instruction.
5. The model predictive outcome data management system as claimed in claim 4, wherein the data preview unit is specifically configured to:
according to the instruction of data preview, displaying the data set name, the description information, the data content and the generation time in the result data with the access right of the user through a visual interface; the preview result data supports preview of text data, structured data and picture data.
6. The model predictive outcome data management system of claim 1, further comprising:
and the data cleaning module is used for cleaning the result data exceeding the retention time or the retention storage space in the result data set according to a specified period according to the set retention time or the cleaning strategy of the retention storage space.
7. A method for managing result data of model prediction, the method comprising:
obtaining result data generated in the model prediction process and storing the result data under a preset path;
receiving a control instruction input in a result data set management page, and processing the result data, wherein the control instruction comprises instructions including data storage, data preview, data maintenance, data deletion and data derivation; wherein, the liquid crystal display device comprises a liquid crystal display device,
storing the result data in a result data set according to the data storage instruction;
previewing, maintaining or deleting the result data according to the data previewing, data maintaining or data deleting instructions;
according to the instruction of data export, exporting corresponding result data, and transmitting the result data to a data user in a file transmission mode;
wherein the method further comprises:
the SDK is used as a mode of storing a result data set, so that access difference of heterogeneous data storage addresses in the AI prediction process is shielded for modeling personnel, and an access right control mechanism is provided; the SDK is provided with a result data set storage interface and a result data set export interface;
when a user initiates a request, verifying the input user name and password; after passing the verification, inquiring whether the user number of the user is stored in a stored user login information table; if not, generating a user number of the user and recording the user number in the user login information table; the user login information table stores user numbers corresponding to each user, result data with management rights and result data with access rights; the user management module is used for checking the current user login or integrating the third party single sign-on, and automatically creating a unique number corresponding to the user and recording the unique number in a user table when a new third party user passes the check and logs in; after successful login, storing the login information of the current user, and checking that the current user is logged in every subsequent request operation for the data set;
managing and checking rights, wherein the rights are divided into management rights and access rights, and after modeling personnel establishes a model and executes a prediction task and obtains result data, the modeling personnel has the management rights of the result data and sets the access rights; wherein, the access authority comprises: designating project disclosure, which is accessible to users within the project, and not accessible to other users; fully disclosed, all can be accessed;
storing result data set management data, a data set internal storage mode and data set authority assignment public personnel mapping and user verification information by utilizing a metadata set; wherein the information for result dataset management includes: the method comprises the steps of a result data set name, a result data set description, an AI research and development task, an AI modeling project, a modeling scheme, a creator, creation time, a management mode, a result data set storage days, a result data set storage size and a disclosure mode; the internal storage mode of the data set comprises the following steps: a data set storage configuration; the dataset rights assignment disclosure person mapping includes: designating a public number; the user verification information includes: the user number, user login name, user password verification and result data set default specified public numbering list.
8. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of claim 7 when executing the computer program.
9. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program which, when executed by a processor, implements the method of claim 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010201917.3A CN111415014B (en) | 2020-03-20 | 2020-03-20 | Model prediction result data management system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010201917.3A CN111415014B (en) | 2020-03-20 | 2020-03-20 | Model prediction result data management system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111415014A CN111415014A (en) | 2020-07-14 |
| CN111415014B true CN111415014B (en) | 2023-10-20 |
Family
ID=71494566
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010201917.3A Active CN111415014B (en) | 2020-03-20 | 2020-03-20 | Model prediction result data management system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111415014B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112380223A (en) * | 2020-12-11 | 2021-02-19 | 深圳供电局有限公司 | Information table cross-region automatic maintenance system based on model unification |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105956029A (en) * | 2016-04-25 | 2016-09-21 | 吉林大学 | Data access method under hybrid storage architecture |
| WO2018107610A1 (en) * | 2016-12-15 | 2018-06-21 | 平安科技（深圳）有限公司 | Service data processing method, system and device, and computer-readable storage medium |
| CN110297869A (en) * | 2019-05-30 | 2019-10-01 | 北京百度网讯科技有限公司 | A kind of AI Data Warehouse Platform and operating method |
| CN110321112A (en) * | 2019-07-02 | 2019-10-11 | 北京百度网讯科技有限公司 | AI ability research/development platform and data processing method |
| CN110502547A (en) * | 2019-09-04 | 2019-11-26 | 河南宏博测控技术有限公司 | Internet of Things detection data management system and Internet of Things detection data management method |
| CN110826398A (en) * | 2019-09-23 | 2020-02-21 | 上海意略明数字科技股份有限公司 | Intelligent image recognition big data acquisition and analysis system and application method |
-
2020
- 2020-03-20 CN CN202010201917.3A patent/CN111415014B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105956029A (en) * | 2016-04-25 | 2016-09-21 | 吉林大学 | Data access method under hybrid storage architecture |
| WO2018107610A1 (en) * | 2016-12-15 | 2018-06-21 | 平安科技（深圳）有限公司 | Service data processing method, system and device, and computer-readable storage medium |
| CN110297869A (en) * | 2019-05-30 | 2019-10-01 | 北京百度网讯科技有限公司 | A kind of AI Data Warehouse Platform and operating method |
| CN110321112A (en) * | 2019-07-02 | 2019-10-11 | 北京百度网讯科技有限公司 | AI ability research/development platform and data processing method |
| CN110502547A (en) * | 2019-09-04 | 2019-11-26 | 河南宏博测控技术有限公司 | Internet of Things detection data management system and Internet of Things detection data management method |
| CN110826398A (en) * | 2019-09-23 | 2020-02-21 | 上海意略明数字科技股份有限公司 | Intelligent image recognition big data acquisition and analysis system and application method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111415014A (en) | 2020-07-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105653368B (en) | System and method for private cloud computing | |
| KR101872748B1 (en) | Managing changes to collections of data | |
| WO2008046218A1 (en) | Method and apparatus for creating a configurable browser-based forms application | |
| US20160239272A1 (en) | Method and device for the automated production and provision of at least one software application | |
| US20150067503A1 (en) | System and method for virtual assistants with agent store | |
| CN110555072A (en) | Data access method, device, equipment and medium | |
| CN107534671A (en) | The polymerization of Distributed Services entity and association is with combining | |
| US20200250587A1 (en) | Framework for multi-tenant data science experiments at-scale | |
| US11907262B2 (en) | System and method for data pruning via dynamic partition management | |
| CN107710202A (en) | With the primary document of cloud of old Tool integration | |
| CN111415014B (en) | Model prediction result data management system and method | |
| US20170116169A1 (en) | Managing data relationships of customizable forms | |
| CN115202729A (en) | Container service-based mirror image generation method, device, equipment and medium | |
| TW200302420A (en) | Visualization processing system, visualization processing method, and visualization processing program product | |
| CN116756711A (en) | Data processing method, device, equipment and medium | |
| JPH1153243A (en) | Computer system and medium recorded with processing program thereof | |
| US20220198304A1 (en) | Providing explainable machine learning model results using distributed ledgers | |
| KR101818745B1 (en) | Drawing and product management system, and Controlling method therefor | |
| CN110308926A (en) | Software information methods of exhibiting, device, computer installation and computer storage medium | |
| TW201725542A (en) | Form management system and method | |
| CN101425011A (en) | Graphic scene case script editing system and method | |
| JP2006260074A (en) | Cad data management device | |
| Woodall et al. | A cloud-based system for scraping data from amazon product reviews at scale | |
| CN109634606A (en) | A kind of method and device of defined function menu | |
| JP2005258591A (en) | Database access control system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220908 Address after: 25 Financial Street, Xicheng District, Beijing 100033 Applicant after: CHINA CONSTRUCTION BANK Corp. Address before: 25 Financial Street, Xicheng District, Beijing 100033 Applicant before: CHINA CONSTRUCTION BANK Corp. Applicant before: Jianxin Financial Science and Technology Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |