CN112817635B - Model processing method and data processing system - Google Patents
Model processing method and data processing system Download PDFInfo
- Publication number
- CN112817635B CN112817635B CN202110130264.9A CN202110130264A CN112817635B CN 112817635 B CN112817635 B CN 112817635B CN 202110130264 A CN202110130264 A CN 202110130264A CN 112817635 B CN112817635 B CN 112817635B
- Authority
- CN
- China
- Prior art keywords
- model
- port
- service
- file
- publishing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/70—Software maintenance or management
- G06F8/71—Version control; Configuration management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44505—Configuring for program initiating, e.g. using registry, configuration files
- G06F9/4451—User profiles; Roaming
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Stored Programmes (AREA)
Abstract
The invention provides a model processing method and a data processing system, which relate to the field of big data processing, and the model processing method comprises the following steps: obtaining a model to be issued and a target port for issuing the model; and issuing the model to be issued as a model service based on the target port issued by the model. The method and the device for releasing the model service have the advantages that the model releasing is simple and convenient to achieve, and the model releasing efficiency is improved.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a model processing method and a data processing system.
Background
With the improvement of social informatization and intelligence level, the method for training the business model by using the big data system and realizing the intelligent processing of big data business by using the trained business model also gradually becomes a general means of big data industry. Usually, the model can be used by the user after being deployed, however, the existing model deployment is complex in implementation process and has high requirements on resources and the like of a data analysis processing system.
Disclosure of Invention
The embodiment of the invention provides a model processing method and a data processing system, which can solve the problems that the model issuing process is complex and the requirement on the data processing system is high in the existing scheme.
In order to solve the above technical problem, the present invention provides a model processing method, including:
obtaining a model to be issued and a target port for issuing the model;
and issuing the model to be issued as a model service based on the target port issued by the model.
Optionally, in the model processing method, the step of obtaining the model to be published and the target port for model publishing includes:
acquiring the running state of the released model;
determining a currently used model issuing port according to the running state of the issued model;
and determining the target port issued by the model according to the currently used model issuing port and the port configuration information.
Optionally, in the model processing method, the step of obtaining the model to be released includes:
displaying a template model list;
determining a target model from the template model list based on a first selection operation;
and importing the first model file into the selected target model to obtain the model to be issued.
Optionally, in the model processing method, the method further includes:
acquiring configured model description information;
and generating a template model based on the model description information.
Optionally, in the above model processing method, before importing the first model file into the selected target model, the method includes,
determining whether the first model file type is matched with a file type supported by the target model;
and if so, importing the first model file into the selected target model.
Optionally, in the model processing method, after the step of importing the first model file into the selected target model, the method further includes:
acquiring an identifier of the first model file;
detecting a second model file corresponding to the identifier in the target model according to the identifier to obtain a detection result;
and determining the version number of the model to be issued based on the detection result.
Optionally, in the model processing method, the step of determining the target port published by the model according to the currently used port published by the model and the port configuration information includes:
displaying at least one candidate issuing port determined according to the currently used model issuing port and the port configuration information;
determining at least one target port from the at least one candidate publishing port based on a selection operation;
the step of issuing the model to be issued as the model service based on the target port of the model issue includes:
and publishing the model to be published as at least one model service based on the at least one target port.
Optionally, in the model processing method, after the step of issuing the model to be issued as the model service based on the target port issued by the model, the method further includes:
obtaining meta-information of the model service,
and configuring the calling address of the model service based on the meta-information of the model service.
Optionally, in the model processing method, the meta information of the model service includes model service version information, and the step of configuring the calling address of the model service based on the meta information of the model service includes:
and configuring a calling address for at least two model services with the same model service version information.
Optionally, in the model processing method, after the step of issuing the model to be issued as the model service based on the target port issued by the model, the method further includes:
responding to a second selection operation, and determining that the release model indicated by the second selection operation is a model to be tested;
determining a model service corresponding to the model to be tested based on the model to be tested;
and running the model service for testing.
Optionally, in the model processing method, after the step of issuing the model to be issued as the model service based on the target port issued by the model, the method further includes:
responding to a third selection operation, and determining that the release model indicated by the third selection operation is a model to be evaluated;
determining a model service corresponding to the model to be evaluated based on the model to be evaluated;
determining an evaluation method based on the type of the model to be evaluated;
and running the model service for evaluation based on the evaluation method.
Optionally, the model processing method further includes:
responding to a fourth selection operation, and determining a model indicated by the fourth selection operation as a model to be migrated;
generating a preset file based on the model description information of the model to be migrated;
and carrying out model migration on the model to be migrated based on the model file of the model to be migrated and the preset file.
The present invention provides a data processing system, comprising:
the model management module is used for acquiring a model to be issued and a target port for issuing the model;
and the model service module is used for issuing the model to be issued as the model service based on the target port issued by the model.
Optionally, the data processing system further includes:
the model monitoring module is used for acquiring the running state of the published model; determining a currently used model issuing port according to the running state of the issued model;
the model management module is also used for receiving the currently used model issuing port sent by the model monitoring module and determining the target port of model issuing according to the currently used model issuing port and the port configuration information.
Optionally, in the data processing system, the model management module further includes:
the first display submodule is used for displaying a template model list;
a first determining submodule for determining a target model from the template model list based on a first selection operation;
and the first generation submodule is used for importing the first model file into the selected target model to obtain the model to be issued.
Optionally, in the data processing system, the model management module further includes:
the first obtaining submodule is used for obtaining the configured model description information;
and the second generation submodule is used for generating a template model based on the model description information.
Optionally, in the data processing system, the model management module further includes:
the verification submodule is used for determining whether the type of the first model file is matched with the file type supported by the target model before the first model file is imported into the selected target model; and if so, importing the first model file into the selected target model.
Optionally, in the data processing system, the model management module further includes:
the second obtaining submodule is used for obtaining the identifier of the first model file;
the detection submodule is used for detecting a second model file corresponding to the identifier in the target model according to the identifier to obtain a detection result;
and the second determining submodule is used for determining the version number of the model to be issued based on the detection result.
Optionally, in the data processing system, the model management module further includes:
the second display submodule is used for displaying at least one candidate issuing port determined according to the currently used model issuing port and the port configuration information;
a selection submodule for determining at least one target port from the at least one candidate publication port based on a selection operation;
the model service module further comprises:
and the issuing sub-module is used for issuing the model to be issued to at least one model service based on the at least one target port.
Optionally, the data processing system further includes:
an obtaining module for obtaining meta-information of the model service,
and the configuration module is used for configuring the calling address of the model service based on the meta-information of the model service.
Optionally, in the data processing system, the meta information of the model service includes model service version information, and the configuration module is further configured to:
and configuring a calling address for at least two model services with the same model service version information.
Optionally, the data processing system further includes:
the third determining module is used for responding to a second selecting operation and determining the issuing model indicated by the second selecting operation as a model to be tested;
the first generation module is used for determining the model service corresponding to the model to be tested based on the model to be tested;
and the test module is used for running the model service to test.
Optionally, the data processing system further includes:
a fourth determining module, configured to determine, in response to a third selection operation, that the publishing model indicated by the third selection operation is the model to be evaluated
A fifth determining module, configured to determine, based on the model to be evaluated, a model service corresponding to the model to be evaluated;
a sixth determining module, configured to determine an evaluation method based on the type of the model to be evaluated;
and the evaluation module is used for operating the model service for evaluation based on the evaluation method.
Optionally, the data processing system further includes:
a seventh determining module, configured to determine, in response to a fourth selecting operation, that a model indicated by the fourth selecting operation is a model to be migrated;
the second generation module is used for generating a preset file based on the model description information of the model to be migrated;
and the migration module is used for carrying out model migration on the model to be migrated based on the model file of the model to be migrated and the preset file.
The invention also provides a data processing system comprising a processor, a memory and a computer program stored on the memory and executable on the processor, the computer program, when executed by the processor, implementing the steps of the model processing method as described above.
The present invention also provides a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the above-mentioned model processing method.
The embodiment of the invention simplifies the realization process of model release by providing the model release port for the model to be released, reduces the requirement on the data processing system, improves the model release efficiency and can realize the suspension and restart of the model service at any time.
The embodiment of the invention can also monitor the release model in real time and know the running state and the performance of the model, thereby monitoring the released model release port and further accurately knowing the available model release port.
The embodiment of the invention can also release the model corresponding to the same model file into a plurality of model services, and different versions of the same model can be independently released into one model service, thereby realizing load balancing.
The embodiment of the invention also supports one-key migration of the model from the test environment to the production environment, leads the data of the model out of the test environment and then into the production environment, does not need a user to set complex parameters, and reduces the operation complexity.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the description of the embodiments of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
FIG. 1 is a flow chart of a model processing method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a model publishing interface provided by an embodiment of the invention;
FIG. 3 is a block diagram of a data processing system provided by an embodiment of the present invention;
fig. 4 is a schematic diagram of a model service publishing architecture according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a flowchart of a model processing method provided by an embodiment of the present invention, where the model processing method is applied to a data processing system, and as shown in fig. 1, the model processing method includes the following steps:
And 102, issuing the model to be issued as a model service based on the target port of the model issuing.
The above steps are specifically explained in conjunction.
Optionally, the step 101 of obtaining the model to be published, and the target port for publishing the model includes: displaying a template model list; and determining a target model from the template model list based on a first selection operation, and importing a first model file into the selected target model to obtain the model to be issued.
Wherein, the template model includes at least: model description information such as template model type, model name, characteristic variable configuration, target characteristic variable configuration and the like.
The imported first model file can be downloaded from other systems by users or use the existing model files in the system.
The model management interface displays a plurality of template models, a user can select a target model from the template models displayed by the model management interface, and in response to an importing operation, a first model file is imported into the target model to generate a model to be published. The model to be issued comprises a first model file.
Optionally, before the step of displaying the template model list, the method further includes:
acquiring configured model description information;
generating a template model based on the model description information
In particular, a user may create a template model in a data processing system. Specifically, a model management interface is displayed, a template model creation interface is displayed in response to a creation operation of a user on the model management interface, and model description information of the template model is input on the template model creation interface, so that the template model is generated according to the model description information.
Wherein the model description information includes but is not limited to at least one of: the method comprises the following steps of template model type, model name, characteristic variable configuration and target characteristic variable configuration. Wherein, the configuration of the characteristic variables comprises the following steps: the field name entered, the description of the field, the field type, whether padding of null values is allowed, whether padding of missing values is allowed, etc. The target feature variable configuration includes a name, a description of a field, a field type, and the like. The feature variable configuration and the target feature variable configuration support configuration in a table form and also support configuration in a CSV form.
It should be noted that the template model that is just newly created is an empty model, that is, there is no model file in the newly created template model, and the model file only includes model description information.
Furthermore, the user can also edit, delete, export and the like the template model.
Alternatively to this, the first and second parts may,
before importing the first model file into the selected object model, the method further comprises:
determining whether the first model file type is matched with a file type supported by the target model;
and if so, importing the first model file into the selected target model.
Specifically, before the first model file is imported, the uploaded model file is verified according to the type of the selected template model, and if the type of the uploaded model file is matched with the type of the selected template model, the model file can be uploaded to obtain the model to be released. . And if the uploaded model file type is not matched with the type of the selected first model, the model file is failed to be uploaded. For example, when the template model type is pmml, the suffix of the uploaded model file must be pmml, otherwise the model file upload fails.
Optionally, before importing the first model file into the selected target model, the method further includes:
acquiring an identifier of the first model file;
detecting a second model file corresponding to the identifier in the target model according to the identifier to obtain a detection result;
and determining the version number of the model to be issued based on the detection result.
In addition, before uploading the model files, the identifiers of the model files are compared, and the fact that no repeated model files exist under the same model is guaranteed. For example, the identification may be an MD5 value, a hash value, or the like. Specifically, a first model file identifier is obtained, the identifier of the first model file is compared with an existing identifier corresponding to a model file in a target model, and if the identifier corresponding to the identifier of the first model file is detected, it is determined that the first model file is imported into the target model; if the identifier corresponding to the identifier of the model file is not detected, it is determined that the first model file is not imported into the target model, so that the model to be released obtained after the first model file is imported into the target model is a new version, and the version number of the model to be released is automatically + 1. By the method, the version number of the generated model to be released is automatically +1 when a new model file is uploaded every time, and the setting by a user is not needed.
For example, if the uploaded model file is the 7 th model file of the first model, the model service that is published using the 7 th model file is equivalent to the 7 th version of the second model; meanwhile, the existing version can be used for model service release (namely, the original model file can also be released).
Optionally, the step of obtaining a target port to be published for model publishing includes:
acquiring the running state of the released model;
determining a currently used model issuing port according to the running state of the issued model;
and determining the target port issued by the model according to the currently used model issuing port and the port configuration information.
The port configuration information at least comprises a port range used for model release.
Specifically, the currently used model publishing port is obtained by monitoring the running state of the published model. And obtaining the model issuing port range from the configuration file, thereby determining the target port of the model issuing according to the condition of the currently used model issuing port and the corresponding model issuing port range.
The obtaining of the currently used model publishing port may be: acquiring all in-use model service ports from a data processing system; alternatively, based on the machine selected by the user, the currently in-use model service port corresponding to the machine may be selected.
Optionally, the step of determining the target port published by the model according to the currently used port published by the model and the port configuration information includes:
displaying at least one candidate issuing port determined according to the currently used model issuing port and the port configuration information;
determining at least one target port from the at least one candidate publishing port based on a selection operation;
optionally, the step 102 of issuing the model to be issued as the model service based on the target port issued by the model includes: and publishing the model to be published as at least one model service based on the at least one target port.
Specifically, as shown in fig. 2, a user selects a machine for publication in the model publication interface, for example, the user selects 172.20.3.83 for publication, and the port configuration information preset by the machine and the current port usage determined according to the running state of the published model are used together to determine the port for publication. Specifically, based on the machine 172.20.3.83 selected by the user, the model publishing port range corresponding to the machine is obtained from the preset port configuration information, so that at least one publishing port is determined according to the determined current port usage and the model publishing port range corresponding to the machine.
Based on the release mode, the model of the same version can be released into a plurality of model services, when the data volume needing to be processed is large, the plurality of model services can be called to be processed simultaneously, load balance is achieved, real-time processing can be achieved, and data processing efficiency is improved.
Further, the allocation may be performed in a round robin manner when load balancing is implemented. That is, when data is allocated to a plurality of model services, the data is forwarded through the gateway, and specifically, the forwarding may be: firstly, the gateway receives a model service calling request, and then the gateway acquires the issued model service from the data management system, so that the data is distributed to the corresponding model service, and the load balance is realized.
Optionally, the step 102 of publishing the model to be published as the model service based on the target port published by the model includes: obtaining model release information corresponding to a model to be released; and issuing model service based on the model issuing information and the target port. Further, a model service address corresponding to the model service is obtained.
The model issuing information comprises model description information, model file information and a model service name. The model description information is the description information of the template model corresponding to the model to be issued; the model file information is information of the first model file, for example, a storage address of the first model file, a name of the first model file, and the like.
In addition, the data processing system also supports gray scale distribution, which is a distribution mode capable of smoothly transiting between black and white. The gray release is a type of incremental release, a new version model service is deployed simultaneously under the condition that the original version model service is available, the new version model service adopting the gray release does not occupy all data, but distributes data respectively in a certain mode together with the original version model service, and the effect comparison is carried out, namely, the method is a shunting method which is used by the same model together with the model services of different versions.
Optionally, after the step 102 of publishing the model to be published as the model service based on the target port published by the model, the method further includes:
obtaining meta-information of the model service,
and configuring the calling address of the model service based on the meta-information of the model service. The meta-information of the model service includes model service version information, and the step of configuring the calling address of the model service based on the meta-information of the model service includes: and configuring a calling address for at least two model services with the same model service version information.
Specifically, one model service corresponds to one model file, the version information of the model service corresponds to the version information of the model file, and the calling address of the model service can be generated through the version information of the model file corresponding to the model service. The plurality of model services published by the same model file use the same calling address, namely, the calling address of one model service is exposed to the outside by the plurality of model services published by the same model file.
The embodiment of the invention can realize the release of the model by providing the model release port for the model to be released without using a container, thereby improving the release efficiency of the model and realizing the suspension and restart of the model service at any time.
The embodiment of the invention can also release the same model as a plurality of model services, and different versions of the same model can be independently released as one model service, thereby realizing load balancing.
Optionally, the model processing method provided in the embodiment of the present invention supports specific testing of the released model, and the method further includes:
responding to a second selection operation, and determining that the release model indicated by the second selection operation is a model to be tested;
determining a model service corresponding to the model to be tested based on the model to be tested;
and running the model service for testing.
Specifically, a model to be tested is selected from a data processing system, a corresponding model service is determined based on model version information of the model to be tested selected by a user, then features to be input filled according to input features filled during model creation are sent to the determined model service, and the model service is operated for testing. Furthermore, after prediction and return, the model file can be displayed on the front-end interface according to the configuration name of the output result, namely after the test is executed by clicking, if the data has no problem, the model file can use the data to predict and display the prediction result on the front-end interface. And if the test result has a problem, displaying corresponding error information.
The embodiment of the invention can also monitor the published model service in real time and know the running state and the performance of the model, thereby monitoring the released model publishing port and further accurately knowing the available model publishing port. The real-time monitoring of published model services further comprises: monitoring the calling frequency tps, the number of successful predictions, the average calling time, and the like.
Optionally, the model processing method provided in the embodiment of the present invention supports evaluation of a published model, and specifically, the method further includes:
responding to a third selection operation, and determining that the release model indicated by the third selection operation is a model to be evaluated;
determining a model service corresponding to the model to be evaluated based on the model to be evaluated;
determining an evaluation method based on the type of the model to be evaluated;
and running the model service for evaluation based on the evaluation method.
Specifically, a model to be evaluated is selected from a data processing system, a corresponding model service is determined based on model version information of the model to be evaluated selected by a user, wherein an evaluation file comprises data to be predicted and an actual result, a corresponding evaluation method is selected according to the selected type of the model to be evaluated before the data processing system evaluates, and then corresponding evaluation is performed based on the selected evaluation method, such as classification model evaluation auc, recall rate and the like, and a regression model evaluates root mean square error, r-squared and the like. Further, before evaluation, an evaluation file for evaluation needs to be uploaded so as to run the model service for evaluation. Wherein the evaluation file contains data to make the prediction and the actual result.
Optionally, the model processing method provided in the embodiment of the present invention supports migration of a model, and the method further includes:
responding to a fourth selection operation, and determining a model indicated by the fourth selection operation as a model to be migrated;
generating a preset file based on the model description information of the to-be-migrated model;
and carrying out model migration on the model to be migrated based on the model file of the model to be migrated and the preset file.
Specifically, when the template model is created, the model description information of the template model is stored in the database. In response to the model migration, model description information corresponding to the model is obtained from the database, and the information is exported to generate a preset file, for example, the preset file may be a json format file, and a model file (i.e., an ontology of the model) is exported at the same time. The json file and the model file can be directly imported into a production environment by exporting the description information of the model into the json file, and because the json file contains the description information of the model, a user does not need to set model parameters again, and the data processing system can directly acquire the parameters of the model by analyzing the json file, so that one-key migration of the model in different environments is realized, and the operation complexity is reduced. Model migration is the migration of a deployed model from one environment to another, for example, from a test environment to a production environment for deployment.
The embodiment of the invention also supports one-key migration of the model from the test environment to the production environment, leads the data of the model service out of the test environment and then into the production environment, does not need a user to set complex parameters, and reduces the operation complexity.
Based on the model processing method provided in the above embodiment, an embodiment of the present invention further provides a data processing system for implementing the method, referring to fig. 3, a data processing system 300 provided in an embodiment of the present invention includes:
the model management module 301 is configured to obtain a model to be published and a target port for publishing the model;
a model service module 302, configured to publish the model to be published as a model service based on the target port published by the model.
The model management module includes port configuration information and model information.
Optionally, the data processing system further includes:
the model monitoring module is used for acquiring the running state of the published model; determining a currently used model issuing port according to the running state of the issued model;
the model management module is also used for receiving the currently used model issuing port sent by the model monitoring module and determining the target port of model issuing according to the currently used model issuing port and the port configuration information.
An exemplary model service publishing architecture in a data processing system is shown in FIG. 4. The model management module comprises preset port configuration information and a template model, wherein no model file exists in the simulation model temporarily, and the port configuration information comprises a model issuing port range. The model management module comprises management operations on the template model and the model file imported into the template model, the management of the template model comprises creation, deletion and other operations, and the management of the model file comprises uploading and storing of the file and other operations. And the model monitoring module is used for monitoring the running state of the published model and storing the currently used model publishing port. And the model service module is used for publishing the model file as the model service and sending the monitoring content of the model service to the model monitoring module. When model service is released, the model management module acquires a currently used model release port from the model monitoring module, and allocates a model release port for a model file to be released currently according to a preset model release port range and the currently used model release port, and the model service module releases the model file as the model service based on the model release port. And the model service module sends the running information of the model service to the monitoring module. The monitoring module determines the port condition currently used based on the monitored operation information, and sends the port condition currently used to the model management module when the model management module sends a request.
A user can select one machine in a cluster to publish model services after clicking publishing services on a model publishing interface, in the publishing process, a model management module firstly obtains ports with created model services from a model monitoring module, so that all the ports capable of publishing new model services are screened out based on the model publishing port range of the machine stored by the model management module and all the ports with created model services, then a random port is selected from the ports, the port information and the model publishing information of a model to be published are sent to a model service module, and the model service module publishes the model as the model services based on the port information and the model publishing information of the model to be published and generates a corresponding model service address. The model publishing information of the model to be published comprises: address of model file, model name, model service name, etc. The address of the model file is the position where the model file is stored, and connection is established when the model is read to obtain the model file.
Wherein, the model management module obtaining the port that has created the model service from the model monitoring module may be: acquiring all in-use model service ports from a model monitoring module; alternatively, based on the selected machine, the currently in-use model service port corresponding to the machine may be selected.
Optionally, the model management module further includes:
the first display submodule is used for displaying a template model list;
a first determining submodule for determining a target model from the template model list based on a first selection operation;
and the first generation submodule is used for importing the first model file into the selected target model to obtain the model to be issued.
Optionally, the model management module further includes:
the first obtaining submodule is used for obtaining the configured model description information;
and the second generation submodule is used for generating a template model based on the model description information.
Optionally, the model management module further includes:
the verification submodule is used for determining whether the type of the first model file is matched with the file type supported by the target model before the first model file is imported into the selected target model; and if so, importing the first model file into the selected target model.
Optionally, the model management module further includes:
the second obtaining submodule is used for obtaining the identifier of the first model file;
the detection submodule is used for detecting a second model file corresponding to the identifier in the target model according to the identifier to obtain a detection result;
and the second determining submodule is used for determining the version number of the model to be issued based on the detection result.
Optionally, the model management module further includes:
the second display submodule is used for displaying at least one candidate issuing port determined according to the currently used model issuing port and the port configuration information;
a selection submodule for determining at least one target port from the at least one candidate publication port based on a selection operation;
the model service module further comprises:
and the issuing sub-module is used for issuing the model to be issued to at least one model service based on the at least one target port.
Optionally, the data processing system further includes:
an obtaining module for obtaining meta-information of the model service,
and the configuration module is used for configuring the calling address of the model service based on the meta-information of the model service.
Optionally, the meta information of the model service includes model service version information, and the configuration module is further configured to:
and configuring a calling address for at least two model services with the same model service version information.
Optionally, the data processing system further includes:
the third determining module is used for responding to a second selecting operation and determining the issuing model indicated by the second selecting operation as a model to be tested;
the first generation module is used for determining the model service corresponding to the model to be tested based on the model to be tested;
and the test module is used for running the model service to test.
Optionally, the data processing system further includes:
a fourth determining module, configured to determine, in response to a third selection operation, that the publishing model indicated by the third selection operation is the model to be evaluated
A fifth determining module, configured to determine, based on the model to be evaluated, a model service corresponding to the model to be evaluated;
a sixth determining module, configured to determine an evaluation method based on the type of the model to be evaluated;
and the evaluation module is used for operating the model service for evaluation based on the evaluation method.
Optionally, the data processing system further includes:
a seventh determining module, configured to determine, in response to a fourth selecting operation, that a model indicated by the fourth selecting operation is a model to be migrated;
the second generation module is used for generating a preset file based on the model description information of the model to be migrated;
and the migration module is used for carrying out model migration on the model to be migrated based on the model file of the model to be migrated and the preset file.
Embodiments of the present invention provide a data processing system, which includes a processor, a memory, and a computer program stored on the memory and executable on the processor, and when executed by the processor, the computer program implements the steps of the model processing method according to the above embodiments.
An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the model processing method according to the above embodiment.
The embodiment of the present invention further provides a readable storage medium, where a computer program is stored on the readable storage medium, and when the computer program is executed by a processor, the computer program implements each process of the data processing method embodiment, and can achieve the same technical effect, and in order to avoid repetition, details are not repeated here. The computer-readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (22)
1. A method of model processing, the method comprising:
obtaining a model to be issued and a target port for issuing the model;
issuing the model to be issued as a model service based on the target port of the model issue;
after the step of issuing the model to be issued as the model service based on the target port of the model issue, the method further includes:
obtaining meta-information of the model service,
configuring a calling address of the model service based on the meta-information of the model service;
the model publishing comprises: selecting a machine for publishing by a user on a model publishing interface, and acquiring a model publishing port range corresponding to the machine from preset port configuration information, so as to determine at least one publishing port according to the determined current port use condition and the model publishing port range corresponding to the machine; and sending the publishing port information and the model publishing information of the model to be published to a model service module, wherein the model service module publishes the model as the model service based on the publishing port information and the model publishing information of the model to be published.
2. The model processing method according to claim 1, wherein the step of obtaining the model to be released comprises:
displaying a template model list;
determining a target model from the template model list based on a first selection operation;
and importing the first model file into the selected target model to obtain the model to be issued.
3. The model processing method of claim 2, characterized in that the method further comprises:
acquiring configured model description information;
and generating a template model based on the model description information.
4. The model processing method according to claim 2, characterized in that before importing a first model file into the selected object model, the method comprises:
determining whether the first model file type is matched with a file type supported by the target model;
and if so, importing the first model file into the selected target model.
5. The model processing method of claim 4, wherein said step of importing a first model file into the selected target model is followed by the method further comprising:
acquiring an identifier of the first model file;
detecting a second model file corresponding to the identifier in the target model according to the identifier to obtain a detection result;
and determining the version number of the model to be issued based on the detection result.
6. The model processing method of claim 1, wherein the step of determining the target port for model publication based on the currently used model publication port and port configuration information comprises:
displaying at least one candidate issuing port determined according to the currently used model issuing port and the port configuration information;
determining at least one target port from the at least one candidate publishing port based on a selection operation;
the step of issuing the model to be issued as the model service based on the target port of the model issue includes:
and publishing the model to be published as at least one model service based on the at least one target port.
7. The model processing method according to claim 1, wherein the meta information of the model service includes model service version information, and the step of configuring the calling address of the model service based on the meta information of the model service includes:
and configuring a calling address for at least two model services with the same model service version information.
8. The model processing method of claim 2, wherein after the step of publishing the model to be published as a model service based on the target port of the model publication, the method further comprises:
responding to a second selection operation, and determining that the release model indicated by the second selection operation is a model to be tested;
determining a model service corresponding to the model to be tested based on the model to be tested;
and running the model service for testing.
9. The model processing method of claim 2, wherein after the step of publishing the model to be published as a model service based on the target port of the model publication, the method further comprises:
responding to a third selection operation, and determining that the release model indicated by the third selection operation is a model to be evaluated;
determining a model service corresponding to the model to be evaluated based on the model to be evaluated;
determining an evaluation method based on the type of the model to be evaluated;
and running the model service for evaluation based on the evaluation method.
10. The model processing method of claim 2, further comprising:
responding to a fourth selection operation, and determining a model indicated by the fourth selection operation as a model to be migrated;
generating a preset file based on the model description information of the model to be migrated;
and carrying out model migration on the model to be migrated based on the model file of the model to be migrated and the preset file.
11. A data processing system, characterized in that the data processing system comprises:
the model management module is used for acquiring a model to be issued and a target port for issuing the model;
the model service module is used for issuing the model to be issued as model service based on the target port issued by the model;
the data processing system further comprises:
an obtaining module for obtaining meta-information of the model service,
the configuration module is used for configuring the calling address of the model service based on the meta-information of the model service;
the model publishing comprises: selecting a machine for publishing by a user on a model publishing interface, and acquiring a model publishing port range corresponding to the machine from preset port configuration information, so as to determine at least one publishing port according to the determined current port use condition and the model publishing port range corresponding to the machine; and sending the publishing port information and the model publishing information of the model to be published to a model service module, wherein the model service module publishes the model as the model service based on the publishing port information and the model publishing information of the model to be published.
12. The data processing system of claim 11, wherein the model management module further comprises:
the first display submodule is used for displaying a template model list;
a first determining submodule for determining a target model from the template model list based on a first selection operation;
and the first generation submodule is used for importing the first model file into the selected target model to obtain the model to be issued.
13. The data processing system of claim 12, wherein the model management module further comprises:
the first obtaining submodule is used for obtaining the configured model description information;
and the second generation submodule is used for generating a template model based on the model description information.
14. The data processing system of claim 12, wherein the model management module further comprises:
the verification submodule is used for determining whether the type of the first model file is matched with the file type supported by the target model before the first model file is imported into the selected target model; and if so, importing the first model file into the selected target model.
15. The data processing system of claim 14, wherein the model management module further comprises:
the second obtaining submodule is used for obtaining the identifier of the first model file;
the detection submodule is used for detecting a second model file corresponding to the identifier in the target model according to the identifier to obtain a detection result;
and the second determining submodule is used for determining the version number of the model to be issued based on the detection result.
16. The data processing system of claim 11, wherein the model management module further comprises:
the second display submodule is used for displaying at least one candidate issuing port determined according to the currently used model issuing port and the port configuration information;
a selection submodule for determining at least one target port from the at least one candidate publication port based on a selection operation;
the model service module further comprises:
and the issuing sub-module is used for issuing the model to be issued to at least one model service based on the at least one target port.
17. The data processing system of claim 11, wherein the meta-information for the model service includes model service version information, the configuration module further configured to:
and configuring a calling address for at least two model services with the same model service version information.
18. The data processing system of claim 12, further comprising:
the third determining module is used for responding to a second selecting operation and determining the issuing model indicated by the second selecting operation as a model to be tested;
the first generation module is used for determining the model service corresponding to the model to be tested based on the model to be tested;
and the test module is used for running the model service to test.
19. The data processing system of claim 12, further comprising:
a fourth determining module, configured to determine, in response to a third selection operation, that the publishing model indicated by the third selection operation is the model to be evaluated
A fifth determining module, configured to determine, based on the model to be evaluated, a model service corresponding to the model to be evaluated;
a sixth determining module, configured to determine an evaluation method based on the type of the model to be evaluated;
and the evaluation module is used for operating the model service for evaluation based on the evaluation method.
20. The data processing system of claim 12, further comprising:
a seventh determining module, configured to determine, in response to a fourth selecting operation, that a model indicated by the fourth selecting operation is a model to be migrated;
the second generation module is used for generating a preset file based on the model description information of the model to be migrated;
and the migration module is used for carrying out model migration on the model to be migrated based on the model file of the model to be migrated and the preset file.
21. A data processing system comprising a processor, a memory and a computer program stored on the memory and executable on the processor, the computer program, when executed by the processor, implementing the steps of the model processing method as claimed in any one of claims 1 to 10.
22. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, carries out the steps of the model processing method according to one of claims 1 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110130264.9A CN112817635B (en) | 2021-01-29 | 2021-01-29 | Model processing method and data processing system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110130264.9A CN112817635B (en) | 2021-01-29 | 2021-01-29 | Model processing method and data processing system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112817635A CN112817635A (en) | 2021-05-18 |
| CN112817635B true CN112817635B (en) | 2022-02-08 |
Family
ID=75860354
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110130264.9A Active CN112817635B (en) | 2021-01-29 | 2021-01-29 | Model processing method and data processing system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112817635B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109088905A (en) * | 2017-06-14 | 2018-12-25 | 浙江贵仁信息科技股份有限公司 | A kind of mathematical model service system and mathematical model management method based on cloud computing |
| CN109299178A (en) * | 2018-09-30 | 2019-02-01 | 北京九章云极科技有限公司 | A kind of application method and data analysis system |
| CN110688539A (en) * | 2019-09-30 | 2020-01-14 | 北京九章云极科技有限公司 | Model management system and method |
| CN111580926A (en) * | 2020-03-25 | 2020-08-25 | 中国平安财产保险股份有限公司 | Model publishing method, model deploying method, model publishing device, model deploying device, model publishing equipment and storage medium |
| CN112015519A (en) * | 2020-08-28 | 2020-12-01 | 江苏银承网络科技股份有限公司 | Model online deployment method and device |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103309985B (en) * | 2013-06-17 | 2016-06-08 | 广东电网公司电力科学研究院 | For business service registration and dissemination method and the system thereof of service register center |
| CN111310934B (en) * | 2020-02-14 | 2023-10-17 | 北京百度网讯科技有限公司 | Model generation method and device, electronic equipment and storage medium |
| CN112015470B (en) * | 2020-09-09 | 2022-02-01 | 平安科技（深圳）有限公司 | Model deployment method, device, equipment and storage medium |
-
2021
- 2021-01-29 CN CN202110130264.9A patent/CN112817635B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109088905A (en) * | 2017-06-14 | 2018-12-25 | 浙江贵仁信息科技股份有限公司 | A kind of mathematical model service system and mathematical model management method based on cloud computing |
| CN109299178A (en) * | 2018-09-30 | 2019-02-01 | 北京九章云极科技有限公司 | A kind of application method and data analysis system |
| CN110688539A (en) * | 2019-09-30 | 2020-01-14 | 北京九章云极科技有限公司 | Model management system and method |
| CN111580926A (en) * | 2020-03-25 | 2020-08-25 | 中国平安财产保险股份有限公司 | Model publishing method, model deploying method, model publishing device, model deploying device, model publishing equipment and storage medium |
| CN112015519A (en) * | 2020-08-28 | 2020-12-01 | 江苏银承网络科技股份有限公司 | Model online deployment method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112817635A (en) | 2021-05-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108153670B (en) | Interface testing method and device and electronic equipment | |
| CN107861870B (en) | Interface test and test data generation method, device, terminal and storage medium | |
| US11870658B2 (en) | Method and apparatus for updating gateway resource, and IOT control platform | |
| CN109075991A (en) | Cloud verifying and test automation | |
| CN110311987A (en) | Node scheduling method, apparatus, equipment and the storage medium of microserver | |
| US10838941B2 (en) | Automated image-based record creation and related database systems | |
| CN111158613A (en) | Data block storage method and device based on access heat and storage equipment | |
| CN110569159A (en) | Baffle generation method, device, equipment and computer storage medium | |
| CN113791759A (en) | Code development processing method and device, electronic equipment and storage medium | |
| CN107733710A (en) | Construction method, device, computer equipment and the storage medium of link call relation | |
| CN113254331A (en) | Model test method, device, storage medium and program product | |
| CN112463588A (en) | Automatic test system and method, storage medium and computing equipment | |
| CN112817635B (en) | Model processing method and data processing system | |
| CN115129574A (en) | Code testing method and device | |
| CN109274533B (en) | Web service fault positioning device and method based on rule engine | |
| CN111090401A (en) | Storage device performance prediction method and device | |
| CN113971191A (en) | Data import method and device and computer readable storage medium | |
| CN115617555A (en) | Information processing system and method | |
| CN112785418B (en) | Credit risk modeling method, apparatus, device and computer readable storage medium | |
| CN113435489B (en) | Method, device, computer readable storage medium and processor for deploying system | |
| CN116089277A (en) | Neural network operator test and live broadcast application method and device, equipment and medium thereof | |
| CN111158654A (en) | Algorithm calling method, device, server and storage medium | |
| CN112749159A (en) | Form generation method and device, electronic equipment and storage medium | |
| KR20200021132A (en) | Method for Processing Prediction by using Artificial Intelligence | |
| CN111210279A (en) | Target user prediction method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |