CN111523810A - Enterprise-level model management method, system, device and storage medium - Google Patents
Enterprise-level model management method, system, device and storage medium Download PDFInfo
- Publication number
- CN111523810A CN111523810A CN202010334283.9A CN202010334283A CN111523810A CN 111523810 A CN111523810 A CN 111523810A CN 202010334283 A CN202010334283 A CN 202010334283A CN 111523810 A CN111523810 A CN 111523810A
- Authority
- CN
- China
- Prior art keywords
- model
- workflow
- management
- task
- verification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007726 management method Methods 0.000 title claims description 116
- 238000003860 storage Methods 0.000 title abstract description 16
- 238000012795 verification Methods 0.000 claims abstract description 66
- 238000000034 method Methods 0.000 claims abstract description 46
- 238000012544 monitoring process Methods 0.000 claims abstract description 42
- 238000011161 development Methods 0.000 claims abstract description 38
- 238000012545 processing Methods 0.000 claims abstract description 27
- 238000004519 manufacturing process Methods 0.000 claims abstract description 18
- 230000000977 initiatory effect Effects 0.000 claims abstract description 9
- 230000000694 effects Effects 0.000 claims description 12
- 238000010200 validation analysis Methods 0.000 claims description 7
- 238000012550 audit Methods 0.000 claims description 6
- 238000005520 cutting process Methods 0.000 claims description 5
- 238000009826 distribution Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 abstract description 31
- 230000018109 developmental process Effects 0.000 description 35
- 238000010586 diagram Methods 0.000 description 13
- 238000011156 evaluation Methods 0.000 description 11
- 238000004891 communication Methods 0.000 description 10
- 230000007246 mechanism Effects 0.000 description 7
- 230000008520 organization Effects 0.000 description 7
- 238000012552 review Methods 0.000 description 6
- 230000002159 abnormal effect Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000012423 maintenance Methods 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000015572 biosynthetic process Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 230000003862 health status Effects 0.000 description 2
- 238000011835 investigation Methods 0.000 description 2
- 230000007774 longterm Effects 0.000 description 2
- 238000012806 monitoring device Methods 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 230000009897 systematic effect Effects 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 241000283153 Cetacea Species 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 230000002860 competitive effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013479 data entry Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000008570 general process Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 239000003999 initiator Substances 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000012384 transportation and delivery Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 230000003442 weekly effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/067—Enterprise or organisation modelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/10—Office automation; Time management
- G06Q10/103—Workflow collaboration or project management
Landscapes
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Strategic Management (AREA)
- Engineering & Computer Science (AREA)
- Entrepreneurship & Innovation (AREA)
- Economics (AREA)
- General Business, Economics & Management (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Development Economics (AREA)
- Educational Administration (AREA)
- Game Theory and Decision Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
A method, system, device and storage medium for managing enterprise-level model, the method mainly includes: establishing an enterprise level model asset library, and registering all models and related documents to the same management platform; monitoring the working state and performance indexes of each model in the asset library in real time in a life cycle according to a pre-configured instrument panel; and performing model task processing on the general model workflow according to different model task requirements, such as tasks of model requirement initiation, model development, model verification, model implementation and deployment, model approval, model online, continuous verification after production, monitoring and the like. By adopting the invention, all models are registered to the same model management platform, the models are shared and reused in the platform according to the authorities of different roles, the indexes or states of any one or more models are presented through the instrument panel, and the centralization, the process and the unification of model management can be realized through the processing of different subtasks in the life cycle of the general model workflow correlation model.
Description
Technical Field
The invention relates to the technical field of model management, in particular to an enterprise-level model management method, an enterprise-level model management system, enterprise-level model management equipment and a storage medium.
Background
In various financial institutions of countries in the world, models are widely developed and applied to scenes such as marketing, wind control and decision making, and become indispensable important tools in various financial services including credit services. However, there is also a risk due to errors or improper use of the model itself, which is the risk of the model. For example, a crash from Long Term Capital Management (LTCM) in 1998, the U.S. sublicense crisis in 2007, the Morgan chase university (JP Morgan chase) "london whale" event 2012, etc. are all associated with model risks, indicating that model risks have had enormous destructiveness as only one operational risk, and the absence of a model risk management system for an institution or even a country would pose an immeasurable systematic risk of destructiveness for the institution or country.
Model risk management is a long-term, systematic, complex and detailed work, and in the absence of efficient methods and tools, a large amount of human resources must be occupied, and the timeliness and quality of the work may still be difficult to guarantee.
In the existing model management, along with diversification of service lines and complexity of service scene types, the number of models in a mechanism and the work task of the models are rapidly increased. Generally, model work can be independently carried out between each service line in the organization, so that model assets in the organization are dispersed, historical information related to the model cannot be traced back, internal audit requirements cannot be met, or compliance risk potential hazards caused by the fact that effective model documents cannot be provided exist; the state of the model cannot be known in time, so that the working state of the model cannot be effectively tracked, and delay is caused, or the operation loss caused by the performance loss of the model cannot be known in time; the model assets cannot be shared and reused across departments, part of model work is repeated, and human resources are consumed. In addition, because the flows of model work among different business departments are not uniform, the model work is possibly too cautious or too rough, so that the model work time is unreasonable or the model quality is uneven, the timeliness is low, the responsibility of each link of the model work flow is fuzzy, a specific responsible person cannot be positioned, and the model work progress is influenced; the verification mechanism is lack after the model is put into production, a periodical verification plan is lacked, a large amount of human resources are consumed to generate reports, the model reports are not standard, and the problems of the model cannot be found and positioned in time.
Disclosure of Invention
Embodiments of the present invention provide an enterprise-level model management method, system, device, and storage medium, which can solve technical problems related to the flow and unification of model management by constructing an enterprise-level model management platform.
A first aspect of an embodiment of the present invention provides an enterprise-level model management method, which may include:
establishing an enterprise level model asset library, and registering all models and related documents to the same management platform;
monitoring the working state and performance indexes of each model in the model asset library in real time in a life cycle according to a pre-configured model instrument panel;
and performing model task processing on the general model workflow according to different model task requirements, wherein the model task at least comprises one or more of model requirement initiation, model development, model verification, model implementation and deployment, model approval, model online, continuous verification after production and monitoring subtasks.
A second aspect of an embodiment of the present invention provides an enterprise-level model management system, which may include:
the system comprises a registration database building module, a database management module and a database management module, wherein the registration database building module is used for building an enterprise-level model asset library and registering all models and related documents to the same management platform;
the instrument panel monitoring module is used for monitoring the model working state and the performance index of each model in the model asset library in a life cycle in real time according to a pre-configured model instrument panel;
and the workflow processing module is used for performing model task processing on the general model workflow according to different model task requirements, wherein the model task at least comprises one or more of model requirement initiation, model development, model verification, model implementation and deployment, model approval, model online, continuous verification after production and monitoring subtasks.
A third aspect of embodiments of the present invention provides a computer storage medium storing a plurality of instructions adapted to be loaded by a processor and to perform the following steps:
establishing an enterprise level model asset library, and registering all models and related documents to the same management platform;
monitoring the working state and performance indexes of each model in the model asset library in real time in a life cycle according to a pre-configured model instrument panel;
and performing model task processing on the general model workflow according to different model task requirements, wherein the model task at least comprises one or more of model requirement initiation, model development, model verification, model implementation and deployment, model approval, model online, continuous verification after production and monitoring subtasks.
A fourth aspect of embodiments of the present invention provides an apparatus, which may include: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to perform the steps of:
establishing an enterprise level model asset library, and registering all models and related documents to the same management platform;
monitoring the working state and performance indexes of each model in the model asset library in real time in a life cycle according to a pre-configured model instrument panel;
and performing model task processing on the general model workflow according to different model task requirements, wherein the model task at least comprises one or more of model requirement initiation, model development, model verification, model implementation and deployment, model approval, model online, continuous verification after production and monitoring subtasks.
The invention has the beneficial effects that:
by positioning at an enterprise level, assisting an organization to establish an enterprise-level model asset library, carrying out centralized management on models applied in different business scenes, and registering all models and related documents thereof on the same platform;
the running state of the model or the working stage of the model can be known in real time according to the configured model instrument panel, a mechanism can timely know the working process of the model and follow up decision links such as artificial examination and approval required in a workflow, for the produced model, when model loss or abnormal conditions occur, an alarm rule is triggered and fed back to the model instrument panel in real time, and a user can intervene early to carry out model investigation work;
according to the given platform authority, a user can easily find needed model assets on the platform, the model assets are shared and reused in the mechanism, and the intelligent development is promoted by mutually referencing;
based on the life cycle of the model, a model workflow of a universal version is defined, all model works can follow relatively consistent flows, each link of the workflow is assigned to a specific responsible person, work results needing to be uploaded are well defined according to different links, and examination and approval links are configured in the workflow according to needs, so that the work quality of each work link is guaranteed in a single model workflow, and the work quality requirements of the models are unified between the workflow and the workflow.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart diagram of a method for managing an enterprise-level model according to an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of an enterprise-level model management system according to an embodiment of the present invention;
FIG. 3 is a block diagram of another enterprise-level model management system provided by an embodiment of the present invention;
FIG. 4 is an architecture diagram of an enterprise-level model management system provided by an embodiment of the present invention;
FIG. 5 is a diagram of dependency relationships between model forms, model workflows, and model registration functions provided by an embodiment of the present invention;
FIG. 6 is an architecture diagram of a process engine provided by an embodiment of the present invention;
FIG. 7 is an architecture diagram of a model evaluation service provided by an embodiment of the present invention;
FIG. 8 is an architecture diagram of a data reflow service provided by an embodiment of the present invention;
FIG. 9 is a block diagram of another enterprise-level model management system provided by an embodiment of the present invention;
FIG. 10 is a block diagram of a workflow creation module according to an embodiment of the present invention;
FIG. 11 is a schematic structural diagram of a registration library building module provided in the embodiment of the present invention;
fig. 12 is a schematic structural diagram of an apparatus according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "including" and "having," and any variations thereof, in the description and claims of this invention and the above-described drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.
Furthermore, the terms "mounted," "disposed," "provided," "connected," and "sleeved" are to be construed broadly. For example, it may be a fixed connection, a removable connection, or a unitary construction; can be a mechanical connection, or an electrical connection; may be directly connected, or indirectly connected through intervening media, or may be in internal communication between two devices, elements or components. The specific meanings of the above terms in the present invention can be understood by those of ordinary skill in the art according to specific situations.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
The device related to the embodiment of the invention can be a mainframe computer, a PC, and other computer processing devices with data processing capability.
As shown in fig. 1, in a first embodiment of the present application, a method for enterprise-level model management includes at least the following steps:
s101, an enterprise level model asset library is constructed, and all models and related documents are registered to the same management platform.
In specific implementation, the system firstly needs to centralize all models in an enterprise to a system platform for management, breaks barriers of data and models between business departments, makes asset sharing and multiplexing possible, and then can perform enterprise-level management. For any model within an enterprise, the model and related documents need to be registered to the platform. Optionally, the enterprise may also transfer the historical models and related files to the unified management platform by means of an interface, so as to perform unified management on all models in one enterprise.
The following illustrates the benefits of building an enterprise-level model asset library for model management in a new business, i.e., cold-start scenario: in a service cold start scene, there is often no way to develop a model corresponding to a new service because of no data, and for the same link in a credit service, it may be considered to multiplex existing models in other services into the new service, and this way may not be perfect, but applying models of different services in the same link will give the new service a certain wind control capability. Specifically, a worker related to the new service may obtain a corresponding right given by the system, the worker may log in the system to check whether a model of another service is applicable and is multiplexed into a decision flow of the new service, the worker may copy and deploy some manual analysis components related to an available model into a production environment of the new service, and the worker also needs to create a new model definition, a new model requirement, and the like in the system.
In a specific implementation, the process of model registration may include the following steps:
1) creating a model definition: model name, model description, database and data sheet corresponding to the model.
2) Defining model metadata: model name, model description, business to which the model belongs, business scenario in which the model is used, model type, model output type, model affiliation person, sample window in model development, model release date, and the like.
3) And connecting the external database corresponding to the model and the data form of the model.
4) Variables used by the model are selected for the data form in the platform to be used for generating a model verification report, and the variables are endowed with a report role, such as the following characteristics for defining a variable as the model use: identifying variables used by the model and assigning roles to model features, identifying other variables needed for reporting and assigning related roles, reporting roles having model call date roles, model score roles, sample weight roles, model target roles, and the like.
5) Configuring a model, uploading a model file, wherein the actual file format of the model is pmml or pkl, the model file is called when a report is generated, and the calling aims at identifying the model when the platform generates the report; defining a data window and a benchmark, defining a model prediction objective and assigning a report role-objective, defining a score distribution and a cutting group to which the model belongs.
6) Optionally, the model-related supporting files may be uploaded at the same time.
7) Optionally, model-related comments may be attached.
And S102, monitoring the model working state and the performance index of each model in the model asset library in a life cycle in real time according to a pre-configured model instrument board.
In the specific implementation, after the model registration and configuration of each model are completed, all models are already centralized on the system platform, and at this time, the performance of all models, that is, the working state and performance index of each model, can be monitored. In the aspect of centralized monitoring, the system mainly reflects the working state of the model and the performance index of the model in a centralized manner in a mode of a model instrument board.
It should be noted that, in the system, the dashboard can display the current status of all registered models, and the user can perform model screening operation in the dashboard to locate the model that needs further operation. Clicking on any model within the dashboard may navigate to a verification results page associated with the model, which may generally be displayed in the form of a sector statistical chart, bar statistical chart, or table. The complete report running behind the verification result (usually a metric) can be downloaded from the system platform, providing the pdf,. xlsx,. html file format.
Optionally, the expression mode of the model state variables and the verification results displayed on the model dashboard may be customized and adjusted according to the user's requirements. Generally, the model dashboard displays the name of the model, the cutting group where the model is located, the working environment to which the model belongs, the alarm level and the number of alarm rule triggers of the model, the general health profile of the model, the working period of the model, the number of days since the last verification of the model, and other information.
Optionally, the system platform may generate an alarm signal/event according to the defined alarm rule and the verification result index and feed back the alarm signal/event to the model dashboard. The user can view the single alarm or all alarms of the model to track the reason of the alarm and arrange for further work. The user can modify the current alarm rules and can add more alarm rules.
Optionally, the person to whom the model belongs may change the health status of the model and add comments according to the verification result. The model health status levels can be classified as healthy, attentive-mild, attentive-moderate, attentive-urgent, requiring immediate treatment, etc.
Alternatively, the relevant person for each model may add relevant comments to record observations of the model while viewing historical activities of the model.
And S103, performing model task processing on the general model workflow according to different model task requirements.
It should be noted that the system may create a general model workflow based on any model work requirement or model work task, so that all model works can follow a relatively consistent flow, each link of the workflow is assigned to a specific responsible person, and according to different links, work results to be uploaded are defined, and approval links are configured in the workflow according to needs, thereby ensuring the work quality of each work link in a single model workflow, and the requirements of the model work quality are uniform between the workflow and the workflow. In addition, according to a binary or continuous model target, a corresponding model standardized report is embedded in the model workflow, a user can customize a report generation period (quarterly, monthly and weekly), a data time range (the first 3 months, the first 6 months or other backtracking time ranges), and the platform automatically generates the report.
In specific implementation, any task is put into a universal model workflow, the model workflow covers sub-links of a plurality of model works, and workflow authors can perform custom modification on the basis of the universal model workflow for different model task requirements, so that all model works are ensured to follow certain normative, but flexibility is provided according to respective model work requirements. When the workflow is released and activated, the workflow automatically follows up the working progress and automatically reminds a person in charge of the sub-link to complete the task.
In an optional embodiment, for a single work task or a combination of multiple work tasks such as model development, model verification, model maintenance, model deployment, model optimization, and the like, a corresponding workflow definition corresponding work range and a required sub-work link can be established. The workflow author defines the subtask links required in the workflow and assigns the subtasks to specific principals or responsible teams. It can be understood that all model work users (including subtask link responsible persons or responsible teams) related to the workflow can see tasks to be completed in the respective systems, and the users can also see where task links responsible for the users are located in the whole model workflow, so that time is arranged in advance, and communication among link responsible persons is promoted.
In a preferred implementation, the step of creating a model workflow for a model work requirement or a model work task may include the following processes:
1) a workflow author creates a workflow and defines subtasks in the workflow, and when the workflow author creates the workflow, the workflow author needs to define the name of the workflow, the type of the workflow and a corresponding general workflow template; when defining the subtasks, the workflow author needs to be aware of the subtasks required by the model task and assign the subtasks to people when creating the workflow.
2) The workflow author issues the workflow, and a system administrator deploys the workflow after auditing the workflow without errors, and the workflow automatically advances according to the defined task links and the responsible persons.
3) And the workflow sends a mail to a responsible person of the next subtask according to the progress, and related workers of the task receive the mail prompt and the system login link and obtain corresponding system permissions conforming to the workers at the same time, so that model lookup, modification and other model related work permissions can be carried out.
4) When the worker responsible for the subtask completes and uploads the document required by the task, the subtask is simply completed and the workflow automatically advances to the next subtask and notifies the relevant responsible person. And (4) repeating the steps 3 and 4 by the workflow until all subtasks of the workflow are completed, and determining whether to close the workflow or not by a workflow author according to the work quality and the workflow progress and marking a completion label.
5) All users authorized to review the model or workflow may review all historical activities and associated records during the validation of the workflow.
6) If the workflow author modifies or updates the workflow during the validation of the workflow, the workflow is updated according to the update, the workflow needs to be reissued, a system administrator needs to be redeployed, and the workflow is restarted from the latest unfinished subtask and pushed to work.
In the embodiment of the application, a set of enterprise-level management method and system based on a life cycle and utilizing configurable and automatic workflows is established, so that risk and analysts of an organization can follow a designed workflow while continuously developing a leading-edge model and a leading-edge strategy with competitive advantages, and information, output and important files related to all models are unified into a library. By setting role authority, different personnel and departments can properly use and share information in the library, so that the requirements from the interior of the organization are met under the condition of saving most human resources. For the working scenario of model development, the workflow guides the user through a standardized flow of recorded actions and decisions. It avoids manual data entry by automatically obtaining segmentation criteria, test results, and other key information. In addition, the workflow can coordinate cross-department model examination and verification, timely inform related personnel and ensure that the next subtask is carried out after the current subtask is approved. The dashboard of the workflow may also provide a clear real-time status for the progress of all models. In response to regulatory and internal reporting or query requirements, the workflow works by importing all data during the model lifecycle into a centralized repository. On such a unified platform containing all model-related information, the task of preparing a user for audits or writing reports for responding to queries can be made faster and more efficient. For example, they can check the most recent verification tests for a model very quickly, and can also easily peruse the entire process history, including each verification that the model is doing, and the results and subsequent operations in each instance.
In summary, the enterprise-level model management platform based on the automated and configurable workflow can realize centralized model management, so that workers can quickly know all histories of development (development data/model development/model verification), deployment (complexity/operation requirement/operation test of the model), and maintenance (verification/modification/iteration after model delivery) of each model. Meanwhile, the bank can be ensured to retain all the information related to the model and support the basis of the related decision, and the inquiry from the internal requirement of the bank and the external supervision is met to the maximum extent. In addition, besides the built-in workflow, the platform can simultaneously carry out the periodic automatic verification of the model and mark the performance of the model according to the change of the performance index of the model. This allows the bank to learn the state of the model more timely, to maintain the model in operation more easily, and to ensure that it is at a high performance level.
As shown in fig. 2, in the second embodiment of the present application, the role authority management in the enterprise-level model management architecture is mainly used for managing the authority of different roles in different business teams, and each role can view interested data through the dashboard according to the respective matched authority, for example, centralized management can be performed through the dashboard 1, content in a model list can be viewed, verification and monitoring can be performed through the dashboard 2, and verification and monitoring results of a single or multiple models can be viewed. In addition, model manifests and validation and monitoring results of single or multiple models can be shared and multiplexed to different business teams.
It should be noted that the different roles in the workflow or system may include at least the model owner, workflow author, model approver, and system administrator. Wherein:
the model belongs to the people: the owner of the model, a specific initiator of the new model requirement, and the owner of the model can have all system role authorities except a system administrator, and can delete, modify, and the like the model with the viewing authority.
Workflow authors: the workflow author creates and releases the model workflow in the platform system, and defines the subtasks in the workflow according to the model work tasks to assign the subtasks in the workflow to the responsible persons, and the workflow author and the person to which the model belongs can be the same person.
The model authors: the model author is responsible for completing subtasks in the model workflow, the model author can be further subdivided into data analysis personnel, model development personnel, model verification personnel and model deployment personnel according to different internal positions of the using mechanism, and other relevant personnel of the model work except for model examination and approval personnel, and the model author is responsible for creating model definition, configuring model metadata, relevant model variables and variable roles, developing the model, creating a model verification plan, composing alarm rules and the like.
Model approver: different from a model author, a model approver is generally not particularly responsible for actual operation of model work, and the model approver is responsible for an approval subtask in a model workflow and is responsible for approving output of the model author and completing the approval subtask.
A system administrator: the system administrator is responsible for authority management, configuring an external database and a data table corresponding to each model, deploying issued workflows, and configuring a custom model instrument board and a custom model verification report.
Self-defining roles: according to the working requirements of the model, the organization can create a self-defined role through a system administrator.
In a specific implementation scenario, the permissions corresponding to different roles complete different tasks, for example:
the model belongs to the people: the person to which the model belongs enters a platform system to inquire the states of all models managed by the person at present, and specific historical activities or present model expression conditions of a certain model can enter the working environment of the model to inquire.
Workflow authors: the workflow author enters a workflow management page of the platform system to create a model workflow according to a new model task, and can also select a general workflow corresponding to the new model development for the new model development task.
The model authors: after the workflow is formally deployed and validated, the model author of the current first subtask receives task mail and system connections from the system as the workflow advances. For example, in a subtask-model development link, a model developer has the authority corresponding to the role of a model author in a system platform, and according to the requirements of the subtask, the model developer completes data cleaning to generate a data quality report, performs index derivation and feature screening to generate a feature report, performs model development and testing to generate a report, uploads the report to the subtask uniformly and submits the subtask.
Model approver: before the new model is on line, the model needs to be tested and verified, and model approvers check the normative and the rationality of the reports in the stages of model development, model development and model verification and give approval for whether the approval is passed. Only after approval by the model approver, the model work can move on to the next link.
A system administrator: after a workflow author creates and uploads a workflow, a system administrator needs to deploy the workflow, and only if the workflow is deployed, the workflow will actually take effect and be displayed on a dashboard. For the responsible person assigned by the subtask in the workflow, the system administrator gives the corresponding system role and the related authority to the responsible person.
In this embodiment, the content management information base may correspond to the enterprise-level model asset base in the above-described embodiment. The process of model registration in the content management information base is as follows: firstly, different business teams create respective model definitions; further defining model metadata which mainly comprises a business line, a business scene, a model type, an output type, a model attribution person, a sample window, a release date and the like; then connecting the corresponding database of the model and the data table, then configuring the model, and setting the verification definition. Wherein the defined model metadata and the created model definition may be embodied in a model manifest. When single or multiple models are verified and monitored, the database and the data table corresponding to the monitored model, the configuration of the model and the definition of configuration verification are mainly verified.
In the third embodiment of the present application, the enterprise and model management system is divided into four important modules, namely, a model management platform, a workflow management system, and a model verification/evaluation system from the perspective of principle and function, and the working relationship between the different modules in the model risk management is as shown in fig. 3:
relation line a: model component-model author-model working environment-content management system
For an existing model, a model author creates a model definition in a model working environment, uploads the relevant parts and relevant documents of the model, and the parts and relevant documents of the model are stored in a content management system of a model risk management platform.
Relation line b: model author-custom report-System administrator-System management-dashboard
Generally, a template of a custom report is designed outside a model risk management platform by a model author, and then is uniformly managed and configured in the platform by a system administrator, after configuration is completed, the model author can use the report to perform verification and monitoring work in a model working environment, and a verification and monitoring result is fed back to a dashboard.
A relation line c; the workflow author-workflow management-dashboard workflow author creates and maintains the workflow in the workflow management, when the workflow is released and deployed, the state of the workflow is collected into the dashboard, and besides, the workflow supervises the model author to enter the model working environment to complete the task according to the work task and the responsible person in the workflow.
Relation line d: the system administrator-system management/authority management system administrator is responsible for management of the daily system platform, maintenance of the platform environment, assistance of workflow authors in workflow deployment, daily platform log maintenance, and management of the role of the platform and the authority corresponding to the role.
Relation line e: model production environment (non-platform environment range), model backflow data (non-platform environment range), model management platform database (non-platform environment range), model verification/monitoring work, verification report and result, content management system, model working environment model are called in production environment to generate model backflow data, the backflow data are processed by ETL and the like to flow back to the database which is externally connected with the model risk management platform, when a model author carries out work such as model verification and monitoring in a platform, a verification and monitoring engine in the platform calls data corresponding to a model in an external database, a verification report and a result are generated by combining information such as model definition, variables and verification report roles in the platform, and the verification report and the result are stored in a content management system, and the model author can check the verification report and the result in a model work environment.
Relation line f: when a model author completes corresponding work or tasks in the model work environment, the completed work or operation is stored in the content management system, the content management system and the workflow management system are communicated with each other, the workflow system identifies the next action, advances the workflow progress forward and returns the progress to the content management system.
In a fourth embodiment of the present application, the architecture of the enterprise-level model management system is shown in fig. 4, and includes a presentation layer, an access gateway layer, a system application service layer, and an infrastructure capability layer. Wherein:
a display layer: the method is mainly used for login used by a user and visual display of a front-end operation interface of the system.
An access gateway layer:
a nginx gateway: the network isolation and the unified management of the API are realized, the public network service is exposed to the outside, the intranet flow is shunted and forwarded internally, the API request protocol conversion is supported, the load of the request is distributed in a balanced manner, and the high-availability and unified security entry management of the system is improved.
2. And (3) access authentication: and performing access authentication on the API access. Preventing illegal request operations.
The system applies the business layer:
model asset management platform:
a) model form: the most basic unit module in the model registration is composed of a plurality of fields, represents the classification combination of the fields, can be expanded according to the service customization and is quoted for the workflow model node.
b) Model workflow: and defining a model life cycle management process link for a carrier in the model registration process.
c) Model registration: registering model instance information, storing model registration details, and containing the content and associated files of the model.
The dependency relationship among the model form, the model workflow and the model registration function is shown in fig. 5: the model forms can be customized and expanded according to the needs of service scenes, and different forms are combined with the workflow to finally form the whole process of model registration. Model nodes in the workflow need to refer to a certain model form. The model workflow is combined with the model form to generate a model registration processing process, and the change of nodes in the whole model registration processing process needs to call the model workflow for processing. In the process of model registration processing, model data can be instantiated, the requirement information, development information, verification information and the like of the model are perfected, model registration is completed, and the instantiation data of the model is also completed.
And (4) reporting the data:
a) operation and operation history records related to the model, such as the model (pmml, pdf, excel,. pkl), the data report (. xlsx,. doc,. pdf,. r), the verification report (. doc,. pdf, model code), etc., are respectively stored in the file object system OSS and the big data platform, and the timing task jobgenerates a BI report required by the service according to the service requirement and sends the BI report through a channel such as a mail.
b) The model operation platform can generate score and other dimensional data to be grounded on a database according to real-time calling conditions of production, the model management platform can acquire variable and data in the database from multiple dimensions in the database, and different dimensional reports are generated by combining dynamic script codes to form model verification or supervision reports.
A flow engine:
the core function of model management is to realize the process standardized management of model production, model registration, model examination and approval and the like, and the model management engine is an engine for driving the full life cycle management of the model, can support the configuration and the expandability of services, is in environment communication connection with a service model factory and a model operation platform, and supports the self-iteration of the model. The architecture of the flow engine is shown in fig. 6: the embedded activiviti open source flow platform develops a stateless springboot application, takes MySQL as a state storage layer, is deployed in an independent application cluster mode, and exposes a uniform domain name to the outside. The method conforms to BPMN standard and realizes a general process platform customized by companies. The flow node supports triggering an external HTTP interface, a restful interface related to the flow can be exposed to the outside, and the upper layer of the service can be directly called. The method comprises the steps of providing a web-version console, visually dragging and editing a process page, and enabling an editor to create or edit an activiti process file (the activiti is of a specific process file format, is essentially a ZIP package and comprises a standard BPMN process definition XML file and related resource files such as icons and the like), and a process instance query interface (API). And after the verification is finished, deploying the process file into the current activiti engine. The process nodes of the process engine can call the designated service interfaces in the service package, and the interfaces need to be authenticated to prevent illegal access and ensure the safety of service calling.
Model assessment and model validation/monitoring service:
model predictive assessment support predictive assessment of standard generic model files in model asset systems, with current designs supporting pmml, py model file type predictive assessment. The overall evaluation architecture is shown in fig. 7:
the front-end page sends the model code, the model data file or the designated model data source to the model evaluation service. The model evaluation service obtains model files and model report templates from the model asset management service according to the content uploaded by the front end, and obtains data from the existing configured data source if the model evaluation service points to the data source. And the model evaluation service analyzes and operates the model file, and a model operation report is produced after the operation is finished. The model evaluation service sends a model operation report to the model asset management service, and the model asset management service sends a notification to a specified mailbox. And the model evaluation service calls an acquisition interface of the big data cloud platform and issues the model operation data to the big data cloud platform.
The data backflow service mainly provides operation data backflow of a three-party model factory and a model operation platform, supports two modes of real-time synchronous backflow and offline batch file backflow, and provides a data source for model supervision, and mainly provides a data source for model development evaluation and backflow effect evaluation as shown in fig. 8. In fig. 8, modeling related data generated by the model development environment (i.e., the model development modeling factory) and model operation environment (i.e., real-time data operation of the model production, some configuration data and workflow data generated by the model management platform) are transmitted to the message middleware kafka or similar data bus by way of data production. Data reflow service: and consuming service data generated by message middleware or similar data buses, performing format conversion and unified processing, and performing relevant preprocessing of service indexes. Data flows into the big data platform. And the big data platform runs a report task, performs spark, hive and other timing calculations, and visually checks and displays the result on the model management platform. The flow approval information and BI report forms related to the flow can reach the user terminal through a three-party information channel in real time.
Base capacity layer:
1) a big data platform: and the data storage and real-time spark stream calculation of a big data scene and the basic storage and calculation capacity such as hive are supported.
2) MySQL: and operations such as basic object fields such as model forms and model registration, storage such as process engine and model circulation states, business query and the like are supported.
3) ELK: the storage of log data is supported, and the viewing capacity is monitored and visualized based on the operation of some rules.
4) And OSS: supporting model-dependent file object storage.
5) Kafka: and the message asynchronous streaming of data between systems and the loose coupling between the systems are supported.
And (3) service monitoring:
and (4) model operation condition, online abnormal API call or abnormal model management flow operation, and real-time scripted calculation and alarm.
In a fifth embodiment of the present application, another enterprise-level model management system is provided for embodiments of the present invention. As shown in fig. 9, the enterprise-level model management system 10 of the present embodiment may include: the system comprises a registry creating module 101, a dashboard monitoring module 102, a workflow processing module 103, a workflow creating module 104, a review feedback module 105, an update restarting module 106 and a dashboard adjusting module 107. As shown in fig. 10, the workflow creating module 104 includes a subtask creating unit 1041, a workflow deploying unit 1042, a subtask processing unit 1043, and a subtask advancing unit 1044. The registry library module 101, as shown in fig. 11, includes a model definition creating unit 1011, a metadata definition unit 1012, a data connection unit 1013, a report processing unit 1014, a model configuration unit 1015, and an asset library forming unit 1016.
And the registration database building module 101 is used for building an enterprise-level model asset library and registering all models and related documents to the same management platform.
In an alternative embodiment, the registration library building module 101 may include the following elements:
a model definition creating unit 1011 for creating a model definition.
A metadata definition unit 1012 for defining model metadata.
And a data connection unit 1013 for connecting an external database corresponding to the model and a data form of the model.
And the report processing unit 1014 is used for generating a model verification report according to the variables used by the data form selection model and giving report roles to the variables.
The model configuration unit 1015 is configured to configure a model, upload a model file corresponding to the model, define a data window and a reference, define a model prediction target, assign a report role-target to which the target belongs, and define a score distribution and a cutting group to which the model belongs.
An asset repository formation unit 1016 for saving all registered models to the enterprise level model asset repository.
And the instrument panel monitoring module 102 is used for monitoring the model working state and the performance index of each model in the model asset library in a life cycle in real time according to the pre-configured model instrument panel.
It is understood that the model dashboard displays the current status of all registered models and displays the verification results for the selected model in a preset display form, including one or more of a sector statistical chart, a bar statistical chart, or a table.
And the instrument panel adjusting module 107 is used for adjusting the model state displayed on the model instrument panel or the expression mode of the verification result according to the user requirement.
A workflow creation module 104 for creating a generic model workflow based on any model work requirement or model work task.
In alternative embodiments, the workflow creation module 104 may include the following elements:
a subtask creating unit 1041, configured to define a subtask in the workflow.
The workflow deployment unit 1042 is configured to deploy the issued workflow after the audit is correct, so that the workflow is automatically advanced according to the predefined task links and the responsible person.
And the subtask processing unit 1043 is configured to send, by the workflow, a reminding message to a responsible person corresponding to a next subtask according to the current progress, and give a system authority required by a task link corresponding to the responsible person according to the set work in the system, where the reminding message at least includes one or more of a mail reminder and a system login link.
And the subtask propelling unit 1044 is used for automatically propelling the workflow to the next subtask after one subtask is completed.
And the workflow processing module 103 is used for performing model task processing on the general model workflow according to different model task requirements, wherein the model task at least comprises one or more of model requirement initiation, model development, model verification, model implementation and deployment, model approval, model online, continuous verification after production and monitoring subtasks.
And a review feedback module 105 for feeding back historical activities and related records of the workflow during validation of the workflow according to review requests conforming to the review authority of the workflow.
And the update restarting module 106 is used for reissuing the workflow and restarting the task from the latest uncompleted subtask when the workflow needs to be modified or the model task needs to be added, deleted and updated during the workflow is in effect.
It should be noted that, for the detailed execution process of each module and unit in the system, reference may be made to the description in the foregoing embodiments, and details are not described here.
In the embodiment of the invention, by positioning at an enterprise level, an organization is assisted to establish an enterprise-level model asset library, models applied to different business scenes are managed in a centralized manner, and all models and related documents thereof are all registered on the same platform; the running state of the model or the working stage of the model can be known in real time according to the configured model instrument panel, a mechanism can timely know the working process of the model and follow up decision links such as artificial examination and approval required in a workflow, for the produced model, when model loss or abnormal conditions occur, an alarm rule is triggered and fed back to the model instrument panel in real time, and a user can intervene early to carry out model investigation work; according to the given platform authority, a user can easily find needed model assets on the platform, the model assets are shared and reused in the mechanism, and the intelligent development is promoted by mutually referencing; based on the life cycle of the model, a model workflow of a universal version is defined, all model works can follow relatively consistent flows, each link of the workflow is assigned to a specific responsible person, work results needing to be uploaded are well defined according to different links, and examination and approval links are configured in the workflow according to needs, so that the work quality of each work link is guaranteed in a single model workflow, and the work quality requirements of the models are unified between the workflow and the workflow.
An embodiment of the present invention further provides a computer storage medium, where the computer storage medium may store a plurality of instructions, where the instructions are suitable for being loaded by a processor and executing the method steps in the embodiments shown in fig. 1 to 8, and a specific execution process may refer to specific descriptions of the embodiments shown in fig. 1 to 8, which are not described herein again.
Fig. 12 is a schematic structural diagram of an apparatus according to an embodiment of the present invention. As shown in fig. 12, the apparatus 1000 may include: at least one processor 1001, such as a CPU, at least one network interface 1004, a user interface 1003, memory 1005, at least one communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display) and a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a standard wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 1005 may optionally be at least one memory device located remotely from the processor 1001. As shown in fig. 12, memory 1005, which is one type of computer storage medium, may include an operating system, a network communication module, a user interface module, and an enterprise-level model management application.
In the apparatus 1000 shown in fig. 12, the user interface 1003 is mainly used as an interface for providing input for a user, and acquiring data input by the user; the network interface 1004 is used for data communication with the user terminal; and processor 1001 may be configured to invoke an enterprise-level model management application stored in memory 1005 and perform the following operations in particular:
establishing an enterprise level model asset library, and registering all models and related documents to the same management platform;
monitoring the working state and performance indexes of each model in the model asset library in real time in a life cycle according to a pre-configured model instrument panel;
and performing model task processing on the general model workflow according to different model task requirements, wherein the model task at least comprises one or more of model requirement initiation, model development, model verification, model implementation and deployment, model approval, model online, continuous verification after production and monitoring subtasks.
In some embodiments, the processor 1001 is further configured to:
a generic model workflow is created based on any model work requirement or model work task.
In some embodiments, the processor 1001 specifically performs the following operations when creating a generic model workflow based on any model work requirement or model work task:
defining subtasks in the workflow;
after the audit is correct, the issued workflow is deployed so that the workflow is automatically propelled according to a predefined task link and a responsible person;
the workflow sends reminding information to a responsible person corresponding to the next subtask according to the current progress, and system authority required by a task link corresponding to the responsible person is given to the responsible person in the system according to the set work, wherein the reminding information at least comprises one or more of mail reminding and system login links;
after one subtask is completed, the workflow advances to the next subtask.
In some embodiments, the processor 1001 is further configured to:
during the workflow validation, according to the workflow lookup request conforming to the lookup authority, feeding back historical activities and related records of the workflow;
and during the effective period of the workflow, if the workflow needs to be modified or the model tasks are added, deleted and updated, the workflow is reissued, and the tasks are restarted from the nearest uncompleted subtasks.
In some embodiments, processor 1001, in building an enterprise level model asset library, registers all models and related documents to the same management platform, specifically performs the following operations:
creating a model definition;
defining model metadata;
connecting an external database corresponding to the model and a data form of the model;
generating a model verification report according to variables used by the data form selection model, and giving report roles to the variables;
configuring a model, uploading a model file corresponding to the model, defining a data window and a benchmark, defining a model prediction target, giving a report role-target to which the target belongs, and defining a score distribution and a cutting group to which the model belongs;
all registered models are saved to the enterprise level model asset library.
In some embodiments, the model dashboard displays the current status of all registered models and displays the verification results for the selected model in a preset display form including one or more of a sector statistical chart, a bar statistical chart, or a table.
In some embodiments, the processor 1001 is further configured to:
and adjusting the model state displayed on the model instrument board or the expression mode of the verification result according to the user requirement.
The enterprise-level model management method is realized based on a model management system, and the model management system comprises the following equipment: the model monitoring system comprises a model unified management device, a model development device, a model operation device and a model monitoring device, wherein the model monitoring device and the model unified management device can be integrated into a whole. The above devices can be understood as a single computer device or a computer cluster, and are set according to specific application requirements.
The model development equipment is used for carrying out model development. The model operation device can be understood as a device which puts the model developed by the model development device into a business environment to operate and complete the business requirement. The model development equipment is in communication connection with the model unified management equipment, the model unified management equipment configures a workflow for the model development equipment according to the development requirements of the model development equipment, and the model development equipment uploads the completion results of each task to the model unified management platform based on the workflow. Therefore, the model data in the workflow formation can be directly uploaded to the model unified management device, on one hand, the standardization of the model development process can be ensured, and on the other hand, the model uploaded to the model unified management device can be allowed to be shared and multiplexed.
The model monitoring equipment is in communication connection with the model operation equipment and the model unified management equipment, the monitoring needs to be configured through the model unified management equipment, and model parameters including but not limited to model production data are obtained from the model operation environment equipment according to the configured monitoring needs to monitor the model state. In order to display the monitoring result conveniently, the model monitoring equipment displays the parameters of each model through an instrument panel. Typically because each model has a data protocol and interface for the respective model. In the centralized management method of the model, a unified management system is constructed, the model is developed from the beginning until the model is cancelled out and becomes invalid, so that the model is unified, and on the other hand, the model monitoring equipment is configured with middleware for unifying protocols, so that the format of data is adjusted to be adapted to the model monitoring equipment.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The above disclosure is only for the purpose of illustrating the preferred embodiments of the present invention, and it is therefore to be understood that the invention is not limited by the scope of the appended claims.
Claims (10)
1. A method for enterprise-level model management, the method comprising:
establishing an enterprise level model asset library, and registering all models and related documents to the same management platform;
monitoring the working state and performance indexes of each model in the model asset library in real time in a life cycle according to a pre-configured model instrument panel;
and performing model task processing on the general model workflow according to different model task requirements, wherein the model task at least comprises one or more of model requirement initiation, model development, model implementation and deployment, model verification, model approval, model online, continuous verification after production and monitoring subtasks.
2. The model management method of claim 1, further comprising:
a generic version of the model workflow is defined based on the model lifecycle.
3. The model management method of claim 2, wherein defining a generic version of a model workflow based on a model lifecycle comprises:
defining subtasks in the workflow;
after the audit is correct, the issued workflow is deployed so that the workflow is automatically propelled according to a predefined task link and a responsible person;
the workflow sends reminding information to a responsible person corresponding to the next subtask according to the current progress, and system authority required by a task link corresponding to the responsible person is given to the responsible person in the system according to the set work, wherein the reminding information at least comprises one or more of mail reminding and system login links;
after completion of one subtask, the workflow automatically advances to the next subtask.
4. The model management method of claim 3, further comprising:
during the workflow validation, feeding back the historical activities and related records of the workflow according to the query request conforming to the query authority of the workflow;
and during the effective period of the workflow, if the workflow needs to be modified or the model tasks are added, deleted and updated, the workflow is reissued, and the tasks are restarted from the nearest uncompleted subtasks.
5. The model management method of claim 1, wherein said building an enterprise level model asset library, registering all models and related documents to the same management platform, comprises:
creating a model definition;
defining model metadata;
connecting an external database corresponding to the model and a data form of the model;
generating a model verification report according to variables used by the data form selection model, and giving report roles to the variables;
configuring a model, uploading a model file corresponding to the model, defining a data window and a benchmark, defining a model prediction target, giving a report role-target to which the target belongs, and defining a score distribution and a cutting group to which the model belongs;
all registered models are saved to the enterprise level model asset library.
6. The model management method according to claim 1, wherein the model dashboard displays the current status of all registered models and displays the verification results for the selected model in a preset display form including one or more of a sector statistical chart, a bar statistical chart, or a table.
7. The model management method of claim 6, further comprising:
and adjusting the model state displayed on the model instrument board or the expression mode of the verification result according to the user requirement.
8. An enterprise-level model management system, the system comprising:
the system comprises a registration database building module, a database management module and a database management module, wherein the registration database building module is used for building an enterprise-level model asset library and registering all models and related documents to the same management platform;
the instrument panel monitoring module is used for monitoring the model working state and the performance index of each model in the model asset library in a life cycle in real time according to a pre-configured model instrument panel;
and the workflow processing module is used for performing model task processing on the general model workflow according to different model task requirements, wherein the model task at least comprises one or more of model requirement initiation, model development, model verification, model implementation and deployment, model approval, model online, continuous verification after production and monitoring subtasks.
9. The management system of claim 8, wherein the system further comprises:
and the workflow creating module is used for creating a universal model workflow based on any model work requirement or model work task.
10. The management system of claim 9, wherein the workflow creation module comprises:
the subtask creating unit is used for defining subtasks in the workflow;
the workflow deployment unit is used for deploying the issued workflow after the audit is correct so as to enable the workflow to automatically advance according to a predefined task link and a responsible person;
the subtask processing unit is used for sending reminding information to a responsible person corresponding to the next subtask according to the current progress by the workflow, and endowing the responsible person with system authority required by a task link corresponding to the responsible person according to the set work in the system, wherein the reminding information at least comprises one or more of mail reminding and system login links;
and the subtask propelling unit is used for automatically propelling the workflow to the next subtask after one subtask is completed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010334283.9A CN111523810A (en) | 2020-04-24 | 2020-04-24 | Enterprise-level model management method, system, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010334283.9A CN111523810A (en) | 2020-04-24 | 2020-04-24 | Enterprise-level model management method, system, device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111523810A true CN111523810A (en) | 2020-08-11 |
Family
ID=71904099
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010334283.9A Pending CN111523810A (en) | 2020-04-24 | 2020-04-24 | Enterprise-level model management method, system, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111523810A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112131495A (en) * | 2020-09-11 | 2020-12-25 | 重庆誉存大数据科技有限公司 | Webpage display method, device and equipment based on decision flow result and storage medium |
| CN113641405A (en) * | 2021-08-31 | 2021-11-12 | 重庆允成互联网科技有限公司 | Workflow rapid configuration method, device, equipment and storage medium |
| CN113919757A (en) * | 2021-11-22 | 2022-01-11 | 重庆允成互联网科技有限公司 | Workflow-based backlog handler matching method and system |
| CN114118875A (en) * | 2021-12-07 | 2022-03-01 | 中原银行股份有限公司 | Model full life cycle management system and method |
| CN114781346A (en) * | 2022-04-21 | 2022-07-22 | 广东电网有限责任公司 | Audit model sharing method and device, computer equipment and computer program product |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100100561A1 (en) * | 2008-10-15 | 2010-04-22 | Workscape, Inc. | Benefits management for enterprise-level human capital management |
| US20110153293A1 (en) * | 2009-12-17 | 2011-06-23 | International Business Machines Corporation | Managing and maintaining scope in a service oriented architecture industry model repository |
| CN105893522A (en) * | 2016-03-30 | 2016-08-24 | 电子科技大学 | System for developing, generating and managing large-data analysis model business |
| CN106920067A (en) * | 2017-01-18 | 2017-07-04 | 上海爱韦讯信息技术有限公司 | The organization assetses management system and method for customizable |
| CN107437164A (en) * | 2016-05-25 | 2017-12-05 | 北京京东尚科信息技术有限公司 | E-mail sending method and device based on workflow |
| CN107958321A (en) * | 2016-10-18 | 2018-04-24 | 穆西格马交易方案私人有限公司 | Model verifies system and method |
| US20180374015A1 (en) * | 2017-05-30 | 2018-12-27 | Open Text Sa Ulc | Systems and methods of activity flows in lifecycle models |
-
2020
- 2020-04-24 CN CN202010334283.9A patent/CN111523810A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100100561A1 (en) * | 2008-10-15 | 2010-04-22 | Workscape, Inc. | Benefits management for enterprise-level human capital management |
| US20110153293A1 (en) * | 2009-12-17 | 2011-06-23 | International Business Machines Corporation | Managing and maintaining scope in a service oriented architecture industry model repository |
| CN105893522A (en) * | 2016-03-30 | 2016-08-24 | 电子科技大学 | System for developing, generating and managing large-data analysis model business |
| CN107437164A (en) * | 2016-05-25 | 2017-12-05 | 北京京东尚科信息技术有限公司 | E-mail sending method and device based on workflow |
| CN107958321A (en) * | 2016-10-18 | 2018-04-24 | 穆西格马交易方案私人有限公司 | Model verifies system and method |
| CN106920067A (en) * | 2017-01-18 | 2017-07-04 | 上海爱韦讯信息技术有限公司 | The organization assetses management system and method for customizable |
| US20180374015A1 (en) * | 2017-05-30 | 2018-12-27 | Open Text Sa Ulc | Systems and methods of activity flows in lifecycle models |
Non-Patent Citations (1)
| Title |
|---|
| 车争等: "业务流程生命周期建模与分析", 《工业工程与管理》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112131495A (en) * | 2020-09-11 | 2020-12-25 | 重庆誉存大数据科技有限公司 | Webpage display method, device and equipment based on decision flow result and storage medium |
| CN112131495B (en) * | 2020-09-11 | 2023-05-12 | 重庆誉存大数据科技有限公司 | Webpage display method, device, equipment and storage medium based on decision flow result |
| CN113641405A (en) * | 2021-08-31 | 2021-11-12 | 重庆允成互联网科技有限公司 | Workflow rapid configuration method, device, equipment and storage medium |
| CN113919757A (en) * | 2021-11-22 | 2022-01-11 | 重庆允成互联网科技有限公司 | Workflow-based backlog handler matching method and system |
| CN113919757B (en) * | 2021-11-22 | 2022-11-15 | 重庆允成互联网科技有限公司 | Workflow-based backlog handler matching method and system |
| CN114118875A (en) * | 2021-12-07 | 2022-03-01 | 中原银行股份有限公司 | Model full life cycle management system and method |
| CN114781346A (en) * | 2022-04-21 | 2022-07-22 | 广东电网有限责任公司 | Audit model sharing method and device, computer equipment and computer program product |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10373084B2 (en) | Integrated resource tracking system | |
| CN111523810A (en) | Enterprise-level model management method, system, device and storage medium | |
| CN111523812A (en) | Model life cycle management method and system, equipment and storage medium | |
| Xu | Information architecture for supply chain quality management | |
| US8214235B2 (en) | Method and apparatus for enterprise risk management | |
| US10152692B2 (en) | Governing exposing services in a service model | |
| US9058577B2 (en) | Network centric structured communications network | |
| KR20110139706A (en) | Method and system for workflow integration | |
| US8027859B2 (en) | Analysis of a model of a complex system, based on two models of the system, wherein the two models represent the system with different degrees of detail | |
| US20070162496A1 (en) | Comparison of models of a complex system | |
| CN102982396A (en) | General process modeling framework | |
| US7941301B2 (en) | Modelling a complex system | |
| CN111523811A (en) | Model verification and monitoring method, system, equipment and storage medium | |
| Koziolek et al. | Assessing software product line potential: an exploratory industrial case study | |
| CN113448693B (en) | SAAS cloud platform of digital factory | |
| CN111028120A (en) | Closed-loop automation system for handling official business | |
| Rautenburger et al. | Process mining | |
| CN111523808B (en) | Model centralized management method, system, equipment and storage medium | |
| US20190266544A1 (en) | Techniques for managing process-flows across an enterprise | |
| KR101076419B1 (en) | Project management method using activity based project management system | |
| CN114444832A (en) | Management department grading linkage responsibility billboard and application method thereof | |
| Alshammari | Analytical Evaluation of SOA and SCRUM Business Process Management Approaches for IoT‐Based Services Development | |
| Houssin et al. | Information system based on a working situation model for a new design approach in concurrent engineering | |
| Chung et al. | An ANN‐based DSS system for quality assurance in production network | |
| Lau et al. | Quality management framework for a virtual enterprise network: a multi‐agent approach |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information |
Inventor after: Shen Lizhong Inventor after: Li Wanhua Inventor after: Zhan Weihua Inventor before: Huang Haoxiang Inventor before: Liu Chongkun |
|
| CB03 | Change of inventor or designer information | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20201229 Address after: 25 Financial Street, Xicheng District, Beijing 100033 Applicant after: China Construction Bank Corp. Address before: Room 704, building 18, No. 998, Wenyi West Road, Wuchang Street, Yuhang District, Hangzhou City, Zhejiang Province Applicant before: TONGDUN HOLDINGS Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200811 |
|
| RJ01 | Rejection of invention patent application after publication |