CN108399458B - Deep learning model training system constructed based on SAAS - Google Patents
Deep learning model training system constructed based on SAAS Download PDFInfo
- Publication number
- CN108399458B CN108399458B CN201810250383.6A CN201810250383A CN108399458B CN 108399458 B CN108399458 B CN 108399458B CN 201810250383 A CN201810250383 A CN 201810250383A CN 108399458 B CN108399458 B CN 108399458B
- Authority
- CN
- China
- Prior art keywords
- training
- saas
- server
- gateway
- deep learning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a method and a system for constructing a safety system based on a domain mechanism by a deep learning model training system constructed based on SAAS (software as a service), wherein the method comprises the following steps: the SAAS consumer is connected with the training server through the gateway to upload and manage a training set, and the training set is distributed to the trainers to run training after being checked by the training manager; the application server comprises an application manager and a plurality of users, the SAAS consumer is connected with the application server through a gateway, the application manager distributes or starts an application person to run a trained model, and then the SAAS consumer transmits input data through the gateway and calls the trained model on the application person to process the input data. The invention does not require the consumer to know much expertise about deep learning; expensive hardware cost is saved; the customer can pay according to the needs, and convenient to use is nimble.
Description
Technical Field
The invention relates to a deep learning model training system, in particular to a deep learning model training system constructed based on SAAS (software as a service), and belongs to the technical field of computers.
Background
The concept of deep learning is derived from the research of artificial neural networks, and is a new field in machine learning research, and the motivation is to establish and simulate a neural network for analyzing and learning of human brain, which simulates the mechanism of human brain to interpret data, such as images, sounds and texts.
Many people want the self-trained deep learning model to help them solve more detailed problems, but the process requires consumers to know much professional knowledge about deep learning and expensive hardware support, which greatly hinders the popularization and application of deep learning.
Therefore, the deep learning model training system constructed based on the SAAS (software, namely service) is developed, and the deep learning model for training the personal customized training system and the SAAS system for paying on demand related to the deep learning model are provided, so that the system is very necessary and has an important application prospect.
Disclosure of Invention
The invention aims at the problems in the prior art, discloses a deep learning model training system constructed based on SAAS, and provides a deep learning model for training a personal customized training system and an SAAS system for paying according to needs of the deep learning model.
In order to achieve the above object, the technical scheme adopted by the invention is as follows:
a deep learning model training system constructed based on SAAS comprises: a gateway (1), a training server (2), an application server (3), a distributed file system (4) and a database (5),
the training server (2) comprises a training manager (21) and a plurality of trainers (22), SAAS consumers are connected with the training server (2) through the gateway (2) to upload and manage a training set, the training set is distributed to the trainers (22) to run and train after being checked by the training manager (21), and trained models are stored after the training set is finished;
the application server (3) comprises an application manager (31) and a plurality of users (32), an SAAS consumer is connected with the application server (3) through the gateway (2), the application manager (31) distributes or starts one of the users (32) to run the trained model, and then the SAAS consumer transmits input data through the gateway (2), calls the trained model on the user (32) to process the input data, and returns a result after the input data is processed;
the distributed file system (4) stores a training set and a trained model uploaded by SAAS consumers; the database (5) stores and manages metadata.
In the deep learning model training system constructed based on the SAAS, the application manager (31) allocates, starts, supervises and stops idle applications, and the training model in the applications is uninstalled through the application manager (32).
In the deep learning model training system constructed based on the SAAS, the training server (2) is a server or a server cluster.
In the deep learning model training system constructed based on the SAAS, the application server (3) is a server or a server cluster.
In the deep learning model training system constructed based on the SAAS, the gateway (1) is a gateway including a load balancing function and a charging function.
In the deep learning model training system constructed based on SAAS, the distributed file system (4) includes, but is not limited to, the following file systems: NFS, AFS, KFS, DFS.
Compared with the prior art, the invention has the advantages that:
(1) the consumer is not required to know much expertise about deep learning;
(2) expensive hardware cost is saved;
(3) the customer can pay according to the needs, and the use is convenient and flexible.
Drawings
FIG. 1 is a system diagram illustrating an embodiment of a deep learning model training system based on SAAS construction according to the present invention;
FIG. 2 is a flow chart of model training of one embodiment of the SAAS-based deep learning model training system of FIG. 1;
FIG. 3 is a flow chart of a model application of one embodiment of the deep learning model training system constructed based on SAAS in FIG. 1.
Wherein:
1-gateway 2-training server
21-training manager 22-trainer
3-application Server 31-application manager
32-user 4-distributed file system
5-database.
Detailed Description
The invention is described in detail below with reference to the figures and the embodiments.
Referring to fig. 1, fig. 2 and fig. 3, the deep learning model training system constructed based on SAAS of the present invention includes: a gateway 1, a training server 2, an application server 3, a distributed file system 4 and a database 5,
the training server 2 comprises a training manager 21 and a plurality of trainers 22, SAAS consumers are connected with the training server 2 through a gateway 2 to upload and manage a training set, the training set is distributed to the trainers 22 to run and train after being checked by the training manager 21, and trained models are stored after the training is finished;
the training set is a set of data that has been classified (labeled) for training a machine learning model, for example, training a machine learning model that distinguishes between cat and dog photos, requiring a set of cat photos and a set of dog photos as the training set; for another example, the training distinguishes between a machine learning model with a high-ranking and a low-ranking, and the words of the three types of ranking are required to be used as a training set.
The application server 3 comprises an application manager 31 and a plurality of application persons 32, the SAAS consumer is connected with the application server 3 through the gateway 2, the application manager 31 distributes or starts one of the application persons 32 to run the trained model, then the SAAS consumer transmits input data through the gateway 2, calls the trained model on the application persons 32 to process the input data, and returns the result after the input data is finished;
the distributed file system 4 stores the training set and the trained model uploaded by the SAAS consumer; the database 5 stores and manages metadata; metadata is data of data, and in the present system, the summary information, training configuration information, and the like of a training set and a model, a model name, a classification of the training set, a sample data size included in each classification, the number of iterations of the training model, a learning rate, and the like are referred to.
As a preferred solution, the application manager 31 allocates, starts, supervises and stops idle applications, and unloads the training models in the applications through the application users 32.
As a preferred solution, the training server 2 is a server or a server cluster.
As a preferred solution, the application server 3 is a server or a server cluster.
As a preferred solution, the gateway 1 is a gateway comprising a load balancing function and a charging function.
As a preferred approach, the distributed file system 4 includes, but is not limited to, the following types of file systems: NFS, AFS, KFS, DFS, preferably NFS.
It should be understood that the above-mentioned embodiments are merely preferred embodiments of the present invention, and not intended to limit the present invention, and any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (4)
1. A deep learning model training system constructed based on SAAS (software as a service), comprising: a gateway (1), a training server (2), an application server (3), a distributed file system (4) and a database (5),
the training server (2) comprises a training manager (21) and a plurality of trainers (22), SAAS consumers are connected with the training server (2) through the gateway (2) to upload and manage a training set, the training set is distributed to the trainers (22) to run and train after being checked by the training manager (21), and trained models are stored after the training is finished;
the application server (3) comprises an application manager (31) and a plurality of users (32), the SAAS consumer is connected with the application server (3) through the gateway (2), the application manager (31) distributes or starts one of the users (32) to run the trained model, and then the SAAS consumer transmits input data through the gateway (2), calls the trained model on the user (32) to process the input data, and returns the result after the input data is finished;
the distributed file system (4) stores a training set and a trained model uploaded by SAAS consumers; the database (5) stores and manages metadata, wherein:
the application manager (31) allocates, starts, supervises and stops idle applications, uninstalls training models in applications by the user (32);
the gateway (1) is a gateway comprising a load balancing function and a charging function.
2. A SAAS-based deep learning model training system according to claim 1, wherein the training server (2) is a server or a server cluster.
3. A SAAS-based deep learning model training system according to claim 1, wherein the application server (3) is a server or a server cluster.
4. A SAAS-based deep learning model training system according to claim 1, wherein the distributed file system (4) includes but is not limited to the following types of file systems: NFS, AFS, KFS, DFS.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810250383.6A CN108399458B (en) | 2018-03-26 | 2018-03-26 | Deep learning model training system constructed based on SAAS |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810250383.6A CN108399458B (en) | 2018-03-26 | 2018-03-26 | Deep learning model training system constructed based on SAAS |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108399458A CN108399458A (en) | 2018-08-14 |
| CN108399458B true CN108399458B (en) | 2022-07-08 |
Family
ID=63093141
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810250383.6A Active CN108399458B (en) | 2018-03-26 | 2018-03-26 | Deep learning model training system constructed based on SAAS |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108399458B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109241141B (en) * | 2018-09-04 | 2022-04-29 | 北京百度网讯科技有限公司 | Deep learning training data processing method and device |
| CN109460826A (en) * | 2018-10-31 | 2019-03-12 | 北京字节跳动网络技术有限公司 | For distributing the method, apparatus and model modification system of data |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104598239A (en) * | 2015-01-19 | 2015-05-06 | 中国传媒大学 | Software issuing and using system, and software using and issuing method based on system |
| CN106529673A (en) * | 2016-11-17 | 2017-03-22 | 北京百度网讯科技有限公司 | Deep learning network training method and device based on artificial intelligence |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160125039A1 (en) * | 2014-10-30 | 2016-05-05 | Szegedi Tudományegyetem | Data mining method and apparatus, and computer program product for carrying out the method |
| CN107370796B (en) * | 2017-06-30 | 2021-01-08 | 深圳致星科技有限公司 | Intelligent learning system based on Hyper TF |
| CN107563417A (en) * | 2017-08-18 | 2018-01-09 | 北京天元创新科技有限公司 | A kind of deep learning artificial intelligence model method for building up and system |
-
2018
- 2018-03-26 CN CN201810250383.6A patent/CN108399458B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104598239A (en) * | 2015-01-19 | 2015-05-06 | 中国传媒大学 | Software issuing and using system, and software using and issuing method based on system |
| CN106529673A (en) * | 2016-11-17 | 2017-03-22 | 北京百度网讯科技有限公司 | Deep learning network training method and device based on artificial intelligence |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108399458A (en) | 2018-08-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20230222341A1 (en) | Targeted crowd sourcing for metadata management across data sets | |
| US12093837B2 (en) | Building a federated learning framework | |
| US11176318B2 (en) | Medical network | |
| US10923115B2 (en) | Dynamically generated dialog | |
| CN108734293B (en) | Task management system, method and device | |
| CN107145395B (en) | Method and device for processing task | |
| US11102276B2 (en) | System and method for providing more appropriate question/answer responses based upon profiles | |
| CN109040157A (en) | A kind of recorded broadcast data Learning-memory behavior method Internet-based | |
| US10664929B2 (en) | Extracting data for professional event and participant grouping recommendations | |
| US11789980B2 (en) | Method, system, and non-transitory computer readable record medium for providing multi profile | |
| CN107910060A (en) | Method and apparatus for generating information | |
| CN108399458B (en) | Deep learning model training system constructed based on SAAS | |
| KR20210150838A (en) | System for providing virtual space renting service using virtual space archive | |
| US20180053431A1 (en) | Computer architecture for customizing the content of publications and multimedia | |
| CN109101309A (en) | For updating user interface method and device | |
| CN110490450A (en) | Biological information management system based on mixed cloud | |
| WO2015039105A1 (en) | System and method for distributed virtual assistant platforms | |
| KR102641638B1 (en) | Method, device, and system for providing a platform service for writing and managing manuscripts based on a generative artificial intelligence model | |
| US11455555B1 (en) | Methods, mediums, and systems for training a model | |
| Damerow et al. | The giles ecosystem–storage, text extraction, and OCR of documents | |
| CN116956117A (en) | Method, device, equipment, storage medium and program product for identifying label | |
| CN116955698A (en) | Matching model training method and device, electronic equipment and storage medium | |
| CN115730812A (en) | Index information generation method and device, electronic equipment and readable medium | |
| CN104636412A (en) | Method and system for personalizing data for device | |
| JP7530528B1 (en) | Information processing method, information processing device, and information processing program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |