CN108399458B

CN108399458B - Deep learning model training system constructed based on SAAS

Info

Publication number: CN108399458B
Application number: CN201810250383.6A
Authority: CN
Inventors: 廖志杰
Original assignee: Hangzhou Shufeng Technology Co ltd; Chengdu Ruima Technology Co ltd
Current assignee: Hangzhou Shufeng Technology Co ltd; Chengdu Ruima Technology Co ltd
Priority date: 2018-03-26
Filing date: 2018-03-26
Publication date: 2022-07-08
Anticipated expiration: 2038-03-26
Also published as: CN108399458A

Abstract

The invention discloses a method and a system for constructing a safety system based on a domain mechanism by a deep learning model training system constructed based on SAAS (software as a service), wherein the method comprises the following steps: the SAAS consumer is connected with the training server through the gateway to upload and manage a training set, and the training set is distributed to the trainers to run training after being checked by the training manager; the application server comprises an application manager and a plurality of users, the SAAS consumer is connected with the application server through a gateway, the application manager distributes or starts an application person to run a trained model, and then the SAAS consumer transmits input data through the gateway and calls the trained model on the application person to process the input data. The invention does not require the consumer to know much expertise about deep learning; expensive hardware cost is saved; the customer can pay according to the needs, and convenient to use is nimble.

Description

Deep learning model training system constructed based on SAAS

Technical Field

The invention relates to a deep learning model training system, in particular to a deep learning model training system constructed based on SAAS (software as a service), and belongs to the technical field of computers.

Background

The concept of deep learning is derived from the research of artificial neural networks, and is a new field in machine learning research, and the motivation is to establish and simulate a neural network for analyzing and learning of human brain, which simulates the mechanism of human brain to interpret data, such as images, sounds and texts.

Many people want the self-trained deep learning model to help them solve more detailed problems, but the process requires consumers to know much professional knowledge about deep learning and expensive hardware support, which greatly hinders the popularization and application of deep learning.

Therefore, the deep learning model training system constructed based on the SAAS (software, namely service) is developed, and the deep learning model for training the personal customized training system and the SAAS system for paying on demand related to the deep learning model are provided, so that the system is very necessary and has an important application prospect.

Disclosure of Invention

The invention aims at the problems in the prior art, discloses a deep learning model training system constructed based on SAAS, and provides a deep learning model for training a personal customized training system and an SAAS system for paying according to needs of the deep learning model.

In order to achieve the above object, the technical scheme adopted by the invention is as follows:

a deep learning model training system constructed based on SAAS comprises: a gateway (1), a training server (2), an application server (3), a distributed file system (4) and a database (5),

the training server (2) comprises a training manager (21) and a plurality of trainers (22), SAAS consumers are connected with the training server (2) through the gateway (2) to upload and manage a training set, the training set is distributed to the trainers (22) to run and train after being checked by the training manager (21), and trained models are stored after the training set is finished;

the application server (3) comprises an application manager (31) and a plurality of users (32), an SAAS consumer is connected with the application server (3) through the gateway (2), the application manager (31) distributes or starts one of the users (32) to run the trained model, and then the SAAS consumer transmits input data through the gateway (2), calls the trained model on the user (32) to process the input data, and returns a result after the input data is processed;

the distributed file system (4) stores a training set and a trained model uploaded by SAAS consumers; the database (5) stores and manages metadata.

In the deep learning model training system constructed based on the SAAS, the application manager (31) allocates, starts, supervises and stops idle applications, and the training model in the applications is uninstalled through the application manager (32).

In the deep learning model training system constructed based on the SAAS, the training server (2) is a server or a server cluster.

In the deep learning model training system constructed based on the SAAS, the application server (3) is a server or a server cluster.

In the deep learning model training system constructed based on the SAAS, the gateway (1) is a gateway including a load balancing function and a charging function.

In the deep learning model training system constructed based on SAAS, the distributed file system (4) includes, but is not limited to, the following file systems: NFS, AFS, KFS, DFS.

Compared with the prior art, the invention has the advantages that:

(1) the consumer is not required to know much expertise about deep learning;

(2) expensive hardware cost is saved;

(3) the customer can pay according to the needs, and the use is convenient and flexible.

Drawings

FIG. 1 is a system diagram illustrating an embodiment of a deep learning model training system based on SAAS construction according to the present invention;

FIG. 2 is a flow chart of model training of one embodiment of the SAAS-based deep learning model training system of FIG. 1;

FIG. 3 is a flow chart of a model application of one embodiment of the deep learning model training system constructed based on SAAS in FIG. 1.

Wherein:

1-gateway 2-training server

21-training manager 22-trainer

3-application Server 31-application manager

32-user 4-distributed file system

5-database.

Detailed Description

The invention is described in detail below with reference to the figures and the embodiments.

Referring to fig. 1, fig. 2 and fig. 3, the deep learning model training system constructed based on SAAS of the present invention includes: a gateway 1, a training server 2, an application server 3, a distributed file system 4 and a database 5,

the training server 2 comprises a training manager 21 and a plurality of trainers 22, SAAS consumers are connected with the training server 2 through a gateway 2 to upload and manage a training set, the training set is distributed to the trainers 22 to run and train after being checked by the training manager 21, and trained models are stored after the training is finished;

the training set is a set of data that has been classified (labeled) for training a machine learning model, for example, training a machine learning model that distinguishes between cat and dog photos, requiring a set of cat photos and a set of dog photos as the training set; for another example, the training distinguishes between a machine learning model with a high-ranking and a low-ranking, and the words of the three types of ranking are required to be used as a training set.

The application server 3 comprises an application manager 31 and a plurality of application persons 32, the SAAS consumer is connected with the application server 3 through the gateway 2, the application manager 31 distributes or starts one of the application persons 32 to run the trained model, then the SAAS consumer transmits input data through the gateway 2, calls the trained model on the application persons 32 to process the input data, and returns the result after the input data is finished;

the distributed file system 4 stores the training set and the trained model uploaded by the SAAS consumer; the database 5 stores and manages metadata; metadata is data of data, and in the present system, the summary information, training configuration information, and the like of a training set and a model, a model name, a classification of the training set, a sample data size included in each classification, the number of iterations of the training model, a learning rate, and the like are referred to.

As a preferred solution, the application manager 31 allocates, starts, supervises and stops idle applications, and unloads the training models in the applications through the application users 32.

As a preferred solution, the training server 2 is a server or a server cluster.

As a preferred solution, the application server 3 is a server or a server cluster.

As a preferred solution, the gateway 1 is a gateway comprising a load balancing function and a charging function.

As a preferred approach, the distributed file system 4 includes, but is not limited to, the following types of file systems: NFS, AFS, KFS, DFS, preferably NFS.

It should be understood that the above-mentioned embodiments are merely preferred embodiments of the present invention, and not intended to limit the present invention, and any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A deep learning model training system constructed based on SAAS (software as a service), comprising: a gateway (1), a training server (2), an application server (3), a distributed file system (4) and a database (5),

the training server (2) comprises a training manager (21) and a plurality of trainers (22), SAAS consumers are connected with the training server (2) through the gateway (2) to upload and manage a training set, the training set is distributed to the trainers (22) to run and train after being checked by the training manager (21), and trained models are stored after the training is finished;

the application server (3) comprises an application manager (31) and a plurality of users (32), the SAAS consumer is connected with the application server (3) through the gateway (2), the application manager (31) distributes or starts one of the users (32) to run the trained model, and then the SAAS consumer transmits input data through the gateway (2), calls the trained model on the user (32) to process the input data, and returns the result after the input data is finished;

the distributed file system (4) stores a training set and a trained model uploaded by SAAS consumers; the database (5) stores and manages metadata, wherein:

the application manager (31) allocates, starts, supervises and stops idle applications, uninstalls training models in applications by the user (32);

the gateway (1) is a gateway comprising a load balancing function and a charging function.

2. A SAAS-based deep learning model training system according to claim 1, wherein the training server (2) is a server or a server cluster.

3. A SAAS-based deep learning model training system according to claim 1, wherein the application server (3) is a server or a server cluster.

4. A SAAS-based deep learning model training system according to claim 1, wherein the distributed file system (4) includes but is not limited to the following types of file systems: NFS, AFS, KFS, DFS.