CN110941421A - Development machine learning device and using method thereof - Google Patents
Development machine learning device and using method thereof Download PDFInfo
- Publication number
- CN110941421A CN110941421A CN201911205340.7A CN201911205340A CN110941421A CN 110941421 A CN110941421 A CN 110941421A CN 201911205340 A CN201911205340 A CN 201911205340A CN 110941421 A CN110941421 A CN 110941421A
- Authority
- CN
- China
- Prior art keywords
- platform
- algorithm
- management
- machine learning
- deep learning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/20—Software design
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention discloses a development machine learning device, comprising: the machine learning platform is a platform for mining value information from mass data based on Huawei fusion instrumentation HD distributed storage and parallel computing technology; the deep learning platform is an enterprise-level deep learning modeling platform, and can enable client algorithm developers to efficiently manage data sets, develop algorithm codes, evaluate models and predict service release experience, and reduce deep learning modeling thresholds; and the reasoning platform is mainly used for completing multi-algorithm unified management and task containerization heterogeneous resource unified scheduling, and assisting the clients to realize cluster computing power sharing and reduce the operation and maintenance cost of the AI system. The invention provides a one-stop development platform for developers, provides massive data preprocessing and semi-automatic labeling, large-scale distributed training, automatic model generation and end-edge-cloud model on-demand deployment capability, helps users to quickly create and deploy models, and manages full-period AI workflow.
Description
Technical Field
The invention belongs to the technical field of artificial intelligence, and particularly relates to a development machine learning device and a using method thereof.
Background
AI technology, especially machine learning technology represented by deep learning, has been rapidly developed in recent years, and gradually falls to multiple industries to achieve a good application effect. Landing of AI benefits from three aspects:
data is petroleum of AI, data acquisition means are more and more abundant, data processing cost is more and more low, and the data volume of accumulating of each trade is exponential growth, provides the most firm guarantee for the application of AI technique.
AI chips have been increasingly powerful in computing power, and in recent years, Nvidia companies have successively introduced P4, P40, P100, and V100 series GPU cards. Domestic AI chips are also successively introduced by companies represented by Hua, and the competition and prosperity of AI hardware promote the continuous increase of AI computing power.
The algorithm is the core of AI. After the traditional deep learning CNN/RNN series classical model, the reinforcement learning and confrontation network algorithm model is continuously emerged. The success of AlphaGo, Master should be mainly attributed to the new powerful AI algorithm.
The existing learning device has the disadvantages of larger data volume, expensive acceleration resources, difficulty in obtaining, time-consuming calculation process, various tools, long learning period and more complex model, and therefore a development machine learning device and a use method thereof are provided to solve the problems mentioned in the background technology.
Disclosure of Invention
The present invention is directed to a development machine learning apparatus and a method for using the same to solve the problems of the background art.
In order to achieve the purpose, the invention provides the following technical scheme: a development machine learning apparatus comprising:
the machine learning platform is a platform for mining value information from mass data based on Huawei fusion instrumentation HD distributed storage and parallel computing technology;
the deep learning platform is an enterprise-level deep learning modeling platform, integrates mainstream TensorFlow, MxNet, Pythrch and Caffe frames, enables client algorithm developers to efficiently manage data sets, develop algorithm codes, evaluate models and predict service release experience, and reduces deep learning modeling thresholds;
the inference platform is mainly used for completing multi-algorithm unified management and task containerization heterogeneous resource unified scheduling, is suitable for deploying online inference and offline batch processing applications based on framework deep learning algorithms such as TensorFlow, Pythrch, Caffe and MxNet, can be widely applied to large-scale parallel task computing scenes such as video analysis, image processing and log analysis, and can assist customers to achieve cluster computing power sharing and reduce operation and maintenance cost of an AI system.
Preferably, the machine learning platform presets an algorithm model, and provides end-to-end capabilities of data preprocessing, feature engineering, visualization and interactive modeling, model evaluation and model deployment.
Preferably, the deep learning platform provides end-to-end modeling development capabilities of data set management, notebook environment code development, model training and evaluation management, model management and prediction service release management for developers with a certain algorithm basis.
Preferably, the reasoning platform comprises an algorithm bin and a Batch, and the algorithm bin is responsible for unified management of multiple manufacturers and multiple algorithms; the Batch is responsible for uniformly managing heterogeneous resources such as a CPU, a memory and a GPU and uniformly scheduling tasks.
Preferably, system management is also included, including user management, security management, service management, and integrated management.
The invention also provides a use method for developing the machine learning device, which specifically comprises the following steps:
s1, mining value information from the mass data by the machine learning platform;
s2, a deep learning platform integrates mainstream TensorFlow, MxNet, Pythrch and Caffe frames, and enables client algorithm developers to efficiently manage data sets, develop algorithm codes, evaluate models and predict service release experience;
and S3, completing multi-algorithm unified management and task containerization heterogeneous resource unified scheduling.
Compared with the prior art, the invention has the beneficial effects that: the invention provides a development machine learning device and a using method thereof, and the development machine learning device is a one-stop development platform for developers, provides mass data preprocessing and semi-automatic labeling, large-scale distributed training, automatic model generation and end-edge-cloud model on-demand deployment capability, helps users to quickly create and deploy models, and manages a full-period AI workflow.
Drawings
FIG. 1 is a diagram illustrating a system architecture for developing a machine learning apparatus according to the present invention;
FIG. 2 is a diagram of a machine learning platform architecture according to the present invention;
FIG. 3 is a diagram illustrating a deep learning platform architecture according to the present invention;
FIG. 4 is a diagram of the inference platform architecture of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1-4, the embodiment is as follows:
a development machine learning apparatus comprising:
the machine learning platform is a platform for mining value information from mass data based on Huawei fusion instrumentation HD distributed storage and parallel computing technology;
the deep learning platform is an enterprise-level deep learning modeling platform, integrates mainstream TensorFlow, MxNet, Pythrch and Caffe frames, enables client algorithm developers to efficiently manage data sets, develop algorithm codes, evaluate models and predict service release experience, and reduces deep learning modeling thresholds;
the inference platform is mainly used for completing multi-algorithm unified management and task containerization heterogeneous resource unified scheduling, is suitable for deploying online inference and offline batch processing applications based on framework deep learning algorithms such as TensorFlow, Pythrch, Caffe and MxNet, can be widely applied to large-scale parallel task computing scenes such as video analysis, image processing and log analysis, and can assist customers to achieve cluster computing power sharing and reduce operation and maintenance cost of an AI system.
Specifically, the machine learning platform presets an algorithm model, and provides end-to-end capabilities of data preprocessing, feature engineering, visualization and interactive modeling, model evaluation and model deployment.
Specifically, the deep learning platform provides end-to-end modeling development capabilities of data set management, notebook environment code development, model training and evaluation management, model management and prediction service release management for developers with a certain algorithm basis.
Specifically, the reasoning platform comprises an algorithm bin and a Batch, wherein the algorithm bin is responsible for unified management of multiple manufacturers and multiple algorithms; the Batch is responsible for uniformly managing heterogeneous resources such as a CPU, a memory and a GPU and uniformly scheduling tasks.
Specifically, system management is also included, which includes user management, security management, service management, and integrated management.
The invention also provides a use method for developing the machine learning device, which specifically comprises the following steps:
s1, mining value information from the mass data by the machine learning platform;
s2, a deep learning platform integrates mainstream TensorFlow, MxNet, Pythrch and Caffe frames, and enables client algorithm developers to efficiently manage data sets, develop algorithm codes, evaluate models and predict service release experience;
and S3, completing multi-algorithm unified management and task containerization heterogeneous resource unified scheduling.
In summary, compared with the prior art, the invention is a developer-oriented one-stop development platform, and provides massive data preprocessing, semi-automatic labeling, large-scale distributed training, automatic model generation, and end-edge-cloud model on-demand deployment capability, so as to help users to quickly create and deploy models and manage full-period AI workflows.
Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that modifications may be made to the embodiments or portions thereof without departing from the spirit and scope of the invention.
Claims (6)
1. A development machine learning apparatus, comprising:
the machine learning platform is a platform for mining value information from mass data based on Huawei fusion instrumentation HD distributed storage and parallel computing technology;
the deep learning platform is an enterprise-level deep learning modeling platform, integrates mainstream TensorFlow, MxNet, Pythrch and Caffe frames, enables client algorithm developers to efficiently manage data sets, develop algorithm codes, evaluate models and predict service release experience, and reduces deep learning modeling thresholds;
the inference platform is mainly used for completing multi-algorithm unified management and task containerization heterogeneous resource unified scheduling, is suitable for deploying online inference and offline batch processing applications based on framework deep learning algorithms such as TensorFlow, Pythrch, Caffe and MxNet, can be widely applied to large-scale parallel task computing scenes such as video analysis, image processing and log analysis, and can assist customers to achieve cluster computing power sharing and reduce operation and maintenance cost of an AI system.
2. The development machine learning apparatus according to claim 1, characterized in that: the machine learning platform presets an algorithm model and provides end-to-end capabilities of data preprocessing, feature engineering, visualization and interactive modeling, model evaluation and model deployment.
3. The development machine learning apparatus according to claim 1, characterized in that: the deep learning platform provides end-to-end modeling development capabilities of data set management, notebook environment code development, model training and evaluation management, model management and prediction service release management for developers with a certain algorithm basis.
4. The development machine learning apparatus according to claim 1, characterized in that: the reasoning platform comprises an algorithm bin and a Batch, and the algorithm bin is responsible for unified management of multiple manufacturers and multiple algorithms; the Batch is responsible for uniformly managing heterogeneous resources such as a CPU, a memory and a GPU and uniformly scheduling tasks.
5. The development machine learning apparatus according to claim 1, characterized in that: system management is also included, including user management, security management, service management, and integrated management.
6. A method of using the development machine learning apparatus according to claim 1, characterized in that: the method specifically comprises the following steps:
s1, mining value information from the mass data by the machine learning platform;
s2, a deep learning platform integrates mainstream TensorFlow, MxNet, Pythrch and Caffe frames, and enables client algorithm developers to efficiently manage data sets, develop algorithm codes, evaluate models and predict service release experience;
and S3, completing multi-algorithm unified management and task containerization heterogeneous resource unified scheduling.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911205340.7A CN110941421A (en) | 2019-11-29 | 2019-11-29 | Development machine learning device and using method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911205340.7A CN110941421A (en) | 2019-11-29 | 2019-11-29 | Development machine learning device and using method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110941421A true CN110941421A (en) | 2020-03-31 |
Family
ID=69909453
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911205340.7A Pending CN110941421A (en) | 2019-11-29 | 2019-11-29 | Development machine learning device and using method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110941421A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111752554A (en) * | 2020-05-18 | 2020-10-09 | 南京认知物联网研究院有限公司 | Multi-model cooperation system and method based on model arrangement |
| CN111897664A (en) * | 2020-08-03 | 2020-11-06 | 中关村科学城城市大脑股份有限公司 | Allocation system and method for AI algorithm and AI model applied to urban brain |
| CN112445462A (en) * | 2020-11-16 | 2021-03-05 | 北京思特奇信息技术股份有限公司 | Artificial intelligence modeling platform and method based on object-oriented design |
| CN113590953A (en) * | 2021-07-30 | 2021-11-02 | 郑州轻工业大学 | Deep learning-based recommendation algorithm library |
| US11520564B2 (en) | 2021-01-20 | 2022-12-06 | International Business Machines Corporation | Intelligent recommendations for program code |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108881446A (en) * | 2018-06-22 | 2018-11-23 | 深源恒际科技有限公司 | A kind of artificial intelligence plateform system based on deep learning |
| CN109272116A (en) * | 2018-09-05 | 2019-01-25 | 郑州云海信息技术有限公司 | A kind of method and device of deep learning |
| US20190065994A1 (en) * | 2017-08-23 | 2019-02-28 | Boe Technology Group Co., Ltd. | Deep learning-based image recognition method and apparatus |
-
2019
- 2019-11-29 CN CN201911205340.7A patent/CN110941421A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190065994A1 (en) * | 2017-08-23 | 2019-02-28 | Boe Technology Group Co., Ltd. | Deep learning-based image recognition method and apparatus |
| CN108881446A (en) * | 2018-06-22 | 2018-11-23 | 深源恒际科技有限公司 | A kind of artificial intelligence plateform system based on deep learning |
| CN109272116A (en) * | 2018-09-05 | 2019-01-25 | 郑州云海信息技术有限公司 | A kind of method and device of deep learning |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111752554A (en) * | 2020-05-18 | 2020-10-09 | 南京认知物联网研究院有限公司 | Multi-model cooperation system and method based on model arrangement |
| CN111752554B (en) * | 2020-05-18 | 2021-03-12 | 南京认知物联网研究院有限公司 | Multi-model cooperation system and method based on model arrangement |
| CN111897664A (en) * | 2020-08-03 | 2020-11-06 | 中关村科学城城市大脑股份有限公司 | Allocation system and method for AI algorithm and AI model applied to urban brain |
| CN112445462A (en) * | 2020-11-16 | 2021-03-05 | 北京思特奇信息技术股份有限公司 | Artificial intelligence modeling platform and method based on object-oriented design |
| US11520564B2 (en) | 2021-01-20 | 2022-12-06 | International Business Machines Corporation | Intelligent recommendations for program code |
| CN113590953A (en) * | 2021-07-30 | 2021-11-02 | 郑州轻工业大学 | Deep learning-based recommendation algorithm library |
| CN113590953B (en) * | 2021-07-30 | 2023-07-18 | 郑州轻工业大学 | Recommendation algorithm system based on deep learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110941421A (en) | Development machine learning device and using method thereof | |
| CN109491790B (en) | Container-based industrial Internet of things edge computing resource allocation method and system | |
| CN105046327B (en) | A kind of intelligent grid information system and method based on machine learning techniques | |
| CN108255605A (en) | Image recognition cooperative computing method and system based on neural network | |
| Bellavista et al. | Machine learning for predictive diagnostics at the edge: An IIoT practical example | |
| CN115511501A (en) | Data processing method, computer equipment and readable storage medium | |
| CN112581578A (en) | Cloud rendering system based on software definition | |
| Onoufriou et al. | Nemesyst: A hybrid parallelism deep learning-based framework applied for internet of things enabled food retailing refrigeration systems | |
| CN111062521B (en) | Online prediction method, system and server | |
| CN113516331A (en) | Building data processing method and device | |
| Kumar et al. | Association learning based hybrid model for cloud workload prediction | |
| CN112036483A (en) | Object prediction classification method and device based on AutoML, computer equipment and storage medium | |
| WO2023129164A1 (en) | Digital twin sequential and temporal learning and explaining | |
| CN117240887B (en) | Wisdom thing networking energy management platform system | |
| CN113627032A (en) | Intelligent decision method for equipment design/maintenance scheme based on digital twin | |
| CN116341131B (en) | Remanufacturing design simulation system, method, equipment and medium based on digital twin | |
| CN110766163B (en) | System for implementing machine learning process | |
| Zhou et al. | Cushion: A proactive resource provisioning method to mitigate SLO violations for containerized microservices | |
| Li et al. | Research and applications of cloud manufacturing in China | |
| CN113190328A (en) | System identification-oriented containerized cloud workflow processing system and method | |
| Baig et al. | Bit rate reduction in cloud gaming using object detection technique | |
| CN114596054A (en) | Service information management method and system for digital office | |
| CN114819367A (en) | Public service platform based on industrial internet | |
| CN106530110A (en) | Big-data-based oceanographic engineering management system and method | |
| CN113157252B (en) | Electromagnetic signal general distributed intelligent processing and analyzing platform and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200331 |