CN109117266B - Video artificial intelligence training platform based on multilayer framework - Google Patents
Video artificial intelligence training platform based on multilayer framework Download PDFInfo
- Publication number
- CN109117266B CN109117266B CN201810770495.4A CN201810770495A CN109117266B CN 109117266 B CN109117266 B CN 109117266B CN 201810770495 A CN201810770495 A CN 201810770495A CN 109117266 B CN109117266 B CN 109117266B
- Authority
- CN
- China
- Prior art keywords
- platform
- node
- resource
- slave
- resources
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013473 artificial intelligence Methods 0.000 title claims abstract description 55
- 238000012549 training Methods 0.000 title claims abstract description 23
- 238000004891 communication Methods 0.000 claims abstract description 5
- 230000010354 integration Effects 0.000 claims abstract description 4
- 230000003993 interaction Effects 0.000 claims abstract description 4
- 238000002372 labelling Methods 0.000 claims description 16
- 238000004422 calculation algorithm Methods 0.000 claims description 14
- 238000000034 method Methods 0.000 claims description 8
- 238000007619 statistical method Methods 0.000 claims description 7
- 238000004458 analytical method Methods 0.000 claims description 3
- 238000005457 optimization Methods 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
Abstract
The invention provides a video artificial intelligence training platform based on a multilayer framework, which comprises: the system comprises a plurality of platform nodes, a plurality of network nodes and a plurality of network nodes, wherein each platform node has a fixed authority value, and after each platform node establishes communication, the affiliation of each platform node is determined according to the respective authority value; when the authority values of two platform nodes are equal, the two platform nodes are only used as data integration and interaction of the nodes, and the authority of data resources is not distributed between the two platform nodes; when the authority value of one platform node is larger than that of the other platform node, the platform superior node with the larger authority value is the platform superior node; the lower authority value is the lower node of the platform; after the platform upper node obtains the resource management distribution authority of the platform lower node, scheduling all resources of the platform lower node; wherein the resources of each platform node include: the system comprises video input resources, artificial intelligence computing resources of each node, system storage resources and label marking resources.
Description
Technical Field
The invention relates to the technical field of videos, in particular to a video artificial intelligence training platform based on a multilayer framework.
Background
In the current process of processing video identification by artificial intelligence and optimizing the artificial intelligence algorithm of the video, a computer with high computing power is basically adopted as an artificial intelligence computing unit, after a certain artificial intelligence algorithm is developed, the artificial intelligence algorithm is loaded to the computing unit, and the artificial intelligence algorithm is optimized through a large amount of data samples, so that a more approved artificial intelligence algorithm is obtained finally. The optimization and learning process of the artificial intelligence algorithm is greatly limited, and in the aspect of a large number of data samples, the work of acquiring a large number of data samples often exists; however, in the learning and training process, the artificial intelligence learning and training effect is not ideal due to insufficient data acquisition samples or insufficient special information covered by the data acquisition samples, so that the artificial intelligence algorithm cannot cope with some special situations in actual application. Moreover, even if enough artificial intelligence data samples exist, the problem that the result of the artificial intelligence algorithm needs to be manually judged to be wrong is solved, and if the wrong result of the artificial intelligence algorithm is determined by insufficient manpower, the optimization and the deep learning of the artificial intelligence algorithm are not caused.
Disclosure of Invention
Based on this, an object of the embodiments of the present invention is to provide a video artificial intelligence training platform based on a multi-layer architecture, so as to solve the existing problems.
In order to achieve the purpose, the embodiment of the invention adopts the following technical scheme:
a video artificial intelligence training platform based on a multi-layer architecture comprises: each platform node is provided with an artificial intelligence computing unit, a video label labeling unit and a video signal access unit;
the artificial intelligence computing unit can execute different artificial intelligence algorithms to realize different structural analyses of multi-channel video input; the video label labeling unit can label the accessed video signal;
after each platform node establishes communication, determining the subordination relation of each platform node according to the respective authority value;
when the authority values of two platform nodes are equal, the two platform nodes are only used as data integration and interaction of the nodes, and the authority of data resources is not distributed between the two platform nodes;
when the authority value of one platform node is larger than that of the other platform node, the platform superior node with the larger authority value is the platform superior node; the lower authority value is the lower node of the platform;
after the platform upper node obtains the resource management distribution authority of the platform lower node, scheduling all resources of the platform lower node; wherein the resources of each platform node include: the system comprises video input resources, artificial intelligence computing resources of each node, system storage resources and label marking resources.
Wherein each of the platform nodes comprises a master server and a slave server; the main server comprises a video input resource main service, an artificial intelligent computing resource main service of each node, a system storage resource main service and a label marking resource main service; the slave server comprises a video input resource slave service, an artificial intelligence computing resource slave service, a system storage resource slave service and a label labeling resource slave service;
the scheduling of all resources of the platform subordinate node includes: a platform upper node starts a main server, and a platform lower node starts a slave server; and performing resource scheduling on the slave servers of the lower nodes of the platform through the master server of the upper nodes of the platform.
Wherein, the resource scheduling of the slave server of the platform lower node by the master server of the platform upper node comprises:
the method comprises the following steps that a main server of a platform superior node simultaneously calls video input resource slave services, artificial intelligence computing resource slave services, system storage resource slave services and label labeling resource slave services of slave servers of a plurality of platform subordinate nodes;
and the platform superior node performs statistical analysis on the output result of the called platform subordinate node.
Wherein, the resource scheduling of the slave server of the platform lower node by the master server of the platform upper node comprises:
the main server of the platform superior node calls corresponding video input resource slave service, artificial intelligence computing resource slave service, system storage resource slave service and label labeling resource slave service according to the resource occupancy rate of the slave server of the platform subordinate node; and the platform superior node performs statistical analysis on the output result of the called platform subordinate node.
By utilizing the scheme of the invention, a large number of data samples can be obtained through the video input and other information input of each node; meanwhile, each node of the training platform has artificial intelligence computing capacity, and large-scale operation set can be realized through unified distribution of top-level nodes, so that huge operation capacity is obtained; a large amount of label marking information is also concentrated in the training platform, and the workload of a large amount of artificial judgment results in artificial intelligence learning can be reduced through the marking information, so that the optimization capability and the optimization time of artificial intelligence are greatly accelerated by the training platform.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings, there is shown in the drawings,
fig. 1 is a schematic structural diagram of a video artificial intelligence training platform based on a multi-layer architecture according to the present invention.
Fig. 2 is a schematic diagram of a platform upper node calling a platform lower node according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the detailed description and specific examples, while indicating the scope of the invention, are intended for purposes of illustration only and are not intended to limit the scope of the invention.
A video artificial intelligence training platform based on a multi-layer architecture comprises: each platform node is provided with an artificial intelligence computing unit, a video label labeling unit and a video signal access unit;
the artificial intelligence computing unit can execute different artificial intelligence algorithms to realize different structural analyses of multi-channel video input; the video label labeling unit can label the accessed video signal;
after each platform node establishes communication, determining the subordination relation of each platform node according to the respective authority value;
when the authority values of two platform nodes are equal, the two platform nodes are only used as data integration and interaction of the nodes, and the authority of data resources is not distributed between the two platform nodes;
when the authority value of one platform node is larger than that of the other platform node, the platform superior node with the larger authority value is the platform superior node; the lower authority value is the lower node of the platform;
after the platform upper node obtains the resource management distribution authority of the platform lower node, scheduling all resources of the platform lower node; wherein the resources of each platform node include: the system comprises video input resources, artificial intelligence computing resources of each node, system storage resources and label marking resources.
As an illustration, a structure diagram of a video artificial intelligence training platform based on a multi-layer architecture can be seen in fig. 1. Including platform node E, F, A, B, C, where the authority value of the platform node: e is more than F, A is more than B is more than C; then, platform node E is the upper node of platform node F, platform node F is the upper node of platform node A, B, C, and platform node A, B, C is the same level node.
Wherein each of the platform nodes comprises a master server and a slave server; the main server comprises a video input resource main service, an artificial intelligent computing resource main service of each node, a system storage resource main service and a label marking resource main service; the slave server comprises a video input resource slave service, an artificial intelligence computing resource slave service, a system storage resource slave service and a label labeling resource slave service;
the scheduling of all resources of the platform subordinate node includes: a platform upper node starts a main server, and a platform lower node starts a slave server; and performing resource scheduling on the slave servers of the lower nodes of the platform through the master server of the upper nodes of the platform.
The superior node of the platform has the functions of receiving input requests for artificial intelligence training and outputting artificial intelligence calculation training. The specific implementation process is as follows:
when the platform superior node receives the training input request, relevant resources in a resource pool of the master server and the slave server of the platform inferior node are checked, and data communication is carried out between the master server and the slave server of the platform inferior node, so that resource scheduling is realized. The number of the platform lower level nodes corresponding to the platform upper level nodes is possibly very large, so that the calculation function of the platform lower level nodes can be started to perform parallel calculation according to the training requirement of one path of video input, label marking resources can be simultaneously distributed with marks, a plurality of result outputs of a certain video input can be rapidly realized, the corresponding result outputs are subjected to statistical analysis at the upper level nodes, the result output of the artificial intelligence algorithm in a plurality of calculations can be obtained, and the optimization and improvement of the training result can be rapidly realized.
Wherein, the resource scheduling of the slave server of the platform lower node by the master server of the platform upper node comprises:
the method comprises the following steps that a main server of a platform superior node simultaneously calls video input resource slave services, artificial intelligence computing resource slave services, system storage resource slave services and label labeling resource slave services of slave servers of a plurality of platform subordinate nodes;
and the platform superior node performs statistical analysis on the output result of the called platform subordinate node.
Referring specifically to fig. 2, following the structure shown in fig. 1, when the platform node F invokes the platform subordinate node A, B, C, the master server of the platform node F invokes the video input resource slave service, the artificial intelligence computing resource slave service, the system storage resource slave service, and the tag annotation resource slave service of the slave server of the platform subordinate node A, B, C at the same time.
Preferably, the resource call of the platform node may be set to allow the resource call request of the platform upper node only after the resource call of the platform node needs to satisfy the resource request of the platform upper node, wherein the resource scheduling of the slave server of the platform lower node by the master server of the platform upper node includes:
the main server of the platform superior node calls corresponding video input resource slave service, artificial intelligence computing resource slave service, system storage resource slave service and label labeling resource slave service according to the resource occupancy rate of the slave server of the platform subordinate node; and the platform superior node performs statistical analysis on the output result of the called platform subordinate node.
By utilizing the scheme of the invention, a large number of data samples can be obtained through the video input and other information input of each node; meanwhile, each node of the training platform has artificial intelligence computing capacity, and large-scale operation set can be realized through unified distribution of top-level nodes, so that huge operation capacity is obtained; a large amount of label marking information is also concentrated in the training platform, and the workload of a large amount of artificial judgment results in artificial intelligence learning can be reduced through the marking information, so that the optimization capability and the optimization time of artificial intelligence are greatly accelerated by the training platform.
Any combination of the various embodiments of the present invention should be considered as disclosed in the present invention, unless the inventive concept is contrary to the present invention; within the scope of the technical idea of the invention, any combination of various simple modifications and different embodiments of the technical solution without departing from the inventive idea of the present invention shall fall within the protection scope of the present invention.
Claims (1)
1. A video artificial intelligence training platform based on a multi-layer architecture is characterized by comprising: each platform node is provided with an artificial intelligence computing unit, a video label labeling unit and a video signal access unit;
the artificial intelligence computing unit can execute different artificial intelligence algorithms to realize different structural analyses of multi-channel video input; the video label labeling unit can label the accessed video signal;
after each platform node establishes communication, determining the subordination relation of each platform node according to the respective authority value;
when the authority values of two platform nodes are equal, the two platform nodes are only used as data integration and interaction of the nodes, and the authority of data resources is not distributed between the two platform nodes;
when the authority value of one platform node is larger than that of the other platform node, the platform superior node with the larger authority value is the platform superior node; the lower authority value is the lower node of the platform;
after the platform upper node obtains the resource management distribution authority of the platform lower node, scheduling all resources of the platform lower node; wherein the resources of each platform node include: the method comprises the following steps of (1) video input resources, artificial intelligent computing resources of each node, system storage resources and label labeling resources;
wherein each of the platform nodes comprises a master server and a slave server; the main server comprises a video input resource main service, an artificial intelligent computing resource main service of each node, a system storage resource main service and a label marking resource main service; the slave server comprises a video input resource slave service, an artificial intelligence computing resource slave service, a system storage resource slave service and a label labeling resource slave service;
the scheduling of all resources of the platform subordinate node includes: a platform upper node starts a main server, and a platform lower node starts a slave server; performing resource scheduling on a slave server of a lower-level node of a platform through a master server of a higher-level node of the platform;
wherein, the resource scheduling of the slave server of the platform lower node by the master server of the platform upper node comprises:
the method comprises the following steps that a main server of a platform superior node simultaneously calls video input resource slave services, artificial intelligence computing resource slave services, system storage resource slave services and label labeling resource slave services of slave servers of a plurality of platform subordinate nodes;
the platform upper node performs statistical analysis on the output result of the called platform lower node;
wherein, the resource scheduling of the slave server of the platform lower node by the master server of the platform upper node comprises:
the main server of the platform superior node calls corresponding video input resource slave service, artificial intelligence computing resource slave service, system storage resource slave service and label labeling resource slave service according to the resource occupancy rate of the slave server of the platform subordinate node; and the platform superior node performs statistical analysis on the output result of the called platform subordinate node.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810770495.4A CN109117266B (en) | 2018-07-13 | 2018-07-13 | Video artificial intelligence training platform based on multilayer framework |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810770495.4A CN109117266B (en) | 2018-07-13 | 2018-07-13 | Video artificial intelligence training platform based on multilayer framework |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109117266A CN109117266A (en) | 2019-01-01 |
| CN109117266B true CN109117266B (en) | 2021-10-19 |
Family
ID=64862879
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810770495.4A Active CN109117266B (en) | 2018-07-13 | 2018-07-13 | Video artificial intelligence training platform based on multilayer framework |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109117266B (en) |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101782976B (en) * | 2010-01-15 | 2013-04-10 | 南京邮电大学 | Automatic selection method for machine learning in cloud computing environment |
| CN105321346A (en) * | 2015-09-18 | 2016-02-10 | 成都融创智谷科技有限公司 | Method for utilizing cloud computing basic resource pool to control urban intelligent traffic |
| CN106851184A (en) * | 2015-12-07 | 2017-06-13 | 北京航天长峰科技工业集团有限公司 | A kind of outlying district monitoring system architecture design method |
| US11080616B2 (en) * | 2016-09-27 | 2021-08-03 | Clarifai, Inc. | Artificial intelligence model and data collection/development platform |
| US20180196866A1 (en) * | 2017-01-06 | 2018-07-12 | Microsoft Technology Licensing, Llc | Topic nodes |
| CN108062246B (en) * | 2018-01-25 | 2019-06-14 | 北京百度网讯科技有限公司 | Resource regulating method and device for deep learning frame |
-
2018
- 2018-07-13 CN CN201810770495.4A patent/CN109117266B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109117266A (en) | 2019-01-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9838483B2 (en) | Methods, systems, and computer readable media for a network function virtualization information concentrator | |
| CN105940377B (en) | Methods, systems, and computer readable media for cloud-based virtualization orchestrators | |
| CN107241281B (en) | Data processing method and device | |
| CN109150572B (en) | Method, device and computer readable storage medium for realizing alarm association | |
| Di Mauro et al. | Availability evaluation of multi-tenant service function chaining infrastructures by multidimensional universal generating function | |
| WO2016204804A1 (en) | Virtualized network function monitoring | |
| CN105574082A (en) | Storm based stream processing method and system | |
| CN103401947A (en) | Method and device for allocating tasks to multiple servers | |
| CN111966289A (en) | Partition optimization method and system based on Kafka cluster | |
| CN103067486B (en) | Based on the large data processing method of PaaS platform | |
| CN105321346A (en) | Method for utilizing cloud computing basic resource pool to control urban intelligent traffic | |
| CN105071994A (en) | Mass data monitoring system | |
| CN108924228B (en) | Industrial internet optimization system based on edge calculation | |
| CN104811922A (en) | Adjacent node registration method and the device and cross-node registration method and system | |
| CN105007200B (en) | The analysis method and system of network packet | |
| CN109117266B (en) | Video artificial intelligence training platform based on multilayer framework | |
| CN106878356B (en) | Scheduling method and computing node | |
| CN106571935B (en) | Resource scheduling method and equipment | |
| CN106790693A (en) | The analysis method and system of a kind of network packet | |
| CN106790368A (en) | Resource regulating method and device in a kind of distributed system | |
| CN109376509A (en) | A kind of KVM task distributes system and automatic distributing method automatically | |
| CN112260946B (en) | Link failure processing method and device, terminal equipment and storage medium | |
| CN114513553A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN111294553B (en) | Method, device, equipment and storage medium for processing video monitoring service signaling | |
| Lee et al. | mIoTwins: design and evaluation of mIoT framework for private edge networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: A Video Artificial Intelligence Training Platform Based on Multilayer Architecture Effective date of registration: 20231026 Granted publication date: 20211019 Pledgee: Societe Generale Bank Limited by Share Ltd. Guangzhou branch Pledgor: SHIYUN RONGJU (GUANGZHOU) TECHNOLOGY Co.,Ltd. Registration number: Y2023980062916 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right |