TWI773100B - Neural network structure search method and apparatus - Google Patents
Neural network structure search method and apparatus Download PDFInfo
- Publication number
- TWI773100B TWI773100B TW110102763A TW110102763A TWI773100B TW I773100 B TWI773100 B TW I773100B TW 110102763 A TW110102763 A TW 110102763A TW 110102763 A TW110102763 A TW 110102763A TW I773100 B TWI773100 B TW I773100B
- Authority
- TW
- Taiwan
- Prior art keywords
- training
- retrieval
- task
- tasks
- configuration information
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5066—Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
- Measuring Pulse, Heart Rate, Blood Pressure Or Blood Flow (AREA)
Abstract
Description
本發明涉及深度學習領域,尤其涉及神經網絡架構檢索領域。 The invention relates to the field of deep learning, in particular to the field of neural network architecture retrieval.
隨著人工智能技術的發展,深度學習神經網絡架構檢索技術逐漸成熟。深度學習神經網絡架構檢索技術可以自動構建滿足需求的深度學習神經網絡,降低人力投入,提高人力效率,具有很高的工業價值。 With the development of artificial intelligence technology, deep learning neural network architecture retrieval technology has gradually matured. The deep learning neural network architecture retrieval technology can automatically build a deep learning neural network that meets the needs, reduce manpower input, improve manpower efficiency, and has high industrial value.
然而,目前的深度學習神經網絡架構檢索技術,檢索任務被以單個任務的方式提交到集群管理系統中,在單台機器上執行檢索任務,難以在多台機器組成的集群上以分布式的方式併行執行檢索任務,導致檢索效率較低,並且集群資源的利用率較差。 However, with the current deep learning neural network architecture retrieval technology, the retrieval task is submitted to the cluster management system as a single task, and the retrieval task is executed on a single machine, which is difficult to perform in a distributed manner on a cluster composed of multiple machines. The retrieval tasks are executed in parallel, resulting in lower retrieval efficiency and poor utilization of cluster resources.
本發明實施例提供一種神經網絡架構檢索方法以及裝置,以解決相關技術存在的問題,技術方案如下: Embodiments of the present invention provide a neural network architecture retrieval method and device to solve the problems existing in the related art. The technical solutions are as follows:
第一方面,提供了一種神經網絡架構檢索方法,包括:獲取檢索任務的配置信息,根據配置信息生成多個候選模型結構以及對應的參數;根據多個候選模型結構以及參數創建多個訓練任務;將多個訓練任務調度至對應的物理節點,以使各物理節點併行執行對應的訓練任務,並統計多個訓練任務的訓練結果; 在訓練結果滿足配置信息中的預設訓練條件的情況下,得到檢索任務對應的檢索結果。 A first aspect provides a neural network architecture retrieval method, comprising: acquiring configuration information of a retrieval task, generating multiple candidate model structures and corresponding parameters according to the configuration information; creating multiple training tasks according to the multiple candidate model structures and parameters; Scheduling multiple training tasks to the corresponding physical nodes, so that each physical node executes the corresponding training tasks in parallel, and counts the training results of the multiple training tasks; Under the condition that the training result satisfies the preset training condition in the configuration information, the retrieval result corresponding to the retrieval task is obtained.
在一種實施方式中,還包括:在訓練結果未滿足配置信息中的預設訓練條件的情況下,返回執行根據配置信息生成多個候選模型結構以及對應的參數的步驟。 In an embodiment, the method further includes: if the training result does not meet the preset training condition in the configuration information, returning to execute the step of generating multiple candidate model structures and corresponding parameters according to the configuration information.
在一種實施方式中,配置信息是用戶輸入的配置信息,配置信息包括檢索任務信息和訓練任務信息。 In one embodiment, the configuration information is configuration information input by a user, and the configuration information includes retrieval task information and training task information.
在一種實施方式中,根據配置信息生成多個候選模型結構以及對應的參數,包括:根據檢索任務信息創建對應的算法實例;將訓練任務信息加載至算法實例中,生成多個候選模型結構以及對應的參數。 In one embodiment, generating multiple candidate model structures and corresponding parameters according to the configuration information includes: creating a corresponding algorithm instance according to the retrieval task information; loading the training task information into the algorithm instance to generate multiple candidate model structures and corresponding parameters. parameter.
在一種實施方式中,檢索任務信息包括檢索空間和傳統模型結構的檢索算法;根據檢索任務信息創建對應的算法實例,包括:根據檢索空間和傳統模型結構的檢索算法,創建對應的算法實例。 In one embodiment, the retrieval task information includes a retrieval space and a retrieval algorithm of a traditional model structure; creating a corresponding algorithm instance according to the retrieval task information includes: creating a corresponding algorithm instance according to the retrieval space and the retrieval algorithm of the traditional model structure.
在一種實施方式中,檢索任務信息包括檢索空間和單次模型結構的檢索算法;根據檢索任務信息創建對應的算法實例,包括:根據檢索空間和單次模型結構的檢索算法,創建對應的算法實例。 In one embodiment, the retrieval task information includes a retrieval algorithm of a retrieval space and a single model structure; creating a corresponding algorithm instance according to the retrieval task information includes: creating a corresponding algorithm instance according to the retrieval algorithm of the retrieval space and the single model structure .
在一種實施方式中,還包括:在候選模型結構需要調優的情況下,根據多個候選模型結構以及對應的參數創建多個模型調優任務; 將多個模型調優任務調度至對應的物理節點,以使各物理節點併行執行對應的模型調優任務,得到多個優化後的候選模型結構,並統計多個優化後的候選模型結構的訓練結果。 In one embodiment, the method further includes: when the candidate model structure needs to be tuned, creating a plurality of model tuning tasks according to the plurality of candidate model structures and corresponding parameters; Schedule multiple model tuning tasks to the corresponding physical nodes, so that each physical node executes the corresponding model tuning tasks in parallel, obtains multiple optimized candidate model structures, and counts the training of multiple optimized candidate model structures result.
在一種實施方式中,將多個訓練任務調度至對應的物理節點,包括:根據檢索任務對應的全部訓練任務所使用的資源數量,計算多個訓練任務對應的多個資源類型權重;根據多個訓練任務對應的多個資源權重,確定多個訓練任務的調度順序;根據調度順序將多個訓練任務調度至對應的物理節點。 In one embodiment, scheduling multiple training tasks to corresponding physical nodes includes: calculating multiple resource type weights corresponding to multiple training tasks according to the number of resources used by all training tasks corresponding to the retrieval task; The multiple resource weights corresponding to the training tasks determine the scheduling sequence of the multiple training tasks; the multiple training tasks are scheduled to the corresponding physical nodes according to the scheduling sequence.
在一種實施方式中,訓練結果包括實際訓練次數和訓練指標的實際值,預設訓練條件包括最大檢索次數和訓練指標的期望值;在多個訓練任務的訓練結果滿足配置信息中的預設訓練條件的情況下,得到檢索任務對應的檢索結果,包括:在實際訓練次數大於或等於最大檢索次數,或訓練指標的實際值大於或等於訓練指標的期望值的情況下,得到最優網絡結構以及對應的參數,最優網絡結構以及對應的參數作為檢索結果。 In one embodiment, the training result includes actual training times and actual values of the training index, and the preset training conditions include the maximum retrieval times and the expected value of the training index; the training results of multiple training tasks satisfy the preset training conditions in the configuration information In the case of , obtaining the retrieval result corresponding to the retrieval task, including: obtaining the optimal network structure and the corresponding The parameters, the optimal network structure and the corresponding parameters are used as the retrieval results.
在一種實施方式中,還包括:針對每個訓練任務創建對應的臨時任務,臨時任務用於將訓練結果存儲至數據存儲器中。 In an embodiment, the method further includes: creating a corresponding temporary task for each training task, where the temporary task is used to store the training result in the data storage.
第二方面,本發明實施例提供一種神經網絡架構檢索裝置,包括:配置信息獲取模組,用於獲取檢索任務的配置信息; 候選模型結構生成模組,用於根據配置信息生成多個候選模型結構以及對應的參數;訓練任務創建模組,用於根據多個候選模型結構以及參數創建多個訓練任務;訓練任務調度模組,用於將多個訓練任務調度至對應的物理節點,以使各物理節點併行執行對應的訓練任務,並統計多個訓練任務的訓練結果;檢索結果生成模組,用於在訓練結果滿足配置信息中的預設訓練條件的情況下,得到檢索任務對應的檢索結果。 In a second aspect, an embodiment of the present invention provides a neural network architecture retrieval device, including: a configuration information acquisition module for acquiring configuration information of a retrieval task; The candidate model structure generation module is used to generate multiple candidate model structures and corresponding parameters according to the configuration information; the training task creation module is used to create multiple training tasks according to the multiple candidate model structures and parameters; the training task scheduling module , which is used to schedule multiple training tasks to the corresponding physical nodes, so that each physical node executes the corresponding training tasks in parallel, and counts the training results of multiple training tasks; In the case of the preset training conditions in the information, the retrieval result corresponding to the retrieval task is obtained.
在一種實施方式中,還包括:迭代檢索觸發模組,用於在訓練結果未滿足配置信息中的預設訓練條件的情況下,觸發所述候選模型結構生成模組執行根據配置信息生成多個候選模型結構以及對應的參數的步驟。 In one embodiment, it further includes: an iterative retrieval triggering module, configured to trigger the candidate model structure generation module to perform the generation of multiple Steps for candidate model structures and corresponding parameters.
在一種實施方式中,配置信息是用戶輸入的配置信息,配置信息包括檢索任務信息和訓練任務信息。 In one embodiment, the configuration information is configuration information input by a user, and the configuration information includes retrieval task information and training task information.
在一種實施方式中,候選模型結構生成模組,包括:算法實例創建子模組,用於根據檢索任務信息創建對應的算法實例;候選模型結構生成子模組,用於將訓練任務信息加載至算法實例中,生成多個候選模型結構以及對應的參數。 In one embodiment, the candidate model structure generation module includes: an algorithm instance creation submodule for creating a corresponding algorithm instance according to the retrieval task information; a candidate model structure generation submodule for loading the training task information into In the algorithm instance, multiple candidate model structures and corresponding parameters are generated.
在一種實施方式中,檢索任務信息包括檢索空間和傳統模型結構的檢索算法,算法實例創建子模組,包括:第一算法實例創建單元,用於根據檢索空間和傳統模型結構的檢索算法,創建對應的算法實例。 In one embodiment, the retrieval task information includes a retrieval space and a retrieval algorithm of a traditional model structure, and the algorithm instance creation sub-module includes: a first algorithm instance creation unit for creating a retrieval algorithm based on the retrieval space and the traditional model structure. The corresponding algorithm example.
在一種實施方式中,檢索任務信息包括檢索空間和單次模型結構的檢索算法,算法實例創建子模組,還包括:第二算法實例創建單元,用於根據檢索空間和單次模型結構的檢索算法,創建對應的算法實例。 In one embodiment, the retrieval task information includes a retrieval space and a retrieval algorithm for a single model structure, an algorithm instance creation sub-module further includes: a second algorithm instance creation unit for retrieval based on the retrieval space and the single model structure Algorithm, create the corresponding algorithm instance.
在一種實施方式中,還包括:模型調優任務創建模組,用於在候選模型結構需要調優的情況下,根據多個候選模型結構以及對應的參數創建多個模型調優任務;模型調優任務調度模組,用於將多個模型調優任務調度至對應的物理節點,以使各物理節點併行執行對應的模型調優任務,得到多個優化後的候選模型結構,並統計多個優化後的候選模型結構的訓練結果。 In one embodiment, it further includes: a model tuning task creation module, configured to create multiple model tuning tasks according to multiple candidate model structures and corresponding parameters when the candidate model structure needs to be tuned; The optimal task scheduling module is used to schedule multiple model tuning tasks to the corresponding physical nodes, so that each physical node executes the corresponding model tuning tasks in parallel, obtains multiple optimized candidate model structures, and counts multiple The training results of the optimized candidate model structure.
在一種實施方式中,訓練任務調度模組,包括:資源類型權重計算子模組,用於根據檢索任務對應的全部訓練任務所使用的資源數量,計算多個訓練任務對應的多個資源類型權重;調度順序確定子模組,用於根據多個訓練任務對應的多個資源權重,確定多個訓練任務的調度順序;訓練任務調度子模組,用於根據調度順序將多個訓練任務調度至對應的物理節點。 In one embodiment, the training task scheduling module includes: a resource type weight calculation sub-module, configured to calculate multiple resource type weights corresponding to multiple training tasks according to the number of resources used by all training tasks corresponding to the retrieval task ; The scheduling order determination sub-module is used to determine the scheduling order of multiple training tasks according to multiple resource weights corresponding to multiple training tasks; the training task scheduling sub-module is used to schedule multiple training tasks to the corresponding physical node.
在一種實施方式中,訓練結果包括實際訓練次數和訓練指標的實際值,預設訓練條件包括最大檢索次數和訓練指標的期望值,檢索結果生成模組,包括: 檢索結果生成子模組,用於在實際訓練次數大於或等於最大檢索次數,或訓練指標的實際值大於或等於訓練指標的期望值的情況下,得到最優網絡結構以及對應的參數,最優網絡結構以及對應的參數作為檢索結果。 In one embodiment, the training result includes the actual number of times of training and the actual value of the training index, the preset training condition includes the maximum number of retrieval times and the expected value of the training index, and the retrieval result generation module includes: The retrieval result generation sub-module is used to obtain the optimal network structure and corresponding parameters when the actual training times are greater than or equal to the maximum retrieval times, or the actual value of the training index is greater than or equal to the expected value of the training index. The structure and the corresponding parameters are used as the retrieval result.
在一種實施方式中,還包括:臨時任務創建模組,用於針對每個訓練任務創建對應的臨時任務,臨時任務用於將訓練結果存儲至數據存儲器中。 In one embodiment, the method further includes: a temporary task creation module, configured to create a corresponding temporary task for each training task, and the temporary task is used to store the training result in the data storage.
第三方面,提供了一種電子設備,包括:至少一個處理器;以及與至少一個處理器通信連接的存儲器;其中,存儲器存儲有可被至少一個處理器執行的指令,指令被至少一個處理器執行,以使至少一個處理器能夠執行上述任一項的方法。 In a third aspect, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor , to enable at least one processor to perform any of the above methods.
第四方面,提供了一種存儲有計算機指令的非瞬時計算機可讀存儲介質,計算機指令用於使計算機執行上述任一項的方法。 In a fourth aspect, there is provided a non-transitory computer-readable storage medium storing computer instructions, the computer instructions being used to cause a computer to perform any of the above methods.
上述申請中的一個實施例具有如下優點或有益效果:由於利用檢索任務的配置信息動態地生成多個候選模型結構以及對應的參數,並創建多個候選模型結構的訓練任務,將多個訓練任務調度之對應的物理節點,併行地執行多個訓練任務,進而得到檢索結果,所以不僅提高了檢索效率,還提高了集群資源的利用率,使用的硬體資源和硬體加速資源擴展性增強,滿足大規模檢索的需求。上述可選方式所具有的其他效果將在下文中結合具體實施例加以說明。 An embodiment in the above application has the following advantages or beneficial effects: because the configuration information of the retrieval task is used to dynamically generate multiple candidate model structures and corresponding parameters, and create training tasks for multiple candidate model structures, the multiple training tasks are The physical node corresponding to the scheduling executes multiple training tasks in parallel, and then obtains the retrieval results, so not only improves the retrieval efficiency, but also improves the utilization rate of cluster resources, and the used hardware resources and hardware acceleration resources are more scalable. Meet the needs of large-scale retrieval. Other effects of the above-mentioned optional manners will be described below with reference to specific embodiments.
110:配置信息獲取模組 110: Configuration information acquisition module
120:候選模型結構生成模組 120: Candidate model structure generation module
121:算法實例創建子模組 121: Algorithm instance creation submodule
122:候選模型結構生成子模組 122: Candidate model structure generation submodule
130:訓練任務創建模組 130: Training task creation module
140:訓練任務調度模組 140: Training task scheduling module
141:資源類型權重計算子模組 141: Resource type weight calculation sub-module
142:調度順序確定子模組 142: scheduling order to determine submodules
143:訓練任務調度子模組 143: Training Task Scheduling Submodule
150:檢索結果生成模組 150: Retrieval result generation module
151:檢索結果生成子模組 151: Search result generation submodule
160:迭代檢索觸發模組 160: Iterative retrieval trigger module
170:模型調優任務創建模組 170: Model tuning task creation module
180:模型調優任務調度模組 180: Model Tuning Task Scheduling Module
190:臨時任務創建模組 190: Temporary task creation mod
901:處理器 901: Processor
902:存儲器 902: Memory
903:輸入裝置 903: Input Device
904:輸出裝置 904: Output device
S110:獲取檢索任務的配置信息 S110: Obtain the configuration information of the retrieval task
S120:根據配置信息生成多個候選模型結構以及對應的參數 S120: Generate multiple candidate model structures and corresponding parameters according to the configuration information
S130:根據多個候選模型結構以及參數創建多個訓練任務 S130: Create multiple training tasks according to multiple candidate model structures and parameters
S140:將多個訓練任務調度至對應的物理節點,以使各物理節點並行執行對應 的訓練任務,並統計多個訓練任務的訓練結果 S140: Schedule multiple training tasks to corresponding physical nodes, so that each physical node executes the corresponding training tasks, and statistics the training results of multiple training tasks
S150:在訓練結果滿足配置信息中的預設訓練條件的情況下,得到檢索任務對應的檢索結果 S150: Obtain a retrieval result corresponding to the retrieval task under the condition that the training result satisfies the preset training condition in the configuration information
S121:根據檢索空間和單次模型結構的檢索算法,創建對應的算法實例 S121: Create a corresponding algorithm instance according to the retrieval space and the retrieval algorithm of the single model structure
141:根據檢索任務對應的全部訓練任務所使用的資源數量,計算多個訓練任務對應的多個資源類型權重 141: Calculate the weights of multiple resource types corresponding to multiple training tasks according to the number of resources used by all training tasks corresponding to the retrieval task
142:根據多個訓練任務對應的多個資源權重,確定多個訓練任務的調度順序 142: Determine the scheduling order of multiple training tasks according to multiple resource weights corresponding to multiple training tasks
143:根據調度順序將多個訓練任務調度至對應的物理節點 143: Schedule multiple training tasks to the corresponding physical nodes according to the scheduling order
S151:在實際訓練次數大於或等於最大檢索次數,或訓練指標的實際值大於或等於訓練指標的期望值的情況下,得到最優網絡結構以及對應的參數,最優網絡結構以及對應的參數作為檢索結果 S151: Under the condition that the actual training times is greater than or equal to the maximum retrieval times, or the actual value of the training index is greater than or equal to the expected value of the training index, obtain the optimal network structure and corresponding parameters, and the optimal network structure and corresponding parameters are used as retrieval result
S160:在候選模型結構需要調優的情況下,根據多個候選模型結構以及對應的參數創建多個模型調優任務 S160: When the candidate model structure needs to be tuned, create multiple model tuning tasks according to the multiple candidate model structures and corresponding parameters
S170:將多個模型調優任務調度至對應的物理節點,以使各物理節點併行執行對應的模型調優任務,得到多個優化後的候選模型結構,並統計多個優化後的候選模型結構的訓練結果 S170: Scheduling multiple model tuning tasks to corresponding physical nodes, so that each physical node executes the corresponding model tuning tasks in parallel, obtains multiple optimized candidate model structures, and counts the multiple optimized candidate model structures The training result of
S180:針對每個訓練任務創建對應的臨時任務,臨時任務用於將訓練結果存儲至數據存儲器中 S180: Create a corresponding temporary task for each training task, and the temporary task is used to store the training result in the data storage
[圖1]是根據本發明一實施例的一種神經網絡架構檢索方法的示意圖;[圖2]是根據本發明一實施例的一種神經網絡架構檢索場景的示意圖;[圖3]是根據本發明一實施例的另一種神經網絡架構檢索方法的示意圖;[圖4]是根據本發明一實施例的另一種神經網絡架構檢索方法的示意圖;[圖5]是根據本發明一實施例的訓練任務調度方法的示意圖;[圖6]是根據本發明一實施例的一種神經網絡架構檢索裝置的示意圖;[圖7]是根據本發明另一實施例的一種神經網絡架構檢索裝置的示意圖;[圖8]是根據本發明一實施例的訓練任務調度模組的示意圖;[圖9]是用來實現本發明實施例的一種神經網絡架構檢索方法的電子設備的框圖。 [Fig. 1] is a schematic diagram of a neural network architecture retrieval method according to an embodiment of the present invention; [Fig. 2] is a schematic diagram of a neural network architecture retrieval scenario according to an embodiment of the present invention; [Fig. 3] is according to the present invention A schematic diagram of another neural network architecture retrieval method according to an embodiment; [FIG. 4] is a schematic diagram of another neural network architecture retrieval method according to an embodiment of the present invention; [FIG. 5] is a training task according to an embodiment of the present invention A schematic diagram of a scheduling method; [Fig. 6] is a schematic diagram of a neural network architecture retrieval device according to an embodiment of the present invention; [Fig. 7] is a schematic diagram of a neural network architecture retrieval device according to another embodiment of the present invention; [Fig. 8] is a schematic diagram of a training task scheduling module according to an embodiment of the present invention; [Fig. 9] is a block diagram of an electronic device used to implement a neural network architecture retrieval method according to an embodiment of the present invention.
以下結合附圖對本發明的示範性實施例做出說明,其中包括本發明實施例的各種細節以助於理解,應當將它們認為僅僅是示範性的。因此,本領域普通技術人員應當認識到,可以對這裏描述的實施例做出各種改變和修改,而不會背離本發明的範圍和精神。同樣,為了清楚和簡明,以下的描述中省略了對公知功能和結構的描述。 Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, which include various details of the embodiments of the present invention to facilitate understanding and should be considered as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted from the following description for clarity and conciseness.
如圖1所示,在一種具體的實施方式中,提供一種神經網絡架構檢索方法,包括:步驟S110:獲取檢索任務的配置信息;步驟S120:根據配置信息生成多個候選模型結構以及對應的參數;步驟S130:根據多個候選模型結構以及參數創建多個訓練任務;步驟S140:將多個訓練任務調度至對應的物理節點,以使各物理節點併行執行對應的訓練任務,並統計多個訓練任務的訓練結果; 步驟S150:在訓練結果滿足配置信息中的預設訓練條件的情況下,得到檢索任務對應的檢索結果。 As shown in FIG. 1 , in a specific implementation manner, a neural network architecture retrieval method is provided, including: step S110 : obtaining configuration information of a retrieval task; step S120 : generating multiple candidate model structures and corresponding parameters according to the configuration information Step S130: create multiple training tasks according to multiple candidate model structures and parameters; Step S140: schedule multiple training tasks to corresponding physical nodes, so that each physical node executes the corresponding training tasks in parallel, and counts multiple training tasks The training result of the task; Step S150: Obtain a retrieval result corresponding to the retrieval task when the training result satisfies the preset training condition in the configuration information.
一種示例中,如圖2所示,可以選擇在K8S系統(Kubernetes,容器集群管理系)中運行控制器、調度器、數據存儲器和API(應用程序)服務器等組件,這些組件以Docker(開源的應用容器引擎)容器的方式運行在K8S系統的主控節點上。 In one example, as shown in Figure 2, you can choose to run components such as controllers, schedulers, data storage, and API (application) servers in the K8S system (Kubernetes, container cluster management system). The application container engine) container runs on the master node of the K8S system.
如圖3所示,首先,控制器獲取檢索任務的配置信息。配置信息可以是存儲在配置文件中的默認的配置信息,或者用戶通過界面操作輸入的配置信息,又或者是用戶在客戶端通過命令行輸入的檢索任務信息。配置信息中包括檢索任務信息和訓練任務信息等。對配置信息添加缺失的信息或者默認的初始值等,將配置信息設置完整。並對完整的配置信息進行校驗,例如,確定用戶輸入的檢索空間是適當的檢索空間:如果用戶選擇了整形變量,則檢索空間需要提供最小整數和最大整數;如果用戶選擇了離散變量,則檢索空間需要提供一個離散的列表,寫明所有的檢索選擇;如果用戶選擇了浮點數變量,則檢索空間需要提供最小值、最大值以及步長。校驗是否選擇了合法的操作類型:如果操作類型是卷積操作,則卷積操作對應的參數包括過濾器的尺寸、過濾器的數量、步幅等。在對配置信息進行完校驗後,控制器通過Kubernetes提供的Golang客戶端向API服務器發送創建訓練任務的請求,API服務器將創建訓練任務的請求持久化到數據存儲器中。API服務器依賴數據存儲器,數據存儲器是由ETCD(分布式一致性鍵值存儲系統)提供的存儲功能。API服務器將創建訓練任務的請求發送給檢索任務的處理器進行下一步處理。 As shown in FIG. 3, first, the controller obtains the configuration information of the retrieval task. The configuration information may be default configuration information stored in a configuration file, or configuration information input by the user through an interface operation, or retrieval task information input by the user through a command line on the client. The configuration information includes retrieval task information and training task information. Add missing information or default initial values to the configuration information, and set the configuration information completely. And verify the complete configuration information, for example, determine that the search space entered by the user is an appropriate search space: if the user selects an integer variable, the search space needs to provide the minimum and maximum integers; if the user selects a discrete variable, then The search space needs to provide a discrete list, indicating all search options; if the user selects a floating-point variable, the search space needs to provide the minimum value, maximum value and step size. Check whether a valid operation type is selected: if the operation type is a convolution operation, the parameters corresponding to the convolution operation include the size of the filter, the number of filters, and the stride. After verifying the configuration information, the controller sends a request to create a training task to the API server through the Golang client provided by Kubernetes, and the API server persists the request to create a training task to the data storage. The API server relies on the data store, which is the storage function provided by the ETCD (distributed consistent key-value store). The API server sends the request to create the training task to the processor of the retrieval task for further processing.
檢索任務的處理器利用K8S系統中的Informer(帶有本地緩存和索引機制的客戶端工具)的組件,監控API服務器中關於檢索任務的事件。 The processor of the retrieval task uses the component of Informer (a client tool with a local cache and indexing mechanism) in the K8S system to monitor the events of the retrieval task in the API server.
Informer組件註册了檢索任務、訓練任務和算法等事件的回調處理器,當檢索任務、訓練任務和算法的事件中的任意事件發生變化時,處理器利用K8S系統的客戶端來處理這些變化。處理器在收到上述事件後,首先將這些事件放入維護在內存中的事件隊列中,由專門的Worker(工作服務器)協程負責從事件隊列中取出事件。處理器根據事件的引用對象,通過K8S系統的客戶端從API服務器處取得檢索任務的配置信息。 The Informer component registers callback handlers for events such as retrieval tasks, training tasks, and algorithms. When any events in the retrieval tasks, training tasks, and algorithm events change, the processor uses the client of the K8S system to process these changes. After receiving the above events, the processor first puts these events into the event queue maintained in the memory, and a special Worker (worker server) coroutine is responsible for fetching events from the event queue. The processor obtains the configuration information of the retrieval task from the API server through the client of the K8S system according to the reference object of the event.
其次,檢索任務的處理器接收到創建訓練任務的請求後,通過K8S系統提供的客戶端從API服務器中獲取檢索任務的配置信息。處理器根據檢索任務信息中用戶定義的檢索空間和檢索算法,創建算法實例,並利用算法實例來運行檢索算法。例如,處理器根據不同類型的檢索算法(傳統模型結構的檢索算法、單次模型結構的檢索算法以及優化後的單次模型結構的檢索算法等),創建對應的算法實例。根據算法的架構不同,算法實例會採用不同的實現。處理器在算法實例初始化完成後,會通過gRPC(google remote procedure call,遠程過程調用)框架的通信機制,使得控制器和處理器產生通信交互,將控制器中的訓練任務信息與處理器中的算法實例進行交互,生成多個候選模型結構以及對應的參數(例如,權重)。處理器通過HTTP接口訪問算法實例,通過HTTP接口獲得多個候選模型結構以及對應的參數。 Secondly, after receiving the request to create a training task, the processor of the retrieval task obtains the configuration information of the retrieval task from the API server through the client provided by the K8S system. The processor creates an algorithm instance according to the retrieval space and retrieval algorithm defined by the user in the retrieval task information, and uses the algorithm instance to run the retrieval algorithm. For example, the processor creates corresponding algorithm instances according to different types of retrieval algorithms (retrieval algorithms for traditional model structures, retrieval algorithms for single-shot model structures, and retrieval algorithms for optimized single-shot model structures, etc.). Depending on the architecture of the algorithm, the algorithm instance will use different implementations. After the initialization of the algorithm instance is completed, the processor will communicate with the controller and the processor through the communication mechanism of the gRPC (google remote procedure call, remote procedure call) framework, and transfer the training task information in the controller with the training task information in the processor. Algorithm instances interact to generate multiple candidate model structures and corresponding parameters (eg, weights). The processor accesses the algorithm instance through the HTTP interface, and obtains multiple candidate model structures and corresponding parameters through the HTTP interface.
然後,處理器通過訪問算法實例獲取多個候選模型結構以及對應的參數後,創建候選模型結構的訓練任務。一個候選模型結構對應創建一個訓練任務,一個訓練任務分發多個容器組(pod)上的物理節點併行執行。並將多個 候選模型結構的訓練任務發送至API服務器。由於處理器可以監控API服務器,當API服務器中存在創建了訓練任務的變化時,處理器通過K8S系統的客戶端從API服務器處取得配置信息中的訓練任務信息,並根據訓練任務信息,創建承載訓練負載的容器組(pod),容器組(pod)中運行有物理節點,這一功能由Kubeflow實現。在容器組(pod)被創建後,調度器會根據訓練任務信息,確定多個訓練任務需要被調度到的物理節點,使得這些訓練任務可以被多個物理節點併行執行。處理器會根據訓練任務信息確定訓練任務是否需要GPU或CPU等硬體加速器資源,提高使用的硬體資源和硬體加速資源擴展性。每個訓練任務會接受候選模型結構作為輸入,利用訓練任務信息中的訓練代碼進行訓練。在訓練任務完成後,訓練任務退出,處理器修改訓練任務的狀態為完成。隨後,處理器會統計訓練結果,訓練結果包括:同一檢索任務中一共完成的訓練任務的次數,每個訓練任務得到的訓練指標的實際值等。 Then, after the processor obtains multiple candidate model structures and corresponding parameters by accessing the algorithm instance, a training task of the candidate model structures is created. A candidate model structure corresponds to creating a training task, and a training task is distributed to physical nodes on multiple container groups (pods) for parallel execution. and multiple The training task of the candidate model structure is sent to the API server. Since the processor can monitor the API server, when there is a change in the creation of training tasks in the API server, the processor obtains the training task information in the configuration information from the API server through the client of the K8S system, and creates a bearer based on the training task information. The container group (pod) of the training load. The container group (pod) runs with physical nodes. This function is implemented by Kubeflow. After the container group (pod) is created, the scheduler determines the physical nodes to which multiple training tasks need to be scheduled according to the training task information, so that these training tasks can be executed by multiple physical nodes in parallel. The processor will determine whether the training task requires hardware accelerator resources such as GPU or CPU according to the training task information, so as to improve the used hardware resources and the scalability of hardware acceleration resources. Each training task will accept the candidate model structure as input, and use the training code in the training task information to train. After the training task is completed, the training task exits, and the processor modifies the state of the training task to complete. Subsequently, the processor will count the training results, and the training results include: the total number of training tasks completed in the same retrieval task, the actual value of the training indicators obtained by each training task, and the like.
最後,處理器根據檢索任務信息中的最大檢索次數和訓練指標的期望值,與訓練結果中的實際訓練次數和訓練指標的實際值進行比對,如果訓練結果大於檢索任務信息中的對應值,則處理器標示這一檢索任務的狀態為完成。 Finally, the processor compares the maximum retrieval times in the retrieval task information and the expected value of the training index with the actual training times in the training result and the actual value of the training index. If the training result is greater than the corresponding value in the retrieval task information, then The processor marks the status of this retrieval task as complete.
本實施方式中,深度學習神經網絡架構檢索技術利用K8S系統的分布式執行的能力,實現大規模的深度學習神經網絡架構檢索。在進行檢索任務時,用戶僅需提供檢索任務的配置信息,例如,檢索任務信息和訓練任務信息,就可發起深度學習神經網絡架構的檢索任務。用戶無需自行對深度學習神經網絡架構檢索的運行環境進行構建,無需自行申請訓練任務所需的集群資源,無需自行設計檢索算法相關的內容,而是由K8S系統自動實現,提高了檢索的便利性、適應性以及檢索速度。根據用戶輸入的檢索任務的配置信息動態 地生成多個候選模型結構以及對應的參數,並創建多個候選模型結構的訓練任務,利用K8S系統將多個訓練任務調度之對應的物理節點,並行地執行多個訓練任務,進而得到檢索結果。不僅提高了檢索效率,還提高了集群資源的利用率,使用的硬體資源和硬體加速資源擴展性增強,滿足大規模檢索的需求。 In this embodiment, the deep learning neural network architecture retrieval technology utilizes the distributed execution capability of the K8S system to realize large-scale deep learning neural network architecture retrieval. When performing a retrieval task, the user only needs to provide configuration information of the retrieval task, such as retrieval task information and training task information, to initiate a retrieval task of the deep learning neural network architecture. Users do not need to build the operating environment for deep learning neural network architecture retrieval by themselves, do not need to apply for cluster resources required for training tasks, and do not need to design content related to retrieval algorithms by themselves, but are automatically implemented by the K8S system, which improves the convenience of retrieval. , adaptability, and retrieval speed. Dynamically based on the configuration information of the retrieval task entered by the user Generate multiple candidate model structures and corresponding parameters, and create training tasks for multiple candidate model structures, use the K8S system to schedule the corresponding physical nodes of multiple training tasks, execute multiple training tasks in parallel, and then obtain retrieval results . It not only improves the retrieval efficiency, but also improves the utilization rate of cluster resources, and the used hardware resources and hardware acceleration resources have enhanced scalability to meet the needs of large-scale retrieval.
在一種實施方式中,如圖4所示,還包括:在訓練結果未滿足配置信息中的預設訓練條件的情況下,返回執行根據配置信息生成多個候選模型結構以及對應的參數的步驟。 In an embodiment, as shown in FIG. 4 , the method further includes: if the training result does not meet the preset training conditions in the configuration information, returning to execute the step of generating multiple candidate model structures and corresponding parameters according to the configuration information.
一種示例中,處理器根據檢索任務信息中的最大檢索次數和訓練指標的期望值,與訓練結果中的實際訓練次數和訓練指標的實際值進行比對,如果訓練結果小於檢索任務信息中的對應值,則處理器觸發返回執行步驟S120-步驟S140。具體的,處理器會以歷史訓練任務的訓練指標和候選模型結構作為輸入,重新與算法實例交互,算法實例會根據歷史訓練任務的訓練指標和候選模型結構,推薦新的候選模型結構,直到最大檢索次數小於或者等於實際訓練次數,或者訓練指標的實際值優於期望值為止。 In one example, the processor compares the maximum number of retrievals in the retrieval task information and the expected value of the training index with the actual number of trainings in the training result and the actual value of the training index, if the training result is less than the corresponding value in the retrieval task information. , the processor triggers to return to step S120-step S140. Specifically, the processor will use the training indicators and candidate model structures of the historical training tasks as input, and re-interact with the algorithm instance. The algorithm instance will recommend new candidate model structures based on the training indicators and candidate model structures of the historical training tasks, until the maximum The retrieval times are less than or equal to the actual training times, or the actual value of the training index is better than the expected value.
在一種實施方式中,配置信息是用戶輸入的配置信息,配置信息包括檢索任務信息和訓練任務信息。 In one embodiment, the configuration information is configuration information input by a user, and the configuration information includes retrieval task information and training task information.
一種示例中,用戶輸入的配置信息可以包括:檢索任務信息和訓練任務信息。其中,檢索任務信息包括:檢索任務所在的命名空間(NameSpace)、檢索任務的名字、檢索任務需要使用的數據集以及版本、執行檢索任務所需的模型結構以及版本、檢索任務的訓練指標名稱以及訓練指標的期望值、檢索空間的定義(操作,如卷積操作等;操作類型,如離散變量,整形變量等)、檢索失敗次數、檢索次數最大值、檢索算法。檢索算法可以包 括:檢索算法名稱(如使用自定義算法時提供檢索算法的鏡像)、檢索算法類型(如一次性算法類型,傳統算法類型等)、檢索算法使用的硬體資源(如GPU、CPU、持久化存儲、內存等)。訓練任務信息包括:單次執行訓練任務的訓練代碼、單次執行訓練任務的硬體資源(如CPU、內存、GPU等)、單次執行訓練任務的訓練模式(如TensorFlow基於數據流編程的分布式模式、Parameter Server參數服務器、Worker工作服務器模式等)等。 In an example, the configuration information input by the user may include: retrieval task information and training task information. The retrieval task information includes: the namespace where the retrieval task is located (NameSpace), the name of the retrieval task, the data set and version that the retrieval task needs to use, the model structure and version required to execute the retrieval task, the training index name of the retrieval task, and Expected values of training indicators, definition of retrieval space (operations, such as convolution operations, etc.; operation types, such as discrete variables, shaping variables, etc.), retrieval failure times, maximum retrieval times, retrieval algorithm. The retrieval algorithm can include Including: search algorithm name (such as providing a mirror image of the search algorithm when using a custom algorithm), search algorithm type (such as one-time algorithm type, traditional algorithm type, etc.), hardware resources used by the search algorithm (such as GPU, CPU, persistent storage, memory, etc.). The training task information includes: the training code for a single execution of the training task, the hardware resources (such as CPU, memory, GPU, etc.) for the single execution of the training task, the training mode for the single execution of the training task (such as the distribution of TensorFlow based on data flow programming mode, Parameter Server parameter server, Worker work server mode, etc.) and so on.
在一種實施方式中,如圖4所示,步驟S120,包括:步驟S121:根據檢索任務信息創建對應的算法實例;步驟S122:將訓練任務信息加載至算法實例中,生成多個候選模型結構以及對應的參數。 In one embodiment, as shown in FIG. 4 , step S120 includes: step S121 : creating a corresponding algorithm instance according to the retrieval task information; step S122 : loading the training task information into the algorithm instance, generating multiple candidate model structures and corresponding parameters.
一種示例中,如圖3所示,處理器根據檢索任務信息中的算法類型,創建對應的算法實例。例如,針對傳統模型結構的檢索算法,處理器利用檢索任務信息中的算法代碼,創建出對應的算法實例。針對如DARTS(基於梯度反傳的神經網絡檢索技術的框架)等僅需要一次模型訓練的檢索算法,處理器利用檢索任務信息中的算法代碼,創建出對應的算法實例。然後,處理器將訓練任務信息加載至創建出來的算法實例中,生成多個候選模型結構以及對應的參數。 In an example, as shown in FIG. 3 , the processor creates a corresponding algorithm instance according to the algorithm type in the retrieval task information. For example, for the retrieval algorithm of the traditional model structure, the processor uses the algorithm code in the retrieval task information to create a corresponding algorithm instance. For retrieval algorithms that require only one model training, such as DARTS (framework of neural network retrieval technology based on gradient backpropagation), the processor uses the algorithm code in the retrieval task information to create a corresponding algorithm instance. Then, the processor loads the training task information into the created algorithm instance, and generates multiple candidate model structures and corresponding parameters.
本實施方式中,在進行檢索任務時,用戶僅需提供檢索任務的配置信息,例如,檢索任務信息和訓練任務信息,就可發起深度學習神經網絡架構的檢索任務,提高了檢索的便利性。同時,由於根據不同的算法類型創建對應的算法實例,進而生成多個候選模型結構以及對應的參數,所以提高了檢索的適應性。 In this embodiment, when performing a retrieval task, the user only needs to provide configuration information of the retrieval task, for example, retrieval task information and training task information, to initiate a retrieval task of a deep learning neural network architecture, which improves the convenience of retrieval. At the same time, since corresponding algorithm instances are created according to different algorithm types, and then multiple candidate model structures and corresponding parameters are generated, the adaptability of retrieval is improved.
在一種實施方式中,檢索任務信息包括檢索空間和傳統模型結構的檢索算法;步驟S121,包括:根據檢索空間和傳統模型結構的檢索算法,創建對應的算法實例。 In one embodiment, the retrieval task information includes a retrieval space and a retrieval algorithm of the traditional model structure; step S121 includes: creating a corresponding algorithm instance according to the retrieval space and the retrieval algorithm of the traditional model structure.
一種示例中,如圖3所示,如果算法類型為傳統模型結構的檢索算法,例如,用戶選擇的隨機檢索的檢索算法,算法實例會隨機地在檢索空間中選取取值。處理器會根據配置信息中的鏡像列表,尋找到隨機檢索對應的鏡像,並且利用這一鏡像創建出算法容器,算法容器以RESTful(REpresentational State Transfer,表現層狀態轉移)形式的API服務器的方式對外提供服務。 In an example, as shown in Figure 3, if the algorithm type is a retrieval algorithm with a traditional model structure, for example, a retrieval algorithm selected by the user for random retrieval, the algorithm instance will randomly select values in the retrieval space. The processor will find the corresponding image for random retrieval according to the image list in the configuration information, and use this image to create an algorithm container. The algorithm container is an API server in the form of RESTful (REpresentational State Transfer) externally. Provide services.
另一示例中,用戶選擇了基於深度學習的檢索算法,處理器會根據配置信息中相關信息建立算法實例。算法實例會建立一個LSTM(Long Short-Term Memory,長短期記憶網絡),進而生成的候選模型結構為深度學習的模型結構。 In another example, the user selects a retrieval algorithm based on deep learning, and the processor will create an algorithm instance according to the relevant information in the configuration information. The algorithm instance will establish an LSTM (Long Short-Term Memory, long short-term memory network), and then the generated candidate model structure is the model structure of deep learning.
在一種實施方式中,檢索任務信息包括檢索空間和單次模型結構的檢索算法;步驟S121,包括:根據檢索空間和單次模型結構的檢索算法,創建對應的算法實例。 In one embodiment, the retrieval task information includes a retrieval space and a retrieval algorithm for the single model structure; step S121 includes: creating a corresponding algorithm instance according to the retrieval space and the retrieval algorithm for the single model structure.
一種示例中,如圖3所示,如果算法類型為單次模型結構的檢索算法,例如,用戶選擇的DARTS等的子圖檢索算法,既產生模型結構又產生模型權重。處理器會創建算法實例來完成訓練。在訓練完成後,處理器利用得到的子圖,創建模型調優任務,並將模型調優任務發送至K8S的API服務器。如果用戶不需要調優,則將檢索到的模型直接推送到數據存儲器中或者其他持久化存儲器中。 In an example, as shown in FIG. 3 , if the algorithm type is a retrieval algorithm of a single model structure, for example, a subgraph retrieval algorithm such as DARTS selected by the user, both the model structure and the model weight are generated. The processor creates an instance of the algorithm to complete the training. After the training is completed, the processor uses the obtained subgraph to create a model tuning task, and sends the model tuning task to the API server of K8S. If the user does not need tuning, the retrieved model is pushed directly to the data store or other persistent store.
另一示例中,如果用戶選擇的是只產生模型結構,不產生模型權重的子圖檢索算法,處理器會創建算法實例完成訓練。在訓練完成後,處理器利用得到的模型子結構創建訓練任務,並將訓練任務發送至K8S系統API服務器。 In another example, if the user selects a subgraph retrieval algorithm that only generates model structures and does not generate model weights, the processor will create an algorithm instance to complete the training. After the training is completed, the processor uses the obtained model substructure to create a training task, and sends the training task to the K8S system API server.
在一種實施方式中,如圖4所示,還包括:步驟S160:在候選模型結構需要調優的情況下,根據多個候選模型結構以及對應的參數創建多個模型調優任務;步驟S170:將多個模型調優任務調度至對應的物理節點,以使各物理節點併行執行對應的模型調優任務,得到多個優化後的候選模型結構,並統計多個優化後的候選模型結構的訓練結果。 In one embodiment, as shown in FIG. 4 , the method further includes: Step S160 : in the case that the candidate model structure needs to be tuned, create a plurality of model tuning tasks according to the plurality of candidate model structures and corresponding parameters; Step S170 : Schedule multiple model tuning tasks to the corresponding physical nodes, so that each physical node executes the corresponding model tuning tasks in parallel, obtains multiple optimized candidate model structures, and counts the training of multiple optimized candidate model structures result.
一種示例中,如果用戶輸入的配置信息中包含有候選模型結構相關的調優信息,那麽處理器獲取調優信息後,將創建模型調優任務的請求發送至API服務器,在API服務器收到創建模型調優任務的請求後,處理器監聽到來自API服務器的創建模型調優任務的事件,並根據調優信息,利用多個候選模型結構以及對應的參數創建多個模型調優任務。例如,根據子圖創建出完整的全圖,或者利用用戶輸入的完整的全圖進行模型優化和評估。 In one example, if the configuration information input by the user contains the tuning information related to the candidate model structure, after obtaining the tuning information, the processor sends a request to create a model tuning task to the API server, and the API server receives the request to create a model tuning task. After the model tuning task is requested, the processor listens to the event of creating a model tuning task from the API server, and creates multiple model tuning tasks using multiple candidate model structures and corresponding parameters according to the tuning information. For example, create a complete full graph from subgraphs, or use the full full graph entered by the user for model optimization and evaluation.
在一種實施方式中,如圖5所示,步驟140:包括:步驟141:根據檢索任務對應的算全部訓練任務所使用的資源數量,計算多個訓練任務對應的多個資源類型權重;步驟142:根據多個訓練任務對應的多個資源權重,確定多個訓練任務的調度順序;步驟143:根據調度順序將多個訓練任務調度至對應的物理節點。
In one embodiment, as shown in FIG. 5 ,
一種示例中,調度器在執行調度任務時,可以利用如下公式計算所有檢索任務正在使用的資源:
其中,Resource表示資源類型,可以為內存、CPU或硬體加速器資源等;i為檢索任務;j為檢索任務i的訓練任務;k為檢索任務i定義的最大檢索次數;Resourcejobj表示第j個任務所用的資源。 Among them, Resource represents the resource type, which can be memory, CPU or hardware accelerator resources, etc.; i is the retrieval task; j is the training task of retrieval task i; k is the maximum retrieval times defined by retrieval task i; Resourcejobj represents the jth task resources used.
調度器計算每個訓練任務j的資源權重:W j =w CPU .CPU i +w Mem .Mem i +w GPU .GPU i The scheduler calculates the resource weight of each training task j: W j = w CPU . CPU i + w Mem . Mem i + w GPU . GPU i
其中,wcpu,wMem,wGPU分別是CPU、內存和GPU的資源權重,都小於1;CPUi是檢索任務i使用的CPU資源數量,Memi是檢索任務i使用的內存資源數量,GPUi是檢索任務i使用的GPU資源數量。 Among them, wcpu, wMem, and wGPU are the resource weights of CPU, memory and GPU respectively, all of which are less than 1; CPUi is the amount of CPU resources used by retrieval task i, Memi is the amount of memory resources used by retrieval task i, and GPUi is the amount of memory resources used by retrieval task i. The number of GPU resources.
調度器在資源不足以運行隊列中的多個訓練任務時,會根據這一權重公式來决定運行哪一個任務。當高權重的任務資源無法被滿足時,調度器會順延調度第二高權重的訓練或調優任務,以此類推。 When the resources are insufficient to run multiple training tasks in the queue, the scheduler will decide which task to run based on this weighting formula. When the high-weight task resources cannot be satisfied, the scheduler will defer scheduling the second-highest-weight training or tuning task, and so on.
在一種實施方式中,如圖4所示,訓練結果包括實際訓練次數和訓練指標的實際值,預設訓練條件包括最大檢索次數和訓練指標的期望值;步驟S150,包括:步驟S151:在實際訓練次數大於或等於最大檢索次數,或訓練指標的實際值大於或等於訓練指標的期望值的情況下,得到最優網絡結構以及對應的參數,最優網絡結構以及對應的參數作為檢索結果。 In one embodiment, as shown in FIG. 4 , the training result includes the actual training times and the actual value of the training index, and the preset training condition includes the maximum number of retrieval times and the expected value of the training index; step S150 includes: step S151 : in the actual training When the number of times is greater than or equal to the maximum number of retrieval times, or the actual value of the training index is greater than or equal to the expected value of the training index, the optimal network structure and corresponding parameters are obtained, and the optimal network structure and corresponding parameters are used as the retrieval results.
一種示例中,訓練指標是指精度或速度等性能評估指標,包括誤差、精准度以及方差等。 In one example, the training metrics refer to performance evaluation metrics such as accuracy or speed, including error, accuracy, and variance.
在一種實施方式中,如圖4所示,還包括:步驟S180:針對每個訓練任務創建對應的臨時任務,臨時任務用於將訓練結果存儲至數據存儲器中。 In an embodiment, as shown in FIG. 4 , the method further includes: Step S180 : Create a corresponding temporary task for each training task, and the temporary task is used to store the training result in the data storage.
一種示例中,在調度完成後,訓練任務會被執行。在執行時,處理器會為每個訓練任務創建一個臨時任務,臨時任務與訓練任務在同一個命名空間下,臨時任務負責通過用K8S系統的客戶端從API服務器處獲得訓練任務或模型調優任務的日志,並將訓練結果存儲至數據存儲器中。例如,在流式的訓練任務的日志中,針對不同的訓練框架利用不同的處理機制得到訓練指標,臨時任務將訓練指標反饋到數據存儲器中。 In one example, training tasks are executed after scheduling is complete. During execution, the processor will create a temporary task for each training task. The temporary task and the training task are in the same namespace. The temporary task is responsible for obtaining training tasks or model tuning from the API server through the client of the K8S system. log of the task and store the training results in the data store. For example, in the logs of streaming training tasks, different processing mechanisms are used to obtain training indicators for different training frameworks, and temporary tasks feed back the training indicators to the data storage.
在另一種具體實施方式中,如圖6所示,提供一種神經網絡架構檢索裝置,包括:配置信息獲取模組110,用於獲取檢索任務的配置信息;候選模型結構生成模組120,用於根據配置信息生成多個候選模型結構以及對應的參數;訓練任務創建模組130,用於根據多個候選模型結構以及參數創建多個訓練任務;訓練任務調度模組140,用於將多個訓練任務調度至對應的物理節點,以使各物理節點併行執行對應的訓練任務,並統計多個訓練任務的訓練結果;檢索結果生成模組150,用於在訓練結果滿足配置信息中的預設訓練條件的情況下,得到檢索任務對應的檢索結果。
In another specific embodiment, as shown in FIG. 6 , a neural network architecture retrieval apparatus is provided, including: a configuration
在一種實施方式中,如圖7所示,還包括:
迭代檢索觸發模組160,用於在訓練結果未滿足配置信息中的預設訓練條件的情況下,觸發候選模型結構生成模組120執行根據配置信息生成多個候選模型結構以及對應的參數的步驟。
In one embodiment, as shown in Figure 7, it also includes:
The iterative
在一種實施方式中,配置信息是用戶輸入的配置信息,配置信息包括檢索任務信息和訓練任務信息。 In one embodiment, the configuration information is configuration information input by a user, and the configuration information includes retrieval task information and training task information.
在一種實施方式中,如圖7所示,候選模型結構生成模組120,包括:算法實例創建子模組121,用於根據檢索任務信息創建對應的算法實例;候選模型結構生成子模組122,用於將訓練任務信息加載至算法實例中,生成多個候選模型結構以及對應的參數。
In one embodiment, as shown in FIG. 7 , the candidate model
在一種實施方式中,檢索任務信息包括檢索空間和傳統模型結構的檢索算法,算法實例創建子模組121,包括:第一算法實例創建單元,用於根據檢索空間和傳統模型結構的檢索算法,創建對應的算法實例。 In one embodiment, the retrieval task information includes a retrieval space and a retrieval algorithm of the traditional model structure, and the algorithm instance creation sub-module 121 includes: a first algorithm instance creation unit, which is used for the retrieval algorithm according to the retrieval space and the traditional model structure, Create the corresponding algorithm instance.
在一種實施方式中,檢索任務信息包括檢索空間和單次模型結構的檢索算法,算法實例創建子模組121,還包括:第二算法實例創建單元,用於根據檢索空間和單次模型結構的檢索算法,創建對應的算法實例。 In one embodiment, the retrieval task information includes a retrieval algorithm for the retrieval space and the single-shot model structure, and the algorithm instance creation sub-module 121 further includes: a second algorithm instance creation unit for searching according to the retrieval space and the single-shot model structure. Retrieve the algorithm and create the corresponding algorithm instance.
在一種實施方式中,如圖7所示,還包括:模型調優任務創建模組170,用於在候選模型結構需要調優的情況下,根據多個候選模型結構以及對應的參數創建多個模型調優任務;
模型調優任務調度模組180,用於將多個模型調優任務調度至對應的物理節點,以使各物理節點併行執行對應的模型調優任務,得到多個優化後的候選模型結構,並統計多個優化後的候選模型結構的訓練結果。
In one embodiment, as shown in FIG. 7 , it further includes: a model tuning
在一種實施方式中,如圖8所示,訓練任務調度模組140,包括:資源類型權重計算子模組141,用於根據檢索任務對應的全部訓練任務所使用的資源數量,計算多個訓練任務對應的多個資源類型權重;調度順序確定子模組142,用於根據多個訓練任務對應的多個資源權重,確定多個訓練任務的調度順序;訓練任務調度子模組143,用於根據調度順序將多個訓練任務調度至對應的物理節點。
In one embodiment, as shown in FIG. 8 , the training
在一種實施方式中,如圖7所示,訓練結果包括實際訓練次數和訓練指標的實際值,預設訓練條件包括最大檢索次數和訓練指標的期望值,檢索結果生成模組150,包括:
檢索結果生成子模組151,用於在實際訓練次數大於或等於最大檢索次數,或訓練指標的實際值大於或等於訓練指標的期望值的情況下,得到最優網絡結構以及對應的參數,最優網絡結構以及對應的參數作為檢索結果。
In one embodiment, as shown in FIG. 7 , the training result includes the actual training times and the actual value of the training index, the preset training condition includes the maximum number of retrieval times and the expected value of the training index, and the retrieval
在一種實施方式中,如圖7所示,還包括:臨時任務創建模組190,用於針對每個訓練任務創建對應的臨時任務,臨時任務用於將訓練結果存儲至數據存儲器中。
In one embodiment, as shown in FIG. 7 , it further includes: a temporary
請實施例各裝置中的各模組的功能可以參見上述方法中的對應描述,在此不再贅述。 Please refer to the corresponding description in the above method for the functions of each module in each device of the embodiment, and details are not repeated here.
根據本發明的實施例,本發明還提供了一種電子設備和一種可讀存儲介質。 According to an embodiment of the present invention, the present invention further provides an electronic device and a readable storage medium.
如圖9所示,是根據本發明實施例的一種神經網絡架構檢索方法的電子設備的框圖。電子設備旨在表示各種形式的數字計算機,諸如,膝上型計算機、臺式計算機、工作臺、個人數字助理、服務器、刀片式服務器、大型計算機、和其它適合的計算機。電子設備還可以表示各種形式的移動裝置,諸如,個人數字處理、蜂窩電話、智能電話、可穿戴設備和其它類似的計算裝置。本文所示的部件、它們的連接和關係、以及它們的功能僅僅作為示例,並且不意在限制本文中描述的和/或者要求的本發明的實現。 As shown in FIG. 9 , it is a block diagram of an electronic device of a neural network architecture retrieval method according to an embodiment of the present invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are by way of example only, and are not intended to limit implementations of the inventions described and/or claimed herein.
如圖9所示,該電子設備包括:一個或多個處理器901、存儲器902,以及用於連接各部件的接口,包括高速接口和低速接口。各個部件利用不同的總線互相連接,並且可以被安裝在公共主板上或者根據需要以其它方式安裝。處理器可以對在電子設備內執行的指令進行處理,包括存儲在存儲器中或者存儲器上以在外部輸入/輸出裝置(諸如,耦合至接口的顯示設備)上顯示GUI的圖形信息的指令。在其它實施方式中,若需要,可以將多個處理器和/或多條總線與多個存儲器和多個存儲器一起使用。同樣,可以連接多個電子設備,各個設備提供部分必要的操作(例如,作為服務器陣列、一組刀片式服務器、或者多處理器系統)。圖9中以一個處理器901為例。
As shown in FIG. 9, the electronic device includes: one or
存儲器902即為本發明所提供的非瞬時計算機可讀存儲介質。其中,所述存儲器存儲有可由至少一個處理器執行的指令,以使所述至少一個處理器執行本申請所提供的一種神經網絡架構檢索方法。本發明的非瞬時計算機可讀
存儲介質存儲計算機指令,該計算機指令用於使計算機執行本發明所提供的一種神經網絡架構檢索方法。
The
存儲器902作為一種非瞬時計算機可讀存儲介質,可用於存儲非瞬時軟件程序、非瞬時計算機可執行程序以及模組,如本發明實施例中的一種神經網絡架構檢索方法對應的程序指令/模組(例如,附圖6所示的配置信息獲取模組110、候選模型結構生成模組120、訓練任務創建模組130、訓練任務調度模組140、檢索結果生成模組150)。處理器901通過運行存儲在存儲器902中的非瞬時軟件程序、指令以及模組,從而執行服務器的各種功能應用以及數據處理,即實現上述方法實施例中的一種神經網絡架構檢索方法。
As a non-transitory computer-readable storage medium, the
存儲器902可以包括存儲程序區和存儲數據區,其中,存儲程序區可存儲操作系統、至少一個功能所需要的應用程序;存儲數據區可存儲根據一種神經網絡架構檢索方法的電子設備的使用所創建的數據等。此外,存儲器902可以包括高速隨機存取存儲器,還可以包括非瞬時存儲器,例如至少一個磁盤存儲器件、閃存器件、或其他非瞬時固態存儲器件。在一些實施例中,存儲器902可選包括相對於處理器901遠程設置的存儲器,這些遠程存儲器可以通過網絡連接至一種神經網絡架構檢索方法的電子設備。上述網絡的實例包括但不限於互聯網、企業內部網、局域網、移動通信網及其組合。
The
一種神經網絡架構檢索方法的電子設備還可以包括:輸入裝置903和輸出裝置904。處理器901、存儲器902、輸入裝置903和輸出裝置904可以通過總線或者其他方式連接,圖9中以通過總線連接為例。
The electronic device of a neural network architecture retrieval method may further include: an
輸入裝置903可接收輸入的數字或字符信息,以及產生與一種神經網絡架構檢索方法的電子設備的用戶設置以及功能控制有關的鍵信號輸入,例如
觸摸螢幕、小鍵盤、鼠標、軌迹板、觸摸板、指示桿、一個或者多個鼠標按鈕、軌迹球、操縱桿等輸入裝置。輸出裝置904可以包括顯示設備、輔助照明裝置(例如,LED)和觸覺反饋裝置(例如,振動電機)等。該顯示設備可以包括但不限於,液晶顯示器(LCD)、發光二極管(LED)顯示器和等離子體顯示器。在一些實施方式中,顯示設備可以是觸摸螢幕。
The
此處描述的系統和技術的各種實施方式可以在數字電子電路系統、集成電路系統、專用ASIC(專用集成電路)、計算機硬體、固件、軟體、和/或它們的組合中實現。這些各種實施方式可以包括:實施在一個或者多個計算機程序中,該一個或者多個計算機程序可在包括至少一個可編程處理器的可編程系統上執行和/或解釋,該可編程處理器可以是專用或者通用可編程處理器,可以從存儲系統、至少一個輸入裝置、和至少一個輸出裝置接收數據和指令,並且將數據和指令傳輸至該存儲系統、該至少一個輸入裝置、和該至少一個輸出裝置。 Various implementations of the systems and techniques described herein can be implemented in digital electronic circuitry, integrated circuit systems, application specific ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs executable and/or interpretable on a programmable system including at least one programmable processor that may is a special-purpose or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
這些計算程序(也稱作程序、軟體、軟體應用、或者代碼)包括可編程處理器的機器指令,並且可以利用高級過程和/或面向對象的編程語言、和/或彙編/機器語言來實施這些計算程序。如本文使用的,術語“機器可讀介質”和“計算機可讀介質”指的是用於將機器指令和/或數據提供給可編程處理器的任何計算機程序產品、設備、和/或裝置(例如,磁盤、光盤、存儲器、可編程邏輯裝置(PLD)),包括,接收作為機器可讀信號的機器指令的機器可讀介質。術語“機器可讀信號”指的是用於將機器指令和/或數據提供給可編程處理器的任何信號。 These computational programs (also referred to as programs, software, software applications, or codes) include machine instructions for programmable processors, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages calculation program. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or apparatus for providing machine instructions and/or data to a programmable processor ( For example, magnetic disks, optical disks, memories, programmable logic devices (PLDs), including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
為了提供與用戶的交互,可以在計算機上實施此處描述的系統和技術,該計算機具有:用於向用戶顯示信息的顯示裝置(例如,CRT(陰極射線管)或者LCD(液晶顯示器)監視器);以及鍵盤和指向裝置(例如,鼠標或者軌迹球),用戶可以通過該鍵盤和該指向裝置來將輸入提供給計算機。其它種類的裝置還可以用於提供與用戶的交互;例如,提供給用戶的反饋可以是任何形式的傳感反饋(例如,視覺反饋、聽覺反饋、或者觸覺反饋);並且可以用任何形式(包括聲輸入、語音輸入或者、觸覺輸入)來接收來自用戶的輸入。 To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having a display device (eg, a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user ); and a keyboard and pointing device (eg, a mouse or trackball) through which a user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (eg, visual feedback, auditory feedback, or tactile feedback); and can be in any form (including acoustic input, voice input, or tactile input) to receive input from the user.
可以將此處描述的系統和技術實施在包括後臺部件的計算系統(例如,作為數據服務器)、或者包括中間件部件的計算系統(例如,應用服務器)、或者包括前端部件的計算系統(例如,具有圖形用戶界面或者網絡瀏覽器的用戶計算機,用戶可以通過該圖形用戶界面或者該網絡瀏覽器來與此處描述的系統和技術的實施方式交互)、或者包括這種後臺部件、中間件部件、或者前端部件的任何組合的計算系統中。可以通過任何形式或者介質的數字數據通信(例如,通信網絡)來將系統的部件相互連接。通信網絡的示例包括:局域網(LAN)、廣域網(WAN)和互聯網。 The systems and techniques described herein may be implemented on a computing system that includes back-end components (eg, as a data server), or a computing system that includes middleware components (eg, an application server), or a computing system that includes front-end components (eg, a user's computer having a graphical user interface or web browser through which a user may interact with implementations of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system. The components of the system may be interconnected by any form or medium of digital data communication (eg, a communication network). Examples of communication networks include: Local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
計算機系統可以包括客戶端和服務器。客戶端和服務器一般遠離彼此並且通常通過通信網絡進行交互。通過在相應的計算機上運行並且彼此具有客戶端-服務器關係的計算機程序來產生客戶端和服務器的關係。 A computer system can include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other.
應該理解,可以使用上面所示的各種形式的流程,重新排序、增加或删除步驟。例如,本發明中記載的各步驟可以並行地執行也可以順序地執行也 可以不同的次序執行,只要能夠實現本發明公開的技術方案所期望的結果,本文在此不進行限制。 It should be understood that steps may be reordered, added or deleted using the various forms of flow shown above. For example, the steps described in the present invention may be executed in parallel or sequentially or in sequence. It can be performed in different orders, as long as the desired results of the technical solutions disclosed in the present invention can be achieved, and are not limited herein.
上述具體實施方式,並不構成對本發明保護範圍的限制。本領域技術人員應該明白的是,根據設計要求和其他因素,可以進行各種修改、組合、子組合和替代。任何在本發明的精神和原則之內所作的修改、等同替換和改進等,均應包含在本發明保護範圍之內。 The above-mentioned specific embodiments do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.
S110:獲取檢索任務的配置信息 S110: Obtain the configuration information of the retrieval task
S120:根據配置信息生成多個候選模型結構以及對應的參數 S120: Generate multiple candidate model structures and corresponding parameters according to the configuration information
S130:根據多個候選模型結構以及參數創建多個訓練任務 S130: Create multiple training tasks according to multiple candidate model structures and parameters
S140:將多個訓練任務調度至對應的物理節點,以使各物理節點並行執行對應的訓練任務,並統計多個訓練任務的訓練結果 S140: Schedule multiple training tasks to corresponding physical nodes, so that each physical node executes the corresponding training tasks in parallel, and counts the training results of the multiple training tasks
S150:在訓練結果滿足配置信息中的預設訓練條件的情況下,得到檢索任務對應的檢索結果 S150: Obtain a retrieval result corresponding to the retrieval task under the condition that the training result satisfies the preset training condition in the configuration information
Claims (20)
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010829782.5 | 2020-08-18 | ||
CN202010829782.5A CN112000450A (en) | 2020-08-18 | 2020-08-18 | Neural network architecture searching method and device |
Publications (2)
Publication Number | Publication Date |
---|---|
TW202209152A TW202209152A (en) | 2022-03-01 |
TWI773100B true TWI773100B (en) | 2022-08-01 |
Family
ID=73472626
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
TW110102763A TWI773100B (en) | 2020-08-18 | 2021-01-26 | Neural network structure search method and apparatus |
Country Status (3)
Country | Link |
---|---|
CN (1) | CN112000450A (en) |
TW (1) | TWI773100B (en) |
WO (1) | WO2022037039A1 (en) |
Families Citing this family (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112000450A (en) * | 2020-08-18 | 2020-11-27 | 中国银联股份有限公司 | Neural network architecture searching method and device |
CN112819138A (en) * | 2021-01-26 | 2021-05-18 | 上海依图网络科技有限公司 | Optimization method and device of image neural network structure |
CN114089889B (en) * | 2021-02-09 | 2024-04-09 | 京东科技控股股份有限公司 | Model training method, device and storage medium |
CN112965803A (en) * | 2021-03-22 | 2021-06-15 | 共达地创新技术(深圳)有限公司 | AI model generation method and electronic equipment |
CN115563063A (en) * | 2021-07-01 | 2023-01-03 | 马上消费金融股份有限公司 | Model construction method and device and electronic equipment |
CN114141255A (en) * | 2021-11-24 | 2022-03-04 | 中国电信股份有限公司 | Training method and device of voiceprint recognition model and voiceprint recognition method and device |
CN114640695B (en) * | 2022-04-24 | 2023-04-07 | 上海交通大学 | High-frequency time sequence data effective transmission method based on long sequence double prediction and informar for intelligent factory |
CN114816669A (en) * | 2022-04-29 | 2022-07-29 | 北京百度网讯科技有限公司 | Distributed training method and data processing method of model |
CN114968520B (en) * | 2022-05-19 | 2023-11-24 | 北京百度网讯科技有限公司 | Task searching method and device, server and storage medium |
US20240034353A1 (en) * | 2022-07-28 | 2024-02-01 | Baidu Usa Llc | Automatic generation of corner scenarios data for tuning autonomous vehicles |
CN115220899A (en) * | 2022-08-20 | 2022-10-21 | 抖音视界有限公司 | Model training task scheduling method and device and electronic equipment |
CN116954873B (en) * | 2023-09-21 | 2024-01-23 | 浪潮电子信息产业股份有限公司 | Heterogeneous computing system, and method, device, equipment and medium for selecting power nodes of heterogeneous computing system |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111198754A (en) * | 2018-11-19 | 2020-05-26 | 中移(杭州)信息技术有限公司 | Task scheduling method and device |
CN111325338A (en) * | 2020-02-12 | 2020-06-23 | 暗物智能科技(广州)有限公司 | Neural network structure evaluation model construction and neural network structure search method |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9390370B2 (en) * | 2012-08-28 | 2016-07-12 | International Business Machines Corporation | Training deep neural network acoustic models using distributed hessian-free optimization |
CN110543944B (en) * | 2019-09-11 | 2022-08-02 | 北京百度网讯科技有限公司 | Neural network structure searching method, apparatus, electronic device, and medium |
CN111325356A (en) * | 2019-12-10 | 2020-06-23 | 四川大学 | Neural network search distributed training system and training method based on evolutionary computation |
CN111063000B (en) * | 2019-12-15 | 2023-12-26 | 中国科学院深圳先进技术研究院 | Magnetic resonance rapid imaging method and device based on neural network structure search |
CN111324630B (en) * | 2020-03-04 | 2023-07-25 | 中科弘云科技(北京)有限公司 | MPI-based neural network architecture search parallelization method and equipment |
CN112000450A (en) * | 2020-08-18 | 2020-11-27 | 中国银联股份有限公司 | Neural network architecture searching method and device |
-
2020
- 2020-08-18 CN CN202010829782.5A patent/CN112000450A/en active Pending
-
2021
- 2021-01-26 TW TW110102763A patent/TWI773100B/en active
- 2021-03-12 WO PCT/CN2021/080497 patent/WO2022037039A1/en active Application Filing
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111198754A (en) * | 2018-11-19 | 2020-05-26 | 中移(杭州)信息技术有限公司 | Task scheduling method and device |
CN111325338A (en) * | 2020-02-12 | 2020-06-23 | 暗物智能科技(广州)有限公司 | Neural network structure evaluation model construction and neural network structure search method |
Also Published As
Publication number | Publication date |
---|---|
TW202209152A (en) | 2022-03-01 |
CN112000450A (en) | 2020-11-27 |
WO2022037039A1 (en) | 2022-02-24 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
TWI773100B (en) | Neural network structure search method and apparatus | |
US20220224752A1 (en) | Automated server workload management using machine learning | |
JP7042897B2 (en) | Model parameter update method and equipment | |
US20220166850A1 (en) | Adaptive computation and faster computer operation | |
US20210248469A1 (en) | Method and apparatus for scheduling deep learning reasoning engines, device, and medium | |
WO2022000802A1 (en) | Deep learning model adaptation method and apparatus and electronic device | |
US9513833B2 (en) | Asynchronous processing of mapping information | |
CN111782147B (en) | Method and device for cluster expansion and contraction capacity | |
CN111488492B (en) | Method and device for searching graph database | |
US10972555B2 (en) | Function based dynamic traffic management for network services | |
US11769125B2 (en) | Method and apparatus for processing transaction requests in blockchain, device and medium | |
US10768998B2 (en) | Workload management with data access awareness in a computing cluster | |
CN111461343B (en) | Model parameter updating method and related equipment thereof | |
US10102230B1 (en) | Rate-limiting secondary index creation for an online table | |
CN111914010B (en) | Method, device, equipment and storage medium for processing business | |
CN111966361B (en) | Method, device, equipment and storage medium for determining model to be deployed | |
CN115335821B (en) | Offloading statistics collection | |
CN111782341B (en) | Method and device for managing clusters | |
EP4172769A1 (en) | Auto-sizing for stream processing applications | |
CN112579897A (en) | Information searching method and device | |
CN113778973B (en) | Data storage method and device | |
US11671370B2 (en) | Method and apparatus for adjusting network flow | |
CN112101447B (en) | Quality evaluation method, device, equipment and storage medium for data set | |
CN111340222B (en) | Neural network model searching method and device and electronic equipment | |
CN113778610B (en) | Method and device for determining resources |