CN116628279A

CN116628279A - Search method, apparatus, device, and computer-readable storage medium

Info

Publication number: CN116628279A
Application number: CN202310189017.5A
Authority: CN
Inventors: 冯乾泰; 赵娟萍
Original assignee: Zeku Technology Shanghai Corp Ltd
Current assignee: Weiguang Co ltd
Priority date: 2023-03-01
Filing date: 2023-03-01
Publication date: 2023-08-22

Abstract

The embodiment of the application discloses a searching method, a searching device, searching equipment and a computer readable storage medium. The method comprises the following steps: selecting chromosomes meeting fixed constraints in a super network corresponding to the architecture search space, and determining an initial population; the super network comprises at least two sub networks, and each chromosome corresponds to one sub network; determining a parent population based on the initial population and the initial target evolution direction, and mutating and crossing chromosomes included in the parent population to determine a target population; continuing to determine a next parent population based on the target population and the next target evolution direction, and mutating and crossing a next chromosome included in the next parent population to determine the next target population until the iteration times are reached, so as to realize multi-target evolution and obtain a last target population; and determining a target chromosome from the last target population to realize multi-target architecture search, and obtaining a target sub-network corresponding to the target chromosome.

Description

Search method, apparatus, device, and computer-readable storage medium

Technical Field

The present application relates to the field of computer technologies, and in particular, to a search method, apparatus, device, and computer readable storage medium.

Background

The neural network architecture search technique (Neural Network Architecture Search, NAS) refers to the design of an optimal architecture for a neural network model through algorithm automation, instead of manual design by an algorithm engineer. The Hardware aware neural architecture search technology (Hardware-aware Neural Network Architecture Search, HW NAS) refers to adding Hardware aware functionality on the basis of NAS. In practical applications, neural Networks (NNs) typically need to be deployed on specific target hardware. For example, the chip needs to reach optimal states of accuracy, delay, power consumption, storage, etc. at the same time when deploying a specific scene NN. Therefore, when the NN network is deployed on the target hardware, not only the performance of the model is required to be considered, but also the hardware indexes such as time delay, power consumption and the like of the model on the hardware are weighed, so that the product achieves the best effect.

In the related art, a pareto optimal solution (Pareto Optimal Solution) is obtained in architecture search by manually setting fixed hardware index constraints and utilizing an evolutionary algorithm (Evolution Algorithm, EA).

However, in the architectural search given the hardware index constraint c (e.g., delay), it is possible that the performance Px of a certain searched model x is consistent with the performance Py of another model y: px=py, or Px is only slightly larger than Py, however, the hardware constraints are clearly c > Cx > Cy, so that the searched model x is not the most needed by the user. If there are models with consistent or nearly equal performance but lower latency, the user would prefer to select the model with lower latency, i.e., model y. Therefore, the search method in the related art reduces the accuracy of the search result.

Disclosure of Invention

The embodiment of the application provides a searching method, a searching device, searching equipment and a computer readable storage medium, which improve the accuracy of a searching result.

The technical scheme of the embodiment of the application is realized as follows:

in a first aspect, an embodiment of the present application provides a search method, where the method includes: selecting chromosomes meeting fixed constraints in a super network corresponding to the architecture search space, and determining an initial population; the super network comprises at least two sub networks, and each chromosome corresponds to one sub network; determining a parent population based on the initial population and the initial target evolution direction, and mutating and crossing chromosomes included in the parent population to determine a target population; continuing to determine a next parent population based on the target population and the next target evolution direction, and mutating and crossing a next chromosome included in the next parent population to determine the next target population until the iteration times are reached, so as to realize multi-target evolution and obtain a last target population; and determining a target chromosome from the last target population to realize multi-target architecture search, and obtaining a target sub-network corresponding to the target chromosome.

In a second aspect, an embodiment of the present application provides a search apparatus, including: the population determining module is used for selecting chromosomes meeting fixed constraints in the super network corresponding to the architecture search space to determine an initial population; the super network comprises at least two sub networks, and each chromosome corresponds to one sub network; determining a parent population based on the initial population and an initial target evolution direction; the evolution module is used for mutating and crossing chromosomes included in the parent population to determine a target population; the population determining module is further used for continuously determining the next parent population based on the target population and the next target evolution direction; the evolution module is further used for carrying out mutation and crossover on a next chromosome included in the next parent population to determine a next target population until iteration times are reached, multi-target evolution is realized, and a last target population is obtained; and the target sub-network determining module is used for determining a target chromosome from the last target population so as to realize multi-target architecture search and obtain a target sub-network corresponding to the target chromosome.

In a third aspect, an embodiment of the present application provides a search apparatus, including: a memory for storing an executable computer program; and a processor, configured to implement the search method according to the first aspect when executing the executable computer program stored in the memory.

In a fourth aspect, an embodiment of the present application provides a computer readable storage medium storing a computer program for implementing the search method according to the first aspect, when the computer program is executed by a processor.

The embodiment of the application provides a searching method, a searching device, searching equipment and a computer readable storage medium. According to the scheme provided by the embodiment of the application, the method comprises the following steps: selecting chromosomes meeting fixed constraints in a super network corresponding to the architecture search space, and determining an initial population; the super network comprises at least two sub networks, and each chromosome corresponds to one sub network; and determining a parent population based on the initial population and the initial target evolution direction, wherein initial chromosomes in the initial population meet fixed constraints, selecting chromosomes which are better in the initial target evolution direction from the initial population, and constructing the parent population. And the chromosomes included in the parent population are mutated and crossed to determine the target population, and the evolution process of the initial target evolution direction in the current round is completed. And continuously determining a next parent population based on the target population and the next target evolution direction, and mutating and crossing the next chromosome included in the next parent population to determine the next target population, thereby completing the next round of evolution process about the next target evolution direction. Until the iteration times are reached, through a plurality of evolutionary processes about different target evolutionary directions, multi-target evolutionary is realized, a last target population is obtained, a target chromosome is determined from the last target population, so that multi-target architecture search is realized, and a target sub-network corresponding to the target chromosome is obtained. Through the evolution direction of the targets in multiple rounds in the scheme, the searched target sub-network can meet the hardware requirement and achieve better model performance. The method and the device realize collaborative optimization of multiple hardware targets such as model accuracy, delay, power consumption and the like while meeting the search efficiency, thereby meeting the multi-dimensional requirement in an actual scene and improving the accuracy of search results.

Drawings

Fig. 1 is an exemplary schematic diagram of a pareto optimal solution according to an embodiment of the present application;

fig. 2 is an exemplary schematic diagram of a single target pareto optimal solution according to an embodiment of the present application;

FIG. 3 is a flowchart illustrating optional steps of a searching method according to an embodiment of the present application;

FIG. 4 is a flowchart illustrating alternative steps of another searching method according to an embodiment of the present application;

FIG. 5 is a flowchart illustrating alternative steps of a search method according to an embodiment of the present application;

FIG. 6 is a flowchart illustrating alternative steps of a search method according to an embodiment of the present application;

FIG. 7 is a schematic diagram of an adaptive adjustment strategy for cross variation according to an embodiment of the present application;

FIG. 8 is a flowchart of an optional step of a greedy strategy-based multi-objective evolutionary algorithm, according to an embodiment of the application;

fig. 9 is an alternative schematic structural diagram of a search device according to an embodiment of the present application;

fig. 10 is a schematic diagram of a composition structure of a search device according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. It should be understood that some embodiments described herein are merely for explaining the technical solution of the present application, and are not intended to limit the technical scope of the present application.

In order to better understand the searching method provided in the embodiment of the present application, before the technical solution of the embodiment of the present application is introduced, an application background and related technologies are described.

The advantages of NAS are mainly manifested in the following aspects: firstly, the cost of manually designing the network can be greatly reduced, which is particularly obvious when the situation is complex and the number of tasks is huge; secondly, the NAS can excavate a potential optimal model through an automatic means to the greatest extent; thirdly, the defects caused by manual network design can be avoided to a great extent, for example, the inherent knowledge system of human beings and the deviation caused by cognition on model design are reduced. Therefore, NAS is becoming more and more widely used, and NAS mainly contains three contents: search space design, search strategy design, and model evaluation strategy.

Based on the HW-NAS of the NAS, in order to simultaneously consider the performance of the model, the time delay of the model on hardware, the power consumption and other hardware indexes, the HW-NAS automatically adjusts the search strategy through the performance feedback of specific target hardware, thereby searching the model with the best comprehensive indexes such as accuracy, time delay, power consumption and the like. Under the condition that demands of embedded equipment, edge computing and the like are gradually increased, higher requirements are put on reasoning delay, power consumption and the like of an NN model, and HW-NAS and multi-target search technology are generated.

The Evolutionary Algorithm (EA) is a common heuristic algorithm in mathematics, is derived from an evolution mechanism of organisms, simulates the processes of reproduction, mutation, genetic recombination, natural selection and the like of the organisms in the nature, and thus finds the individuals which are most suitable for the environment from candidate populations. One of the outstanding advantages is the ability to find a relatively optimal solution to the optimization problem to be solved within an acceptable time. Evolutionary algorithms include genetic algorithms (Genetic Algorithm, GA), evolutionary planning (Evolutionary Programming, EP, genetic planning (Genetic Programming, GP) and evolutionary strategies (Evolution Strategy, ES). Genetic Algorithms (GA) are also very widely used in HW-NAS.

The source-for-All (OFA) proposed in the related art is to find the best model of the actual hardware perception by Genetic Algorithm (GA). The method can realize one-time training and multiple-time hardware deployment in a short time. Illustratively, the OFA first trains a super-network containing a large number of sub-networks, and then adopts an evolutionary algorithm to obtain the best sub-model meeting the requirements from the super-network for actual deployment according to the hardware requirements (e.g., delay) to be deployed.

Based on OFA or the likeIn the network architecture search of genetic algorithms, the super network training and searching processes are usually decoupled. That is, prior to searching, a super network training is performed. A super-network can be regarded as a network pool comprising a large number of sub-networks. In the evolution algorithm of the OFA, a target hardware constraint c is required to be given, and the optimization performance is searched under the condition that the target hardware constraint c is met, so that a model architecture of an optimal sub-network, namely a pareto optimal solution in the mathematical sense, is obtained. Wherein, the pareto optimal solution (pareto optimal solution) is defined as follows: if one solution x is called the pareto optimal solution, if and only if x is not dominated by the other solutions. Wherein the dominant definition is as follows: x dominates y, denoted as x < y, if and only if f _i (x)≤f _i (y), and->s.t.f _j (x)<f _j (y), s.t. represents a constraint.

Based on the definition of the pareto optimal solution, the optimal model (i.e., the pareto optimal solution) obtained in the architecture search needs to satisfy the following conditions: there is no model with better effect on all targets. The current goal cannot be made better without sacrificing other goals. That is, to further optimize the current objective, it is necessary to make a loss to some other objective. Fig. 1 is an exemplary schematic diagram of a pareto optimal solution provided by an embodiment of the present application, where the abscissa of fig. 1 represents a hardware index c, the ordinate represents a performance p, the curve represents a pareto front, and by giving a hardware target constraint (the hardware constraint c in fig. 1), a search in a performance direction (the OFA optimization direction in fig. 1) is performed, and a single hardware target is used as a fixed constraint, so as to obtain an optimal model.

Based on the above-mentioned fig. 1, the ofa uses a single hardware target as a fixed strong constraint, which is difficult to achieve the pareto optimal solution in the application scenario of multi-target optimization. As shown in fig. 2, fig. 2 is an exemplary schematic diagram of a single-target pareto optimal solution according to an embodiment of the present application, where the abscissa of fig. 2 represents a hardware index c, the ordinate represents a performance p, the curve represents a pareto front edge, and the performance p is not always improved along with the relaxation of the hardware index c, and is slower and slower in the later stage. As can be seen from fig. 2, in the architectural search of the given hardware constraint c, the performance Px of the searched optimal model x (the optimal model x obtained by OFA in fig. 2) is almost the same as the performance Py of another model y (Px is slightly larger than Py), but the hardware requirement of model x is higher, and the user may prefer to select model y (model y that the user may need in fig. 2) more than model x. That is, if multiple models are relatively close in hardware index, the OFA is very easy to search out the best model that is not needed by the user.

However, in practical applications, there are some non-idealities in the architecture search process. One situation is that relaxation of the hardware index does not necessarily lead to an improvement in the search effect experience. This is caused by the influence of the training of the super-network, the evaluation algorithm, the model training randomness, etc. Another situation is that the improvement of the model performance is slower and slower along with the relaxation of the hardware index. In this case, the user typically does not wish to trade off excessive hardware metrics for a weak precision boost, and the optimal ratio between targets depends on the actual scenario requirements. An illustration of such a case is given in fig. 2, where after the hardware index is above a certain value, the performance increases very slowly. In scenarios where performance requirements are not very stringent, users often wish to sacrifice performance of small magnitude in exchange for larger hardware index optimization space.

The architecture searching method based on OFA in the related art has the following technical defects: (1) Architectural searches based on the OFA family do not optimize the hardware metrics sufficiently. The processing of the hardware index by the OFA is regarded as a fixed constraint and can be regarded as single-objective optimization of performance under a certain hardware objective feasible region. The OFA algorithm requires manual setting of a fixed hardware index constraint, such as delay constraint c. In the evolution algorithm process, all the rigid index constraints of candidate models generated by random generation, crossing and mutation only need to meet the condition smaller than c. This logic results in the model searched not necessarily being the pareto optimal solution. The meaning of finding the pareto optimal solution in the architecture search is that the current target cannot be better without sacrificing other targets. That is, to further optimize the current objective, it is necessary to make a loss to some other objective. For example, in an architectural search given a hardware constraint c, it may happen that the performance Px of a certain searched model x is consistent with the performance Py of another model y: px=py, or Px is only slightly larger than Py, but the hardware constraints are clearly c > Cx > Cy, so that the searched model x is not the most needed by the user.

(2) A scheme for multi-objective optimization is lacking. In the architectural search process of OFA, the best model is searched for only a single specific hardware constraint (i.e., a single objective). However, in the practical application scenario, the model to be searched out can be optimized under various indexes (i.e., multiple targets) such as precision, delay, power consumption, etc., so as to meet multiple requirements. Therefore, HW-NAS needs to meet the need for multi-objective optimization.

In summary, the technical problems of architecture search performed by the evolutionary algorithm adopted by the OFA include: (1) the search results may not be pareto optimal solutions; (2) OFA does not support architecture search tasks that are performed concurrently by multiple targets (e.g., performance, latency, power consumption).

The embodiment of the application provides a searching method, as shown in fig. 3, fig. 3 is a step flow chart of the searching method provided by the embodiment of the application, and the searching method comprises the following steps:

s101, selecting chromosomes meeting fixed constraints in a super network corresponding to an architecture search space, and determining an initial population; the super-network comprises at least two sub-networks, one for each chromosome.

In an embodiment of the application, the trained superconnel comprises at least two subnetworks, each subnetwork being considered as a chromosome, from which chromosomes are screened for the presence of a chromosome satisfying a fixed constraint. The fixed constraints may be parameters about model metrics, including, but not limited to, model performance, latency, power consumption, computation, memory space, etc., that are custom set by those skilled in the art according to actual needs, where model performance may be understood as the accuracy of model predictions, e.g., for a network of classification functions, the higher the accuracy of classification results, the better the model performance. The fixed constraint may be a preset value of a certain index or at least two or more indexes, predicting index information of each sub-network in the super-network, and adding the index information of the sub-network into the initial population until the size of the initial population is satisfied when the index information of the sub-network satisfies the fixed constraint, thereby generating the initial population.

In the embodiment of the application, the index information of each sub-network in the super-network is predicted, the hardware requirement of the sub-network can be tested through hardware testing, and the model performance of the sub-network can be tested through a test sample set. Each sub-network can also be input into a target predictor for prediction, and hardware demand parameters and model performance parameters of the sub-network are output.

In the embodiment of the application, a person skilled in the art not only sets a fixed constraint in a self-defined manner, but also defines relevant parameters of an evolutionary algorithm, wherein the relevant parameters of the evolutionary algorithm include but are not limited to: initial population size (e.g., setting an initial population comprising N initial chromosomes), iteration round (e.g., number of iterations E), crossover scaling factor, variation scaling factor (e.g., variation scaling factor is M/N, crossover scaling factor is (N-M)/N, M number of mutations are performed, (N-M) crossover, thereby ensuring the number of chromosomes in the population is N), mutation probability, etc.

In the embodiment of the application, the super network in the architecture search space comprises at least two sub-networks, samples are randomly carried out in the architecture search space, a plurality of chromosomes (namely the architecture configuration of the sub-networks) are continuously generated, index information of the chromosomes under each index (the index information can be understood as calculating target values of the chromosomes under each target) is calculated, and the chromosomes meeting fixed constraints are added into the initial population until the size of the initial population is met, so that the initial population is generated. If there is a fixed constraint under a certain index, selecting a chromosome satisfying the constraint.

S102, determining a parent population based on the initial population and the initial target evolution direction, and mutating and crossing chromosomes included in the parent population to determine a target population.

In the embodiment of the application, the initial population comprises N initial chromosomes, and the index information of each chromosome under each index is predicted, wherein the initial target evolution direction is any index or at least two indexes with correlation or conflict. And sequencing index information of N initial chromosomes in the initial target evolution direction, and selecting the first k chromosomes as parent populations. Crossing chromosomes included in the parent population according to the defined crossing proportion coefficient, mutating the chromosomes included in the parent population according to the defined mutation proportion coefficient, and adding the crossed and mutated chromosomes into the target population to obtain the target population, wherein the target population comprises N evolutionary chromosomes. The target population may also be referred to as elite population, and the evolving chromosomes may also be referred to as elite chromosomes.

In the embodiment of the application, the round of evolution process generates a target population by selecting the optimal chromosome for mutation and crossing, and completes the selection of the evolution direction based on the greedy strategy. The evolutionary direction selection based on greedy strategy includes: in each round of evolution process, sampling the evolution direction, wherein the sampled target evolution direction is any index, aiming at different chromosomes, taking index information corresponding to the target evolution direction (namely a target value corresponding to the selected evolution direction) as fitness score of the chromosome, and storing and adding the chromosome with the highest fitness score into a parent population. And the chromosomes in the parent population are mutated and crossed according to the cross-mutation proportion self-adaptive adjustment strategy to obtain a new population (namely a target population). The cross-variation ratio adaptive adjustment strategy includes: the coefficient of variation and the number of crossover ratios in the first round are default (custom set by those skilled in the art according to actual needs), and the second round and later are adjusted in real time according to the distribution rules (e.g., mean and variance) of chromosomes in the parent population.

S103, continuously determining a next parent population based on the target population and the next target evolution direction, and carrying out mutation and crossover on a next chromosome included in the next parent population to determine the next target population until the iteration times are reached, so as to realize multi-target evolution and obtain the last target population.

In the embodiment of the present application, the step of generating the parent population based on the target evolution direction in S102 and determining the target population by crossing and mutation is repeatedly performed to determine the next target population, thereby completing the next round of evolution process about the next target evolution direction. And sequentially cycling until the iteration number E is reached, and outputting the last target population (the last target population can be called elite set).

The scheme provides a greedy strategy-based evolutionary direction selection, and in a greedy strategy-based multi-target evolutionary algorithm, a target evolutionary direction needs to be selected in each round. The greedy strategy is that in each round of iteration, a single target is selected as the optimization direction of the round, and the optimization effect of multiple targets is realized by continuously searching local optima under the single target.

In the embodiment of the application, each round of evolution process can randomly select a target evolution direction or sequentially select the target evolution direction, each round of evolution process comprises crossing and mutation, and the selection-mutation-crossing process is repeatedly performed until a set iteration round (i.e. iteration times) is reached. If the fixed constraint exists under the index corresponding to a certain target evolution direction, the mutation process and the crossover process both select chromosomes meeting the fixed constraint. After multiple rounds of evolution process, the multi-target evolution direction selection based on greedy strategy is completed, and the last target population is obtained.

In the embodiment of the application, the multi-objective optimization is realized by adding a plurality of objective hardware constraints, and better balance (trade-off) of model performance and efficiency is realized. The model performance is ensured, and meanwhile, the working efficiency is improved, for example, the delay is low, the power consumption is low, the calculated amount is small, and the like.

S104, determining a target chromosome from the last target population to realize multi-target architecture search, and obtaining a target sub-network corresponding to the target chromosome.

In the embodiment of the application, the last target population comprises a plurality of candidate chromosomes, and index information of sub-networks corresponding to the candidate chromosomes is predicted, wherein the index information comprises model performance parameters and hardware requirement parameters (such as delay, power consumption, calculated amount, storage space and the like). After the last target population is obtained, candidate chromosomes with optimal model performance parameters (e.g., highest accuracy) may be selected in the last target population as target chromosomes. The candidate chromosome with the optimal hardware requirement parameter (any one of delay, power consumption, calculation amount and storage space) can also be selected from the last target population to serve as the target chromosome. A subnetwork satisfying the fixed constraint may also be selected in the last target population as target chromosome, the number of target chromosomes being at least one. The selection mode and the number of the target chromosomes are not limited, and the scheme for determining the last target population and further obtaining the target chromosomes is within the protection scope of the application as long as the method is optimized through multiple targets.

Illustratively, selecting a subnetwork satisfying the fixed constraint in the last target population is described taking the example that the fixed constraint is less than c as an example, and the chromosome with the delay less than c in the index information in the last target population is taken as a target chromosome. Taking a fixed constraint that the accuracy corresponding to the model performance is greater than d as an example, taking a chromosome with the accuracy greater than d in the index information in the last target population as a target chromosome. Thereby enabling multi-objective architectural searches.

In the embodiment of the application, a target chromosome is determined from the last target population, and a search result is returned, namely, a chromosome with optimal model performance or hardware requirement is returned, and the candidate model architecture of the corresponding sub-network of the chromosome and the corresponding weight parameter are returned, so that the target sub-network is obtained.

The architecture searching method in the embodiment of the application replaces the original evolution algorithm adopted by the OFA by the multi-objective evolution algorithm based on the greedy strategy, and outputs and searches to obtain the objective model (corresponding to the objective sub-network). Through a multi-objective evolutionary algorithm based on greedy strategy, multi-objective co-optimization is achieved. The single-target evolution algorithm of the OFA is improved, and a hardware-aware multi-target neural network architecture searching method based on a greedy strategy is provided, so that multi-target architecture searching is realized. Compared with an evolution algorithm adopted by the OFA, the scheme can realize more efficient compression of hardware constraint, find a sub-network with lower power consumption (or lower delay and smaller calculation amount) in the super-network, and improve the accuracy of search results.

According to the scheme provided by the embodiment of the application, the method comprises the following steps: selecting chromosomes meeting fixed constraints in a super network corresponding to the architecture search space, and determining an initial population; the super network comprises at least two sub networks, and each chromosome corresponds to one sub network; and determining a parent population based on the initial population and the initial target evolution direction, wherein initial chromosomes in the initial population meet fixed constraints, selecting chromosomes which are better in the initial target evolution direction from the initial population, and constructing the parent population. And the chromosomes included in the parent population are mutated and crossed to determine the target population, and the evolution process of the initial target evolution direction in the current round is completed. And continuously determining a next parent population based on the target population and the next target evolution direction, and mutating and crossing the next chromosome included in the next parent population to determine the next target population, thereby completing the next round of evolution process about the next target evolution direction. Until the iteration times are reached, through a plurality of evolutionary processes about different target evolutionary directions, multi-target evolutionary is realized, a last target population is obtained, a target chromosome is determined from the last target population, so that multi-target architecture search is realized, and a target sub-network corresponding to the target chromosome is obtained. Through the evolution direction of the targets in multiple rounds in the scheme, the searched target sub-network can meet the hardware requirement and achieve better model performance. The method and the device realize collaborative optimization of multiple hardware targets such as model accuracy, delay, power consumption and the like while meeting the search efficiency, thereby meeting the multi-dimensional requirement in an actual scene and improving the accuracy of search results.

In some embodiments, determining the target chromosome from the last target population in S104 of fig. 3 described above may also be accomplished in the following manner. Predicting the model performance of each candidate chromosome in the last target population to obtain the model performance of each candidate chromosome; and determining the target chromosome according to the model performance of each candidate chromosome.

In the embodiment of the application, the index information of the sub-network corresponding to each candidate chromosome in the last target population is predicted, and the model performance of the sub-network can be tested through a test sample set; each sub-network can also be input into a target predictor for prediction, and model performance parameters of the sub-network can be output. The chromosome with the best model performance (e.g., highest accuracy) in the last target population is then determined as the target chromosome.

In the embodiment of the application, through the multi-round evolutionary process about different target evolutionary directions, multi-target evolutionary is realized, the condition that the sub-network is optimal only on one index and other indexes are sacrificed infinitely is reduced, and then the chromosome with optimal model performance is selected from the final target population, and the accuracy of the search result is improved.

In some embodiments, the step of determining the parent population in S102 of FIG. 3 described above may further include S1021-S1023. As shown in fig. 4, fig. 4 is a flowchart illustrating optional steps of another searching method according to an embodiment of the present application.

S1021, predicting each index of each initial chromosome in the initial population to obtain at least two index information of each initial chromosome; the metrics include model performance and hardware requirements of the subnetwork.

In the embodiment of the application, each initial chromosome corresponds to one sub-network, the hardware requirement of the sub-network can be tested through hardware testing, the sub-network is operated on hardware equipment, the hardware requirement (such as delay, power consumption, calculated amount, storage space and the like) is tested, for example, the hardware index is obtained through testing on target hardware, and the efficiency of architecture searching is accelerated. The test sample set is input into the sub-network through the test sample set for testing the model performance of the sub-network, and the test accuracy is determined through comparing the label value of the test sample set with the prediction result output by the sub-network. Each sub-network can also be input into a target predictor for prediction, and hardware demand parameters and model performance parameters of the sub-network are output.

In some embodiments, S1021 in fig. 4 described above may also be implemented in the following manner. Predicting each index of each initial chromosome in the initial population based on the index prediction model to obtain at least two index information of each initial chromosome; the index prediction model is used for predicting each index of a model architecture of the sub-network, and comprises a performance index prediction model and a hardware index prediction model.

In the embodiment of the application, any index is used as a target to be optimized, the index prediction model can be also understood as a target predictor, the target predictor is trained in advance, and the target predictor is used for predicting the model performance and hardware requirements of the model architecture of the sub-network. The target predictor comprises a performance index predictor and a hardware index predictor, the predictor can carry out regression by a machine learning method such as a neural network and the like, a candidate model framework is input, and a predicted value of a candidate model related index can be output to replace an actual evaluation strategy.

S1022, sorting the initial chromosomes included in the initial population according to index information corresponding to the initial target evolution direction in at least two index information of each initial chromosome, and determining a sorting result; the initial target evolution direction is any one index or at least two indexes with correlation or conflict.

In the embodiment of the application, the evolution direction of the initial target is any index, and the initial chromosomes in the initial population are ranked according to the index information of the initial chromosomes under the index from big to small (or from small to big, or from high to low, or from low to high, or from good to bad, or from bad to good), so as to determine the ranking result.

In the embodiment of the application, for the technical problem of multi-objective optimization, the correlation or conflict among different objectives can be analyzed, and the algorithm is further optimized. Other multi-objective optimization algorithm schemes may also be employed. Such as particle swarm optimization, multi-objective optimization based on correlations, etc.

In the embodiment of the application, the initial target evolution direction is at least two indexes with correlation or conflict, the correlation value or conflict value of each initial chromosome under the at least two indexes with correlation or conflict is calculated, and the initial chromosomes in the initial population are ordered according to the magnitude of the correlation value or conflict value, so as to determine the ordering result.

S1023, selecting a first preset number of chromosomes according to a preset sequence and constructing a parent population according to the sequencing result.

In the embodiment of the application, taking k as an example as a first preset number, when the sequencing result is from big to small (or from high to low or from good to bad), selecting chromosomes of k in front, and constructing a parent population; when the ordering result is from small to large (or from low to high, or from inferior to superior), the chromosome of the latter k is selected, and a parent population is constructed. The parent population is the chromosome with optimal local index, and the number of the chromosomes in the parent population is smaller than that of the chromosomes in the initial population.

Illustratively, each chromosome corresponds to at least two indexes, the multi-round evolution process corresponds to multiple target evolution directions (i.e., multiple indexes), and multiple targets f exist in the hardware-aware architecture search ₁ 、f ₂ 、f ₃ … …, x represents any subnetwork in the population, f _i (x) Representing the value of x on the target i (corresponding to the index information). In each iteration, any one of the targets is selected as the optimization direction of the present round, e.g. f is selected _t For the target evolution direction at round t, at f by all candidate models (corresponding to chromosomes included in the parent population) _t As a fitness of the ranking. Based on the evolutionary algorithm of the multi-target fitness ordering method, a hardware perception multi-target neural network architecture searching method based on greedy strategies can be formed.

It should be noted that, in the evolution direction selection, greedy strategies can be improved, so as to promote the convergence of the algorithm. Other methods than greedy strategies may be tried for selection of the evolution direction, e.g. recursive algorithm (recursion algorithm), divide-and-conquer method (divide and conquer method), backtracking method (back track method), branch-and-bound method (branch and bound method), etc.

In some embodiments, the parent population comprises a first predetermined number of chromosomes; the step of determining the target population in S102 in fig. 3 may further include S1024-S1026. As shown in fig. 5, fig. 5 is a flowchart illustrating optional steps of a search method according to another embodiment of the present application.

S1024, based on the first preset number of chromosomes, performing mutation according to the initial mutation proportionality coefficient to determine a first number of mutated chromosomes; the first number is the product of the initial variation scaling factor and a second predetermined number.

In the embodiment of the application, a chromosome is randomly sampled in a first preset number of chromosomes, mutation is carried out on the chromosome once to obtain a mutated chromosome, and the mutated chromosome is sequentially circulated and randomly sampled for M times (the first number is M) to obtain a first number of mutated chromosomes.

In the embodiment of the application, the person skilled in the art also needs to set the initial variation proportion coefficient and the initial crossover proportion coefficient in a self-defined manner according to the actual requirement, the sum of the initial variation proportion coefficient and the initial crossover proportion coefficient is 1, the initial variation proportion coefficient and the initial crossover proportion coefficient can also be default coefficients, the default coefficients can be set by the person skilled in the art according to the actual requirement through a large amount of experimental data, and the default coefficients can be adjusted in a self-adaptive manner in the subsequent evolution process.

For example, taking k as a first preset number, N as a second preset number, M/N as an initial mutation scaling factor, and (N-M)/N as an initial crossover scaling factor, the first number is M/n×n=m, during the first round of evolution, randomly sampling k chromosomes according to the default scaling factor, each sampling a chromosome, performing a mutation, and sequentially cycling until M mutations are performed, thereby obtaining M mutated chromosomes (corresponding to the first number of mutated chromosomes).

The mutation refers to a mutation of a single parent candidate model chromosome to generate a new chromosome, and the new chromosome has a single parent model gene, so that part of genes are replaced with new genes generated by a generation algorithm (such as random generation), and compared with crossover, less information is reserved. In the case of random sampling, the same chromosome may be sampled, but the mutation directions are not uniform, so that the obtained mutation chromosomes are different.

S1025, crossing according to an initial crossing proportion coefficient based on the first preset number of chromosomes, and determining a second number of crossed chromosomes; the initial crossover scaling factor is the difference between 1 and the initial variation scaling factor.

In the embodiment of the application, two chromosomes are randomly sampled in a first preset number of chromosomes, and are crossed once to obtain a crossed chromosome, and the crossed chromosomes are sequentially circulated and randomly sampled (N-M) times (namely, the second number is N-M) to obtain a second number of crossed chromosomes.

Illustratively, taking k as a first preset number, N as a second preset number, and (N-M)/N as an initial crossover scaling factor, and (N-M)/n×n=n-M as a second number, during a first round of evolution, randomly sampling k chromosomes according to the default scaling factor, each sampling a chromosome, performing a crossover, and sequentially cycling until (N-M)/multiple crossover is performed, thereby obtaining (N-M) crossover chromosomes (corresponding to the second number of crossover chromosomes).

The crossover refers to crossing chromosomes of two parent candidate models to generate a new chromosome, and the new chromosome retains more information of genes of the two parent models. In random sampling, there are cases where two identical chromosomes are sampled, but genes sampled by crossing are not identical, and thus, the resulting crossing chromosomes are not identical.

S1026, determining a target population according to the first number of variant chromosomes and the second number of crossed chromosomes; the target population includes a second predetermined number of evolutionary chromosomes.

In the embodiment of the application, a first number of variant chromosomes is added into a target population, a second number of intersecting chromosomes is added into the target population, the target population is generated after the mutation and intersecting processes, and the target population comprises a second preset number of evolutionary chromosomes, so that the number of the chromosomes in the population is consistent (for example, N). The target population may also be referred to as elite population, and the evolving chromosomes may also be referred to as elite chromosomes.

In the embodiment of the application, the target population is generated by mutation and crossover based on the first preset number of chromosomes according to the initial mutation proportionality coefficient and the initial crossover proportionality coefficient, and a round of evolution process is completed. The evolutionary chromosomes in the target population can be used in the next round of evolutionary process (selecting the optimal k as the father, carrying out the variation and the crossover of the next round), and the method loops in turn until the iteration times are reached, so as to obtain the last target population, and realize the multi-target evolutionary.

In some embodiments, the search method further includes a cross-scaling factor adaptive adjustment strategy, including S201-S205, for the scaling factors of variation and cross scaling factor employed in each round. As shown in fig. 6, fig. 6 is a flowchart illustrating optional steps of a search method according to another embodiment of the present application.

S201, determining the mean value and the variance according to index information of each current chromosome in the current parent population in the current target evolution direction.

In the embodiment of the application, for a parent population (corresponding to a current parent population) in each round of evolution process, the parent population comprises a plurality of chromosomes (corresponding to current chromosomes), each chromosome corresponds to at least two index information, and the mean value and the variance are calculated according to the index information of each chromosome in the target evolution direction (any index) of the round of evolution process. That is, statistics are performed on the target values of the parent population, including mean and variance.

In the embodiment of the application, different cross variation ratios are adopted according to the mean value and the variance obtained by statistics, as shown in fig. 7, fig. 7 is an exemplary schematic diagram of a cross variation ratio adaptive adjustment strategy provided in the embodiment of the application; and comparing the average value with the average value in the parent statistical parameter of the previous round, and comparing the average value with a related target value of the maximum network or the minimum network in the super network to judge the quality of the statistical parameter. In the strategy section, the crossover and mutation ratios may be given a predetermined default value (i.e., an initial mutation ratio coefficient and an initial crossover ratio coefficient), where high and low in fig. 7 refer to adjustments relative to the default value.

S202, adjusting an initial variation proportion coefficient to be a first coefficient under the condition that the average value is smaller than a preset average value and the variance is smaller than a preset variance; the first coefficient is greater than the initial coefficient of variation.

In the embodiment of the present application, as shown in fig. 7, when the mean value is inferior and the variance is low, it is indicated that the target value of the parent model is inferior (i.e., the index information of the current chromosome included in the current parent population in the current target evolution direction is poor), and the current parent population has a certain popularity, so that the mutation ratio can be properly increased (i.e., the initial mutation ratio coefficient is adjusted to be the first coefficient), and more new excellent chromosomes can be generated as much as possible. Since the sum of the variation ratio coefficient and the crossover ratio coefficient is 1, the variation ratio is increased, that is, the crossover ratio is reduced.

S203, adjusting the initial cross proportion coefficient to be a second coefficient under the condition that the average value is larger than a preset average value and the variance is smaller than the preset variance; the second coefficient is greater than the initial crossover scaling coefficient.

In the embodiment of the present application, as shown in fig. 7, when the mean value is excellent and the variance is low, it is indicated that the target value of the parent model is excellent (i.e., the index information of the current chromosome included in the current parent population in the current target evolution direction is excellent), and the present application has a certain popularity, and the ratio of crossing can be properly increased (i.e., the initial crossing ratio coefficient is adjusted to the second coefficient), so that more excellent parent genes are retained. Since the sum of the variation ratio coefficient and the crossover ratio coefficient is 1, the variation ratio is increased, that is, the variation ratio is reduced.

S204, under the condition that the average value is smaller than a preset average value and the variance is larger than the preset variance, the initial cross proportionality coefficient and the initial variation proportionality coefficient are reserved.

In the embodiment of the application, as shown in fig. 7, when the mean value is inferior and the variance is high, the target value of the parent model is inferior, the universality is weak, a certain good chromosome can be possibly provided, and the default proportion of the cross variation can be reserved.

S205, when the average value is larger than a preset average value and the variance is larger than a preset variance, the initial cross proportionality coefficient and the initial variation proportionality coefficient are reserved.

In the embodiment of the application, as shown in fig. 7, when the mean value is excellent and the variance is high, the target value of the parent model is excellent, the universality is weak, and the default proportion of the cross variation can be reserved.

It should be noted that, in the above-mentioned fig. 6, S202-S205 are four parallel schemes, and there is no sequence of execution, and in the course of one round of evolution, one of the comparison coefficients (the variation scaling coefficient and the crossover scaling coefficient) may be selected for adjustment.

The cross-variation ratio self-adaptive adjustment strategy provided by the embodiment of the application can further optimize the evolutionary algorithm so as to improve the convergence of the evolutionary algorithm. In the process of carrying out cross mutation on the father, compared with the process of setting a fixed cross-mutation ratio, the scheme provides a cross-mutation adjustment strategy with self-adaptive cross-mutation ratio to realize optimization of an evolutionary algorithm so as to better realize optimization of a population.

In some embodiments, the preset mean and the preset variance in S202-S205 in fig. 6 may be implemented by determining the preset mean and the preset variance according to the index information of each previous chromosome in the previous parent population in the current target evolution direction.

In the embodiment of the application, as the current parent population is obtained after the better chromosome is selected from the target population obtained by intersecting and mutating the previous parent population, the distribution rule (for example, the mean value and the variance) of the chromosome in the previous parent population in the current target evolution direction adopted by the current round has the referential property for the current round of evolution process. For the parent population (corresponding to the previous parent population) in the previous round of evolution process, the parent population comprises a plurality of chromosomes (corresponding to the previous chromosome), each chromosome corresponds to at least two index information, and according to the index information of each chromosome in the current target evolution direction (because of the statistical comparison of the current round, the distribution rule of each chromosome in the previous round in the current target evolution direction needs to be calculated, the mean value and the variance are calculated, and the mean value and the variance are used as the preset mean value and the preset variance.

In some embodiments, the preset mean and the preset variance in S202-S205 in fig. 6 may be implemented by determining the preset mean according to the index information of the maximum network or the minimum network in the super-network in the current target evolution direction.

In the embodiment of the application, the super network comprises a plurality of sub networks, and the size of each sub network can be determined according to at least one of the following parameters: resolution, dimension, number of repetitions per block (block), convolution kernel size, etc. The maximum network has larger parameter quantity, better model performance, but poorer hardware demand parameters, such as larger delay, more power consumption, larger calculation amount and the like. The minimum network has smaller parameters, better hardware demand parameters, but poorer model performance, such as lower accuracy, etc.

In the embodiment of the application, the quality and the quality of the mean value can be compared with the mean value in the parent statistical parameter of the previous round, and also can be compared with the related target value of the maximum network or the minimum network in the super network, so as to judge the quality of the statistical parameter, and improve the diversity of the conditions according to the self-adaptive adjustment.

In some embodiments, the step of determining the next parent population in S103 in fig. 3 described above may also be implemented in the following manner. Continuously predicting each index of each evolutionary chromosome in the target population to obtain at least two index information of each evolutionary chromosome; sequencing the evolutionary chromosomes included in the target population according to index information corresponding to the next target evolution direction in at least two index information of each evolutionary chromosome, and determining a next sequencing result; the evolution direction of the next target is any index or at least two indexes with correlation or conflict; and selecting a first preset number of next chromosomes according to a preset sequence according to a next sequencing result, and constructing a next parent population.

In the embodiment of the present application, the at least two index information of the predictive evolutionary chromosome may be implemented in the following manner. And predicting each index of each evolutionary chromosome in the target population based on the index prediction model to obtain at least two index information of each evolutionary chromosome.

In the embodiment of the application, the selection of the evolution direction can be performed by means of random sampling, target-by-target cycle traversal and the like. That is, for the target evolution direction in each round of evolution process, one index may be randomly selected as the current target evolution direction, or the target evolution directions may be sequentially selected. The steps of index prediction, sequencing and selection of the evolving chromosomes in the target population to construct the next parent population are similar to the S1021-S1023 principle of fig. 4, and the implementation method and the achieved technical effects can be referred to the description process of index prediction, sequencing and selection of the initial chromosomes in the initial population to construct the parent population.

In the embodiment of the application, each evolutionary chromosome corresponds to one sub-network, the hardware requirement of the sub-network can be tested through hardware testing, the sub-network is operated on hardware equipment, and the hardware requirement (such as delay, power consumption, calculated amount, storage space and the like) of the sub-network is tested. The test sample set is input into the sub-network through the test sample set for testing the model performance of the sub-network, and the test accuracy is determined through comparing the label value of the test sample set with the prediction result output by the sub-network. Each sub-network can also be input into a target predictor for prediction, and hardware demand parameters and model performance parameters of the sub-network are output.

In the embodiment of the application, when the next target evolution direction is any index, according to the index information of the evolution chromosomes under the index, sequencing the evolution chromosomes in the target population according to the big to small (or the big to big, or the high to low, or the low to high, or the good to bad, or the bad to good), and determining the sequencing result. When the next target evolution direction is at least two indexes with correlation or conflict, calculating the correlation value or conflict value of each evolution chromosome under the at least two indexes with correlation or conflict, sequencing the evolution chromosomes in the target population according to the correlation value or conflict value, and determining the sequencing result. And selecting k chromosomes with better ranking results (corresponding to the first preset number) and constructing the next parent population. The next parent population is the chromosome with the optimal local index, and the number of the chromosomes in the next parent population is smaller than the number of the chromosomes in the target population.

In some embodiments, the next parent population comprises a first predetermined number of next chromosomes; the step of determining the next target population in S103 in fig. 3 described above may also be implemented in the following manner. Based on the first preset number of next chromosomes, performing mutation according to the next mutation scaling factor, and determining a third number of mutated chromosomes; the third number is the product of the next variation ratio coefficient and the second preset number; crossing according to a next crossing proportion coefficient based on the first preset number of next chromosomes, and determining a fourth number of crossed chromosomes; the next crossover scaling factor is the difference between 1 and the next variation scaling factor; determining a next target population based on the third number of variant chromosomes and the fourth number of cross chromosomes; the next target population includes a second predetermined number of next evolving chromosomes.

In the embodiment of the present application, the steps of crossing and mutating the next chromosome in the next parent population to generate the next target population are similar to the principle of S1024-S1026 in fig. 5, and the implementation method and the achieved technical effects can refer to the description process of crossing and mutating the chromosome in the parent population to generate the target population. Randomly sampling one chromosome in the next chromosome of the first preset number, mutating the chromosome once to obtain a mutated chromosome, and sequentially circulating and randomly sampling the chromosome for P times (the third number is P) to obtain a third number of mutated chromosomes. The third number may or may not be identical to the first number, and is identical to the first number when the cross scaling factor is identical.

For example, taking k as the first preset number, N as the second preset number, P/N as the next variation scaling factor, and (N-P)/N as the next crossover scaling factor, P/n×n=p as the third number, and (N-P)/n×n=n-P as the fourth number, during this round of evolution, k chromosomes are randomly sampled according to the current round scaling factor, each chromosome is sampled, one variation is performed, and the steps are sequentially repeated until P variations are performed, so as to obtain P variant chromosomes (corresponding to the third number of variant chromosomes). The k chromosomes are randomly sampled, one chromosome per sample, and the crossover is performed, and the cycle is sequentially performed until (N-P)/crossover is performed, so that (N-P) crossover chromosomes (corresponding to a fourth number of crossover chromosomes) are obtained.

In the embodiment of the application, the third number of variant chromosomes is added into the next target population, the fourth number of crossed chromosomes is added into the next target population, the next target population is generated after the mutation and crossing processes, the next target population still comprises the second preset number of evolutionary chromosomes, the consistent number (for example, N) of chromosomes in the population is ensured, and the multi-target evolution is realized. Wherein the next target population may also be referred to as the next elite population, and the next evolving chromosome may also be referred to as the next elite chromosome.

In some embodiments, the search method further comprises a training process of the super network, generating an initial super network based on preset parameters of the architecture search space; the initial super network comprises at least two initial sub networks; training at least two initial sub-networks to determine a super-network.

In the embodiment of the present application, those skilled in the art also need to define the architecture search space of the super network, that is, set preset parameters of the architecture search space of the super network, so as to generate an initial super network according to the preset parameters. Defined architectural search spaces include, but are not limited to: resolution search space, convolution kernel search space per layer, width (number of convolution kernel channels) search space, depth (number of layer stacks) search space. According to preset parameters, generating a plurality of sub-networks with different levels (also known as layer stacking times), different convolution kernels, different channel numbers and different image resolutions, thereby obtaining an initial super-network. The initial super network comprises at least two initial sub networks; and training the initial subnetworks to obtain trained subnetworks.

In the embodiment of the application, the super network trains: the super network can be trained by sandwich rule and progressive shrinkage. The training method of the sandwich rule is as follows: in each training phase, 1 maximum sub-network is sampled, 1 minimum sub-network is sampled again at random, and in each training iteration, the maximum sub-network with the largest channel width, the minimum sub-network with the smallest channel width and the two random sub-networks are sampled respectively, and gradient is accumulated. The training method of progressive shrinkage is as follows: the method comprises the steps of training a network with the maximum convolution kernel size (kernel size), depth (depth) and width (width) to support a subnet, namely adding a small-sized subnet into a sampling space, and adopting a knowledge distillation method to enable the maximum super-network to guide the learning of the subnet. The method for training the super network is not particularly limited.

In some embodiments, the searching method further includes an application process of the searched target sub-network, and any one of the following processing is performed on the data to be processed by using the target sub-network: generating, classifying, clustering, predicting and identifying to obtain a data processing result.

In the embodiment of the application, the super network is generated and trained according to the actual demands of users, so after the target sub network is searched by using the greedy strategy-based hardware-aware multi-target neural network architecture searching method provided by the scheme, the target sub network can be used for carrying out data processing on the data to be processed, wherein the data processing comprises but is not limited to: generating, classifying, clustering, predicting and identifying. For example, if the requirement of the user in building the super network is data classification, the target sub-network may be used to classify the data to be processed, so as to obtain a classification result. Because the target sub-network not only meets the hardware requirement, but also has better model performance, the data processing efficiency is improved and the accuracy of the data processing result is improved when the target sub-network is utilized for data processing.

In the following, an exemplary application of the embodiment of the present application in a practical application scenario will be described.

In the embodiment of the present application, as shown in fig. 8, fig. 8 is a flowchart of optional steps of a greedy strategy-based multi-objective evolutionary algorithm provided in the embodiment of the present application, including S301-S309.

S301, generating an initial population.

In the embodiment of the application, sampling is carried out randomly in the architecture search space, a plurality of chromosomes (namely the architecture configuration of the subnetwork) are continuously generated, target values under all targets of the chromosomes are calculated, and the chromosomes meeting fixed constraints are added into the initial population until the size of the initial population is met. If there is a fixed constraint under a certain target, a chromosome satisfying the constraint is selected.

S302, inputting the sub-networks included in the population into a target index predictor to obtain various target indexes.

S303, judging whether the cycle number reaches the iteration number E.

If yes, S309 is executed, and if no, S304 is executed.

S304, selecting and obtaining a current optimization target t in the evolution direction based on the greedy strategy.

In the embodiment of the application, the current target evolution direction is a t index, and the evolution direction selection based on greedy strategies: sampling the evolution direction, selecting the selected evolution direction based on greedy strategy according to different chromosomes, and taking the selected evolution direction, namely the selected target value, as the fitness score of the chromosome. Chromosomes with highest fitness scores are saved and added to elite populations (corresponding to the target population).

The direction of evolution based on greedy strategy is selected as follows: in a multi-objective evolutionary algorithm based on greedy strategy, the direction of evolution needs to be selected in each round. The greedy strategy is that in each round of iteration, a single target is selected as the optimization direction of the round, and the optimization effect of multiple targets is realized by continuously searching local optima under the single target. Assume that there are multiple targets f in the search for the hardware aware architecture ₁ 、f ₂ 、f ₃ … …, x represents any of the populationsMeaning model, f _i (x) Representing the value of x on target i. In each iteration, a single target is chosen as the optimization direction for the round, e.g., f _t For the evolution direction at round t, at f by all candidate models _t As the fitness of the rank. The selection of the evolution direction may be performed by means of random sampling, target-by-target loop traversal, etc.

S305, establishing a new population.

The new population created is an empty set for storing the crossed chromosomes and mutated chromosomes.

S306, sorting the populations according to the t index, and taking the first K populations as parent populations.

A parent population may also be referred to as a set of parents.

S307, randomly sampling in the parent population, executing M times of mutation, and adding the mutation to the new population.

S308, randomly sampling in the parent population, intersecting N-M times, and adding the intersecting points into the new population.

In the embodiment of the present application, after executing S308, S302 is executed, and S302-S308 are executed in a loop until the number of loops reaches the number of iterations E, and S309 is executed.

According to the cross-variation ratio adaptive adjustment strategy (corresponding to fig. 6 and 7) proposed by the scheme, elites are selected for variation and crossover to obtain a new population. The selection-mutation-crossover process is repeated until a set iteration round. If there is a fixed constraint under a certain target, both the mutation process and the crossover process select chromosomes that meet the constraint.

S309, returning to the best candidate network in the population.

In the embodiment of the application, the optimal chromosome (the optimal candidate model architecture) and the weight parameters corresponding to the optimal chromosome are returned.

In summary, the technical scheme of the scheme is as follows: the method comprises the steps of selecting an evolution direction based on a greedy strategy, combining a cross-variation proportion self-adaptive adjustment strategy, obtaining a greedy strategy-based multi-target evolution algorithm, and then obtaining a greedy strategy-based hardware perception multi-target neural network architecture searching method.

The scheme provides a hardware perception multi-target neural network architecture searching method based on a greedy strategy, which can search a better performance model and simultaneously consider more hardware target constraints to optimize. The scheme improves the evolution algorithm in the OFA, provides a multi-target architecture searching method based on a greedy strategy, and achieves multi-target collaborative optimization of model accuracy, delay, power consumption and the like while meeting searching efficiency, so that optimal user experience is achieved in an actual scene.

In the embodiment of the application, if a hardware perception multi-target neural network architecture searching method based on a greedy strategy exists, a user is required to set judgment logic of target value quality; or, information such as "greedy strategy architecture search", "greedy multi-object NAS", "hill climbing method multi-object architecture search" and the like is described in a Product instruction manual and/or a Product description (PR), which indicates that the searching method provided by the embodiment of the present application is applied to this scheme.

In order to implement the searching method according to the embodiment of the present application, the embodiment of the present application further provides a searching device, as shown in fig. 9, fig. 9 is an optional schematic structural diagram of the searching device according to the embodiment of the present application, where the searching device 90 includes: the population determining module 901 is configured to select chromosomes that satisfy a fixed constraint in a super network corresponding to the architecture search space, and determine an initial population; the super network comprises at least two sub networks, and each chromosome corresponds to one sub network; determining a parent population based on the initial population and an initial target evolution direction; an evolution module 902, configured to mutate and cross chromosomes included in the parent population to determine a target population; the population determining module 901 is further configured to determine a next parent population based on the target population and a next target evolution direction; the evolution module 902 is further configured to mutate and cross a next chromosome included in the next parent population to determine a next target population, until the iteration number is reached, and implement multi-target evolution, so as to obtain a final target population; the target sub-network determining module 903 is configured to determine a target chromosome from the last target population, so as to implement multi-target architecture search, and obtain a target sub-network corresponding to the target chromosome.

In some embodiments, the population determining module 901 is further configured to predict each index of each initial chromosome in the initial population, so as to obtain at least two index information of each initial chromosome; the indexes comprise model performance and hardware requirements of the sub-network; sorting the initial chromosomes included in the initial population according to index information corresponding to the initial target evolution direction in at least two index information of each initial chromosome, and determining a sorting result; the initial target evolution direction is any index or at least two indexes with correlation or conflict; and selecting a first preset number of chromosomes according to a preset sequence according to the sequencing result, and constructing the parent population.

In some embodiments, the parent population comprises a first predetermined number of chromosomes;

the evolution module 902 is further configured to determine a first number of variant chromosomes by performing mutation according to an initial mutation scaling factor based on the first preset number of chromosomes; the first number is the product of the initial variation ratio coefficient and a second preset number; crossing according to an initial crossing proportion coefficient based on the first preset number of chromosomes, and determining a second number of crossed chromosomes; the initial crossover proportionality coefficient is the difference between 1 and the initial variation proportionality coefficient; determining the target population based on the first number of variant chromosomes and the second number of cross chromosomes; the target population includes the second predetermined number of evolutionary chromosomes.

In some embodiments, the search apparatus 90 further comprises an adjustment module 904;

the adjustment module 904 is configured to determine a mean and a variance according to index information of each current chromosome in the current parent population in a current target evolution direction; under the condition that the mean value is smaller than a preset mean value and the variance is smaller than a preset variance, the initial variation proportion coefficient is adjusted to be a first coefficient; the first coefficient is greater than the initial coefficient of variation; when the mean value is larger than the preset mean value and the variance is smaller than the preset variance, the initial cross proportion coefficient is adjusted to be a second coefficient; the second coefficient is greater than the initial cross scaling coefficient; when the mean value is smaller than the preset mean value and the variance is larger than the preset variance, the initial cross proportionality coefficient and the initial variation proportionality coefficient are reserved; and under the condition that the average value is larger than the preset average value and the variance is larger than the preset variance, the initial cross proportion coefficient and the initial variation proportion coefficient are reserved.

In some embodiments, the adjusting module 904 is further configured to determine the preset mean and the preset variance according to index information of each previous chromosome in the previous parent population in the current target evolution direction.

In some embodiments, the adjusting module 904 is further configured to determine the preset average value according to index information of a maximum network or a minimum network in the super network in the current target evolution direction.

In some embodiments, the population determining module 901 is further configured to continuously predict each index of each evolutionary chromosome in the target population, so as to obtain at least two index information of each evolutionary chromosome; sequencing the evolutionary chromosomes included in the target population according to index information corresponding to the next target evolution direction in at least two index information of each evolutionary chromosome, and determining a next sequencing result; the next target evolution direction is any one index or at least two indexes with correlation or conflict; and selecting a first preset number of next chromosomes according to a preset sequence according to the next sequencing result, and constructing the next parent population.

In some embodiments, the next parent population comprises a first predetermined number of next chromosomes;

the evolution module 902 is further configured to determine a third number of variant chromosomes by performing variation according to a next variation scaling factor based on the first preset number of next chromosomes; the third number is the product of the next coefficient of variation and a second preset number; crossing according to a next crossing proportion coefficient based on the first preset number of next chromosomes, and determining a fourth number of crossed chromosomes; the next crossover scaling factor is the difference between 1 and the next variation scaling factor; determining the next target population based on the third number of variant chromosomes and the fourth number of cross chromosomes; the next target population includes the second predetermined number of next evolving chromosomes.

In some embodiments, the population determining module 901 is further configured to predict each index of each initial chromosome in the initial population based on an index prediction model, so as to obtain at least two index information of each initial chromosome; the index prediction model is used for predicting each index of a model architecture of the sub-network, and comprises a performance index prediction model and a hardware index prediction model.

In some embodiments, search device 90 also includes training module 905;

training module 905, configured to generate an initial super-network based on preset parameters of the architecture search space; the initial super network comprises at least two initial sub networks; training the at least two initial sub-networks to determine the super-network.

In some embodiments, the target subnetwork determining module 903 is further configured to predict a model performance of each candidate chromosome in the last target population, to obtain a model performance of each candidate chromosome; and determining the target chromosome according to the model performance of each candidate chromosome.

In some embodiments, the search apparatus 90 further comprises a processing module 906;

A processing module 906, configured to process, with the target subnetwork, data to be processed by any one of the following: generating, classifying, clustering, predicting and identifying to obtain a data processing result.

It should be noted that, in the searching device provided in the foregoing embodiment, only the division of each program module is used for illustration, and in practical application, the process allocation may be performed by different program modules according to needs, that is, the internal structure of the device is divided into different program modules, so as to complete all or part of the processes described above. In addition, the search device and the search method provided in the foregoing embodiments belong to the same concept, and specific implementation processes and beneficial effects thereof are detailed in the method embodiments, which are not described herein again. For technical details not disclosed in the present apparatus embodiment, please refer to the description of the method embodiment of the present application for understanding.

In the embodiment of the present application, fig. 10 is a schematic diagram illustrating the composition structure of a search device according to the embodiment of the present application, and as shown in fig. 10, the search device 100 according to the embodiment of the present application includes a processor 1001, a memory 1002 storing an executable computer program, and the processor 1001 is configured to implement the search method according to the embodiment of the present application when executing the executable computer program stored in the memory 1002.

In some embodiments, the search device 100 may further include a communication interface 1003, and a bus 1004 for connecting the processor 1001, the memory 1002, and the communication interface 1003.

In an embodiment of the present application, the processor 1001 may be at least one of an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a digital signal processor (Digital Signal Processor, DSP), a digital signal processing device (Digital Signal Processing Device, DSPD), a programmable logic device (ProgRAMmable Logic Device, PLD), a field programmable gate array (Field ProgRAMmable Gate Array, FPGA), a central processing unit (Central Processing Unit, CPU), a controller, a microcontroller, and a microprocessor. It will be appreciated that the electronics for implementing the above-described processor functions may be other for different devices, and embodiments of the present application are not particularly limited.

In the embodiment of the present application, the bus 1004 is used to connect the communication interface 1003, the processor 1001, and the memory 1002, so as to implement mutual communication between these devices.

The memory 1002 is used to store executable computer programs and data, including computer operating instructions, the memory 1002 may comprise high speed RAM memory, and may also include non-volatile memory, such as at least two disk memories. In practical applications, the Memory 1002 may be a volatile Memory (RAM), such as a Random-Access Memory (RAM); or a nonvolatile Memory (non-volatile Memory), such as a Read-Only Memory (ROM), a flash Memory (flash Memory), a Hard Disk (HDD) or a Solid State Drive (SSD); or a combination of the above-mentioned types of memory and provide executable computer programs and data to the processor 1001.

In addition, each functional module in the present embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional modules.

The integrated units, if implemented in the form of software functional modules, may be stored in a computer-readable storage medium, if not sold or used as separate products, and based on this understanding, the technical solution of the present embodiment may be embodied essentially or partly in the form of a software product, or all or part of the technical solution may be embodied in a storage medium, which includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) or processor (processor) to perform all or part of the steps of the method of the present embodiment. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

An embodiment of the present application provides a computer-readable storage medium storing a computer program for implementing the search method according to any one of the embodiments above when executed by a processor.

For example, the program instruction corresponding to one search method in the present embodiment may be stored on a storage medium such as an optical disc, a hard disc, or a usb disk, and when the program instruction corresponding to one search method in the storage medium is read or executed by an electronic device, the search method described in any one of the above embodiments may be implemented.

It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, magnetic disk storage, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of implementations of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block and/or flow of the flowchart illustrations and/or block diagrams, and combinations of blocks and/or flow diagrams in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart block or blocks and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart block or blocks and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks and/or block diagram block or blocks.

The foregoing description is only of the preferred embodiments of the present application, and is not intended to limit the scope of the present application.

Claims

1. A method of searching, the method comprising:

selecting chromosomes meeting fixed constraints in a super network corresponding to the architecture search space, and determining an initial population; the super network comprises at least two sub networks, and each chromosome corresponds to one sub network;

Determining a parent population based on the initial population and the initial target evolution direction, and mutating and crossing chromosomes included in the parent population to determine a target population;

continuing to determine a next parent population based on the target population and the next target evolution direction, and mutating and crossing a next chromosome included in the next parent population to determine the next target population until the iteration times are reached, so as to realize multi-target evolution and obtain a last target population;

and determining a target chromosome from the last target population to realize multi-target architecture search, and obtaining a target sub-network corresponding to the target chromosome.

2. The method of claim 1, wherein the determining a parent population based on the initial population and an initial target evolution direction comprises:

predicting each index of each initial chromosome in the initial population to obtain at least two index information of each initial chromosome; the indexes comprise model performance and hardware requirements of the sub-network;

sorting the initial chromosomes included in the initial population according to index information corresponding to the initial target evolution direction in at least two index information of each initial chromosome, and determining a sorting result; the initial target evolution direction is any index or at least two indexes with correlation or conflict;

And selecting a first preset number of chromosomes according to a preset sequence according to the sequencing result, and constructing the parent population.

3. The method of claim 1, wherein the parent population comprises a first predetermined number of chromosomes;

the mutating and crossing chromosomes included in the parent population to determine a target population comprises:

based on the first preset number of chromosomes, performing mutation according to an initial mutation scaling factor, and determining a first number of mutated chromosomes; the first number is the product of the initial variation ratio coefficient and a second preset number;

crossing according to an initial crossing proportion coefficient based on the first preset number of chromosomes, and determining a second number of crossed chromosomes; the initial crossover proportionality coefficient is the difference between 1 and the initial variation proportionality coefficient;

determining the target population based on the first number of variant chromosomes and the second number of cross chromosomes; the target population includes the second predetermined number of evolutionary chromosomes.

4. A method according to any one of claims 1-3, wherein the method further comprises:

determining the mean value and the variance according to the index information of each current chromosome in the current parent population in the current target evolution direction;

Under the condition that the mean value is smaller than a preset mean value and the variance is smaller than a preset variance, the initial variation proportion coefficient is adjusted to be a first coefficient; the first coefficient is greater than the initial coefficient of variation;

when the mean value is larger than the preset mean value and the variance is smaller than the preset variance, the initial cross proportion coefficient is adjusted to be a second coefficient; the second coefficient is greater than the initial cross scaling coefficient;

when the mean value is smaller than the preset mean value and the variance is larger than the preset variance, the initial cross proportionality coefficient and the initial variation proportionality coefficient are reserved;

and under the condition that the average value is larger than the preset average value and the variance is larger than the preset variance, the initial cross proportion coefficient and the initial variation proportion coefficient are reserved.

5. The method according to claim 4, wherein the method further comprises:

and determining the preset mean value and the preset variance according to index information of each previous chromosome in the previous parent population in the current target evolution direction.

6. The method according to claim 4, wherein the method further comprises:

And determining the preset average value according to index information of the maximum network or the minimum network in the super network in the current target evolution direction.

7. A method according to any one of claims 1-3, wherein said continuing to determine a next parent population based on said target population and a next target direction of evolution comprises:

continuously predicting each index of each evolutionary chromosome in the target population to obtain at least two index information of each evolutionary chromosome;

sequencing the evolutionary chromosomes included in the target population according to index information corresponding to the next target evolution direction in at least two index information of each evolutionary chromosome, and determining a next sequencing result; the next target evolution direction is any one index or at least two indexes with correlation or conflict;

and selecting a first preset number of next chromosomes according to a preset sequence according to the next sequencing result, and constructing the next parent population.

8. A method according to any one of claims 1-3, wherein the next parent population comprises a first preset number of next chromosomes;

The mutating and crossing the next chromosome included in the next parent population to determine a next target population comprises:

based on the first preset number of next chromosomes, performing mutation according to a next mutation scaling factor, and determining a third number of mutated chromosomes; the third number is the product of the next coefficient of variation and a second preset number;

crossing according to a next crossing proportion coefficient based on the first preset number of next chromosomes, and determining a fourth number of crossed chromosomes; the next crossover scaling factor is the difference between 1 and the next variation scaling factor;

determining the next target population based on the third number of variant chromosomes and the fourth number of cross chromosomes; the next target population includes the second predetermined number of next evolving chromosomes.

9. The method of claim 2, wherein predicting each indicator of each initial chromosome in the initial population to obtain at least two indicator information for each initial chromosome comprises:

predicting each index of each initial chromosome in the initial population based on an index prediction model to obtain at least two index information of each initial chromosome; the index prediction model is used for predicting each index of a model architecture of the sub-network, and comprises a performance index prediction model and a hardware index prediction model.

10. A method according to any one of claims 1-3, wherein the method further comprises:

generating an initial super network based on preset parameters of the architecture search space; the initial super network comprises at least two initial sub networks;

training the at least two initial sub-networks to determine the super-network.

11. A method according to any one of claims 1-3, wherein said determining a target chromosome from said last target population comprises:

predicting the model performance of each candidate chromosome in the last target population to obtain the model performance of each candidate chromosome;

and determining the target chromosome according to the model performance of each candidate chromosome.

12. A method according to any one of claims 1-3, wherein the method further comprises:

and processing the data to be processed by using the target sub-network, wherein the data to be processed comprises any one of the following steps: generating, classifying, clustering, predicting and identifying to obtain a data processing result.

13. A search apparatus, the apparatus comprising:

the population determining module is used for selecting chromosomes meeting fixed constraints in the super network corresponding to the architecture search space to determine an initial population; the super network comprises at least two sub networks, and each chromosome corresponds to one sub network; determining a parent population based on the initial population and an initial target evolution direction;

The evolution module is used for mutating and crossing chromosomes included in the parent population to determine a target population;

the population determining module is further used for continuously determining the next parent population based on the target population and the next target evolution direction;

the evolution module is further used for carrying out mutation and crossover on a next chromosome included in the next parent population to determine a next target population until iteration times are reached, multi-target evolution is realized, and a last target population is obtained;

and the target sub-network determining module is used for determining a target chromosome from the last target population so as to realize multi-target architecture search and obtain a target sub-network corresponding to the target chromosome.

14. A search apparatus, the apparatus comprising:

a memory for storing an executable computer program;

a processor for implementing the method of any of claims 1-12 when executing an executable computer program stored in said memory.

15. A computer readable storage medium, characterized in that a computer program is stored for implementing the method of any one of claims 1-12 when executed by a processor.