CN110598629B - Super-network search space construction method and device and electronic equipment - Google Patents
Super-network search space construction method and device and electronic equipment Download PDFInfo
- Publication number
- CN110598629B CN110598629B CN201910861037.6A CN201910861037A CN110598629B CN 110598629 B CN110598629 B CN 110598629B CN 201910861037 A CN201910861037 A CN 201910861037A CN 110598629 B CN110598629 B CN 110598629B
- Authority
- CN
- China
- Prior art keywords
- layer
- feature
- layers
- search space
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000010276 construction Methods 0.000 title claims description 29
- 238000013528 artificial neural network Methods 0.000 claims abstract description 70
- 238000010586 diagram Methods 0.000 claims abstract description 48
- 238000000034 method Methods 0.000 claims abstract description 21
- 230000015654 memory Effects 0.000 claims description 17
- 238000013527 convolutional neural network Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 210000004027 cell Anatomy 0.000 description 6
- 238000004891 communication Methods 0.000 description 4
- 238000004590 computer program Methods 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 239000004973 liquid crystal related substance Substances 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 210000002569 neuron Anatomy 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001151 other effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000004549 pulsed laser deposition Methods 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
- G06F18/24137—Distances to cluster centroïds
- G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
- G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
- G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
- G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biodiversity & Conservation Biology (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- General Engineering & Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a method and a device for constructing a search space of a super network and electronic equipment, and relates to the field of neural network search. The specific implementation scheme is as follows: selecting two layers of characteristic diagrams from the multiple layers of characteristic diagrams of the deep neural network; adding a connecting layer between the two selected layers of feature images, wherein the connecting layer is used for merging the two selected layers of feature images; and constructing a search space of the super network by using the connection layer. The addition of the connection layer changes the number of layers of the feature layer, and thus the depth of the deep neural network. And constructing a new search space by using the connection layer, and further obtaining the deep neural network with any depth under the constraint conditions of any model size, model speed and model precision.
Description
Technical Field
The present application relates to the field of computer vision, and in particular, to the field of neural network searching.
Background
Deep learning technology has achieved tremendous success in many directions, and NAS technology (Neural Architecture Search, neural network architecture search) has become a research hotspot in recent years. The NAS replaces complex manual operation by an algorithm, and an optimal neural network architecture is automatically searched in a huge search space. The step of conducting an architecture search of the neural network includes: first, a search space is defined and a search space is determined. The search strategy is then determined based on the optimization algorithm employed, such as with reinforcement learning, evolutionary algorithm, bayesian optimization, and the like. And finally, searching to obtain the speed of the model structure and the performance of the model.
The supernetwork based search space is typically a stack based on cell structures (cell blocks). The cell structure includes two basic cell structures, one is a general cell structure and one is a downsampling cell structure. The stacking times of the two basic unit structures are designed in advance according to manual priori information, so that the model depth of the super network is fixed. The model depth of the super network refers to the total number of layers of the deep neural network. For example, the main difference between the residual networks such as the residual networks 18, 34, 50, 101, 152 is that the model depths are different, and the performance differences between the networks with different depths are very large.
At present, since the model depth in the super network is always fixed, under the constraint condition of the model size and the speed, the fixed model depth cannot realize automatic search of the model with any depth. Further, the speed and accuracy of searching the super-network search space to obtain the model are affected.
Disclosure of Invention
The embodiment of the application provides a method and a device for constructing a search space of a super network and electronic equipment, so as to solve at least one technical problem.
In a first aspect, a method for constructing a search space of a super network is provided, including:
selecting two layers of characteristic diagrams from the multiple layers of characteristic diagrams of the deep neural network;
adding a connecting layer between the two selected layers of feature images, wherein the connecting layer is used for merging the two selected layers of feature images;
and constructing a search space of the super network by using the connection layer.
In this embodiment, the addition of the connection layer changes the number of layers of the feature layer, and thus the depth of the deep neural network. And a new search space is constructed by utilizing the connection layer, so that a deep neural network with any depth can be obtained under the constraint conditions of any model size, model speed and model precision, and meanwhile, the parameter sharing of the super network is not influenced.
In one embodiment, selecting a two-layer feature map includes:
inputting the first layer of characteristic diagram into a plurality of layers of operation layers, outputting an Mth layer of characteristic diagram, wherein M is more than or equal to 2;
and selecting any two adjacent layers of feature maps from the first layer of feature maps to the M layer of feature maps.
In one embodiment, a connection layer is added between the two selected feature maps, comprising:
and respectively copying the characteristic information of the two selected adjacent characteristic graphs, and placing the copied characteristic information in the connecting layer.
In this embodiment, the feature information of the two-layer feature map connected by the connection layer is copied, and the two layers can be regarded as one layer. Thereby reducing the number of hidden layers and the depth of the neural network.
In one embodiment, selecting a two-layer feature map includes:
inputting the first layer of feature map to a multi-layer operation layer, and outputting an M-th layer of feature map;
any two layers of non-adjacent feature maps are selected from the first layer of feature maps to the M layer of feature maps.
In one embodiment, a connection layer is added between the two selected feature maps, comprising:
performing recursion operation on the feature information of the selected two layers of non-adjacent feature graphs and the feature information of the feature graph between the two layers of non-adjacent feature graphs to obtain the recursion feature information;
and placing the recursive characteristic information in a connection layer.
In the present embodiment, the feature information of all feature maps between two feature maps connected by the connection layer is recursively calculated. Thereby reducing the number of hidden layers and the depth of the neural network.
In one embodiment, constructing a search space of a super network using a connection layer includes:
obtaining the maximum network layer number of the deep neural network;
obtaining the depth of the deep neural network according to the maximum network layer number and the connection layer;
and constructing a search space of the super network by using the depth of the deep neural network.
In this embodiment, the depth of the deep neural network satisfying the constraint condition of search is calculated from the connection layer and the initial depth M, and the search space of the super network is constructed using the calculated depth of the deep neural network. In the search space, the optimal deep neural network is searched, and the search efficiency is improved.
In a second aspect, there is provided a search space construction apparatus of a super network, including:
the feature map selecting module is used for selecting two layers of feature maps from the multi-layer feature maps of the deep neural network;
the connecting layer adding module is used for adding a connecting layer between the two selected layers of feature graphs, and the connecting layer is used for;
and the search space construction module is used for constructing the search space of the super network by utilizing the connection layer.
In one embodiment, the feature map selection module includes:
the feature map generating unit is used for inputting the first layer of feature map into the multi-layer operation layer and outputting the M layer of feature map, wherein M is more than or equal to 2;
the first feature map selecting unit is used for selecting any two adjacent feature maps from the first layer feature map to the M layer feature map.
In one embodiment, the connection layer adding module includes:
and the first connecting layer adding unit is used for respectively copying the characteristic information of the two selected adjacent characteristic graphs and placing the copied characteristic information in the connecting layer.
In one embodiment, the feature map selection module includes:
and the second characteristic diagram selecting unit is used for selecting any two layers of non-adjacent characteristic diagrams from the first layer of characteristic diagrams to the M layer of characteristic diagrams.
In one embodiment, the connection layer adding module includes:
and the second connection layer adding unit is used for carrying out recursive operation on the characteristic information of the selected two layers of non-adjacent characteristic diagrams and the characteristic information of the characteristic diagram between the two layers of non-adjacent characteristic diagrams to obtain the characteristic information after recursion, and placing the characteristic information after recursion in the connection layer.
In one embodiment, the search space construction module includes:
the maximum network layer number acquisition unit is used for acquiring the maximum network layer number of the deep neural network;
the depth calculation unit is used for obtaining the depth of the depth neural network according to the maximum network layer number and the operation layer number;
and the search space construction unit is used for constructing the search space of the super network by utilizing the depth of the deep neural network.
One embodiment of the above application has the following advantages or benefits: because the connecting layer is added between the two different layers of feature images, the connecting layer is used for merging the two selected layers of feature images, the addition of the connecting layer changes the layer number of the feature layers, and the technical means of depth of the deep neural network is changed, the technical problem that the depth of any deep neural network cannot be searched in the traditional super network search space based on unit structure stacking is solved, the search efficiency is improved, and the technical effects of meeting various search constraint conditions are achieved.
Other effects of the above alternative will be described below in connection with specific embodiments.
Drawings
The drawings are for better understanding of the present solution and do not constitute a limitation of the present application. Wherein:
FIG. 1 is a schematic flow diagram of a method for constructing a search space of a super network according to an embodiment of the present application;
FIG. 2 is a search space construction scenario diagram of a super network in which embodiments of the present application may be implemented.
FIG. 3 is a flowchart of another method for constructing a search space of a super network according to an embodiment of the present application;
FIG. 4 is a flowchart of another method for constructing a search space of a super network according to an embodiment of the present application;
FIG. 5 is a flowchart of another method for constructing a search space of a super network according to an embodiment of the present application;
FIG. 6 is a block diagram of a search space construction device of a super network according to an embodiment of the present application;
FIG. 7 is a block diagram of another super-network search space construction device according to an embodiment of the present application;
FIG. 8 is a block diagram of another super-network search space construction apparatus according to an embodiment of the present application;
FIG. 9 is a block diagram of another search space construction apparatus of a super network according to an embodiment of the present application;
fig. 10 is a block diagram of an electronic device for implementing a search space construction method of a super network according to an embodiment of the present application.
Detailed Description
Exemplary embodiments of the present application are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Example one …
In a specific embodiment, as shown in fig. 1, a method for constructing a search space of a super network is provided, including:
step S10: selecting two layers of characteristic diagrams from the multiple layers of characteristic diagrams of the deep neural network;
step S20: adding a connecting layer between the two selected layers of feature images, wherein the connecting layer is used for merging any two layers of feature images;
step S30: and constructing a search space of the super network by using the connection layer.
In one example, a deep neural network may include an input layer, a hidden layer, and an output layer, with a plurality of neurons in each layer, with neurons interconnecting each layer to each layer. In computer vision, the hidden layer is mainly used for extracting features, so the hidden layer can also be called a feature layer. As shown in fig. 2, in the convolutional neural network, data exists in three dimensions at each convolutional layer. Convolutional neural networks can be seen as a stack of two-dimensional pictures, each of which is called a feature map. Each feature layer includes one or more feature maps. For example, in the input layer, in the case of a gray picture, there are only 1 feature map, and in the case of a color picture, 3 feature maps (red, green, blue) are typically included in the input layer. Between feature layers there are multiple convolution kernels (kernel), which may be represented by multiple OPs, including op_1 … … op_n, for example, as shown in fig. 2. Each feature map in the upper layer is convolved with each convolution kernel to produce a feature map of the lower layer. With many convolution kernels, a convolution produces many feature maps. For example, op_1 represents an optional 1 st operation, such as op_1 representing a convolution with a convolution kernel size of 3*3, convolution channel number inputs and outputs of 256 and 512, respectively, and a convolution step size of 1. Op_2 represents an optional 2 nd operation, op_2 represents a convolution with a convolution kernel size of 5*5, convolution channel number inputs and outputs of 256 and 512, respectively, and a convolution step size of 1. The arrow indicates that the feature map of the M-1 layer is subjected to certain operation to obtain the feature map of the M layer. The layer 1 feature map can generate a second layer feature map through any operation of the OP_ … … OP_N. And similarly, outputting the M layer characteristic diagram.
The super network comprises a plurality of deep neural networks, from the 1 st layer characteristic diagram to the M th layer characteristic diagram, through a plurality of layers of OP, as each layer of OP has a plurality of different options, the deep neural networks are formed through the arrangement and combination of the layers of OP. Each OP corresponds to a parameter of a deep neural network. The depth in each deep neural network refers to the number of hidden layers, i.e., the depth is M.
As shown in fig. 2, two layers may be selected among the M feature layers, that is, two layers of feature maps may be selected. A connection layer (skip_layer) is added in the two-layer special graph, the connection layer realizes an operation realization of F (x) =x, and through the operation, cross-layer direct connection can be realized, so that the depth of the network is changed. For example, if the depth M is 5, each layer OP includes n=4 convolution kernels, one of which is an operation corresponding to the connection layer, the number of deep neural networks in the super network is 4 5 =1024 deep neural networks, each with a depth of 5. Adding a first connection layer between the layer 1 and layer 2 feature mapsA second connecting layer is added between the layer 2 characteristic diagram and the layer 3 characteristic diagram, a third connecting layer is added between the layer 3 characteristic diagram and the layer 4 characteristic diagram, and a fourth connecting layer is added between the layer 4 characteristic diagram and the layer 5 characteristic diagram. If the first connection layer is used, the layer 1 and layer 2 feature maps become one layer, and the depth becomes 4. If the first connection layer and the third connection layer are used, the depth becomes 3 with the layer 1, layer 2, and layer 3 feature maps as one layer. In this embodiment, the parameter sharing under the fixed depth may be that the parameters corresponding to each OP are adapted to a plurality of deep neural networks. After the connection layer is set, taking the first layer feature map to the third layer feature map as an example, when the second layer feature map is cross-layer, the parameter sharing mode may be that the first layer is directly connected to the third layer. The parameter sharing mode with variable layer number only needs to memorize more sharing information, and does not influence the essence of parameter sharing.
The user proposes different search constraints, such as convolution kernel size, search speed, and accuracy. To meet the search needs of the user, the ways to change the fixed depth of the deep neural network include: selecting different two layers of feature images, adding a connecting layer between the two layers of feature images, adding the connecting layer between the two layers of feature images, and combining the two layers of feature images, wherein the adding of the connecting layer changes the layer number of the feature layers, and the depth of the deep neural network. And a new search space is constructed by utilizing the connection layer, so that a deep neural network with any depth can be obtained under the constraint conditions of any model size, model speed and model precision, and meanwhile, the parameter sharing of the super network is not influenced. The technical problem that the depth of a neural network at any depth cannot be searched in the traditional super network search space based on unit structure stacking is solved.
In one embodiment, as shown in fig. 3, step S10 includes:
step S110: inputting the first layer of characteristic diagram into a plurality of layers of operation layers, outputting an Mth layer of characteristic diagram, wherein M is more than or equal to 2;
step S111: and selecting any two adjacent layers of feature maps from the first layer of feature maps to the M layer of feature maps.
In one embodiment, as shown in fig. 3, step S20 includes:
step S210: and respectively copying the characteristic information of the two selected adjacent characteristic graphs, and placing the copied characteristic information in the connecting layer.
In one example, two adjacent feature layers may be selected, for example, a first layer feature map and a second layer feature map … …, an M-1 layer feature map and an M layer feature map. The feature information of the two-layer feature map connected by the connection layer is copied, and the two layers can be regarded as one layer. Thereby reducing the number of hidden layers and the depth of the neural network.
It should be noted that the basis for selecting two adjacent feature layers can be to search the model size, speed and precision of the deep neural network, and multiple groups of feature images can be selected, and each group of feature images is added into the connection layer respectively, so that the depth of the deep neural network is diversified, and various requirements are met.
In one embodiment, as shown in fig. 4, step S10 includes:
step S120: inputting the first layer of characteristic diagram into a plurality of layers of operation layers, outputting an Mth layer of characteristic diagram, wherein M is more than or equal to 2;
step S121: any two layers of non-adjacent feature maps are selected from the first layer of feature maps to the M layer of feature maps.
In one embodiment, step S20 shown in fig. 4 includes:
step S220: and carrying out recursion operation on the characteristic information of the selected two layers of non-adjacent characteristic diagrams and the characteristic information of the characteristic diagram between the two layers of non-adjacent characteristic diagrams to obtain the recursion characteristic information, and placing the recursion characteristic information in the connection layer.
In one example, two feature layers that are not adjacent, for example, a layer 1 feature map and a layer 3 feature map, or a layer 5 feature map and a layer 8 feature map … …, or an M-5 feature map and an M layer feature map, may be selected. And carrying out recursion calculation on the feature information of all the feature graphs between the two layers of feature graphs connected by the connection layer. Thereby reducing the number of hidden layersDepth of the low neural network. For example, selecting a layer 1 feature map and a layer 4 feature map, proceeding from layer 1 to two successive layers, performing an operation f in the connection layer 4 (f 3 (f 2 (x)))＝f 4 (f 3 (x))＝f 4 (x) The equation x corresponds to the layer 1 feature map directly passing to the layer 4 feature map, the layer 2 feature map directly passing to the layer 4 feature map, and the layer 3 feature map directly passing to the layer 4 feature map.
It should be noted that the basis for selecting two non-adjacent feature layers can be to search the model size, speed and precision of the deep neural network, and multiple groups of feature images can be selected, wherein each group of feature images is added into a connecting layer respectively, so that the depth of the deep neural network is diversified, and various requirements are met.
In one embodiment, as shown in fig. 5, step S30 includes:
step S310: obtaining the maximum network layer number of the deep neural network;
step S320: obtaining the depth of the deep neural network according to the maximum network layer number and the operation layer number;
step S330: and constructing a search space of the super network by using the depth of the deep neural network.
In one example, as shown in fig. 2, the maximum number of network layers of the deep neural network, i.e., the initial depth M, is obtained. The two layers of the feature images are selected to form a group, multiple groups of feature images can be obtained according to the constraint condition of searching, and a connecting layer is added into each group of feature images. And calculating the depth of the depth neural network meeting the constraint condition of searching according to the connection layer and the initial depth M, and constructing a search space of the super network by utilizing the depth of the calculated depth neural network. In the search space, the optimal deep neural network is searched, and the search efficiency is improved.
Example two
In another embodiment, as shown in fig. 6, there is provided a search space construction apparatus 100 of a super network, including:
a feature map selecting module 110, configured to select two layers of feature maps from the multiple layers of feature maps of the deep neural network;
a connection layer adding module 120, configured to add a connection layer between the two selected feature maps, where the connection layer is used for;
a search space construction module 130 for constructing a search space of the super network using the connection layer.
In one embodiment, as shown in fig. 7, there is provided a search space construction apparatus 200 of a super network, and the feature map selection module 110 includes:
a feature map generating unit 1101, configured to input a first layer feature map to a multi-layer operation layer, output an mth layer feature map, and M is greater than or equal to 2;
a first feature map selecting unit 1102, configured to select any two adjacent feature maps from the first layer feature map to the mth layer feature map.
In one embodiment, as shown in fig. 7, the connection layer adding module 120 includes:
the first connection layer adding unit 1201 is configured to copy the feature information of the two adjacent feature graphs, and place the copied feature information in the connection layer.
In one embodiment, a search space construction apparatus 300 of a super network is provided, as shown in fig. 8, the feature map selection module 110 includes:
the second feature map selecting unit 1103 is configured to select any two non-adjacent feature maps from the first layer feature map to the mth layer feature map.
In one embodiment, as shown in fig. 8, there is provided a search space construction apparatus 300 of a super network, and the connection layer adding module 120 includes:
the second connection layer adding unit 1202 is configured to recursively operate the feature information of the selected two non-adjacent feature graphs and the feature information of the feature graph between the two non-adjacent feature graphs to obtain recursive feature information, and place the recursive feature information in the connection layer.
In one embodiment, as shown in fig. 9, there is provided a search space construction apparatus 400 of a super network, the search space construction module 130 including:
a maximum network layer number obtaining unit 1301, configured to obtain a maximum network layer number of the deep neural network;
a depth calculating unit 1302, configured to obtain the depth of the depth neural network according to the maximum network layer number and the operation layer number;
the search space construction unit 1303 is configured to construct a search space of the super network using the depth of the deep neural network.
According to embodiments of the present application, an electronic device and a readable storage medium are also provided.
Fig. 10 is a block diagram of an electronic device according to a method for constructing a search space of a super network according to an embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the application described and/or claimed herein.
As shown in fig. 10, the electronic device includes: one or more processors 1001, memory 1002, and interfaces for connecting the components, including a high-speed interface and a low-speed interface. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the electronic device, including instructions stored in or on memory to display graphical information of a graphical user interface (Graphical User Interface, GUI) on an external input/output device, such as a display device coupled to the interface. In other embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple electronic devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 1001 is illustrated in fig. 10.
The memory 1002 is used as a non-transitory computer readable storage medium, and may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules (e.g., the feature map selection module 110, the connection layer addition module 120, and the search space construction module 130 shown in fig. 6) corresponding to a method for constructing a search space of a super network in an embodiment of the present application. The processor 1001 executes various functional applications of the server and data processing by executing non-transitory software programs, instructions, and modules stored in the memory 1002, that is, implements a search space construction method of a super network in the above-described method embodiment.
An electronic device of search space construction of a super network may further include: an input device 1003 and an output device 1004. The processor 1001, memory 1002, input device 1003, and output device 1004 may be connected by a bus or other means, for example by a bus connection in fig. 10.
The input device 1003 may receive input numeric or character information and generate key signal inputs related to user settings and function control of an electronic device constructed of a search space of a super network, such as a touch screen, keypad, mouse, trackpad, touch pad, pointer stick, one or more mouse buttons, trackball, joystick, etc. The output means 1004 may include a display device, auxiliary lighting means (e.g., LEDs), tactile feedback means (e.g., vibration motors), and the like. The display device may include, but is not limited to, a liquid crystal display (Liquid Crystal Display, LCD), a light emitting diode (Light Emitting Diode, LED) display, and a plasma display. In some implementations, the display device may be a touch screen.
Various implementations of the systems and techniques described here can be implemented in digital electronic circuitry, integrated circuitry, application specific integrated circuits (Application Specific Integrated Circuits, ASIC), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
These computing programs (also referred to as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, programmable logic devices (programmable logic device, PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., CRT (Cathode Ray Tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area network (Local Area Network, LAN), wide area network (Wide Area Network, WAN) and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
According to the technical scheme of the embodiment of the application, different two layers of feature images are selected, a connecting layer is added between the two selected layers of feature images, the connecting layer is used for combining the two selected layers of feature images, the number of layers of the feature layers is changed by adding the connecting layer, and the depth of the deep neural network is changed. And a new search space is constructed by utilizing the connection layer, so that a deep neural network with any depth can be obtained under the constraint conditions of any model size, model speed and model precision, and meanwhile, the parameter sharing of the super network is not influenced. The technical problem that the depth of a neural network at any depth cannot be searched in the traditional super network search space based on unit structure stacking is solved.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present application may be performed in parallel, sequentially, or in a different order, provided that the desired results of the technical solutions disclosed in the present application can be achieved, and are not limited herein.
The above embodiments do not limit the scope of the application. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application are intended to be included within the scope of the present application.
Claims (6)
1. A method for constructing a search space of a super network, comprising:
selecting two layers of characteristic diagrams from the multiple layers of characteristic diagrams of the deep neural network; the multi-layer feature map is data existing in a hidden layer in the deep neural network, each layer of feature map comprises one or more feature maps, and the hidden layer is used for extracting features in computer vision; the feature map is a two-dimensional picture contained in three-dimensional data in a convolutional neural network;
adding a connecting layer between the two selected feature images, wherein the connecting layer is used for merging the two selected feature images;
constructing a search space of the super network by utilizing the connection layer;
in the multi-layer feature map of the deep neural network, selecting two layers of feature maps comprises:
inputting the first layer of characteristic diagram into a plurality of layers of operation layers, outputting an Mth layer of characteristic diagram, wherein M is more than or equal to 2;
selecting any two adjacent or non-adjacent feature maps from the first layer feature map to the Mth layer feature map;
wherein, add the tie layer between two-layer feature map that choose, including:
when the two layers of the selected feature images are adjacent feature images, the feature information of the two adjacent layers of the selected feature images is copied respectively, and the copied feature information is placed in the connecting layer;
under the condition that the two layers of the selected feature images are non-adjacent feature images, carrying out recursion operation on the feature information of the two layers of the selected non-adjacent feature images and the feature information of the feature images between the two layers of the selected non-adjacent feature images to obtain the feature information after recursion;
and placing the recursive characteristic information in the connection layer.
2. The method of claim 1, wherein constructing a search space of a super network using the connection layer comprises:
obtaining the maximum network layer number of the deep neural network;
obtaining the depth of the deep neural network according to the maximum network layer number and the connection layer;
and constructing a search space of the super network by utilizing the depth of the deep neural network.
3. A search space construction apparatus of a super network, comprising:
the feature map selecting module is used for selecting two layers of feature maps from the multi-layer feature maps of the deep neural network; the multi-layer feature map is data existing in a hidden layer in the deep neural network, and each layer of feature map comprises one or more feature maps; the hidden layer is used for extracting features in computer vision; the feature map is a two-dimensional picture contained in three-dimensional data in a convolutional neural network;
the connecting layer adding module is used for adding a connecting layer between the two selected layers of feature graphs, and the connecting layer is used for;
the search space construction module is used for constructing a search space of the super network by utilizing the connection layer;
wherein, the feature map selection module includes:
the feature map generating unit is used for inputting the first layer of feature map into the multi-layer operation layer and outputting the M layer of feature map, wherein M is more than or equal to 2;
a first feature map selecting unit, configured to select any two adjacent or non-adjacent feature maps from the first layer feature map to the M-th layer feature map;
wherein, the connection layer adding module includes:
the first connecting layer adding unit is used for respectively copying the characteristic information of the two adjacent layers of the selected characteristic images under the condition that the two adjacent layers of the selected characteristic images are adjacent characteristic images, and placing the copied characteristic information in the connecting layer;
and the second connection layer adding unit is used for carrying out recursion operation on the characteristic information of the selected two layers of non-adjacent characteristic diagrams and the characteristic information of the characteristic diagram between the two layers of non-adjacent characteristic diagrams under the condition that the selected two layers of characteristic diagrams are non-adjacent characteristic diagrams to obtain the recursion characteristic information, and placing the recursion characteristic information in the connection layer.
4. The apparatus of claim 3, wherein the search space construction module comprises:
the maximum network layer number acquisition unit is used for acquiring the maximum network layer number of the deep neural network;
the depth calculation unit is used for obtaining the depth of the depth neural network according to the maximum network layer number and the operation layer number;
and the search space construction unit is used for constructing the search space of the super network by utilizing the depth of the deep neural network.
5. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of claim 1 or 2.
6. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of claim 1 or 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910861037.6A CN110598629B (en) | 2019-09-11 | 2019-09-11 | Super-network search space construction method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910861037.6A CN110598629B (en) | 2019-09-11 | 2019-09-11 | Super-network search space construction method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110598629A CN110598629A (en) | 2019-12-20 |
| CN110598629B true CN110598629B (en) | 2023-06-06 |
Family
ID=68859037
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910861037.6A Active CN110598629B (en) | 2019-09-11 | 2019-09-11 | Super-network search space construction method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110598629B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111275172B (en) * | 2020-01-21 | 2023-09-01 | 复旦大学 | Feedforward neural network structure searching method based on search space optimization |
| CN111680599B (en) * | 2020-05-29 | 2023-08-08 | 北京百度网讯科技有限公司 | Face recognition model processing method, device, equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108021983A (en) * | 2016-10-28 | 2018-05-11 | 谷歌有限责任公司 | Neural framework search |
| CN108564166A (en) * | 2018-03-22 | 2018-09-21 | 南京大学 | Based on the semi-supervised feature learning method of the convolutional neural networks with symmetrical parallel link |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3583553A1 (en) * | 2017-07-21 | 2019-12-25 | Google LLC | Neural architecture search for convolutional neural networks |
| US10776668B2 (en) * | 2017-12-14 | 2020-09-15 | Robert Bosch Gmbh | Effective building block design for deep convolutional neural networks using search |
-
2019
- 2019-09-11 CN CN201910861037.6A patent/CN110598629B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108021983A (en) * | 2016-10-28 | 2018-05-11 | 谷歌有限责任公司 | Neural framework search |
| CN108564166A (en) * | 2018-03-22 | 2018-09-21 | 南京大学 | Based on the semi-supervised feature learning method of the convolutional neural networks with symmetrical parallel link |
Non-Patent Citations (2)
| Title |
|---|
| Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning;Christian Szegedy et al.;《arXiv:1602.07261v2》;20160823;第1-12页 * |
| Neural Architecture Search with Reinforcement Learning;Barret Zoph, Quoc V.Le;《ICLR 2017》;20171231;第1-16页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110598629A (en) | 2019-12-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110826696B (en) | Super-network search space construction method and device and electronic equipment | |
| CN110909877B (en) | Neural network model structure searching method and device, electronic equipment and storage medium | |
| CN111753961B (en) | Model training method and device, prediction method and device | |
| CN111667057B (en) | Method and apparatus for searching model structures | |
| CN111275190B (en) | Compression method and device of neural network model, image processing method and processor | |
| CN110569972A (en) | search space construction method and device of hyper network and electronic equipment | |
| US11727676B2 (en) | Feature fusion method and apparatus for image processing, electronic device and storage medium | |
| CN112560875B (en) | Depth information complement model training method, device, equipment and storage medium | |
| CN111967297B (en) | Image semantic segmentation method and device, electronic equipment and medium | |
| CN110580520A (en) | model structure sampling device based on hyper-network and electronic equipment | |
| CN112150462B (en) | Method, device, equipment and storage medium for determining target anchor point | |
| CN110852449B (en) | Model migration method and electronic equipment | |
| JP7242994B2 (en) | Video event identification method, apparatus, electronic device and storage medium | |
| CN110598629B (en) | Super-network search space construction method and device and electronic equipment | |
| CN110766089A (en) | Model structure sampling method and device of hyper network and electronic equipment | |
| CN111553279B (en) | Method, device, equipment and storage medium for learning and identifying characterization of interest points | |
| US11983849B2 (en) | Image filling method and apparatus, device, and storage medium | |
| CN111932464B (en) | Super-resolution model using and training method, device, equipment and medium | |
| CN111783951B (en) | Model acquisition method, device, equipment and storage medium based on super network | |
| CN111652354B (en) | Method, apparatus, device and storage medium for training super network | |
| CN111966925B (en) | Building interest point weight judging method and device, electronic equipment and storage medium | |
| CN111680597A (en) | Face recognition model processing method, device, equipment and storage medium | |
| CN112580723B (en) | Multi-model fusion method, device, electronic equipment and storage medium | |
| CN112634401B (en) | Plane track drawing method, device, equipment and storage medium | |
| EP3872704A2 (en) | Header model for instance segmentation, instance segmentation model, image segmentation method and apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |