CN111767990A - Neural network processing method and device - Google Patents
Neural network processing method and device Download PDFInfo
- Publication number
- CN111767990A CN111767990A CN202010601501.0A CN202010601501A CN111767990A CN 111767990 A CN111767990 A CN 111767990A CN 202010601501 A CN202010601501 A CN 202010601501A CN 111767990 A CN111767990 A CN 111767990A
- Authority
- CN
- China
- Prior art keywords
- search
- neural networks
- candidate neural
- parameter values
- feature fusion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 191
- 238000003672 processing method Methods 0.000 title abstract description 26
- 230000004927 fusion Effects 0.000 claims abstract description 113
- 230000000694 effects Effects 0.000 claims abstract description 57
- 238000000034 method Methods 0.000 claims abstract description 33
- 238000012545 processing Methods 0.000 claims abstract description 19
- 238000001514 detection method Methods 0.000 claims description 95
- 238000004590 computer program Methods 0.000 claims description 7
- 238000013140 knowledge distillation Methods 0.000 claims description 5
- 238000004821 distillation Methods 0.000 claims description 2
- 238000013473 artificial intelligence Methods 0.000 abstract description 2
- 238000013135 deep learning Methods 0.000 abstract description 2
- 230000015654 memory Effects 0.000 description 16
- 238000010586 diagram Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 9
- 230000008569 process Effects 0.000 description 9
- 238000004891 communication Methods 0.000 description 7
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 239000003795 chemical substances by application Substances 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000000513 principal component analysis Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a processing method and device of a neural network, relates to the technical field of artificial intelligence, cloud computing and deep learning, and can be used for image processing. The specific implementation mode comprises the following steps: acquiring identifiers of a plurality of candidate neural networks trained in advance and each candidate neural network in the candidate neural networks; determining a search space, wherein the search space is used for selecting the identifiers of at least two candidate neural networks for feature fusion from the identifiers and determining a feature fusion mode; acquiring parameter values of search parameters of a search space; searching the identifications of at least two candidate neural networks in a search space by adopting the parameter values; and taking the parameter value associated with the feature fusion as a feature fusion mode from the parameter values corresponding to the turn of the identification of the at least two candidate neural networks. The method and the device can improve the fusion effect of the neural network in a searching space searching mode, and can improve the fusion speed by obtaining the parameter values of the searching parameters.
Description
Technical Field
The embodiment of the application relates to the technical field of computers, in particular to the technical field of artificial intelligence, cloud computing and deep learning, and particularly relates to a processing method and device of a neural network.
Background
The deep neural network can be applied to various scenes, such as intelligent transportation, face recognition and the like. With the application and development of the deep neural network, the application of the deep neural network is more and more popular.
The deep neural network may detect images and the like. The detection effect of a single deep neural network is limited, in order to improve the detection effect of the deep neural network, the constraint of the single neural network is broken in the related technology, the technology of fusing a plurality of deep neural networks is adopted, and model fusion is realized. Therefore, when the object such as an image is detected, the multiple models can be used for detecting cooperatively, so that a better detection effect is achieved.
Disclosure of Invention
A processing method and apparatus of a neural network, an electronic device, and a storage medium are provided.
According to a first aspect, there is provided a processing method of a neural network, comprising: acquiring identifiers of a plurality of candidate neural networks trained in advance and each candidate neural network in the candidate neural networks; determining a search space, wherein the search space is used for selecting the identifiers of at least two candidate neural networks for feature fusion from the identifiers and determining a feature fusion mode; obtaining parameter values of search parameters of a search space, wherein the search parameters comprise candidate neural networks adopted in the search; searching the identifications of at least two candidate neural networks in a search space by using the parameter values, wherein each round of search uses the parameter values corresponding to the round of search; and taking the parameter value associated with the feature fusion as a feature fusion mode from the parameter values corresponding to the turn of the identification of the at least two candidate neural networks.
According to a second aspect, there is provided a processing apparatus of a neural network, comprising: an identification acquisition unit configured to acquire an identification of each of a plurality of candidate neural networks trained in advance and a plurality of candidate neural networks; the space determining unit is configured to determine a search space, wherein the search space is used for selecting the identifiers of at least two candidate neural networks for feature fusion from the identifiers and determining a feature fusion mode; a value acquisition unit configured to acquire parameter values of search parameters of a search space, wherein the search parameters include candidate neural networks employed in a search; a search unit configured to search a search space for identities of at least two candidate neural networks using the parameter values, wherein each round of search uses the parameter values corresponding to the round of search; and the mode determining unit is configured to take the parameter value associated with the feature fusion as the feature fusion mode from the parameter values corresponding to the identification turns of the at least two candidate neural networks.
According to a third aspect, there is provided an electronic device comprising: one or more processors; a storage device to store one or more programs that, when executed by one or more processors, cause the one or more processors to implement a method of any embodiment of a processing method, such as a neural network.
According to a fourth aspect, there is provided a computer-readable storage medium, on which a computer program is stored which, when executed by a processor, implements a method as any one of the embodiments of the processing method of the neural network.
According to the scheme of the application, the fusion effect of the neural network can be improved in a mode of searching in the search space. And, by obtaining the parameter value of the search parameter, the fusion speed can be improved. In some cases, the fused neural network with improved fusion effect is used for image processing, so that the detection precision and detection speed of the image can be improved. Furthermore, the method is beneficial to reducing the time delay of the execution equipment and reducing the occupation and consumption of computing resources such as a memory and the like.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
FIG. 1 is an exemplary system architecture diagram to which some embodiments of the present application may be applied;
FIG. 2 is a flow diagram of one embodiment of a processing method of a neural network according to the present application;
FIG. 3 is a schematic diagram of an application scenario of a processing method of a neural network according to the present application;
FIG. 4 is a flow diagram of yet another embodiment of a processing method of a neural network according to the present application;
FIG. 5 is a schematic block diagram of one embodiment of a processing device of a neural network according to the present application;
fig. 6 is a block diagram of an electronic device for implementing a processing method of a neural network according to an embodiment of the present application.
Detailed Description
The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
Fig. 1 shows an exemplary system architecture 100 to which embodiments of the neural network processing method or the neural network processing device of the present application may be applied.
As shown in fig. 1, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The user may use the terminal devices 101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like. Various communication client applications, such as video applications, live applications, instant messaging tools, mailbox clients, social platform software, and the like, may be installed on the terminal devices 101, 102, and 103.
Here, the terminal apparatuses 101, 102, and 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various electronic devices having a display screen, including but not limited to smart phones, tablet computers, e-book readers, laptop portable computers, desktop computers, and the like. When the terminal apparatuses 101, 102, 103 are software, they can be installed in the electronic apparatuses listed above. It may be implemented as multiple pieces of software or software modules (e.g., multiple pieces of software or software modules to provide distributed services) or as a single piece of software or software module. And is not particularly limited herein.
The server 105 may be a server providing various services, such as a background server providing support for the terminal devices 101, 102, 103. The background server may analyze and perform other processing on the received data such as the identifiers of the multiple candidate neural networks, and feed back a processing result (for example, the identifiers and the feature fusion modes of at least two candidate neural networks) to the terminal device.
It should be noted that the processing method of the neural network provided in the embodiment of the present application may be executed by the server 105 or the terminal devices 101, 102, and 103, and accordingly, the processing apparatus of the neural network may be disposed in the server 105 or the terminal devices 101, 102, and 103.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
With continued reference to FIG. 2, a flow 200 of one embodiment of a processing method of a neural network in accordance with the present application is shown. The processing method of the neural network comprises the following steps:
In this embodiment, an execution subject (for example, the server or the terminal device shown in fig. 1) on which the processing method of the neural network operates may acquire a plurality of candidate neural networks trained in advance, and may further acquire an identifier of each candidate neural network among the candidate neural networks. Specifically, each candidate neural network has its corresponding identification (i.e., ID).
The neural network can be various neural networks for detecting images, such as deep neural networks for detecting images, such as convolutional neural networks, residual neural networks, and the like.
In this embodiment, the execution body may determine a search space. In particular, the executing entity or other electronic device may search in a search space to select, from the identifications of the plurality of candidate neural networks, identifications of at least two candidate neural networks participating in feature fusion. In addition, the execution subject may also search for a feature fusion mode, that is, a mode used for feature fusion. Feature fusion is a way of fusion between neural networks, and refers to fusion of features generated by using at least two neural network detection images.
In this embodiment, the executing body may obtain a parameter value of the search parameter of the search space. The search parameter is a parameter used for searching in the search space. Specifically, the executing body may obtain one parameter value for each search parameter for each search turn. In practice, the parameter values of the obtained search parameters may indicate candidate neural networks for feature fusion in each search process. For example, there are 4 neural networks respectively No. 1, No. 2, No. 3, and No. 4, and the obtained parameter values may include two neural networks for feature fusion in the first round of search: number 1 and number 3.
After the search space is determined, candidate parameter values of each search parameter are determined, that is, all permutation combinations of the parameter values of each search parameter can be determined, and each permutation combination in the permutation combinations corresponds to one round of search. In step 203, the execution body acquires some permutation combinations of the permutation combinations to perform a search.
And step 204, searching the identifiers of at least two candidate neural networks in the search space by using the parameter values, wherein each round of search uses the parameter values corresponding to the round of search.
In this embodiment, the executing entity may perform a search in the search space based on the parameter value to search for the identities of the at least two candidate neural networks in the plurality of candidate neural networks. In each round of searching process, the executing body adopts the parameter value corresponding to the round of searching. For example, the first round of search uses parameter value a of search parameter X and parameter value a of search parameter Y. The second search round uses parameter value a of search parameter X and parameter value b of search parameter Y.
In this embodiment, the executing entity may determine parameter values corresponding to a round of search in which the identifiers of the at least two candidate neural networks are searched, and use a parameter value associated with feature fusion in the parameter values as the feature fusion manner. The round corresponds to the round search. The parameter values associated with feature fusion refer to parameter values required for feature fusion. Such as parameter values of a feature fusion algorithm.
The method provided by the embodiment of the application can improve the fusion effect of the neural network in a searching mode in a searching space. And, by obtaining the parameter value of the search parameter, the fusion speed can be improved.
In some optional implementations of this embodiment, the search parameter further includes at least one of: and searching the adopted feature fusion mode and the dimension of the fused features.
In these alternative implementations, the search parameters may further include a feature fusion algorithm corresponding to each search and/or a dimension of the fused features. The feature fusion algorithm corresponding to each round of search refers to a feature fusion algorithm adopted for feature fusion of the candidate neural network corresponding to each round of search in the process of each round of search. Specifically, the parameter value of the feature fusion algorithm as the search parameter may be a Principal Component Analysis (PCA) or may be a Linear Discriminant Analysis (LDA).
Specifically, the feature fusion algorithm corresponding to each search and the dimension of the fused feature are search parameters associated with feature fusion, and the parameter values of the search parameters and the dimension of the fused feature are parameter values associated with feature fusion.
These implementations may result in a more comprehensive and complete feature fusion approach by defining parameter values associated with feature fusion.
In some optional implementations of this embodiment, the method may further include: acquiring a target detection model, wherein the detection time consumption of the target detection model is less than a preset time consumption threshold value, and the number of parameters of the target detection model is less than a preset number threshold value; and (3) taking a model fusing the at least two candidate neural networks in the characteristic fusion mode as a teacher model, taking a target detection model as a student model, and carrying out knowledge distillation to obtain a distilled target detection model.
In these optional implementation manners, the executing entity may use the model including the at least two candidate neural networks as a teacher model, and perform knowledge distillation on the target detection model with short time consumption and few parameters, so that the target detection model has the characteristics of short time consumption and few parameters, and may have detection accuracy similar to that of the teacher model, thereby improving the detection accuracy of the target detection model.
In some optional implementations of this embodiment, step 203 may include: obtaining a parameter value determination model, wherein the parameter value determination model is used for determining a parameter value for each search parameter for each round of search; and inputting the candidate parameter values of the search parameters into the parameter value determination model to generate output values, wherein the output values comprise the parameter values of the search parameters corresponding to each round of search.
In these alternative implementations, the execution body may obtain a parameter value determination model. The parameter value determination model may be a function, an algorithm or a neural network, such as a deep neural network, which may be a convolutional neural network, a residual neural network, for example.
The executing body can determine the parameter value adopted by each round of search before searching by using the parameter value determination model. The parameter value determination model may determine, for each search parameter, a parameter value to be used in each search round as an output value from candidate parameter values for the search parameter.
For example, the search parameter may include a total number of search rounds, that is, a number of search rounds, and the candidate parameter value of the number of search rounds is a number of specific rounds. The output value corresponding to the number of search rounds includes one of these values.
With continued reference to fig. 3, fig. 3 is a schematic diagram of an application scenario of the processing method of the neural network according to the present embodiment. In the application scenario of fig. 3, an executing agent 301 obtains a plurality of candidate neural networks trained in advance and an identification 302 of each candidate neural network of the plurality of candidate neural networks. The execution subject 301 determines a search space 303, where the search space is used to select, from the identifiers 302, identifiers of at least two candidate neural networks for feature fusion and determine a feature fusion manner. The execution subject 301 obtains parameter values 304 for search parameters of a search space, where the search parameters include candidate neural networks employed in the search. The execution subject 301 searches the search space 303 for the identification 305 of at least two candidate neural networks using the parameter values 304, wherein each round of search uses the parameter values corresponding to the round of search. The execution subject 301 takes the parameter value associated with feature fusion from the parameter values corresponding to the rounds of the identifiers 305 of the at least two candidate neural networks as the feature fusion mode 306.
With further reference to fig. 4, a flow 400 of yet another embodiment of a processing method of a neural network is shown. The process 400 includes the following steps:
In this embodiment, an execution subject (for example, the server or the terminal device shown in fig. 1) on which the processing method of the neural network operates may acquire a plurality of candidate neural networks trained in advance, and may further acquire an identifier of each candidate neural network among the candidate neural networks. Specifically, each candidate neural network has its corresponding identification (i.e., ID).
In this embodiment, the execution body may determine a search space. In particular, the executing entity or other electronic device may search in a search space to select, from the identifications of the plurality of candidate neural networks, identifications of at least two candidate neural networks participating in feature fusion. In addition, the execution subject may also search for a feature fusion mode, that is, a mode used for feature fusion. Feature fusion is a way of fusion between neural networks, and refers to fusion of features generated by using at least two neural network detection images.
In step 403, parameter values of search parameters of the search space are obtained, where the search parameters include candidate neural networks used in the search.
In this embodiment, the executing body may obtain a parameter value of the search parameter of the search space. The search parameter is a parameter used for searching in the search space. Specifically, the executing body may obtain one parameter value for each search parameter for each search turn. In practice, the parameter values of the obtained search parameters may indicate candidate neural networks for feature fusion in each search process.
And step 404, for each round of search, based on the parameter value corresponding to the round of search, detecting an image by using the candidate neural network corresponding to the round of search, and performing feature fusion on the generated image features to generate a fusion result.
In this embodiment, the executing entity may detect the image by searching for the corresponding candidate neural network in each round based on the parameter value corresponding to the round search. In the process of detecting the image, an image feature, that is, a feature map (feature map) is generated, and the executing body may fuse the generated feature maps to implement feature fusion, thereby generating a fusion result.
In this embodiment, the execution subject may determine the detection result corresponding to the fusion result, for example, the execution subject may input the fusion result into a preset processing layer to obtain the detection result. The preset treatment layer can include a full connection layer, a pooling layer and/or a normalization layer, etc. Then, the executing body can compare the detection result with the real value of the object to be detected in the image, so as to determine the detection effect.
In practice, the execution body described above may determine the detection effect in various ways. For example, the detection result is the coordinate position of the target to be detected, if the difference between the coordinate position and the coordinate position shown by the true value is smaller than a preset threshold, the detection effect may be determined to be good, otherwise (if the difference between the coordinate position and the coordinate position shown by the true value is not smaller than the threshold), the detection effect may be determined to be poor. In addition, the execution main body may also quantize the detection effect, for example, the detection result is the coordinate position of the object to be detected, and the execution main body may directly use the difference between the coordinate position and the coordinate position shown by the real value as the detection effect. And acquiring a poor target threshold, and taking the detection effect smaller than or equal to the target threshold as a detection result with good detection effect.
The performing agent may determine the identities of the at least two candidate neural networks based on the detection effect in various ways. For example, the execution subject may take the candidate neural networks corresponding to a certain round as the at least two candidate neural networks when the detection effect corresponding to the search result of the round is good.
And 406, taking the parameter value associated with the feature fusion as a feature fusion mode from the parameter values corresponding to the turns of the identifiers of the at least two candidate neural networks.
In this embodiment, the executing entity may determine parameter values corresponding to a round of search in which the identifiers of the at least two candidate neural networks are searched, and use a parameter value associated with feature fusion in the parameter values as the feature fusion manner. The round corresponds to the round search.
The embodiment can determine the detection effect of the neural network, and accurately determine at least two candidate neural networks based on the feedback of the detection effect.
In some optional implementations of this embodiment, the detection effect includes detection time consumption and/or detection accuracy; determining the identities of the at least two candidate neural networks based on the detected effects in step 405 may include: determining whether a detection effect reaching a preset effect value exists in the detection effects corresponding to each round of search; and if the candidate neural networks do not exist, updating the acquired parameter values, continuing searching in the search space by using the updated parameter values, and determining the identifiers of the at least two candidate neural networks.
In these optional implementations, the execution main body may determine whether there is a detection effect within a preset effect range in the detection effects corresponding to each round of search. If not, the execution main body can update the acquired parameter value, and continue searching in the search space by using the updated parameter value. The preset effect range may be the target threshold, or the detection effect determined as being good.
If the detection effect of the detection result generated in all the rounds is not within the preset effect range, the execution main body can update the acquired parameter values, continue to search the identifiers of the at least two candidate neural networks for feature fusion by using the updated parameter values, obtain a new search result, and determine a feature fusion mode corresponding to the new search result.
In practice, the execution subject may update the acquired parameter value in various ways. For example, the executing entity may determine a parameter value that is different from the parameter values in each round of search that has been performed (which may be different for some of the search parameters). Thereafter, the execution body may take the parameter value as an updated parameter value. In addition, the execution body may input the parameter values and the detection effects of each search round into a specified model (for example, a specified convolutional neural network), and obtain updated parameter values output by the specified model.
These implementations can determine a more optimal feature fusion approach and neural networks participating in feature fusion by updating parameter values and continuing the search.
With further reference to fig. 5, as an implementation of the method shown in the above figures, the present application provides an embodiment of a processing apparatus of a neural network, the embodiment of the apparatus corresponds to the embodiment of the method shown in fig. 2, and the embodiment of the apparatus may further include the same or corresponding features or effects as the embodiment of the method shown in fig. 2, except for the features described below. The device can be applied to various electronic equipment.
As shown in fig. 5, the processing apparatus 500 of the neural network of the present embodiment includes: an identification acquisition unit 501, a space determination unit 502, a value acquisition unit 503, a search unit 504, and a manner determination unit 505. The identification obtaining unit 501 is configured to obtain an identification of each candidate neural network of a plurality of candidate neural networks trained in advance; a space determining unit 502 configured to determine a search space, where the search space is used to select, from the identifiers, identifiers of at least two candidate neural networks for feature fusion and determine a feature fusion manner; a value acquisition unit 503 configured to acquire parameter values of search parameters of a search space, wherein the search parameters include candidate neural networks employed in the search; a searching unit 504 configured to search for an identification of at least two candidate neural networks in a search space using the parameter values, wherein each round of search uses the parameter values corresponding to the round of search; and the mode determining unit 505 is configured to take the parameter value associated with the feature fusion as the feature fusion mode from the parameter values corresponding to the identified turns of the at least two candidate neural networks.
In this embodiment, specific processes of the identifier obtaining unit 501, the space determining unit 502, the value obtaining unit 503, the searching unit 504 and the mode determining unit 505 of the processing apparatus 500 of the neural network and technical effects brought by the specific processes may respectively refer to relevant descriptions of step 201, step 202, step 203, step 204 and step 205 in the corresponding embodiment of fig. 2, and are not described herein again.
In some optional implementations of this embodiment, the apparatus includes: an identification acquisition unit configured to acquire an identification of each of a plurality of candidate neural networks trained in advance and a plurality of candidate neural networks; the space determining unit is configured to determine a search space, wherein the search space is used for selecting the identifiers of at least two candidate neural networks for feature fusion from the identifiers and determining a feature fusion mode; a value acquisition unit configured to acquire parameter values of search parameters of a search space, wherein the search parameters include candidate neural networks employed in a search; a search unit configured to search a search space for identities of at least two candidate neural networks using the parameter values, wherein each round of search uses the parameter values corresponding to the round of search; and the mode determining unit is configured to take the parameter value associated with the feature fusion as the feature fusion mode from the parameter values corresponding to the identification turns of the at least two candidate neural networks.
In some optional implementations of this embodiment, the search parameter further includes at least one of: and searching the adopted feature fusion mode and the dimension of the fused features.
In some optional implementations of this embodiment, the searching unit is further configured to perform the searching for the identities of the at least two candidate neural networks in the search space using the parameter values as follows: for each round of search, based on the parameter value corresponding to the round of search, detecting an image by using the candidate neural network corresponding to the round of search, and performing feature fusion on the generated image features to generate a fusion result; and determining the detection effect of the detection result corresponding to the fusion result, and determining the identifications of the at least two candidate neural networks based on the detection effect.
In some optional implementations of this embodiment, the detection effect includes detection time consumption and/or detection accuracy; a search unit further configured to perform determining an identification of at least two candidate neural networks based on the detection effect as follows: determining whether a detection effect reaching a preset effect value exists in the detection effects corresponding to each round of search; and if the candidate neural networks do not exist, updating the acquired parameter values, continuing searching in the search space by using the updated parameter values, and determining the identifiers of the at least two candidate neural networks.
In some optional implementations of this embodiment, the apparatus further includes: a model obtaining unit configured to obtain a target detection model, wherein a detection time of the target detection model is less than a preset time-consuming threshold, and a number of parameters of the target detection model is less than a preset number threshold; and the distillation unit is configured to perform knowledge distillation by taking the model fusing the at least two candidate neural networks in a characteristic fusion mode as a teacher model and taking the target detection model as a student model to obtain a distilled target detection model.
According to an embodiment of the present application, an electronic device and a readable storage medium are also provided.
Fig. 6 is a block diagram of an electronic device according to a processing method of a neural network according to an embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.
As shown in fig. 6, the electronic apparatus includes: one or more processors 601, memory 602, and interfaces for connecting the various components, including a high-speed interface and a low-speed interface. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions for execution within the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input/output apparatus (such as a display device coupled to the interface). In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories and multiple memories, as desired. Also, multiple electronic devices may be connected, with each device providing portions of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system). In fig. 6, one processor 601 is taken as an example.
The memory 602 is a non-transitory computer readable storage medium as provided herein. The memory stores instructions executable by the at least one processor to cause the at least one processor to perform the processing method of the neural network provided by the present application. The non-transitory computer-readable storage medium of the present application stores computer instructions for causing a computer to execute the processing method of the neural network provided by the present application.
The memory 602, which is a non-transitory computer-readable storage medium, may be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions/modules corresponding to the processing method of the neural network in the embodiment of the present application (for example, the identification acquisition unit 501, the space determination unit 502, the value acquisition unit 503, the search unit 504, and the manner determination unit 505 shown in fig. 5). The processor 601 executes various functional applications of the server and data processing, i.e., a processing method of the neural network in the above method embodiment, by executing the non-transitory software programs, instructions, and modules stored in the memory 602.
The memory 602 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created from use of processing electronics of the neural network, and the like. Further, the memory 602 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory 602 optionally includes memory located remotely from the processor 601, which may be connected to the processing electronics of the neural network through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device of the processing method of the neural network may further include: an input device 603 and an output device 604. The processor 601, the memory 602, the input device 603 and the output device 604 may be connected by a bus or other means, and fig. 6 illustrates the connection by a bus as an example.
The input device 603 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the processing electronics of the neural network, such as a touch screen, keypad, mouse, track pad, touch pad, pointer stick, one or more mouse buttons, track ball, joystick, or other input device. The output devices 604 may include a display device, auxiliary lighting devices (e.g., LEDs), and tactile feedback devices (e.g., vibrating motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device can be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present application may be implemented by software or hardware. The described units may also be provided in a processor, and may be described as: a processor includes an identification acquisition unit, a space determination unit, a value acquisition unit, a search unit, and a manner determination unit. Where the names of these units do not in some cases constitute a limitation on the unit itself, for example, the space determination unit may also be described as a "unit that determines the search space".
As another aspect, the present application also provides a computer-readable medium, which may be contained in the apparatus described in the above embodiments; or may be present separately and not assembled into the device. The computer readable medium carries one or more programs which, when executed by the apparatus, cause the apparatus to: acquiring identifiers of a plurality of candidate neural networks trained in advance and each candidate neural network in the candidate neural networks; determining a search space, wherein the search space is used for selecting the identifiers of at least two candidate neural networks for feature fusion from the identifiers and determining a feature fusion mode; obtaining parameter values of search parameters of a search space, wherein the search parameters comprise candidate neural networks adopted in the search; searching the identifications of at least two candidate neural networks in a search space by using the parameter values, wherein each round of search uses the parameter values corresponding to the round of search; and taking the parameter value associated with the feature fusion as a feature fusion mode from the parameter values corresponding to the turn of the identification of the at least two candidate neural networks.
The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention herein disclosed is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the invention. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.
Claims (12)
1. A method of processing a neural network, the method comprising:
acquiring a plurality of candidate neural networks trained in advance and identifiers of each candidate neural network in the candidate neural networks;
determining a search space, wherein the search space is used for selecting identifiers of at least two candidate neural networks for feature fusion from the identifiers and determining a feature fusion mode;
obtaining parameter values of search parameters of the search space, wherein the search parameters comprise candidate neural networks adopted in search;
searching the identifications of the at least two candidate neural networks in the search space by using the parameter values, wherein each round of search uses the parameter values corresponding to the round of search;
and taking the parameter value associated with the feature fusion from the parameter values corresponding to the turn of the identification of the at least two candidate neural networks as the feature fusion mode.
2. The method of claim 1, wherein the search parameters further comprise at least one of:
and searching the adopted feature fusion mode and the dimension of the fused features.
3. The method of claim 1 or 2, wherein said searching the search space for the identities of the at least two candidate neural networks using the parameter values comprises:
for each round of search, based on the parameter value corresponding to the round of search, detecting an image by using the candidate neural network corresponding to the round of search, and performing feature fusion on the generated image features to generate a fusion result;
and determining the detection effect of the detection result corresponding to the fusion result, and determining the identifications of the at least two candidate neural networks based on the detection effect.
4. The method according to claim 4, wherein the detection effect comprises detection time consumption and/or detection precision;
said determining an identity of said at least two candidate neural networks based on said detection effect comprises:
determining whether a detection effect reaching a preset effect value exists in the detection effects corresponding to each round of search;
and if the candidate neural networks do not exist, updating the acquired parameter values, continuing searching in the search space by using the updated parameter values, and determining the identifiers of the at least two candidate neural networks.
5. The method according to claim 1 or 2, wherein the method further comprises:
acquiring a target detection model, wherein the detection time consumption of the target detection model is less than a preset time consumption threshold value, and the number of parameters of the target detection model is less than a preset number threshold value;
and performing knowledge distillation by taking a model fusing the at least two candidate neural networks in the characteristic fusion mode as a teacher model and taking the target detection model as a student model to obtain the distilled target detection model.
6. A processing apparatus of a neural network, the apparatus comprising:
an identification acquisition unit configured to acquire a plurality of candidate neural networks trained in advance and an identification of each of the plurality of candidate neural networks;
the space determining unit is configured to determine a search space, wherein the search space is used for selecting the identifiers of at least two candidate neural networks for feature fusion from the identifiers and determining a feature fusion mode;
a value acquisition unit configured to acquire parameter values of search parameters of the search space, wherein the search parameters include candidate neural networks employed in a search;
a search unit configured to search the search space for the identities of the at least two candidate neural networks using the parameter values, wherein each round of search uses the parameter values corresponding to the round of search;
and the mode determining unit is configured to take the parameter value associated with feature fusion from the parameter values corresponding to the turns of the identification of the at least two candidate neural networks as the feature fusion mode.
7. The apparatus of claim 6, wherein the search parameters further comprise at least one of:
and searching the adopted feature fusion mode and the dimension of the fused features.
8. The apparatus of claim 6 or 7, wherein the search unit is further configured to perform the searching for the identities of the at least two candidate neural networks in the search space using the parameter values as follows:
for each round of search, based on the parameter value corresponding to the round of search, detecting an image by using the candidate neural network corresponding to the round of search, and performing feature fusion on the generated image features to generate a fusion result;
and determining the detection effect of the detection result corresponding to the fusion result, and determining the identifications of the at least two candidate neural networks based on the detection effect.
9. The apparatus of claim 8, wherein the detection effect comprises detection time consumption and/or detection precision;
the search unit is further configured to perform the determining the identities of the at least two candidate neural networks based on the detection effect as follows:
determining whether a detection effect reaching a preset effect value exists in the detection effects corresponding to each round of search;
and if the candidate neural networks do not exist, updating the acquired parameter values, continuing searching in the search space by using the updated parameter values, and determining the identifiers of the at least two candidate neural networks.
10. The apparatus of claim 6 or 7, wherein the apparatus further comprises:
a model obtaining unit configured to obtain a target detection model, wherein a detection time of the target detection model is less than a preset time-consuming threshold, and a number of parameters of the target detection model is less than a preset number threshold;
and the distillation unit is configured to take a model fusing the at least two candidate neural networks in the characteristic fusion mode as a teacher model and take the target detection model as a student model, and perform knowledge distillation to obtain the distilled target detection model.
11. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-5.
12. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, carries out the method according to any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010601501.0A CN111767990A (en) | 2020-06-29 | 2020-06-29 | Neural network processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010601501.0A CN111767990A (en) | 2020-06-29 | 2020-06-29 | Neural network processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111767990A true CN111767990A (en) | 2020-10-13 |
Family
ID=72722539
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010601501.0A Withdrawn CN111767990A (en) | 2020-06-29 | 2020-06-29 | Neural network processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111767990A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113128680A (en) * | 2021-03-12 | 2021-07-16 | 山东英信计算机技术有限公司 | Neural network training method, system, device and medium |
| WO2024040941A1 (en) * | 2022-08-25 | 2024-02-29 | 华为云计算技术有限公司 | Neural architecture search method and device, and storage medium |
-
2020
- 2020-06-29 CN CN202010601501.0A patent/CN111767990A/en not_active Withdrawn
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113128680A (en) * | 2021-03-12 | 2021-07-16 | 山东英信计算机技术有限公司 | Neural network training method, system, device and medium |
| CN113128680B (en) * | 2021-03-12 | 2022-06-10 | 山东英信计算机技术有限公司 | Neural network training method, system, device and medium |
| WO2024040941A1 (en) * | 2022-08-25 | 2024-02-29 | 华为云计算技术有限公司 | Neural architecture search method and device, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111931591B (en) | Method, device, electronic equipment and readable storage medium for constructing key point learning model | |
| CN111221984A (en) | Multimodal content processing method, device, equipment and storage medium | |
| CN110795569B (en) | Method, device and equipment for generating vector representation of knowledge graph | |
| CN111949814A (en) | Searching method, searching device, electronic equipment and storage medium | |
| CN111582477B (en) | Training method and device for neural network model | |
| CN112270711B (en) | Model training and posture prediction method, device, equipment and storage medium | |
| CN113094550A (en) | Video retrieval method, device, equipment and medium | |
| CN110852379B (en) | Training sample generation method and device for target object recognition | |
| CN111667056A (en) | Method and apparatus for searching model structure | |
| CN111539347B (en) | Method and device for detecting target | |
| CN111241838B (en) | Semantic relation processing method, device and equipment for text entity | |
| CN111523007B (en) | Method, device, equipment and storage medium for determining user interest information | |
| CN111782785B (en) | Automatic question and answer method, device, equipment and storage medium | |
| CN111753964A (en) | Neural network training method and device | |
| CN110543558A (en) | question matching method, device, equipment and medium | |
| CN112529180A (en) | Method and apparatus for model distillation | |
| CN112241716A (en) | Training sample generation method and device | |
| CN111241225B (en) | Method, device, equipment and storage medium for judging change of resident area | |
| CN111767990A (en) | Neural network processing method and device | |
| CN113157829A (en) | Method and device for comparing interest point names, electronic equipment and storage medium | |
| CN111524123A (en) | Method and apparatus for processing image | |
| CN112561059A (en) | Method and apparatus for model distillation | |
| CN111488972B (en) | Data migration method, device, electronic equipment and storage medium | |
| CN112529181A (en) | Method and apparatus for model distillation | |
| CN112329732A (en) | Model generation method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20201013 |
|
| WW01 | Invention patent application withdrawn after publication |