WO2021057690A1 - 构建神经网络的方法与装置、及图像处理方法与装置 - Google Patents

构建神经网络的方法与装置、及图像处理方法与装置 Download PDF

Info

Publication number
WO2021057690A1
WO2021057690A1 PCT/CN2020/116673 CN2020116673W WO2021057690A1 WO 2021057690 A1 WO2021057690 A1 WO 2021057690A1 CN 2020116673 W CN2020116673 W CN 2020116673W WO 2021057690 A1 WO2021057690 A1 WO 2021057690A1
Authority
WO
WIPO (PCT)
Prior art keywords
elements
unevaluated
network structure
network
evaluated
Prior art date
Application number
PCT/CN2020/116673
Other languages
English (en)
French (fr)
Inventor
张颐康
钟钊
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN202010588884.2A external-priority patent/CN112633460A/zh
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to EP20868562.8A priority Critical patent/EP4030347A4/en
Publication of WO2021057690A1 publication Critical patent/WO2021057690A1/zh
Priority to US17/700,098 priority patent/US20220222934A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/30—Scenes; Scene-specific elements in albums, collections or shared content, e.g. social network photos or video
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle

Definitions

  • This application relates to the field of artificial intelligence, and more specifically, to a method and device for constructing a neural network, and an image processing method and device.
  • Artificial intelligence is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results.
  • artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a similar way to human intelligence.
  • Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning and decision-making.
  • Research in the field of artificial intelligence includes robotics, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, and basic AI theories.
  • neural networks for example, convolutional neural networks
  • neural networks have also made great achievements in the processing and analysis of various media signals such as images, videos and voices. Achievement.
  • a neural network with good performance often has a sophisticated network structure, but in practical applications, due to different training sets, index requirements, and application goals, the existing network structure cannot be used directly.
  • a common approach is to adjust based on the existing network structure for different tasks, but it is often difficult to obtain a network structure with excellent performance through adjustment; another common approach is based on automated machine learning ( Automated machine learning (AutoML) automatically searches the network structure, but AutoML is a method of designing the network structure from scratch, and the complexity of the task is very high.
  • Automated machine learning AutoML
  • the present application provides a method and device for constructing a neural network, and an image processing method and device, which can efficiently construct a neural network that meets performance requirements.
  • a method for constructing a neural network includes: constructing a search space according to the application requirements of the target neural network, the search space including M elements, and the M elements are used to indicate M networks Structure, each of the M elements includes the number of blocks in the stage of the corresponding network structure and the number of channels of the block, and M is a positive integer; according to the distribution relationship of unassessed elements in the search space from The target network structure is selected from the M network structures.
  • a representative element in the search space can be selected according to the distribution relationship of the unassessed elements. At this time, according to the representative element, it is possible to efficiently select from the M network structures
  • the target network structure can be constructed efficiently to meet the performance requirements of the neural network.
  • each element of the M elements includes the corresponding network structure refers to the network structure indicated by each element of the M elements.
  • each element includes the number of blocks in the stage in the network structure indicated by the element and the number of channels of the block.
  • the application requirements of the target neural network include the operating speed of the target neural network, the parameter quantity of the target neural network, or the structure of the target neural network Requirements, wherein the structural requirements include the number of blocks in each stage of the target neural network structure and the number of channels in each block.
  • the search space is constructed according to the operating speed of the target neural network, the parameter amount of the target neural network, or the structural requirements of the target neural network. Some of the search spaces can be filtered out during the process of constructing the search space. A low-performance network structure can improve the efficiency of building neural networks.
  • the constructing a search space according to the application requirements of the target neural network includes: constructing an initial search space according to the application requirements of the target neural network, the initial The search space includes N initial elements, the N initial elements are used to indicate N initial network structures, and each of the N initial elements includes the number of blocks in the corresponding initial network structure and The number of channels of the block, N is a positive integer greater than or equal to M; the N initial network structures indicated by the N initial elements are screened according to a preset rule to obtain the M in the search space Elements, the preset rule includes: if the number of blocks in each stage of the first initial network structure indicated by the first initial element in the N initial elements is not greater than the number of blocks in the N initial elements The number of blocks in the corresponding stage in the second initial network structure indicated by the second initial element, and the number of channels in each block in each stage of the first initial network structure is not greater than that of the second initial network structure. For the number of channels of each block in the
  • the present application without training the initial network structure indicated by the N initial elements in the initial search space, it is only based on the structure information of the N initial network structures.
  • the low-performance network structure among the N initial network structures can be screened out, and therefore, the efficiency of constructing the neural network can be improved.
  • the selecting a target network structure from the M network structures according to the distribution relationship of the unevaluated elements in the search space includes: The distribution relationship is to determine K elements in the unevaluated elements, where K is a positive integer less than M; select a target network structure from the M network structures according to the K elements.
  • the selecting a target network structure from the M network structures according to the K elements includes: checking the K elements in the unevaluated elements The indicated K network structures are evaluated, and the evaluation results of the evaluated elements are obtained.
  • the evaluation results of the evaluated elements include the evaluation results of the K network structures; according to the evaluation results of the evaluated elements, the evaluation results from the M networks Select the target network structure in the structure.
  • the target network structure that meets the preset requirements is selected among the M network structures, and the evaluation result of the evaluated network structure can be fully utilized, thereby Improve the efficiency of building neural networks.
  • the selecting a target network structure from the M network structures according to the evaluation result of the evaluated element includes: according to the evaluation result of the evaluated element The first unevaluated element is modeled to obtain a model of the first unevaluated element, where the first unevaluated element includes other elements in the search space other than the evaluated element; according to the first A model of unevaluated elements, selecting a target network structure from the M network structures.
  • the first unevaluated element is modeled according to the evaluation result of the evaluated element to obtain the model of the first unevaluated element; using the model of the first unevaluated element is helpful
  • the target network structure can be efficiently selected from the M network structures, so that the target network structure can be efficiently constructed to meet the performance requirements.
  • the method further includes : According to the distribution relationship of the first unevaluated element and the model of the first unevaluated element constructed based on the evaluated element, select a target network structure from the M network structures.
  • the model based on the distribution relationship of the first unevaluated element and the first unevaluated element constructed based on the evaluated element is obtained from the M networks
  • Selecting the target network structure in the structure includes: determining L elements in the first unevaluated element according to the distribution relationship of the first unevaluated element, where L is a positive integer less than M; and according to the L elements and The model of the first non-evaluated element constructed based on the evaluated element selects the target network structure from the M network structures.
  • the target network structure is selected from the M network structures according to the model of the L elements and the first non-evaluated element constructed based on the evaluated elements , Including: determining Q elements from the L elements according to the model of the first unevaluated element constructed based on the evaluated elements, where Q is a positive integer less than L; and indicating Q elements for the Q elements
  • the network structure is evaluated, and the evaluation result of the first evaluated element is obtained.
  • the evaluation result of the first evaluated element includes the evaluation results of the K network structures and the evaluation results of the Q network structures; Evaluate the distribution relationship between the evaluation result of the evaluation element and the second unevaluated element, select a target network structure from the M network structures, and the second unevaluated element includes the search space except for the first evaluated element Other elements.
  • the distribution relationship of the first unevaluated element is a clustering result of the first unevaluated element
  • the L elements are the first Elements in the L clusters included in the clustering result of the unassessed elements.
  • the clustering result of the first unassessed element is used to help select the representative element in the search space.
  • the representative element it can be efficiently obtained from the
  • the target network structure is selected from the M network structures, so that a neural network that meets the performance requirements can be efficiently constructed.
  • the L elements are respectively L elements corresponding to the centers of the L clusters.
  • the L network structures indicated by the L elements corresponding to the centers of the L clusters are the most representative network structures in the search space, and are based on the Q selected from the L elements.
  • the evaluation results of the Q network structures indicated by the elements, the target network structure that meets the preset requirements is selected from the M network structures, and the target that meets the preset requirements can be efficiently selected from the M network structures Network structure, which can improve the efficiency of building neural networks.
  • the method further includes: according to the distribution relationship of the second unevaluated element and the model of the second unevaluated element constructed based on the first evaluated element, from the M Reselect the target network structure in the network structure.
  • the target network structure when the target network structure is not selected from the M network structures according to the distribution relationship of the first unevaluated element and the model of the first unevaluated element constructed based on the evaluated element, reselect the target network structure from the M network structures, which can make full use of the evaluated.
  • the results of the evaluation of the network structure can improve the efficiency of building neural networks.
  • the model of the first unevaluated element constructed based on the evaluated element includes: the model of the first unevaluated element is based on the evaluated network structure The results of the evaluation and the following formula are obtained:
  • x represents the unevaluated network structure in the search space
  • y * represents the accuracy threshold
  • g(x) represents the conditional probability density function of the network structure x whose accuracy is greater than the accuracy threshold y*
  • l(x) represents the conditional probability density function of the network structure x whose accuracy is less than or equal to the accuracy threshold y *
  • x i ) represents the Gaussian mixture
  • ⁇ (x,x i ) represents the Euclidean distance between x and x i
  • ⁇ *(x,x i ) represents the distance function composed of ⁇ (x,x i )
  • x i represents the i-th network structure in the evaluated network structure
  • y i represents the accuracy of the i-th network structure in the evaluated network structure
  • ⁇ i represents G * (x
  • ⁇ represents a hyperparameter of G * (x
  • i
  • the model of the first unevaluated element constructed based on the evaluated element includes: the model of the first unevaluated element is based on the evaluated network structure The results of the evaluation and the following formula are obtained:
  • x represents an unevaluated element in the search space
  • xi represents the i-th element in the evaluated elements
  • y i represents the accuracy of the i-th element in the evaluated elements
  • ⁇ max(y i )
  • f(x) is a random variable that obeys the Gaussian distribution, the mean of f(x) ⁇ (x) and the variance ⁇ (x) of f(x) and It satisfies the following relationship with the input x:
  • ⁇ (x) 1-k T (K+ ⁇ 2 I) -1 k.
  • the distribution relationship of the unevaluated elements is a clustering result of the unevaluated elements, and the K elements are clusters of the unevaluated elements, respectively
  • the result includes the elements in the K clusters.
  • the clustering results of the unassessed elements are used to help select representative elements in the search space.
  • the M elements can be efficiently selected
  • the target network structure is selected from the network structure, so that the neural network that meets the performance requirements can be constructed efficiently.
  • the K elements are respectively K elements corresponding to the centers of the K clusters.
  • the K network structures indicated by the K elements corresponding to the centers of the K clusters are the most representative network structures in the search space, and are based on the K corresponding to the centers of the K clusters.
  • the evaluation results of the K network structures indicated by the elements, the target network structure that meets the preset requirements is selected from the M network structures, and the target network that meets the preset requirements can be efficiently selected from the M network structures Structure, which can improve the efficiency of building neural networks.
  • an image processing method includes: acquiring an image to be processed; classifying the image to be processed according to a target neural network to obtain a classification result of the image to be processed; wherein, the target neural network
  • the network is constructed by the target network structure, and the target network structure is obtained by the method in any one of the foregoing first aspects.
  • the target neural network used in the image processing method in the second aspect performs image classification
  • the target neural network needs to be trained according to the training image, and the trained target neural network can be used to classify the image to be processed. .
  • the neural network structure search method in the first aspect can be used to obtain the target neural network. Then, the target neural network can be trained according to the training image. After the training is completed, the target neural network can be used to perform the processing of the image to be processed. Classified.
  • the target neural network is constructed using the above-mentioned first aspect, it is more in line with or close to the application requirements of the neural network.
  • Using such a neural network for image classification can achieve better image classification effects (for example, The classification result is more accurate, etc.).
  • a device for constructing a neural network including: a construction unit for constructing a search space according to the application requirements of the target neural network.
  • the search space includes M elements, and the M elements are used to indicate M network structures, each of the M elements includes the number of blocks in the phase of the corresponding network structure and the number of channels of the block, and M is a positive integer;
  • the selection unit is used for searching space according to Select the target network structure from the M network structures for the distribution relationship of the unassessed elements in the.
  • a representative element in the search space can be selected according to the distribution relationship of the unassessed elements. At this time, according to the representative element, it is possible to efficiently select from the M network structures
  • the target network structure can be constructed efficiently to meet the performance requirements of the neural network.
  • each element of the M elements includes the corresponding network structure refers to the network structure indicated by each element of the M elements.
  • each element includes the number of blocks in the stage in the network structure indicated by the element and the number of channels of the block.
  • the application requirements of the target neural network include the operating speed of the target neural network, the parameter quantity of the target neural network, or the structure of the target neural network Requirements, wherein the structural requirements include the number of blocks in each stage of the target neural network structure and the number of channels in each block.
  • the search space is constructed according to the operating speed of the target neural network, the parameter amount of the target neural network, or the structural requirements of the target neural network. Some of the search spaces can be filtered out during the process of constructing the search space. A low-performance network structure can improve the efficiency of building neural networks.
  • the construction unit is specifically configured to: construct an initial search space according to the application requirements of the target neural network, the initial search space including N initial elements,
  • the N initial elements are used to indicate N initial network structures, and each of the N initial elements includes the number of blocks in the phase of the corresponding initial network structure and the number of channels of the blocks, N Is a positive integer greater than or equal to M;
  • the N initial network structures indicated by the N initial elements are screened according to a preset rule to obtain the M elements in the search space, the preset rule Including: if the number of blocks in each stage of the first initial network structure indicated by the first initial element among the N initial elements is not greater than the second one indicated by the second initial element among the N initial elements
  • the number of blocks in the corresponding stage in the initial network structure, and the number of channels in each block in each stage of the first initial network structure is not greater than that of each block in the corresponding stage in the second initial network structure.
  • the number of channels of each block, the first initial element is deleted
  • the present application without training the initial network structure indicated by the N initial elements in the initial search space, it is only based on the structure information of the N initial network structures.
  • the low-performance network structure among the N initial network structures can be screened out, and therefore, the efficiency of constructing the neural network can be improved.
  • the selection unit is specifically configured to: determine K elements in the unevaluated elements according to the distribution relationship of the unevaluated elements, where K is less than M A positive integer of; select a target network structure from the M network structures according to the K elements.
  • the selection unit is specifically configured to: evaluate the K network structures indicated by the K elements in the unassessed elements, to obtain an evaluation of the evaluated elements As a result, the evaluation result of the evaluated element includes the evaluation result of the K network structures; according to the evaluation result of the evaluated element, a target network structure is selected from the M network structures.
  • the target network structure that meets the preset requirements is selected among the M network structures, and the evaluation result of the evaluated network structure can be fully utilized, thereby Improve the efficiency of building neural networks.
  • the selection unit is specifically configured to: model the first unassessed element according to the evaluation result of the evaluated element to obtain the first unassessed A model of an element, the first unevaluated element includes other elements in the search space except the evaluated element; according to the model of the first unevaluated element, a target is selected from the M network structures Network structure.
  • the first unevaluated element is modeled according to the evaluation result of the evaluated element to obtain the model of the first unevaluated element; using the model of the first unevaluated element is helpful
  • the target network structure can be efficiently selected from the M network structures, so that the target network structure can be efficiently constructed to meet the performance requirements.
  • the selection unit further It is used for: selecting a target network structure from the M network structures according to the distribution relationship of the first unevaluated element and the model of the first unevaluated element constructed based on the evaluated element.
  • the selection unit is specifically configured to: determine L elements in the first unevaluated element according to the distribution relationship of the first unevaluated element, L is a positive integer less than M; a target network structure is selected from the M network structures according to the L elements and the model of the first unevaluated element constructed based on the evaluated elements.
  • the selection unit is specifically configured to: determine Q from the L elements according to the model of the first non-evaluated element constructed based on the evaluated elements.
  • Q is a positive integer less than L; the Q network structures indicated by the Q elements are evaluated to obtain the evaluation result of the first evaluated element, and the evaluation result of the first evaluated element includes the K The evaluation results of two network structures and the evaluation results of the Q network structures; according to the distribution relationship between the evaluation results of the first evaluated element and the second unevaluated element, the target network structure is selected from the M network structures, so The second unevaluated element includes other elements in the search space except the first evaluated element.
  • the distribution relationship of the first unevaluated element is a clustering result of the first unevaluated element
  • the L elements are the first Elements in the L clusters included in the clustering result of the unassessed elements.
  • the clustering result of the first unassessed element is used to help select the representative element in the search space.
  • the representative element it can be efficiently obtained from the
  • the target network structure is selected from the M network structures, so that a neural network that meets the performance requirements can be efficiently constructed.
  • the L elements are respectively L elements corresponding to the centers of the L clusters.
  • the L network structures indicated by the L elements corresponding to the centers of the L clusters are the most representative network structures in the search space, and are based on the Q determined from the L elements.
  • the evaluation results of the Q network structures indicated by the elements, the target network structure that meets the preset requirements is selected from the M network structures, and the target that meets the preset requirements can be efficiently selected from the M network structures Network structure, which can improve the efficiency of building neural networks.
  • the selection unit in accordance with the distribution relationship of the first unevaluated element and the model of the first unevaluated element constructed based on the evaluated element, from the M network structures
  • the selection unit is further configured to: according to the distribution relationship of the second unevaluated element and the model of the second unevaluated element constructed based on the first evaluated element, from the Reselect the target network structure among the M network structures.
  • the target network structure when the target network structure is not selected from the M network structures according to the distribution relationship of the first unevaluated element and the model of the first unevaluated element constructed based on the evaluated element, reselect the target network structure from the M network structures, which can make full use of the evaluated.
  • the results of the evaluation of the network structure can improve the efficiency of building neural networks.
  • the selection unit is specifically configured to: the model of the first unevaluated element is obtained based on the evaluation result of the evaluated network structure and the following formula:
  • x represents the unevaluated network structure in the search space
  • y * represents the accuracy threshold
  • g(x) represents the conditional probability density function of the network structure x whose accuracy is greater than the accuracy threshold y*
  • l(x) represents the conditional probability density function of the network structure x whose accuracy is less than or equal to the accuracy threshold y *
  • x i ) represents the Gaussian mixture distribution
  • ⁇ (x, x i ) represents the Euclidean distance between x and x i
  • ⁇ *(x, x i ) represents the distance function composed of ⁇ (x, x i )
  • x i represents the i-th network structure in the evaluated network structure
  • y i represents the accuracy of the i-th network structure in the evaluated network structure
  • ⁇ i represents G * (x
  • ⁇ represents a hyperparameter of G * (x
  • the selection unit is specifically configured to: the model of the first unevaluated element is obtained based on the evaluation result of the evaluated network structure and the following formula:
  • x represents an unevaluated element in the search space
  • xi represents the i-th element in the evaluated elements
  • y i represents the accuracy of the i-th element in the evaluated elements
  • ⁇ max(y i )
  • f(x) is a random variable that obeys the Gaussian distribution, the mean of f(x) ⁇ (x) and the variance ⁇ (x) of f(x) and It satisfies the following relationship with the input x:
  • ⁇ (x) 1-k T (K+ ⁇ 2 I) -1 k.
  • the distribution relationship of the unevaluated elements is a clustering result of the unevaluated elements, and the K elements are clusters of the unevaluated elements, respectively
  • the result includes the elements in the K clusters.
  • the clustering results of the unassessed elements are used to help select representative elements in the search space.
  • the M elements can be efficiently selected
  • the target network structure is selected from the network structure, so that the neural network that meets the performance requirements can be constructed efficiently.
  • the K elements are respectively K elements corresponding to the centers of the K clusters.
  • the K network structures indicated by the K elements corresponding to the centers of the K clusters are the most representative network structures in the search space, and are based on the K corresponding to the centers of the K clusters.
  • the evaluation results of the K network structures indicated by the elements, the target network structure that meets the preset requirements is selected from the M network structures, and the target network that meets the preset requirements can be efficiently selected from the M network structures Structure, which can improve the efficiency of building neural networks.
  • an image processing device including: an acquisition unit for acquiring an image to be processed; an image processing unit for classifying the image to be processed according to a target neural network to obtain the image of the image to be processed Classification result; wherein, the target neural network is constructed by a target network structure, and the target network structure is obtained by the method in any one of the above-mentioned first aspects.
  • the target neural network used in the image processing method in the second aspect performs image classification
  • the target neural network needs to be trained according to the training image, and the trained target neural network can be used to classify the image to be processed. .
  • the neural network structure search method in the first aspect can be used to obtain the target neural network. Then, the target neural network can be trained according to the training image. After the training is completed, the target neural network can be used to perform the processing of the image to be processed. Classified.
  • the target neural network is constructed using the above-mentioned first aspect, it is more in line with or close to the application requirements of the neural network.
  • Using such a neural network for image classification can achieve better image classification effects (for example, The classification result is more accurate, etc.).
  • a device for constructing a neural network includes: a memory for storing a program; a processor for executing the program stored in the memory.
  • the device The processor is used to execute the method in any one of the implementation manners in the first aspect.
  • the processor in the fifth aspect mentioned above can be either a central processing unit (CPU), or a combination of a CPU and a neural network processing unit.
  • the neural network processing unit here can include a graphics processing unit (graphics processing unit). unit, GPU), neural-network processing unit (NPU), tensor processing unit (TPU), and so on.
  • TPU is an artificial intelligence accelerator application specific integrated circuit fully customized by Google for machine learning.
  • an image processing device in a sixth aspect, includes: a memory for storing a program; a processor for executing the program stored in the memory, and when the program stored in the memory is executed, the processing The device is used to execute the method in any one of the implementation manners in the second aspect.
  • the processor in the sixth aspect mentioned above can be either a central processing unit (CPU), or a combination of a CPU and a neural network computing processor.
  • the neural network computing processor here can include a graphics processing unit (graphics processing unit). unit, GPU), neural-network processing unit (NPU), tensor processing unit (TPU), and so on.
  • graphics processing unit graphics processing unit
  • NPU neural-network processing unit
  • TPU tensor processing unit
  • TPU is an artificial intelligence accelerator application specific integrated circuit fully customized by Google for machine learning.
  • a computer-readable medium stores program code for device execution, and the program code includes a method for executing any one of the first aspect or the second aspect. .
  • a computer program product containing instructions when the computer program product runs on a computer, it causes the computer to execute the method in any one of the foregoing first aspect or second aspect.
  • a chip in a ninth aspect, includes a processor and a data interface, the processor reads instructions stored in a memory through the data interface, and executes any one of the first aspect or the second aspect above The method in the implementation mode.
  • the chip may further include a memory in which instructions are stored, and the processor is configured to execute instructions stored on the memory.
  • the processor is configured to execute the method in any one of the implementation manners of the first aspect or the second aspect.
  • the aforementioned chip may specifically be a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC).
  • FPGA field-programmable gate array
  • ASIC application-specific integrated circuit
  • a representative element in the search space can be selected according to the distribution relationship of the unassessed elements. At this time, according to the representative element, it is possible to efficiently select from the M network structures
  • the target network structure can be constructed efficiently to meet the performance requirements of the neural network.
  • Fig. 1 is a schematic diagram of an artificial intelligence main frame provided by an embodiment of the present application.
  • FIG. 2 is a schematic structural diagram of a system architecture provided by an embodiment of the application.
  • FIG. 3 is a schematic structural diagram of a convolutional neural network provided by an embodiment of this application.
  • FIG. 4 is a schematic structural diagram of another convolutional neural network provided by an embodiment of the application.
  • FIG. 5 is a schematic diagram of the hardware structure of a chip provided by an embodiment of the application.
  • FIG. 6 is a schematic diagram of a system architecture provided by an embodiment of this application.
  • Fig. 7 is a schematic flowchart of a method for constructing a neural network according to an embodiment of the present application.
  • FIG. 8 is a schematic block diagram of a residual block provided by an embodiment of the present application.
  • Fig. 9 is a schematic flowchart of a method for constructing a neural network according to an embodiment of the present application.
  • FIG. 10 is a schematic block diagram of a network structure in a search space provided by an embodiment of the present application.
  • FIG. 11 is a schematic flowchart of an image processing method according to an embodiment of the present application.
  • Fig. 12 is a schematic block diagram of an apparatus for constructing a neural network according to an embodiment of the present application.
  • FIG. 13 is a schematic block diagram of an image processing device according to an embodiment of the present application.
  • Fig. 14 is a schematic block diagram of a neural network training device according to an embodiment of the present application.
  • Figure 1 shows a schematic diagram of an artificial intelligence main framework, which describes the overall workflow of the artificial intelligence system and is suitable for general artificial intelligence field requirements.
  • Intelligent Information Chain reflects a series of processes from data acquisition to processing. For example, it can be the general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, intelligent execution and output. In this process, the data has gone through the condensing process of "data-information-knowledge-wisdom".
  • the "IT value chain” is the industrial ecological process from the underlying infrastructure and information (providing and processing technology realization) of human intelligence to the system, reflecting the value that artificial intelligence brings to the information technology industry.
  • the infrastructure provides computing power support for the artificial intelligence system, realizes communication with the outside world, and realizes support through the basic platform.
  • the infrastructure can communicate with the outside through sensors, and the computing power of the infrastructure can be provided by smart chips.
  • the smart chip here can be a central processing unit (CPU), a neural-network processing unit (NPU), a graphics processing unit (GPU), and an application specific integrated circuit (application specific).
  • Hardware acceleration chips such as integrated circuit (ASIC) and field programmable gate array (FPGA).
  • the basic platform of infrastructure can include distributed computing framework and network and other related platform guarantees and support, and can include cloud storage and computing, interconnection networks, etc.
  • data can be obtained through sensors and external communication, and then these data can be provided to the smart chip in the distributed computing system provided by the basic platform for calculation.
  • the data in the upper layer of the infrastructure is used to represent the data source in the field of artificial intelligence.
  • the data involves graphics, images, voice, text, and IoT data of traditional devices, including business data of existing systems and sensory data such as force, displacement, liquid level, temperature, and humidity.
  • the above-mentioned data processing usually includes data training, machine learning, deep learning, search, reasoning, decision-making and other processing methods.
  • machine learning and deep learning can symbolize and formalize data for intelligent information modeling, extraction, preprocessing, training, etc.
  • Reasoning refers to the process of simulating human intelligent reasoning in a computer or intelligent system, using formal information to conduct machine thinking and solving problems based on reasoning control strategies.
  • the typical function is search and matching.
  • Decision-making refers to the process of making decisions after intelligent information is inferred, and usually provides functions such as classification, ranking, and prediction.
  • some general capabilities can be formed based on the results of the data processing, such as an algorithm or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image Recognition and so on.
  • Intelligent products and industry applications refer to the products and applications of artificial intelligence systems in various fields. It is an encapsulation of the overall solution of artificial intelligence, productizing intelligent information decision-making and realizing landing applications. Its application fields mainly include: intelligent manufacturing, intelligent transportation, Smart home, smart medical, smart security, autonomous driving, safe city, smart terminal, etc.
  • the embodiments of this application can be applied to many fields in artificial intelligence, for example, smart manufacturing, smart transportation, smart home, smart medical, smart security, automatic driving, safe cities, and other fields.
  • the embodiments of the present application can be specifically applied in fields that require the use of (deep) neural networks, such as automatic driving, image classification, image retrieval, image semantic segmentation, image quality enhancement, image super-resolution, and natural language processing.
  • deep neural networks such as automatic driving, image classification, image retrieval, image semantic segmentation, image quality enhancement, image super-resolution, and natural language processing.
  • recognizing the images in the album can facilitate the user or the system to classify and manage the album and improve the user experience.
  • a neural network suitable for album classification is constructed, and then the neural network is trained according to the training pictures in the training picture library to obtain the album classification neural network.
  • the album classification neural network can be used to classify the pictures, so that different categories of pictures can be labeled, which is convenient for users to view and find.
  • the classification tags of these pictures can also be provided to the album management system for classification management, saving users management time, improving the efficiency of album management, and enhancing user experience.
  • a neural network suitable for data processing in an autonomous driving scenario can be constructed.
  • the neural network can be trained through the data in the autonomous driving scenario to obtain
  • the sensor data processing network can finally use the sensor processing network to process the input road image, thereby identifying different objects in the road image.
  • a neural network can be composed of neural units.
  • a neural unit can refer to an arithmetic unit that takes x s and intercept 1 as inputs.
  • the output of the arithmetic unit can be:
  • s 1, 2,...n, n is a natural number greater than 1
  • W s is the weight of x s
  • b is the bias of the neural unit.
  • f is the activation function of the neural unit, which is used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into an output signal.
  • the output signal of the activation function can be used as the input of the next convolutional layer, and the activation function can be a sigmoid function.
  • a neural network is a network formed by connecting multiple above-mentioned single neural units together, that is, the output of one neural unit can be the input of another neural unit.
  • the input of each neural unit can be connected with the local receptive field of the previous layer to extract the characteristics of the local receptive field.
  • the local receptive field can be a region composed of several neural units.
  • Deep neural network also known as multi-layer neural network
  • the DNN is divided according to the positions of different layers.
  • the neural network inside the DNN can be divided into three categories: input layer, hidden layer, and output layer.
  • the first layer is the input layer
  • the last layer is the output layer
  • the number of layers in the middle are all hidden layers.
  • the layers are fully connected, that is to say, any neuron in the i-th layer must be connected to any neuron in the i+1th layer.
  • DNN looks complicated, it is not complicated as far as the work of each layer is concerned. Simply put, it is the following linear relationship expression: among them, Is the input vector, Is the output vector, Is the offset vector, W is the weight matrix (also called coefficient), and ⁇ () is the activation function.
  • Each layer is just the input vector After such a simple operation, the output vector is obtained Due to the large number of DNN layers, the coefficient W and the offset vector The number is also relatively large.
  • DNN The definition of these parameters in DNN is as follows: Take coefficient W as an example: Suppose in a three-layer DNN, the linear coefficients from the fourth neuron in the second layer to the second neuron in the third layer are defined as The superscript 3 represents the number of layers where the coefficient W is located, and the subscript corresponds to the output third-level index 2 and the input second-level index 4.
  • the coefficient from the kth neuron of the L-1 layer to the jth neuron of the Lth layer is defined as
  • Convolutional neural network (convolutional neuron network, CNN) is a deep neural network with a convolutional structure.
  • the convolutional neural network contains a feature extractor composed of a convolutional layer and a sub-sampling layer.
  • the feature extractor can be regarded as a filter.
  • the convolutional layer refers to the neuron layer that performs convolution processing on the input signal in the convolutional neural network.
  • a neuron can only be connected to a part of the neighboring neurons.
  • a convolutional layer usually contains several feature planes, and each feature plane can be composed of some rectangularly arranged neural units. Neural units in the same feature plane share weights, and the shared weights here are the convolution kernels.
  • Sharing weight can be understood as the way of extracting image information has nothing to do with location.
  • the convolution kernel can be initialized in the form of a matrix of random size, and the convolution kernel can obtain reasonable weights through learning during the training process of the convolutional neural network.
  • the direct benefit of sharing weights is to reduce the connections between the layers of the convolutional neural network, and at the same time reduce the risk of overfitting.
  • RNN Recurrent Neural Networks
  • RNN can process sequence data of any length.
  • the training of RNN is the same as the training of traditional CNN or DNN.
  • Important equation taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, then the training of the deep neural network becomes a process of reducing this loss as much as possible.
  • the neural network can use an error back propagation (BP) algorithm to correct the size of the parameters in the initial neural network model during the training process, so that the reconstruction error loss of the neural network model becomes smaller and smaller. Specifically, forwarding the input signal until the output will cause error loss, and the parameters in the initial neural network model are updated by backpropagating the error loss information, so that the error loss is converged.
  • the backpropagation algorithm is a backpropagation motion dominated by error loss, and aims to obtain the optimal parameters of the neural network model, such as the weight matrix.
  • an embodiment of the present application provides a system architecture 100.
  • the data collection device 160 is used to collect training data.
  • the training data may include training images and classification results corresponding to the training images, where the results of the training images may be manually pre-labeled results.
  • the data collection device 160 stores the training data in the database 130, and the training device 120 trains to obtain the target model/rule 101 based on the training data maintained in the database 130.
  • the training device 120 processes the input original image and compares the output image with the original image until the output image of the training device 120 differs from the original image. The difference is less than a certain threshold, thereby completing the training of the target model/rule 101.
  • the above-mentioned target model/rule 101 can be used to implement the image processing method of the embodiment of the present application.
  • the target model/rule 101 in the embodiment of the present application may specifically be a neural network.
  • the training data maintained in the database 130 may not all come from the collection of the data collection device 160, and may also be received from other devices.
  • the training device 120 does not necessarily perform the training of the target model/rule 101 completely based on the training data maintained by the database 130. It may also obtain training data from the cloud or other places for model training.
  • the above description should not be used as a reference to this application. Limitations of the embodiment.
  • the target model/rule 101 trained according to the training device 120 can be applied to different systems or devices, such as the execution device 110 shown in FIG. 2, which can be a terminal, such as a mobile phone terminal, a tablet computer, notebook computers, augmented reality (AR) AR/virtual reality (VR), in-vehicle terminals, etc., can also be servers or clouds.
  • the execution device 110 is configured with an input/output (input/output, I/O) interface 112 for data interaction with external devices.
  • the user can input data to the I/O interface 112 through the client device 140.
  • the input data in this embodiment of the present application may include: a to-be-processed image input by the client device.
  • the preprocessing module 113 and the preprocessing module 114 are used to perform preprocessing according to the input data (such as the image to be processed) received by the I/O interface 112.
  • the preprocessing module 113 and the preprocessing module may not be provided.
  • 114 there may only be one preprocessing module, and the calculation module 111 is directly used to process the input data.
  • the execution device 110 may call data, codes, etc. in the data storage system 150 for corresponding processing .
  • the data, instructions, etc. obtained by corresponding processing may also be stored in the data storage system 150.
  • the I/O interface 112 returns the processing result, such as the classification result of the image obtained above, to the client device 140, so as to provide it to the user.
  • the training device 120 can generate corresponding target models/rules 101 based on different training data for different goals or tasks, and the corresponding target models/rules 101 can be used to achieve the above goals or complete The above tasks provide users with the desired results.
  • the user can manually set input data, and the manual setting can be operated through the interface provided by the I/O interface 112.
  • the client device 140 can automatically send input data to the I/O interface 112. If the client device 140 is required to automatically send the input data and the user's authorization is required, the user can set the corresponding authority in the client device 140.
  • the user can view the result output by the execution device 110 on the client device 140, and the specific presentation form may be a specific manner such as display, sound, and action.
  • the client device 140 can also be used as a data collection terminal to collect the input data of the input I/O interface 112 and the output result of the output I/O interface 112 as new sample data, and store it in the database 130 as shown in the figure.
  • the I/O interface 112 directly uses the input data input to the I/O interface 112 and the output result of the output I/O interface 112 as a new sample as shown in the figure.
  • the data is stored in the database 130.
  • FIG. 2 is only a schematic diagram of a system architecture provided by an embodiment of the present application, and the positional relationship between the devices, devices, modules, etc. shown in the figure does not constitute any limitation.
  • the data The storage system 150 is an external memory relative to the execution device 110. In other cases, the data storage system 150 may also be placed in the execution device 110.
  • the target model/rule 101 is obtained by training according to the training device 120.
  • the target model/rule 101 may be the neural network in the present application in the embodiment of the present application.
  • the neural network constructed in the embodiment of the present application It can be CNN, deep convolutional neural networks (DCNN), recurrent neural network (RNNS), and so on.
  • CNN is a very common neural network
  • the structure of CNN will be introduced in detail below in conjunction with Figure 3.
  • a convolutional neural network is a deep neural network with a convolutional structure. It is a deep learning architecture.
  • the deep learning architecture refers to the algorithm of machine learning. Multi-level learning is carried out on the abstract level of the system.
  • CNN is a feed-forward artificial neural network. Each neuron in the feed-forward artificial neural network can respond to the input image.
  • a convolutional neural network (CNN) 200 may include an input layer 210, a convolutional layer/pooling layer 220 (where the pooling layer is optional), and a neural network layer 230.
  • the input layer 210 can obtain the image to be processed, and pass the obtained image to be processed to the convolutional layer/pooling layer 220 and the subsequent neural network layer 230 for processing, and the processing result of the image can be obtained.
  • the convolutional layer/pooling layer 220 may include layers 221-226, for example: in an implementation, layer 221 is a convolutional layer, layer 222 is a pooling layer, and layer 223 is a convolutional layer. Layers, 224 is the pooling layer, 225 is the convolutional layer, and 226 is the pooling layer; in another implementation, 221 and 222 are the convolutional layers, 223 is the pooling layer, and 224 and 225 are the convolutional layers. Layer, 226 is the pooling layer. That is, the output of the convolutional layer can be used as the input of the subsequent pooling layer, or as the input of another convolutional layer to continue the convolution operation.
  • the convolution layer 221 can include many convolution operators.
  • the convolution operator is also called a kernel. Its function in image processing is equivalent to a filter that extracts specific information from the input image matrix.
  • the convolution operator is essentially It can be a weight matrix. This weight matrix is usually pre-defined. In the process of convolution on the image, the weight matrix is usually one pixel after one pixel (or two pixels after two pixels) along the horizontal direction on the input image. ...It depends on the value of stride) to complete the work of extracting specific features from the image.
  • the size of the weight matrix should be related to the size of the image. It should be noted that the depth dimension of the weight matrix and the depth dimension of the input image are the same.
  • the weight matrix will extend to Enter the entire depth of the image. Therefore, convolution with a single weight matrix will produce a single depth dimension convolution output, but in most cases, a single weight matrix is not used, but multiple weight matrices of the same size (row ⁇ column) are applied. That is, multiple homogeneous matrices.
  • the output of each weight matrix is stacked to form the depth dimension of the convolutional image, where the dimension can be understood as determined by the "multiple" mentioned above.
  • Different weight matrices can be used to extract different features in the image. For example, one weight matrix is used to extract edge information of the image, another weight matrix is used to extract specific colors of the image, and another weight matrix is used to eliminate unwanted noise in the image.
  • the multiple weight matrices have the same size (row ⁇ column), the size of the convolution feature maps extracted by the multiple weight matrices of the same size are also the same, and then the multiple extracted convolution feature maps of the same size are merged to form The output of the convolution operation.
  • weight values in these weight matrices need to be obtained through a lot of training in practical applications.
  • Each weight matrix formed by the weight values obtained through training can be used to extract information from the input image, so that the convolutional neural network 200 can make correct predictions. .
  • the initial convolutional layer (such as 221) often extracts more general features, which can also be called low-level features; with the convolutional neural network
  • the features extracted by the subsequent convolutional layers (for example, 226) become more and more complex, such as features such as high-level semantics, and features with higher semantics are more suitable for the problem to be solved.
  • the 221-226 layers as illustrated by 220 in Figure 3 can be a convolutional layer followed by a layer.
  • the pooling layer can also be a multi-layer convolutional layer followed by one or more pooling layers.
  • the sole purpose of the pooling layer is to reduce the size of the image space.
  • the pooling layer may include an average pooling operator and/or a maximum pooling operator for sampling the input image to obtain an image with a smaller size.
  • the average pooling operator can calculate the pixel values in the image within a specific range to generate an average value as the result of the average pooling.
  • the maximum pooling operator can take the pixel with the largest value within a specific range as the result of the maximum pooling.
  • the operators in the pooling layer should also be related to the image size.
  • the size of the image output after processing by the pooling layer can be smaller than the size of the image of the input pooling layer.
  • Each pixel in the image output by the pooling layer represents the average or maximum value of the corresponding sub-region of the image input to the pooling layer.
  • the convolutional neural network 200 After processing by the convolutional layer/pooling layer 220, the convolutional neural network 200 is not enough to output the required output information. Because as mentioned above, the convolutional layer/pooling layer 220 only extracts features and reduces the parameters brought by the input image. However, in order to generate final output information (required class information or other related information), the convolutional neural network 200 needs to use the neural network layer 230 to generate one or a group of required classes of output. Therefore, the neural network layer 230 may include multiple hidden layers (231, 232 to 23n as shown in FIG. 3) and an output layer 240. The parameters contained in the multiple hidden layers can be based on specific task types. The relevant training data of the, for example, the task type can include image recognition, image classification, image super-resolution reconstruction and so on.
  • the output layer 240 After the multiple hidden layers in the neural network layer 230, that is, the final layer of the entire convolutional neural network 200 is the output layer 240.
  • the output layer 240 has a loss function similar to the classification cross entropy, which is specifically used to calculate the prediction error.
  • a convolutional neural network (CNN) 200 may include an input layer 110, a convolutional layer/pooling layer 120 (the pooling layer is optional), and a neural network layer 130.
  • CNN convolutional neural network
  • FIG. 4 multiple convolutional layers/pooling layers in the convolutional layer/pooling layer 120 in FIG. 4 are parallel, and the respectively extracted features are input to the full neural network layer 130 for processing.
  • the convolutional neural network shown in FIGS. 3 and 4 is only used as an example of two possible convolutional neural networks in the image processing method of the embodiment of this application. In specific applications, this application implements The convolutional neural network used in the image processing method of the example can also exist in the form of other network models.
  • the structure of the convolutional neural network obtained by the search method of the neural network structure of the embodiment of the present application may be as shown in the convolutional neural network structure in FIG. 3 and FIG. 4.
  • FIG. 5 is a hardware structure of a chip provided by an embodiment of the application, and the chip includes a neural network processor 50.
  • the chip can be set in the execution device 110 as shown in FIG. 1 to complete the calculation work of the calculation module 111.
  • the chip can also be set in the training device 120 as shown in FIG. 1 to complete the training work of the training device 120 and output the target model/rule 101.
  • the algorithms of each layer in the convolutional neural network as shown in FIG. 3 and FIG. 4 can be implemented in the chip as shown in FIG. 5.
  • the neural network processor NPU 50 is mounted as a co-processor to a main central processing unit (central processing unit, CPU) (host CPU), and the main CPU distributes tasks.
  • the core part of the NPU is the arithmetic circuit 503.
  • the controller 504 controls the arithmetic circuit 503 to extract data from the memory (weight memory or input memory) and perform calculations.
  • the arithmetic circuit 503 includes multiple processing units (process engines, PE). In some implementations, the arithmetic circuit 503 is a two-dimensional systolic array. The arithmetic circuit 503 may also be a one-dimensional systolic array or other electronic circuit capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 503 is a general-purpose matrix processor.
  • the arithmetic circuit fetches the corresponding data of matrix B from the weight memory 502 and caches it on each PE in the arithmetic circuit.
  • the arithmetic circuit takes the matrix A data and matrix B from the input memory 501 to perform matrix operations, and the partial or final result of the obtained matrix is stored in an accumulator 508.
  • the vector calculation unit 507 can perform further processing on the output of the arithmetic circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, and so on.
  • the vector calculation unit 507 can be used for network calculations in the non-convolutional/non-FC layer of the neural network, such as pooling, batch normalization, local response normalization, etc. .
  • the vector calculation unit 507 can store the processed output vector in the unified buffer 506.
  • the vector calculation unit 507 may apply a nonlinear function to the output of the arithmetic circuit 503, such as a vector of accumulated values, to generate the activation value.
  • the vector calculation unit 507 generates a normalized value, a combined value, or both.
  • the processed output vector can be used as an activation input to the arithmetic circuit 503, for example for use in a subsequent layer in a neural network.
  • the unified memory 506 is used to store input data and output data.
  • the weight data directly transfers the input data in the external memory to the input memory 501 and/or the unified memory 506 through the storage unit access controller 505 (direct memory access controller, DMAC), and stores the weight data in the external memory into the weight memory 502, And the data in the unified memory 506 is stored in the external memory.
  • DMAC direct memory access controller
  • the bus interface unit (BIU) 510 is used to implement interaction between the main CPU, the DMAC, and the instruction fetch memory 509 through the bus.
  • An instruction fetch buffer 509 connected to the controller 504 is used to store instructions used by the controller 504;
  • the controller 504 is used to call the instructions cached in the memory 509 to control the working process of the computing accelerator.
  • the unified memory 506, the input memory 501, the weight memory 502, and the instruction fetch memory 509 are all on-chip (On-Chip) memories.
  • the external memory is a memory external to the NPU.
  • the external memory can be a double data rate synchronous dynamic random access memory.
  • Memory double data rate synchronous dynamic random access memory, referred to as DDR SDRAM
  • HBM high bandwidth memory
  • each layer in the convolutional neural network shown in FIG. 3 and FIG. 4 may be executed by the arithmetic circuit 503 or the vector calculation unit 507.
  • the execution device 110 in FIG. 2 introduced above can execute each step of the image processing method of the embodiment of this application.
  • the CNN model shown in FIGS. 3 and 4 and the chip shown in FIG. 5 can also be used to execute the implementation of this application. Examples of the various steps of the image processing method.
  • the method for constructing a neural network in the embodiment of the present application and the image processing method in the embodiment of the present application will be described in detail below with reference to the accompanying drawings.
  • an embodiment of the present application provides a system architecture 300.
  • the system architecture includes a local device 301, a local device 302, an execution device 210 and a data storage system 250, where the local device 301 and the local device 302 are connected to the execution device 210 through a communication network.
  • the execution device 210 may be implemented by one or more servers.
  • the execution device 210 can be used in conjunction with other computing devices, such as data storage, routers, load balancers, and other devices.
  • the execution device 210 may be arranged on one physical site or distributed on multiple physical sites.
  • the execution device 210 can use the data in the data storage system 250 or call the program code in the data storage system 250 to implement the neural network construction method of the embodiment of the present application.
  • the execution device 210 may execute the following process: construct a search space according to the application requirements of the target neural network; cluster multiple elements in the search space to obtain a clustering result; and according to the clustering result, Selecting a target network structure that meets a preset requirement from a plurality of network structures indicated by the multiple elements; and building the target neural network according to the target network structure.
  • a target neural network can be built, and the target neural network can be used for image classification or image processing.
  • Each local device can represent any computing device, such as personal computers, computer workstations, smart phones, tablets, smart cameras, smart cars or other types of cellular phones, media consumption devices, wearable devices, set-top boxes, game consoles, etc.
  • the local device of each user can interact with the execution device 210 through a communication network of any communication mechanism/communication standard.
  • the communication network can be a wide area network, a local area network, a point-to-point connection, etc., or any combination thereof.
  • the local device 301 and the local device 302 obtain the relevant parameters of the target neural network from the execution device 210, deploy the target neural network on the local device 301 and the local device 302, and use the target neural network for image classification Or image processing and so on.
  • the target neural network can be directly deployed on the execution device 210.
  • the execution device 210 obtains the image to be processed from the local device 301 and the local device 302, and classifies the image to be processed or other types of images according to the target neural network. deal with.
  • the foregoing execution device 210 may also be a cloud device. In this case, the execution device 210 may be deployed in the cloud; or, the foregoing execution device 210 may also be a terminal device. In this case, the execution device 210 may be deployed on the user terminal side. This is not limited.
  • the method 700 for constructing a neural network according to an embodiment of the present application will be introduced in detail with reference to FIG. 7.
  • the method shown in FIG. 7 can be executed by a device for building a neural network.
  • the device for building a neural network can be a mobile terminal, a computer, a server, and other devices with sufficient computing power to build a neural network.
  • the method shown in FIG. 7 includes steps 710, 720, and 730, which are described in detail below.
  • S710 Construct a search space according to the application requirements of the target neural network.
  • the search space may include M elements, the M elements may be used to indicate M network structures, and each of the M elements may include a corresponding stage in the network structure.
  • the number of, the number of blocks in the stage, and/or the number of channels of the block, M is a positive integer.
  • each element of the M elements includes the corresponding network structure refers to the network structure indicated by each element of the M elements.
  • each element includes the number of stages in the network structure indicated by the element, the number of blocks in the stage, and/or the The number of channels of the block.
  • the above-mentioned target neural network may be a deep neural network.
  • a convolutional neural network is taken as an example for description.
  • the network structure may include some stages in the convolutional neural network.
  • the network structure may refer to the part of the convolutional neural network that the user wants to adjust (or referred to as the part to be adjusted). Therefore, the network structure may also be referred to as the network structure to be adjusted.
  • the accuracy of the network structure may refer to the accuracy (or accuracy) of the neural network constructed by the network structure.
  • the above-mentioned target neural network may be constructed from at least one of the network structures, and the network structure may include one or more stages, and each stage may include at least one block.
  • a block can be composed of basic atomic units in a convolutional neural network, and these basic atomic units include: convolutional layer, pooling layer, fully connected layer, nonlinear activation layer, and so on.
  • features usually exist in three-dimensional form (length, width, and depth).
  • a feature can be regarded as a superposition of multiple two-dimensional features, where each two-dimensional feature of the feature can be called It is a feature map.
  • a feature map (two-dimensional feature) of the feature can also be referred to as a channel of the feature.
  • the length and width of the feature map can also be referred to as the resolution of the feature map.
  • the number of blocks in different stages can be different.
  • the resolution of the input feature map and the resolution of the output feature map processed at different stages may also be different.
  • the number of channels for different blocks can be different. It should be understood that the number of channels of a block may also be referred to as the width of the block, which is not limited in the embodiment of the present application.
  • the resolution of the input feature map and the resolution of the output feature map processed by different blocks can also be different.
  • the element may also include the resolution of the input feature map of the stage, the resolution of the output feature map of the stage, the resolution of the input feature map of the block, and/or the The resolution of the output feature map of the block.
  • the parameters (of the convolutional neural network) included in the elements are different, they will directly affect the effect and speed of the convolutional neural network. Therefore, in the embodiments of the present application, it can be based on the application requirements of the target neural network.
  • a search space is constructed for these parameters, and a target network structure that meets the preset requirements is selected in the search space.
  • the application requirements of the target neural network may include the operating speed of the target neural network, the parameter amount of the target neural network, or the structural requirements of the target neural network, wherein the structural requirements It includes the number of stages in the target neural network structure, the number of blocks in each stage, and/or the number of channels in each block.
  • the search space may be constructed based on the operating speed of the target neural network and/or the parameter amount of the target neural network.
  • the search space may include all possible network structures that meet the application requirements of the target nerve.
  • the search space is constructed according to the operating speed of the target neural network and/or the parameter amount of the target neural network. Some low-performance network structures can be filtered out during the process of constructing the search space. Improve the efficiency of building neural networks.
  • the application requirements of the target neural network may also include the structural requirements of the network structure.
  • the structural requirements of the network structure may include requirements for the number of stages in the network structure.
  • the number of stages here can be within the range of the number of stages.
  • the number of stages allowed in the network structure is between 2 and 5; or, the number of stages can also be required. Is the specific number of stages, for example, the number of stages in the network structure is 4.
  • the structural requirements of the network structure may also include requirements for the number of blocks in each stage, and/or requirements for the number of channels of the blocks.
  • the search space may include all possible network structures that meet the application requirements of the target nerve.
  • a search space can be constructed based on an existing neural network, and a target network structure that meets preset requirements can be selected in the search space.
  • a search space can be constructed based on existing ResNets, and a target network structure that meets preset requirements can be selected in the search space.
  • ResNets can divide the neural network into six stages according to the resolution of the output feature map.
  • the first stage contains only one convolution operation, and the last stage performs regression probabilities based on the fully connected operation.
  • the second to fifth stages are all stacked by one or more residual blocks, and the structure of the residual block may be as shown in FIG. 8.
  • the second to fifth stages of the aforementioned ResNets can be used as the network structure to be adjusted, the search space is constructed based on the network structure, and the number of channels of the blocks in each stage is specified to be the same.
  • the number of stages and the number of channels of blocks in the embodiments of the present application are only examples and not limited. In practice, the number of stages in the network structure to be adjusted is not limited to 4, nor is it limited. The number of channels of the blocks in each stage is the same, and the structural requirements of the network structure can be determined according to specific application requirements, which is not limited in the embodiment of the present application.
  • a network structure can be coded as follows:
  • BCSC ⁇ D 2 ,D 3 ,D 4 ,D 5 ,C 2 ,C 3 ,C 4 ,C 5 ⁇
  • D 2 represents the number of blocks in the second stage
  • D 3 represents the number of blocks in the third stage
  • D 4 represents the number of blocks in the fourth stage
  • D 5 represents the fifth The number of blocks in the stage
  • C 2 represents the number of channels of the block in the second stage
  • C 3 represents the number of channels of the block in the second stage
  • C 4 represents the number of channels of the block in the second stage
  • C 5 represents the number of channels of the block in the second stage.
  • BCSC BCSC (octet) encoding method
  • octet BCSC (octet) encoding method
  • the M elements in the search space may be M BCSCs, where each BCSC may indicate a network structure.
  • all possible M network structures can be determined according to the application requirements of the target neural network; and BCSC is used to encode each network structure to obtain M BCSCs; at this time, these M BCSCs It can be used as M elements to form a search space.
  • an initial search space including N BCSCs can be obtained.
  • the initial search space can be filtered, and after filtering out the low-performance network structure therein, the search space including M elements can be obtained.
  • an initial search space may be constructed according to the application requirements of the target neural network.
  • the initial search space includes N initial elements, and the N initial elements are used to indicate N initial network structures.
  • Each element in the initial element includes the number of phases in the corresponding initial network structure, the number of blocks in the phase, and/or the number of channels of the block, and N is a positive integer greater than or equal to M
  • the N initial network structures indicated by the N initial elements can be screened according to preset rules to obtain the M elements in the search space.
  • the preset rule may include:
  • the number of blocks in each stage of the first initial network structure indicated by the first initial element among the N initial elements is not greater than the second initial network indicated by the second initial element among the N initial elements.
  • the number of blocks in the corresponding stage in the structure, and the number of channels in each block in each stage in the first initial network structure is not greater than each block in the corresponding stage in the second initial network structure.
  • the number of channels, the first initial element is deleted from the initial search space.
  • Figure 10 shows N network structures, where N is a positive integer greater than 1.
  • stage 1 includes 1 block, the number of channels of the block is 64
  • stage 2 includes 2 blocks, the number of channels of each block is 128, and stage 3 includes 2 blocks, each The number of channels of the block is 256, and the number of channels in phase 4 is 4 blocks, and the number of channels in each block is 512
  • network structure 2 the number of channels in phase 1 is 1 block, the number of channels in the block is 64, and the number of channels in phase 2 is 1
  • the number of channels in the block is 128, the number of channels in each block is 256 in phase 3, and the number of channels in each block is 512 in phase 4.
  • stage 2 includes 2 blocks with 128 channels, and stage 4 includes 4 blocks.
  • the number of channels is 512 blocks.
  • the copied network structure 2 has the same network structure as the network structure 1, which means that the accuracy of the network structure 2 is lower than that of the network structure 1. You can delete the network structure 2 from the initial search space (or in other words, the network structure 2 corresponds to Elements).
  • the BCSC of the first initial element can be directly compared with the BCSC of the second initial element, and each element in the BCSC of the first initial element is more than the corresponding BCSC of the second initial element.
  • the element of is small, it means that the accuracy of the first initial network structure is lower than that of the second initial network structure, and accordingly, the first initial element can be deleted from the initial search space.
  • the present application without training the initial network structure indicated by the N initial elements in the initial search space, it is only based on the structure information of the N initial network structures.
  • the low-performance network structure among the N initial network structures can be screened out, and therefore, the efficiency of constructing the neural network can be improved.
  • S720 Select a target network structure from the M network structures according to the distribution relationship of unassessed elements in the search space.
  • the distribution relationship of the unevaluated elements may be a clustering result of the unevaluated elements.
  • the representative elements of the unassessed elements can be easily selected.
  • the M networks can be efficiently obtained from the The target network structure is selected from the structure, so that the neural network that meets the performance requirements can be constructed efficiently.
  • the candidate set is determined according to the distribution relationship of the unevaluated elements, and the candidate set may include representative elements among the selected unevaluated elements.
  • the candidate set is only an example and not a limitation. In the embodiment of the present application, other methods may be used to determine the candidate set. In the embodiment of the present application, the candidate set is determined based on the distribution relationship of the unassessed elements. The specific method is not limited.
  • a candidate set (including representative elements among the unassessed elements) can be obtained according to the distribution relationship of the unassessed elements in the following several ways.
  • a candidate set (with unevaluated elements) can be constructed so that the distance between the elements in the candidate set is as large as possible. In this way, the distribution of the elements in the set among the unassessed elements can be dispersed as much as possible, so that representative elements among the unassessed elements can be selected.
  • the candidate set may be obtained by the following method:
  • step (7) If the collection If the number of elements is greater than T, then perform step (7), otherwise perform step (3) for iteration;
  • a candidate set (of unevaluated elements) can be constructed so that each element in the candidate set (as far as possible) is located in the center of a certain subset of the unevaluated elements (where each element is located).
  • the candidate set may be obtained by the following method:
  • step (7) If the collection If the number of elements of is less than K, execute step (7), otherwise execute step (2) for iteration;
  • the unevaluated elements can be clustered to obtain the clustering results of the unevaluated elements (the clustering results of the unevaluated elements can also be considered as the distribution relationship of the unevaluated elements), and based on the unevaluated elements The clustering result of, the representative element among the selected unassessed elements, that is, the candidate set.
  • the unevaluated elements in the search space can be clustered to obtain the clustering results of the unevaluated elements, and then the target network structure is selected from the M network structures according to the clustering results of the unevaluated elements .
  • the M elements before clustering the M elements, the M elements (ie M BCSCs) may also be standardized.
  • the specific standardization process may refer to the prior art. This is not limited.
  • the mean value and variance of the M BCSCs in each dimension may be counted.
  • the variance of each dimension can be recorded as among them, Represents the variance of the number of blocks in the second stage, Represents the variance of the number of blocks in the third stage, Represents the variance of the number of blocks in the fourth stage, Represents the variance of the number of blocks in the fifth stage, Represents the variance of the number of channels of the block in the second stage, Represents the variance of the number of channels of the block in the third stage, Represents the variance of the number of channels of the block in the fourth stage, Represents the variance of the number of channels of the block in the fifth stage.
  • the M obtained after standardization can be Perform clustering.
  • the embodiment of the present application does not limit the clustering method used, and the specific clustering method can refer to the prior art, which will not be repeated here.
  • the K-means algorithm can be used to calculate the normalized M Perform clustering.
  • K elements in the unevaluated elements may be determined according to the distribution relationship of the unevaluated elements, where K is a positive integer less than M; select from the M network structures according to the K elements Target network structure.
  • Clustering the M elements can cluster the M elements into (one or) multiple clusters.
  • the clustering result of the M elements may include (one or more) clusters obtained by clustering the M elements. Among them, the elements in the same cluster are more similar to the elements in other clusters.
  • the clustering result may also include the center of each cluster, and the center of each cluster may be considered as the most representative element in the cluster.
  • the clustering result of the unevaluated elements may include K clusters, and K is a positive integer less than M.
  • the K elements in the unevaluated elements determined according to the distribution relationship of the unevaluated elements may be The K elements corresponding to the cluster centers of the K clusters.
  • the M elements in the search space are clustered, and network structures with similar structures indicated by the M elements can be clustered into one cluster, and multiple clusters obtained from the clustering , Which helps to efficiently select a target network structure that meets the preset requirements from the M network structures.
  • the clustering result may include K clusters.
  • one element may be selected from each cluster to obtain the above K elements, and the K network structures indicated by the K elements may be evaluated to obtain all Describe the evaluation results of K network structures.
  • evaluating the K network structures indicated by the K elements may refer to testing the accuracy (or precision) of the K network structures.
  • the K elements may be K elements corresponding to the centers of the K clusters.
  • the selecting a target network structure from the M network structures according to the K elements may include:
  • the K network structures indicated by the K elements in the un-evaluated elements are evaluated, and the evaluation results of the evaluated elements are obtained.
  • the evaluation results of the evaluated elements include the evaluation results of the K network structures;
  • the target network structure is selected from the M network structures.
  • the K network structures indicated by the K elements corresponding to the centers of the K clusters are the most representative network structures in the search space, and are based on the K corresponding to the centers of the K clusters.
  • the evaluation results of the K network structures indicated by the elements, the target network structure that meets the preset requirements is selected from the M network structures, and the target network that meets the preset requirements can be efficiently selected from the M network structures Structure, which can improve the efficiency of building neural networks.
  • the network structure is the target network structure, and the neural network can be constructed according to the network structure.
  • the first unevaluated element may be modeled according to the evaluation result of the evaluated element to obtain a model of the first unevaluated element, where the first unevaluated element includes the search space except for the already evaluated element. Elements other than the evaluated element; according to the model of the first unevaluated element, a target network structure is selected from the M network structures.
  • the first unevaluated element can be modeled according to the evaluation result of the evaluated element and the following formula to obtain the model of the unevaluated network structure:
  • x represents the unevaluated network structure in the search space
  • y * represents the accuracy threshold
  • g(x) represents the conditional probability density function of the network structure x whose accuracy is greater than the accuracy threshold y*
  • l(x) represents the conditional probability density function of the network structure x whose accuracy is less than or equal to the accuracy threshold y *
  • x i ) represents the Gaussian mixture distribution
  • ⁇ (x, x i ) represents the Euclidean distance between x and x i
  • ⁇ *(x, x i ) represents the distance function composed of ⁇ (x, x i )
  • x i represents the i-th network structure in the evaluated network structure
  • y i represents the accuracy of the i-th network structure in the evaluated network structure
  • ⁇ i represents G * (x
  • ⁇ represents a hyperparameter of G * (x
  • x i ) can satisfy: And when y i ⁇ y * , the larger y i is, the smaller ⁇ i is . When y i > y * , the larger y i is, the larger ⁇ i is; Z can satisfy The value of ⁇ can be half of the value of the average Euclidean distance between elements corresponding to the centers of each cluster after clustering.
  • the above It is positively correlated with the performance of the network structure, that is, The larger the value, the The better the performance of the corresponding network structure.
  • the above-mentioned (g(x)-l(x)) 2 is positively correlated with the uncertainty of the network structure performance, that is to say , the larger the value of (g(x)-l(x)) 2 , then the (g(x) -l(x)) 2 corresponds to the higher the uncertainty of the network structure performance.
  • the network structure is the target network structure, and the neural network can be constructed according to the network structure.
  • the target network structure is selected from the M network structures again according to the evaluation result of the evaluated network structure (including the four network structures just selected above).
  • the first unevaluated element may be modeled according to the evaluation result of the evaluated element and the following formula to obtain the model of the first unevaluated element:
  • x represents an unevaluated element in the search space
  • xi represents the i-th element in the evaluated elements
  • y i represents the accuracy of the i-th element in the evaluated elements
  • ⁇ max(y i )
  • f(x) is a random variable that obeys the Gaussian distribution, the mean of f(x) ⁇ (x) and the variance ⁇ (x) of f(x) and It satisfies the following relationship with the input x:
  • ⁇ (x) 1-k T (K+ ⁇ 2 I) -1 k.
  • modeling method is only an example and not a limitation, and other methods can also be used to perform modeling in the embodiments of the present application.
  • the method may further include:
  • a target network structure is selected from the M network structures.
  • the selecting a target network structure from the M network structures according to the distribution relationship of the first unevaluated element and the model of the first unevaluated element constructed based on the evaluated element may include:
  • the model of the evaluation element selects the target network structure from the M network structures.
  • the selecting a target network structure from the M network structures according to the model of the L elements and the first non-evaluated element constructed based on the evaluated elements may include:
  • Q elements are determined from the L elements, where Q is a positive integer less than L; the Q network structures indicated by the Q elements are performed Evaluation, the evaluation result of the first evaluated element is obtained, the evaluation result of the first evaluated element includes the evaluation results of the K network structures and the evaluation results of the Q network structures; according to the first evaluated element
  • the distribution relationship between the evaluation result and the second unevaluated element select a target network structure from the M network structures, and the second unevaluated element includes other elements in the search space except the first evaluated element element.
  • the distribution relationship of the first unevaluated element may be a clustering result of the first unevaluated element
  • the L elements may be respectively L clusters included in the clustering result of the first unevaluated element Elements in.
  • the L elements may be L elements corresponding to the centers of the L clusters, respectively.
  • the Methods can also include:
  • a target network structure is reselected from the M network structures.
  • the above-mentioned method of reselecting a target network structure from the M network structures according to the distribution relationship of the second unevaluated element and the model of the second unevaluated element constructed based on the first evaluated element It is similar to the foregoing method of selecting a target network structure from the M network structures based on the distribution relationship of the first unevaluated element and the model of the first unevaluated element constructed based on the evaluated element, which will not be repeated here.
  • This process will continue iteratively until the target network structure that meets the preset requirements is selected.
  • FIG. 9 shows a schematic flowchart of an image processing method 900 provided by an embodiment of the present application.
  • the method may be executed by a device or device capable of image processing.
  • the method may be executed by a terminal device, a computer, a server, or the like.
  • S910 Construct an initial search space according to the application requirements of the target neural network.
  • the initial search space may include N initial elements, the N initial elements may be used to indicate N network structures, and each element of the M elements may include a corresponding stage in the network structure.
  • the target neural network may be a convolutional neural network.
  • the network structure may include some stages in the convolutional neural network.
  • the network structure may refer to the part of the convolutional neural network that the user wants to adjust (or referred to as the part to be adjusted). Therefore, the network structure may also be referred to as the network structure to be adjusted.
  • the above-mentioned target neural network may be constructed from at least one of the network structures, and the network structure may include one or more stages, and each stage may include at least one block.
  • S920 Filter the initial search space based on preset rules to obtain the search space.
  • the search space may include M elements, the M elements may be used to indicate M network structures, and the elements may include the number of stages in the network structure, and the number of stages in the network structure.
  • the number of blocks and/or the number of channels of the block, M is a positive integer less than or equal to N.
  • the preset rule may include:
  • the number of blocks in each stage of the first initial network structure indicated by the first initial element among the N initial elements is not greater than the second initial network indicated by the second initial element among the N initial elements.
  • the number of blocks in the corresponding stage in the structure, and the number of channels in each block in each stage in the first initial network structure is not greater than each block in the corresponding stage in the second initial network structure.
  • the number of channels, the first initial element is deleted from the initial search space.
  • Figure 10 shows N network structures, where N is a positive integer greater than 1.
  • stage 1 includes 1 block, the number of channels of the block is 64
  • stage 2 includes 2 blocks, the number of channels of each block is 128, and stage 3 includes 2 blocks, each The number of channels of the block is 256, and the number of channels in phase 4 is 4 blocks, and the number of channels in each block is 512
  • network structure 2 the number of channels in phase 1 is 1 block, the number of channels in the block is 64, and the number of channels in phase 2 is 1
  • the number of channels in the block is 128, the number of channels in each block is 256 in phase 3, and the number of channels in each block is 512 in phase 4.
  • stage 2 includes 2 blocks with 128 channels, and stage 4 includes 4 blocks.
  • the number of channels is 512 blocks.
  • the copied network structure 2 has the same network structure as the network structure 1, which means that the accuracy of the network structure 2 is lower than that of the network structure 1. You can delete the network structure 2 from the initial search space (or in other words, the network structure 2 corresponds to Elements).
  • the BCSC of the first initial element can be directly compared with the BCSC of the second initial element, and each element in the BCSC of the first initial element is more than the corresponding BCSC of the second initial element.
  • the element of is small, it means that the accuracy of the first initial network structure is lower than that of the second initial network structure, and accordingly, the first initial element can be deleted from the initial search space.
  • S930 Perform clustering on the unevaluated elements in the search space to obtain K clusters.
  • the elements in the same cluster are more similar to the elements in other clusters.
  • the clustering result may also include the center of each cluster, and the center of each cluster may be considered as the most representative element in the cluster.
  • the unevaluated elements in the search space are M elements in the search space.
  • the M elements ie, M A BCSC
  • the specific standardization process can refer to the prior art, which is not limited in this application.
  • the mean value and variance of the M BCSCs in each dimension may be counted.
  • the variance of each dimension can be recorded as among them, Represents the variance of the number of blocks in the second stage, Represents the variance of the number of blocks in the third stage, Represents the variance of the number of blocks in the fourth stage, Represents the variance of the number of blocks in the fifth stage, Represents the variance of the number of channels of the block in the second stage, Represents the variance of the number of channels of the block in the third stage, Represents the variance of the number of channels of the block in the fourth stage, Represents the variance of the number of channels of the block in the fifth stage.
  • the M obtained after standardization can be Perform clustering.
  • the embodiment of the present application does not limit the clustering method used, and the specific clustering method can refer to the prior art, which will not be repeated here.
  • the K-means algorithm can be used to calculate the normalized M Perform clustering.
  • the network structure is the target network structure, which can be based on the network structure.
  • the structure builds a neural network.
  • S950 Model an unassessed network structure in the search space according to an assessment result of the assessed network structure to obtain a model of the unassessed network structure.
  • the unevaluated network structure among the M network structures may be modeled to obtain the model of the unevaluated network structure.
  • the evaluated network structure may include K network structures indicated by elements corresponding to the centers of the K clusters.
  • the elements in the same cluster are more similar to the elements in other clusters.
  • the clustering result may also include the center of each cluster, and the center of each cluster may be considered as the most representative element in the cluster.
  • the unevaluated elements in the search space are MK elements in the search space; in the process of the second iteration, the elements in the search space The unevaluated elements of are M-2*K elements in the search space. In the subsequent iteration process, the unevaluated elements in the search space are all M elements minus the evaluated elements in the search space element.
  • S970 Select K clusters from the P clusters according to the model of the unassessed network structure.
  • the accuracy of the unevaluated network structure can be evaluated according to the model of the unevaluated network structure to obtain the accuracy of the evaluation; and then K selected clusters with the highest accuracy rate from the P clusters The network structure is evaluated.
  • the unevaluated elements in the search space are always elements other than the evaluated elements.
  • the unevaluated elements in the search space are other elements in the search space except the K elements that have been evaluated, that is, the unevaluated elements at this time are MK elements .
  • the subsequent iteration process is similar to the first iteration, so I won't repeat it here.
  • the unevaluated elements in the search space can be clustered into P clusters according to the method in the above S960.
  • the unevaluated elements can be selected from P clusters according to the model of the unevaluated elements. Select K elements in the cluster, and evaluate these K elements according to the method in S940 above, and P and K are positive integers.
  • the network structure is the target network structure, and the neural network can be constructed according to the network structure.
  • the network structure is the target network structure, and the neural network can be constructed according to the network structure.
  • FIG. 11 shows a schematic flowchart of an image processing method 1100 provided by an embodiment of the present application.
  • the method may be executed by a device or device capable of image processing.
  • the method may be executed by a terminal device, a computer, a server, or the like.
  • the target neural network used in the image processing method 1100 in FIG. 11 may be constructed by the method 700 in FIG. 7 or the method 900 in FIG. Not limited.
  • the image to be processed may be an image captured by a terminal device (or other device or equipment such as a computer or server) through a camera, or the image to be processed may also be an image from a terminal device (or other device or device such as a computer or server).
  • Device internally obtained images (for example, images stored in an album of the terminal device, or images obtained by the terminal device from the cloud), which is not limited in the embodiment of the present application.
  • S1120 Classify the image to be processed according to the target neural network to obtain a classification result of the image to be processed.
  • the target neural network is constructed by a target network structure, and the target network structure is obtained by the method 700 in FIG. 7 or the method 900 in FIG. 9 described above.
  • the target neural network used in the above image processing method performs image classification
  • the target neural network needs to be trained according to the training image, and the trained target neural network can be used to classify the image to be processed.
  • the target neural network can be constructed using the target network structure obtained by the method 700 in FIG. 7 or the method 900 in FIG. 9, and then the target neural network is trained according to the training image. After the training is completed The target neural network can be used to classify the image to be processed.
  • the target neural network is constructed using the target network structure obtained by the method 700 in FIG. 7 or the method 900 in FIG. 9, it is more in line with or close to the application requirements of the neural network, and such a neural network is used.
  • Image classification can achieve better image classification results (for example, more accurate classification results, etc.).
  • FIG. 12 is a schematic diagram of the hardware structure of the apparatus for constructing a neural network provided by an embodiment of the present application.
  • the apparatus 3000 for constructing a neural network shown in FIG. 12 includes a memory 3001, a processor 3002, a communication interface 3003, and a bus 3004.
  • the memory 3001, the processor 3002, and the communication interface 3003 implement communication connections between each other through the bus 3004.
  • the memory 3001 may be a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM).
  • the memory 3001 may store a program. When the program stored in the memory 3001 is executed by the processor 3002, the processor 3002 is configured to execute each step of the method for constructing a neural network in the embodiment of the present application.
  • the processor 3002 may adopt a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), a graphics processing unit (GPU), or one or more
  • the integrated circuit is used to execute related programs to implement the method for constructing a neural network in the method embodiment of the present application.
  • the processor 3002 may also be an integrated circuit chip with signal processing capability. For example, it may be the chip shown in FIG. 5.
  • each step of the method for constructing a neural network of the present application can be completed by an integrated logic circuit of hardware in the processor 3002 or instructions in the form of software.
  • the above-mentioned processor 3002 may also be a general-purpose processor, a digital signal processing (digital signal processing, DSP), an application specific integrated circuit (ASIC), an off-the-shelf programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, Discrete gates or transistor logic devices, discrete hardware components.
  • DSP digital signal processing
  • ASIC application specific integrated circuit
  • FPGA field programmable gate array
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor.
  • the software module can be located in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers.
  • the storage medium is located in the memory 3001, and the processor 3002 reads the information in the memory 3001, combines its hardware to complete the functions required by the units included in the device for constructing a neural network, or executes the neural network construction of the method embodiment of the application. method.
  • the communication interface 3003 uses a transceiver device such as but not limited to a transceiver to implement communication between the device 3000 and other devices or communication networks. For example, the information of the neural network to be constructed and the training data needed in the process of constructing the neural network can be obtained through the communication interface 3003.
  • a transceiver device such as but not limited to a transceiver to implement communication between the device 3000 and other devices or communication networks. For example, the information of the neural network to be constructed and the training data needed in the process of constructing the neural network can be obtained through the communication interface 3003.
  • the bus 3004 may include a path for transferring information between various components of the device 3000 (for example, the memory 3001, the processor 3002, and the communication interface 3003).
  • FIG. 13 is a schematic diagram of the hardware structure of an image processing apparatus according to an embodiment of the present application.
  • the image processing apparatus 4000 shown in FIG. 13 includes a memory 4001, a processor 4002, a communication interface 4003, and a bus 4004. Among them, the memory 4001, the processor 4002, and the communication interface 4003 implement communication connections between each other through the bus 4004.
  • the memory 4001 may be ROM, static storage device and RAM.
  • the memory 4001 may store a program. When the program stored in the memory 4001 is executed by the processor 4002, the processor 4002 and the communication interface 4003 are used to execute each step of the image processing method of the embodiment of the present application.
  • the processor 4002 may adopt a general-purpose CPU, a microprocessor, an ASIC, a GPU, or one or more integrated circuits to execute related programs to realize the functions required by the units in the image processing apparatus of the embodiment of the present application. Or execute the image processing method in the method embodiment of this application.
  • the processor 4002 may also be an integrated circuit chip with signal processing capability. For example, it may be the chip shown in FIG. 5. In the implementation process, each step of the image processing method of the embodiment of the present application can be completed by an integrated logic circuit of hardware in the processor 4002 or instructions in the form of software.
  • the aforementioned processor 4002 may also be a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component.
  • the methods, steps, and logical block diagrams disclosed in the embodiments of the present application can be implemented or executed.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor.
  • the software module can be located in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers.
  • the storage medium is located in the memory 4001, and the processor 4002 reads the information in the memory 4001, and combines its hardware to complete the functions required by the units included in the image processing apparatus of the embodiment of the present application, or perform the image processing of the method embodiment of the present application method.
  • the communication interface 4003 uses a transceiver device such as but not limited to a transceiver to implement communication between the device 4000 and other devices or a communication network.
  • a transceiver device such as but not limited to a transceiver to implement communication between the device 4000 and other devices or a communication network.
  • the image to be processed can be acquired through the communication interface 4003.
  • the bus 4004 may include a path for transferring information between various components of the device 4000 (for example, the memory 4001, the processor 4002, and the communication interface 4003).
  • FIG. 14 is a schematic diagram of the hardware structure of a neural network training device according to an embodiment of the present application. Similar to the aforementioned device 3000 and device 4000, the neural network training device 5000 shown in FIG. 14 includes a memory 5001, a processor 5002, a communication interface 5003, and a bus 5004. Among them, the memory 5001, the processor 5002, and the communication interface 5003 implement communication connections between each other through the bus 5004.
  • the neural network After the neural network is constructed by the device for constructing a neural network shown in FIG. 12, the neural network can be trained by the neural network training device 5000 shown in FIG. 14, and the trained neural network can be used to execute this application.
  • the device shown in FIG. 14 can obtain training data and the neural network to be trained from the outside through the communication interface 5003, and then the processor trains the neural network to be trained according to the training data.
  • the device 3000, the device 4000, and device 5000 only show a memory, a processor, and a communication interface, in the specific implementation process, those skilled in the art should understand that the device 3000, the device 4000, and the device 5000 may also Including other devices necessary for normal operation. At the same time, according to specific needs, those skilled in the art should understand that the device 3000, the device 4000, and the device 5000 may also include hardware devices that implement other additional functions. In addition, those skilled in the art should understand that the device 3000, the device 4000, and the device 5000 may also only include the components necessary to implement the embodiments of the present application, and not necessarily include all the components shown in FIG. 12, FIG. 13 and FIG. 14.
  • the processor in the embodiment of the present application may be a central processing unit (central processing unit, CPU), and the processor may also be other general-purpose processors, digital signal processors (digital signal processors, DSP), and application-specific integrated circuits. (application specific integrated circuit, ASIC), ready-made programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the memory in the embodiments of the present application may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory.
  • the non-volatile memory can be read-only memory (ROM), programmable read-only memory (programmable ROM, PROM), erasable programmable read-only memory (erasable PROM, EPROM), and electrically available Erase programmable read-only memory (electrically EPROM, EEPROM) or flash memory.
  • the volatile memory may be random access memory (RAM), which is used as an external cache.
  • RAM random access memory
  • static random access memory static random access memory
  • DRAM dynamic random access memory
  • DRAM synchronous dynamic random access memory
  • Access memory synchronous DRAM, SDRAM
  • double data rate synchronous dynamic random access memory double data rate SDRAM, DDR SDRAM
  • enhanced synchronous dynamic random access memory enhanced SDRAM, ESDRAM
  • synchronous connection dynamic random access memory Take memory (synchlink DRAM, SLDRAM) and direct memory bus random access memory (direct rambus RAM, DR RAM).
  • the foregoing embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination.
  • the above-mentioned embodiments may be implemented in the form of a computer program product in whole or in part.
  • the computer program product includes one or more computer instructions or computer programs.
  • the computer instructions or computer programs are loaded or executed on the computer, the processes or functions described in the embodiments of the present application are generated in whole or in part.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.
  • the computer instructions may be transmitted from a website, computer, server, or data center.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more sets of available media.
  • the usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium.
  • the semiconductor medium may be a solid state drive.
  • At least one refers to one or more, and “multiple” refers to two or more.
  • the following at least one item (a)” or similar expressions refers to any combination of these items, including any combination of a single item (a) or a plurality of items (a).
  • at least one item (a) of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple .
  • the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not correspond to the embodiments of the present application.
  • the implementation process constitutes any limitation.
  • the disclosed system, device, and method may be implemented in other ways.
  • the device embodiments described above are merely illustrative, for example, the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or It can be integrated into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the technical solution of the present application essentially or the part that contributes to the existing technology or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program code .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Computational Linguistics (AREA)
  • Databases & Information Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Supply And Distribution Of Alternating Current (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

人工智能领域中的一种构建神经网络的方法与装置、及图像处理方法与装置。其中，该构建神经网络的方法包括：根据目标神经网络的应用需求，构建搜索空间，所述搜索空间包括M个元素，所述M个元素用于指示M个网络结构，所述M个元素中的每一个元素包括对应的网络结构中的阶段中的块的个数和所述块的通道数，M为正整数（S710）；根据搜索空间中未评估元素的分布关系从所述M个网络结构中选择目标网络结构（S720）。该方法能够高效地构建满足性能要求的神经网络。

Description

构建神经网络的方法与装置、及图像处理方法与装置
本申请要求于2019年09月24日提交中国专利局、申请号为201910904314.7、申请名称为“构建神经网络的方法与装置、及图像处理方法与装置”的中国专利申请，以及于2020年06月24日提交中国专利局、申请号为202010588884.2、申请名称为“构建神经网络的方法与装置、及图像处理方法与装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及人工智能领域，并且更具体地，涉及一种构建神经网络的方法与装置、及图像处理方法与装置。
背景技术
人工智能(artificial intelligence,AI)是利用数字计算机或者数字计算机控制的机器模拟、延伸和扩展人的智能，感知环境、获取知识并使用知识获得最佳结果的理论、方法、技术及应用系统。换句话说，人工智能是计算机科学的一个分支，它企图了解智能的实质，并生产出一种新的能以人类智能相似的方式作出反应的智能机器。人工智能也就是研究各种智能机器的设计原理与实现方法，使机器具有感知、推理与决策的功能。人工智能领域的研究包括机器人，自然语言处理，计算机视觉，决策与推理，人机交互，推荐与搜索，AI基础理论等。
随着人工智能技术的快速发展，神经网络(例如，卷积神经网络)的性能得到了持续的提升，神经网络在图像、视频以及语音等多种媒体信号的处理与分析中也取得了很大的成就。一个性能优良的神经网络往往拥有精妙的网络结构，但在实际应用中，由于训练集、指标要求及应用的目标不同，往往不能直接用现有的网络结构。
目前，一种常见的做法是，对于不同的任务，基于现有的网络结构进行调整，然而通过调整往往很难得到一个性能优异的网络结构；另一种常见的做法是，基于自动化机器学习(automated machine learning，AutoML)对网络结构进行自动搜索，但AutoML是一个从零开始设计网络结构的方法，任务的复杂度非常高。
因此，如何高效地构建出满足性能要求的神经网络，成为一个亟需解决的技术问题。
发明内容
本申请提供一种构建神经网络的方法与装置、及图像处理方法与装置，能够高效地构建满足性能要求的神经网络。
第一方面，提供了一种构建神经网络的方法，该方法包括：根据目标神经网络的应用需求，构建搜索空间，所述搜索空间包括M个元素，所述M个元素用于指示M个网络结构，所述M个元素中的每一个元素包括对应的网络结构中的阶段中的块的个数和所 述块的通道数，M为正整数；根据搜索空间中未评估元素的分布关系从所述M个网络结构中选择目标网络结构。
在本申请实施例中，根据未评估元素的分布关系可以挑选出搜索空间中的具有代表性的元素，此时，根据该具有代表性的元素，能够高效地从所述M个网络结构中选择出目标网络结构，从而能够高效地构建满足性能要求的神经网络。
可选地，所述M个元素中的每一个元素包括对应的所述网络结构是指，所述M个元素中的每一个元素指示的所述网络结构。
相应地，对于所述M个元素中的每一个元素，所述每一个元素包括该元素指示的所述网络结构中的阶段中的块的个数和所述块的通道数。
结合第一方面，在第一方面的某些实现方式中，所述目标神经网络的应用需求包括所述目标神经网络的运行速度、所述目标神经网络的参数量或所述目标神经网络的结构要求，其中，所述结构要求包括所述目标神经网络结构中的每个阶段的块的个数和每个所述块的通道数。
在本申请实施例中，根据所述目标神经网络的运行速度、所述目标神经网络的参数量或所述目标神经网络的结构要求构建搜索空间，可以在构建搜索空间的过程中就筛选掉一些低性能的网络结构，能够提高构建神经网络的效率。
结合第一方面，在第一方面的某些实现方式中，所述根据目标神经网络的应用需求，构建搜索空间，包括：根据所述目标神经网络的应用需求，构建初始搜索空间，所述初始搜索空间包括N个初始元素，所述N个初始元素用于指示N个初始网络结构，所述N个初始元素中的每一个元素包括对应的初始网络结构中的阶段中的块的个数和所述块的通道数，N为大于或等于M的正整数；根据预设规则对所述N个初始元素指示的所述N个初始网络结构进行筛选，得到所述搜索空间中的所述M个元素，所述预设规则包括：若所述N个初始元素中的第一初始元素指示的第一初始网络结构中的每一个阶段中的块的个数不大于所述N个初始元素中第二初始元素指示的第二初始网络结构中对应阶段中的块的个数，且所述第一初始网络结构中的每一个阶段中的每个块的通道数均不大于所述第二初始网络结构中对应阶段中的每个块的通道数，则从所述初始搜索空间中删除所述第一初始元素。
在本申请实施例中，根据上述预设规则，在不对所述初始搜索空间中的N个初始元素指示的初始网络结构进行训练的情况下，仅根据所述N个初始网络结构的结构信息就可以筛选出所述N个初始网络结构中的低性能网络结构，因此，能够提高构建神经网络的效率。
结合第一方面，在第一方面的某些实现方式中，所述根据搜索空间中未评估元素的分布关系从所述M个网络结构中选择目标网络结构，包括：根据所述未评估元素的分布关系，确定所述未评估元素中的K个元素，K为小于M的正整数；根据所述K个元素从所述M个网络结构中选择目标网络结构。
结合第一方面，在第一方面的某些实现方式中，所述根据所述K个元素从所述M个网络结构中选择目标网络结构，包括：对所述未评估元素中的K个元素指示的K个网络结构进行评估，得到已评估元素的评估结果，所述已评估元素的评估结果包括所述K个网络结构的评估结果；根据已评估元素的评估结果，从所述M个网络结构中选择目标网 络结构。
在本申请实施例中，根据已评估的网络结构的评估结果，在所述M个网络结构中选择满足预设要求的目标网络结构，可以充分地利用已评估的网络结构的评估结果，从而能够提高构建神经网络的效率。
结合第一方面，在第一方面的某些实现方式中，所述根据已评估元素的评估结果，从所述M个网络结构中选择目标网络结构，包括：根据所述已评估元素的评估结果对第一未评估元素进行建模，得到所述第一未评估元素的模型，所述第一未评估元素包括所述搜索空间中除所述已评估元素之外的其他元素；根据所述第一未评估元素的模型，从所述M个网络结构中选择目标网络结构。
在本申请实施例中，根据所述已评估元素的评估结果对第一未评估元素进行建模，得到所述第一未评估元素的模型；使用所述第一未评估元素的模型，有助于挑选出搜索空间中的具有代表性的元素，此时，根据该具有代表性的元素，能够高效地从所述M个网络结构中选择出目标网络结构，从而能够高效地构建满足性能要求的神经网络。
结合第一方面，在第一方面的某些实现方式中，在根据搜索空间中未评估元素的分布关系从所述M个网络结构中未选择出目标网络结构的情况下，所述方法还包括：根据所述第一未评估元素的分布关系与基于已评估元素构建的第一未评估元素的模型，从所述M个网络结构中选择目标网络结构。
结合第一方面，在第一方面的某些实现方式中，所述根据所述第一未评估元素的分布关系与基于已评估元素构建的第一未评估元素的模型，从所述M个网络结构中选择目标网络结构，包括：根据所述第一未评估元素的分布关系，确定所述第一未评估元素中的L个元素，L为小于M的正整数；根据所述L个元素与基于已评估元素构建的第一未评估元素的模型从所述M个网络结构中选择目标网络结构。
结合第一方面，在第一方面的某些实现方式中，所述根据所述L个元素与基于已评估元素构建的第一未评估元素的模型从所述M个网络结构中选择目标网络结构，包括：根据所述基于已评估元素构建的第一未评估元素的模型，从所述L个元素中确定Q个元素，Q为小于L的正整数；对所述Q个元素指示的Q个网络结构进行评估，得到第一已评估元素的评估结果，所述第一已评估元素的评估结果包括所述K个网络结构的评估结果和所述Q个网络结构的评估结果；根据第一已评估元素的评估结果与第二未评估元素的分布关系，从所述M个网络结构中选择目标网络结构，所述第二未评估元素包括所述搜索空间中除所述第一已评估元素之外的其他元素。
结合第一方面，在第一方面的某些实现方式中，所述第一未评估元素的分布关系为所述第一未评估元素的聚类结果，所述L个元素分别为所述第一未评估元素的聚类结果包括的L个簇中的元素。
在本申请实施例中，使用第一未评估元素的聚类结果，有助于挑选出搜索空间中的具有代表性的元素，此时，根据该具有代表性的元素，能够高效地从所述M个网络结构中选择出目标网络结构，从而能够高效地构建满足性能要求的神经网络。
结合第一方面，在第一方面的某些实现方式中，所述L个元素分别为所述L个簇的中心对应的L个元素。
在本申请实施例中，所述L个簇的中心对应的L个元素指示的L个网络结构是所述 搜索空间中最具代表性的网络结构，基于从所述L个元素中选择的Q个元素所指示的Q个网络结构的评估结果，在所述M个网络结构中选择满足预设要求的目标网络结构，可以高效地从所述M个网络结构中选择出满足预设要求的目标网络结构，从而能够提高构建神经网络的效率。
结合第一方面，在第一方面的某些实现方式中，在根据所述第一未评估元素的分布关系与基于已评估元素构建的第一未评估元素的模型，从所述M个网络结构中未选择出目标网络结构的情况下，所述方法还包括：根据所述第二未评估元素的分布关系与基于第一已评估元素构建的第二未评估元素的模型，从所述M个网络结构中重新选择目标网络结构。
在本申请实施例中，在根据所述第一未评估元素的分布关系与基于已评估元素构建的第一未评估元素的模型，从所述M个网络结构中未选择出目标网络结构的情况下，根据所述第二未评估元素的分布关系与基于第一已评估元素构建的第二未评估元素的模型，从所述M个网络结构中重新选择目标网络结构，可以充分地利用已评估的网络结构的评估结果，从而能够提高构建神经网络的效率。
结合第一方面，在第一方面的某些实现方式中，所述基于已评估元素构建的第一未评估元素的模型，包括：所述第一未评估元素的模型是根据已评估的网络结构的评估结果及下述公式得到的：
Figure PCTCN2020116673-appb-000001
和/或(g(x)-l(x)) 2；
其中，x表示所述搜索空间中未评估的网络结构，y *表示精度阈值，g(x)表示精度大于精度阈值y *的网络结构x的条件概率密度函数，
Figure PCTCN2020116673-appb-000002
l(x)表示精度小于或等于精度阈值y *的网络结构x的条件概率密度函数，
Figure PCTCN2020116673-appb-000003
G *(x|x i)表示混合高斯分布，
Figure PCTCN2020116673-appb-000004
κ(x,x i)表示x与x i之间的欧式距离，κ*(x,x i)表示由κ(x,x i)组成的距离函数，
Figure PCTCN2020116673-appb-000005
x i表示所述已评估的网络结构中的第i个网络结构，y i表示所述已评估的网络结构中的第i个网络结构的精度，ω i表示G *(x|x i)对应的权重，Z表示归一化因子，σ表示G *(x|x i)的一个超参，i为正整数，e为自然对数函数的底数。
结合第一方面，在第一方面的某些实现方式中，所述基于已评估元素构建的第一未评估元素的模型，包括：所述第一未评估元素的模型是根据已评估的网络结构的评估结果及下述公式得到的：
Figure PCTCN2020116673-appb-000006
其中，x表示所述搜索空间中的未评估元素；
Figure PCTCN2020116673-appb-000007
x i表示所述已评估元素中的第i个元素，y i表示所述已评估元素中的第i个元素的精度；τ＝max(y i)，
Figure PCTCN2020116673-appb-000008
表示期望函数；f(x)为服从高斯分布的随机变量，f(x)的均值μ(x)及f(x)的方差σ(x)与
Figure PCTCN2020116673-appb-000009
跟输入x满足下述关系：
μ(x)＝k T(K+η 2I) -1Y，
σ(x)＝1-k T(K+η 2I) -1k。
其中，n为所述已评估的网络结构的个数，Y为y i组成的向量，Y∈R n，Y i＝y i，k为κ(x,x i)组成的向量，k∈R n，k i＝κ(x,x i)，K为κ(x i,x j)组成的矩阵为，K∈R n×n，K i，j＝κ(x i,x j)，
Figure PCTCN2020116673-appb-000010
σ为一个超参，e为自然对数函数的底数；I为单位矩阵，η也为一个超参，i,j为正整数。
结合第一方面，在第一方面的某些实现方式中，所述未评估元素的分布关系为所述未评估元素的聚类结果，所述K个元素分别为所述未评估元素的聚类结果包括的K个簇中的元素。
在本申请实施例中，使用未评估元素的聚类结果，有助于挑选出搜索空间中的具有代表性的元素，此时，根据该具有代表性的元素，能够高效地从所述M个网络结构中选择出目标网络结构，从而能够高效地构建满足性能要求的神经网络。
结合第一方面，在第一方面的某些实现方式中，所述K个元素分别为所述K个簇的中心对应的K个元素。
在本申请实施例中，所述K个簇的中心对应的K个元素指示的K个网络结构是所述搜索空间中最具代表性的网络结构，基于所述K个簇的中心对应的K个元素指示的K个网络结构的评估结果，在所述M个网络结构中选择满足预设要求的目标网络结构，可以高效地从所述M个网络结构中选择出满足预设要求的目标网络结构，从而能够提高构建神经网络的效率。
第二方面，提供了一种图像处理方法，该方法包括：获取待处理图像；根据目标神经网络对所述待处理图像进行分类，得到所述待处理图像的分类结果；其中，所述目标神经网络由目标网络结构构建而成，所述目标网络结构是通过上述第一方面中的任意一种实现方式中的方法得到的。
应理解，第二方面中的图像处理方法所采用的目标神经网络在进行图像分类之前，还需要再根据训练图像对该目标神经网络进行训练，训练得到的目标神经网络就可以对待处理图像进行分类。
也就是说，可以采用第一方面中的神经网络结构搜索方法得到目标神经网络，接下来，再根据训练图像对该目标神经网络进行训练，训练完成后就可以用该目标神经网络对待处理图像进行分类了。
本申请中，由于目标神经网络是采用上述第一方面的方面构建得到的，比较符合或者贴近神经网络的应用需求，利用这样的神经网络进行图像分类，能够取得较好的图像分类效果(例如，分类结果更准确，等等)。
第三方面，提供了一种构建神经网络的装置，包括：构建单元，用于根据目标神经网络的应用需求，构建搜索空间，所述搜索空间包括M个元素，所述M个元素用于指示M个网络结构，所述M个元素中的每一个元素包括对应的网络结构中的阶段中的块的个数和所述块的通道数，M为正整数；选择单元，用于根据搜索空间中未评估元素的分布关系从所述M个网络结构中选择目标网络结构。
在本申请实施例中，根据未评估元素的分布关系可以挑选出搜索空间中的具有代表性的元素，此时，根据该具有代表性的元素，能够高效地从所述M个网络结构中选择出目标网络结构，从而能够高效地构建满足性能要求的神经网络。
可选地，所述M个元素中的每一个元素包括对应的所述网络结构是指，所述M个元素中的每一个元素指示的所述网络结构。
相应地，对于所述M个元素中的每一个元素，所述每一个元素包括该元素指示的所述网络结构中的阶段中的块的个数和所述块的通道数。
结合第三方面，在第三方面的某些实现方式中，所述目标神经网络的应用需求包括所述目标神经网络的运行速度、所述目标神经网络的参数量或所述目标神经网络的结构要求，其中，所述结构要求包括所述目标神经网络结构中的每个阶段的块的个数和每个所述块的通道数。
在本申请实施例中，根据所述目标神经网络的运行速度、所述目标神经网络的参数量或所述目标神经网络的结构要求构建搜索空间，可以在构建搜索空间的过程中就筛选掉一些低性能的网络结构，能够提高构建神经网络的效率。
结合第三方面，在第三方面的某些实现方式中，所述构建单元具体用于：根据所述目标神经网络的应用需求，构建初始搜索空间，所述初始搜索空间包括N个初始元素，所述N个初始元素用于指示N个初始网络结构，所述N个初始元素中的每一个元素包括对应的初始网络结构中的阶段中的块的个数和所述块的通道数，N为大于或等于M的正整数；根据预设规则对所述N个初始元素指示的所述N个初始网络结构进行筛选，得到所述搜索空间中的所述M个元素，所述预设规则包括：若所述N个初始元素中的第一初始元素指示的第一初始网络结构中的每一个阶段中的块的个数不大于所述N个初始元素中第二初始元素指示的第二初始网络结构中对应阶段中的块的个数，且所述第一初始网络结构中的每一个阶段中的每个块的通道数均不大于所述第二初始网络结构中对应阶段中的每个块的通道数，则从所述初始搜索空间中删除所述第一初始元素。
在本申请实施例中，根据上述预设规则，在不对所述初始搜索空间中的N个初始元素指示的初始网络结构进行训练的情况下，仅根据所述N个初始网络结构的结构信息就可以筛选出所述N个初始网络结构中的低性能网络结构，因此，能够提高构建神经网络的效率。
结合第三方面，在第三方面的某些实现方式中，所述选择单元具体用于：根据所述未评估元素的分布关系，确定所述未评估元素中的K个元素，K为小于M的正整数；根据所述K个元素从所述M个网络结构中选择目标网络结构。
结合第三方面，在第三方面的某些实现方式中，所述选择单元具体用于：对所述未评估元素中的K个元素指示的K个网络结构进行评估，得到已评估元素的评估结果，所述已评估元素的评估结果包括所述K个网络结构的评估结果；根据已评估元素的评估结果，从所述M个网络结构中选择目标网络结构。
在本申请实施例中，根据已评估的网络结构的评估结果，在所述M个网络结构中选择满足预设要求的目标网络结构，可以充分地利用已评估的网络结构的评估结果，从而能够提高构建神经网络的效率。
结合第三方面，在第三方面的某些实现方式中，所述选择单元具体用于：根据所述已评估元素的评估结果对第一未评估元素进行建模，得到所述第一未评估元素的模型，所述第一未评估元素包括所述搜索空间中除所述已评估元素之外的其他元素；根据所述第一未评估元素的模型，从所述M个网络结构中选择目标网络结构。
在本申请实施例中，根据所述已评估元素的评估结果对第一未评估元素进行建模，得到所述第一未评估元素的模型；使用所述第一未评估元素的模型，有助于挑选出搜索空间中的具有代表性的元素，此时，根据该具有代表性的元素，能够高效地从所述M个网络结构中选择出目标网络结构，从而能够高效地构建满足性能要求的神经网络。
结合第三方面，在第三方面的某些实现方式中，在根据搜索空间中未评估元素的分布关系从所述M个网络结构中未选择出目标网络结构的情况下，所述选择单元还用于：根据所述第一未评估元素的分布关系与基于已评估元素构建的第一未评估元素的模型，从所述M个网络结构中选择目标网络结构。
结合第三方面，在第三方面的某些实现方式中，所述选择单元具体用于：根据所述第一未评估元素的分布关系，确定所述第一未评估元素中的L个元素，L为小于M的正整数；根据所述L个元素与基于已评估元素构建的第一未评估元素的模型从所述M个网络结构中选择目标网络结构。
结合第三方面，在第三方面的某些实现方式中，所述选择单元具体用于：根据所述基于已评估元素构建的第一未评估元素的模型，从所述L个元素中确定Q个元素，Q为小于L的正整数；对所述Q个元素指示的Q个网络结构进行评估，得到第一已评估元素的评估结果，所述第一已评估元素的评估结果包括所述K个网络结构的评估结果和所述Q个网络结构的评估结果；根据第一已评估元素的评估结果与第二未评估元素的分布关系，从所述M个网络结构中选择目标网络结构，所述第二未评估元素包括所述搜索空间中除所述第一已评估元素之外的其他元素。
结合第三方面，在第三方面的某些实现方式中，所述第一未评估元素的分布关系为所述第一未评估元素的聚类结果，所述L个元素分别为所述第一未评估元素的聚类结果包括的L个簇中的元素。
在本申请实施例中，使用第一未评估元素的聚类结果，有助于挑选出搜索空间中的具有代表性的元素，此时，根据该具有代表性的元素，能够高效地从所述M个网络结构中选择出目标网络结构，从而能够高效地构建满足性能要求的神经网络。
结合第三方面，在第三方面的某些实现方式中，所述L个元素分别为所述L个簇的中心对应的L个元素。
在本申请实施例中，所述L个簇的中心对应的L个元素指示的L个网络结构是所述搜索空间中最具代表性的网络结构，基于从所述L个元素中确定的Q个元素所指示的Q个网络结构的评估结果，在所述M个网络结构中选择满足预设要求的目标网络结构，可以高效地从所述M个网络结构中选择出满足预设要求的目标网络结构，从而能够提高构建神经网络的效率。
结合第三方面，在第三方面的某些实现方式中，在根据所述第一未评估元素的分布关系与基于已评估元素构建的第一未评估元素的模型，从所述M个网络结构中未选择出目标网络结构的情况下，所述选择单元还用于：根据所述第二未评估元素的分布关系与基于第一已评估元素构建的第二未评估元素的模型，从所述M个网络结构中重新选择目标网络结构。
在本申请实施例中，在根据所述第一未评估元素的分布关系与基于已评估元素构建的第一未评估元素的模型，从所述M个网络结构中未选择出目标网络结构的情况下，根 据所述第二未评估元素的分布关系与基于第一已评估元素构建的第二未评估元素的模型，从所述M个网络结构中重新选择目标网络结构，可以充分地利用已评估的网络结构的评估结果，从而能够提高构建神经网络的效率。
结合第三方面，在第三方面的某些实现方式中，所述选择单元具体用于：所述第一未评估元素的模型是根据已评估的网络结构的评估结果及下述公式得到的：
Figure PCTCN2020116673-appb-000011
和/或(g(x)-l(x)) 2；
其中，x表示所述搜索空间中未评估的网络结构，y *表示精度阈值，g(x)表示精度大于精度阈值y *的网络结构x的条件概率密度函数，
Figure PCTCN2020116673-appb-000012
l(x)表示精度小于或等于精度阈值y *的网络结构x的条件概率密度函数，
Figure PCTCN2020116673-appb-000013
G *(x|x i)表示混合高斯分布，
Figure PCTCN2020116673-appb-000014
κ(x,x i)表示x与x i之间的欧式距离，κ*(x,x i)表示由κ(x,x i)组成的距离函数，
Figure PCTCN2020116673-appb-000015
x i表示所述已评估的网络结构中的第i个网络结构，y i表示所述已评估的网络结构中的第i个网络结构的精度，ω i表示G *(x|x i)对应的权重，Z表示归一化因子，σ表示G *(x|x i)的一个超参，i为正整数，e为自然对数函数的底数。
结合第三方面，在第三方面的某些实现方式中，所述选择单元具体用于：所述第一未评估元素的模型是根据已评估的网络结构的评估结果及下述公式得到的：
Figure PCTCN2020116673-appb-000016
其中，x表示所述搜索空间中的未评估元素；
Figure PCTCN2020116673-appb-000017
x i表示所述已评估元素中的第i个元素，y i表示所述已评估元素中的第i个元素的精度；τ＝max(y i)，
Figure PCTCN2020116673-appb-000018
表示期望函数；f(x)为服从高斯分布的随机变量，f(x)的均值μ(x)及f(x)的方差σ(x)与
Figure PCTCN2020116673-appb-000019
跟输入x满足下述关系：
μ(x)＝k T(K+η 2I) -1Y，
σ(x)＝1-k T(K+η 2I) -1k。
其中，n为所述已评估的网络结构的个数，Y为y i组成的向量，Y∈R n，Y i＝y i，k为κ(x,x i)组成的向量，k∈R n，k i＝κ(x,x i)，K为κ(x i,x j)组成的矩阵为，K∈R n×n，K i，j＝κ(x i,x j)，
Figure PCTCN2020116673-appb-000020
σ为一个超参，e为自然对数函数的底数；I为单位矩阵，η也为一个超参，i,j为正整数。
结合第三方面，在第三方面的某些实现方式中，所述未评估元素的分布关系为所述未评估元素的聚类结果，所述K个元素分别为所述未评估元素的聚类结果包括的K个簇中的元素。
在本申请实施例中，使用未评估元素的聚类结果，有助于挑选出搜索空间中的具有代表性的元素，此时，根据该具有代表性的元素，能够高效地从所述M个网络结构中选择出目标网络结构，从而能够高效地构建满足性能要求的神经网络。
结合第三方面，在第三方面的某些实现方式中，所述K个元素分别为所述K个簇的中心对应的K个元素。
在本申请实施例中，所述K个簇的中心对应的K个元素指示的K个网络结构是所述搜索空间中最具代表性的网络结构，基于所述K个簇的中心对应的K个元素指示的K个网络结构的评估结果，在所述M个网络结构中选择满足预设要求的目标网络结构，可以高效地从所述M个网络结构中选择出满足预设要求的目标网络结构，从而能够提高构建神经网络的效率。
第四方面，提供了一种图像处理装置，包括：获取单元，用于获取待处理图像；图像处理单元，用于根据目标神经网络对所述待处理图像进行分类，得到所述待处理图像的分类结果；其中，所述目标神经网络由目标网络结构构建而成，所述目标网络结构是通过上述第一方面中的任意一种实现方式中的方法得到的。
应理解，第二方面中的图像处理方法所采用的目标神经网络在进行图像分类之前，还需要再根据训练图像对该目标神经网络进行训练，训练得到的目标神经网络就可以对待处理图像进行分类。
也就是说，可以采用第一方面中的神经网络结构搜索方法得到目标神经网络，接下来，再根据训练图像对该目标神经网络进行训练，训练完成后就可以用该目标神经网络对待处理图像进行分类了。
本申请中，由于目标神经网络是采用上述第一方面的方面构建得到的，比较符合或者贴近神经网络的应用需求，利用这样的神经网络进行图像分类，能够取得较好的图像分类效果(例如，分类结果更准确，等等)。
应理解，在上述第一方面中对相关内容的扩展、限定、解释和说明也适用于第二方面、第三方面和第四方面中相同的内容。
第五方面，提供了一种构建神经网络的装置，该装置包括：存储器，用于存储程序；处理器，用于执行所述存储器存储的程序，当所述存储器存储的程序被执行时，所述处理器用于执行第一方面中的任意一种实现方式中的方法。
上述第五方面中的处理器既可以是中央处理器(central processing unit，CPU)，也可以是CPU与神经网络运算处理器的组合，这里的神经网络运算处理器可以包括图形处理器(graphics processing unit，GPU)、神经网络处理器(neural-network processing unit，NPU)和张量处理器(tensor processing unit，TPU)等等。其中，TPU是谷歌(google)为机器学习全定制的人工智能加速器专用集成电路。
第六方面，提供了一种图像处理装置，该装置包括：存储器，用于存储程序；处理器，用于执行所述存储器存储的程序，当所述存储器存储的程序被执行时，所述处理器用于执行第二方面中的任意一种实现方式中的方法。
上述第六方面中的处理器既可以是中央处理器(central processing unit，CPU)，也可以是CPU与神经网络运算处理器的组合，这里的神经网络运算处理器可以包括图形处理器(graphics processing unit，GPU)、神经网络处理器(neural-network processing unit，NPU)和张量处理器(tensor processing unit，TPU)等等。其中，TPU是谷歌(google)为机器学习全定制的人工智能加速器专用集成电路。
第七方面，提供一种计算机可读介质，该计算机可读介质存储用于设备执行的程序代码，该程序代码包括用于执行第一方面或第二方面中的任意一种实现方式中的方法。
第八方面，提供一种包含指令的计算机程序产品，当该计算机程序产品在计算机上运 行时，使得计算机执行上述第一方面或第二方面中的任意一种实现方式中的方法。
第九方面，提供一种芯片，所述芯片包括处理器与数据接口，所述处理器通过所述数据接口读取存储器上存储的指令，执行上述第一方面或第二方面中的任意一种实现方式中的方法。
可选地，作为一种实现方式，所述芯片还可以包括存储器，所述存储器中存储有指令，所述处理器用于执行所述存储器上存储的指令，当所述指令被执行时，所述处理器用于执行第一方面或第二方面中的任意一种实现方式中的方法。
上述芯片具体可以是现场可编程门阵列(field－programmable gate array，FPGA)或者专用集成电路(application-specific integrated circuit，ASIC)。
在本申请实施例中，根据未评估元素的分布关系可以挑选出搜索空间中的具有代表性的元素，此时，根据该具有代表性的元素，能够高效地从所述M个网络结构中选择出目标网络结构，从而能够高效地构建满足性能要求的神经网络。
附图说明
图1是本申请实施例提供的一种人工智能主体框架示意图。
图2为本申请实施例提供的一种系统架构的结构示意图。
图3为本申请实施例提供的一种卷积神经网络的结构示意图。
图4为本申请实施例提供的另一种卷积神经网络的结构示意图。
图5为本申请实施例提供的一种芯片的硬件结构示意图。
图6为本申请实施例提供的一种系统架构的示意图。
图7是本申请实施例的构建神经网络的方法的示意性流程图。
图8是本申请一个实施例提供的残差块的示意性框图。
图9是本申请实施例的构建神经网络的方法的示意性流程图。
图10是本申请一个实施例提供的搜索空间中的网络结构的示意性框图。
图11是本申请实施例的图像处理方法的示意性流程图。
图12是本申请实施例的构建神经网络的装置的示意性框图。
图13是本申请实施例的图像处理装置的示意性框图。
图14是本申请实施例的神经网络训练装置的示意性框图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
图1示出一种人工智能主体框架示意图，该主体框架描述了人工智能系统总体工作流程，适用于通用的人工智能领域需求。
下面从“智能信息链”(水平轴)和“信息技术(information technology，IT)价值链”(垂直轴)两个维度对上述人工智能主题框架进行详细的阐述。
“智能信息链”反映从数据的获取到处理的一列过程。举例来说，可以是智能信息感知、 智能信息表示与形成、智能推理、智能决策、智能执行与输出的一般过程。在这个过程中，数据经历了“数据—信息—知识—智慧”的凝练过程。
“IT价值链”从人智能的底层基础设施、信息(提供和处理技术实现)到系统的产业生态过程，反映人工智能为信息技术产业带来的价值。
(1)基础设施：
基础设施为人工智能系统提供计算能力支持，实现与外部世界的沟通，并通过基础平台实现支撑。
基础设施可以通过传感器与外部沟通，基础设施的计算能力可以由智能芯片提供。
这里的智能芯片可以是中央处理器(central processing unit，CPU)、神经网络处理器(neural-network processing unit，NPU)、图形处理器(graphics processing unit，GPU)、专门应用的集成电路(application specific integrated circuit，ASIC)以及现场可编程门阵列(field programmable gate array，FPGA)等硬件加速芯片。
基础设施的基础平台可以包括分布式计算框架及网络等相关的平台保障和支持，可以包括云存储和计算、互联互通网络等。
例如，对于基础设施来说，可以通过传感器和外部沟通获取数据，然后将这些数据提供给基础平台提供的分布式计算系统中的智能芯片进行计算。
(2)数据：
基础设施的上一层的数据用于表示人工智能领域的数据来源。该数据涉及到图形、图像、语音、文本，还涉及到传统设备的物联网数据，包括已有系统的业务数据以及力、位移、液位、温度、湿度等感知数据。
(3)数据处理：
上述数据处理通常包括数据训练，机器学习，深度学习，搜索，推理，决策等处理方式。
其中，机器学习和深度学习可以对数据进行符号化和形式化的智能信息建模、抽取、预处理、训练等。
推理是指在计算机或智能系统中，模拟人类的智能推理方式，依据推理控制策略，利用形式化的信息进行机器思维和求解问题的过程，典型的功能是搜索与匹配。
决策是指智能信息经过推理后进行决策的过程，通常提供分类、排序、预测等功能。
(4)通用能力：
对数据经过上面提到的数据处理后，进一步基于数据处理的结果可以形成一些通用的能力，比如可以是算法或者一个通用系统，例如，翻译，文本的分析，计算机视觉的处理，语音识别，图像的识别等等。
(5)智能产品及行业应用：
智能产品及行业应用指人工智能系统在各领域的产品和应用，是对人工智能整体解决方案的封装，将智能信息决策产品化、实现落地应用，其应用领域主要包括：智能制造、智能交通、智能家居、智能医疗、智能安防、自动驾驶，平安城市，智能终端等。
本申请实施例可以应用在人工智能中的很多领域，例如，智能制造、智能交通、智能家居、智能医疗、智能安防、自动驾驶，平安城市等领域。
具体地，本申请实施例可以具体应用在自动驾驶、图像分类、图像检索、图像语义分 割、图像质量增强、图像超分辨率和自然语言处理等需要使用(深度)神经网络的领域。
下面对相册图片分类和自动驾驶这两种应用场景进行简单的介绍。
相册图片分类：
具体地，当用户在终端设备(例如，手机)或者云盘上存储了大量的图片时，通过对相册中图像进行识别可以方便用户或者系统对相册进行分类管理，提升用户体验。
利用本申请实施例的构建神经网络的方法，构建得到适用于相册分类的神经网络，然后再根据训练图片库中的训练图片对神经网络进行训练，就可以得到相册分类神经网络。接下来就可以利用该相册分类神经网络对图片进行分类，从而为不同的类别的图片打上标签，便于用户查看和查找。另外，这些图片的分类标签也可以提供给相册管理系统进行分类管理，节省用户的管理时间，提高相册管理的效率，提升用户体验。
自动驾驶场景下的物体识别：
自动驾驶中有大量的传感器数据需要处理，深度神经网络凭借着其强大的能力在自动驾驶中发挥着重要的作用。然而手工设计相应的数据处理网络费时费力。因此，通过采用本申请实施例的构建神经网络的方法，能够构建得到适用于自动驾驶场景下进行数据处理的神经网络，接下来，通过自动驾驶场景下的数据对该神经网络进行训练，能够得到传感器数据处理网络，最后就可以利用该传感器处理网络对输入的道路画面进行处理，从而识别出道路画面中的不同物体。
由于本申请实施例涉及大量神经网络的应用，为了便于理解，下面先对本申请实施例可能涉及的神经网络的相关术语和概念进行介绍。
(1)神经网络
神经网络可以是由神经单元组成的，神经单元可以是指以x s和截距1为输入的运算单元，该运算单元的输出可以为：
Figure PCTCN2020116673-appb-000021
其中，s＝1、2、……n，n为大于1的自然数，W s为x s的权重，b为神经单元的偏置。f为神经单元的激活函数(activation functions)，用于将非线性特性引入神经网络中，来将神经单元中的输入信号转换为输出信号。该激活函数的输出信号可以作为下一层卷积层的输入，激活函数可以是sigmoid函数。神经网络是将多个上述单一的神经单元联结在一起形成的网络，即一个神经单元的输出可以是另一个神经单元的输入。每个神经单元的输入可以与前一层的局部接受域相连，来提取局部接受域的特征，局部接受域可以是由若干个神经单元组成的区域。
(2)深度神经网络
深度神经网络(deep neural network，DNN)，也称多层神经网络，可以理解为具有多层隐含层的神经网络。按照不同层的位置对DNN进行划分，DNN内部的神经网络可以分为三类：输入层，隐含层，输出层。一般来说第一层是输入层，最后一层是输出层，中间的层数都是隐含层。层与层之间是全连接的，也就是说，第i层的任意一个神经元一定与第i+1层的任意一个神经元相连。
虽然DNN看起来很复杂，但是就每一层的工作来说，其实并不复杂，简单来说就是如下线性关系表达式：
Figure PCTCN2020116673-appb-000022
其中，
Figure PCTCN2020116673-appb-000023
是输入向量，
Figure PCTCN2020116673-appb-000024
是输出向量，
Figure PCTCN2020116673-appb-000025
是偏移 向量，W是权重矩阵(也称系数)，α()是激活函数。每一层仅仅是对输入向量
Figure PCTCN2020116673-appb-000026
经过如此简单的操作得到输出向量
Figure PCTCN2020116673-appb-000027
由于DNN层数多，系数W和偏移向量
Figure PCTCN2020116673-appb-000028
的数量也比较多。这些参数在DNN中的定义如下所述：以系数W为例：假设在一个三层的DNN中，第二层的第4个神经元到第三层的第2个神经元的线性系数定义为
Figure PCTCN2020116673-appb-000029
上标3代表系数W所在的层数，而下标对应的是输出的第三层索引2和输入的第二层索引4。
综上，第L-1层的第k个神经元到第L层的第j个神经元的系数定义为
Figure PCTCN2020116673-appb-000030
需要注意的是，输入层是没有W参数的。在深度神经网络中，更多的隐含层让网络更能够刻画现实世界中的复杂情形。理论上而言，参数越多的模型复杂度越高，“容量”也就越大，也就意味着它能完成更复杂的学习任务。训练深度神经网络的也就是学习权重矩阵的过程，其最终目的是得到训练好的深度神经网络的所有层的权重矩阵(由很多层的向量W形成的权重矩阵)。
(3)卷积神经网络
卷积神经网络(convolutional neuron network，CNN)是一种带有卷积结构的深度神经网络。卷积神经网络包含了一个由卷积层和子采样层构成的特征抽取器，该特征抽取器可以看作是滤波器。卷积层是指卷积神经网络中对输入信号进行卷积处理的神经元层。在卷积神经网络的卷积层中，一个神经元可以只与部分邻层神经元连接。一个卷积层中，通常包含若干个特征平面，每个特征平面可以由一些矩形排列的神经单元组成。同一特征平面的神经单元共享权重，这里共享的权重就是卷积核。共享权重可以理解为提取图像信息的方式与位置无关。卷积核可以以随机大小的矩阵的形式初始化，在卷积神经网络的训练过程中卷积核可以通过学习得到合理的权重。另外，共享权重带来的直接好处是减少卷积神经网络各层之间的连接，同时又降低了过拟合的风险。
(4)循环神经网络(recurrent neural networks，RNN)是用来处理序列数据的。在传统的神经网络模型中，是从输入层到隐含层再到输出层，层与层之间是全连接的，而对于每一层层内之间的各个节点是无连接的。这种普通的神经网络虽然解决了很多难题，但是却仍然对很多问题无能无力。例如，你要预测句子的下一个单词是什么，一般需要用到前面的单词，因为一个句子中前后单词并不是独立的。RNN之所以称为循环神经网路，即一个序列当前的输出与前面的输出也有关。具体的表现形式为网络会对前面的信息进行记忆并应用于当前输出的计算中，即隐含层本层之间的节点不再无连接而是有连接的，并且隐含层的输入不仅包括输入层的输出还包括上一时刻隐含层的输出。理论上，RNN能够对任何长度的序列数据进行处理。对于RNN的训练和对传统的CNN或DNN的训练一样。
既然已经有了卷积神经网络，为什么还要循环神经网络？原因很简单，在卷积神经网络中，有一个前提假设是：元素之间是相互独立的，输入与输出也是独立的，比如猫和狗。但现实世界中，很多元素都是相互连接的，比如股票随时间的变化，再比如一个人说了：我喜欢旅游，其中最喜欢的地方是云南，以后有机会一定要去。这里填空，人类应该都知道是填“云南”。因为人类会根据上下文的内容进行推断，但如何让机器做到这一步？RNN就应运而生了。RNN旨在让机器像人一样拥有记忆的能力。因此，RNN的输出就需要依赖当前的输入信息和历史的记忆信息。
(5)损失函数
在训练深度神经网络的过程中，因为希望深度神经网络的输出尽可能的接近真正想要预测的值，所以可以通过比较当前网络的预测值和真正想要的目标值，再根据两者之间的差异情况来更新每一层神经网络的权重向量(当然，在第一次更新之前通常会有初始化的过程，即为深度神经网络中的各层预先配置参数)，比如，如果网络的预测值高了，就调整权重向量让它预测低一些，不断地调整，直到深度神经网络能够预测出真正想要的目标值或与真正想要的目标值非常接近的值。因此，就需要预先定义“如何比较预测值和目标值之间的差异”，这便是损失函数(loss function)或目标函数(objective function)，它们是用于衡量预测值和目标值的差异的重要方程。其中，以损失函数举例，损失函数的输出值(loss)越高表示差异越大，那么深度神经网络的训练就变成了尽可能缩小这个loss的过程。
(6)反向传播算法
神经网络可以采用误差反向传播(back propagation，BP)算法在训练过程中修正初始的神经网络模型中参数的大小，使得神经网络模型的重建误差损失越来越小。具体地，前向传递输入信号直至输出会产生误差损失，通过反向传播误差损失信息来更新初始的神经网络模型中参数，从而使误差损失收敛。反向传播算法是以误差损失为主导的反向传播运动，旨在得到最优的神经网络模型的参数，例如权重矩阵。
如图2所示，本申请实施例提供了一种系统架构100。在图2中，数据采集设备160用于采集训练数据。针对本申请实施例的图像处理方法来说，训练数据可以包括训练图像以及训练图像对应的分类结果，其中，训练图像的结果可以是人工预先标注的结果。
在采集到训练数据之后，数据采集设备160将这些训练数据存入数据库130，训练设备120基于数据库130中维护的训练数据训练得到目标模型/规则101。
下面对训练设备120基于训练数据得到目标模型/规则101进行描述，训练设备120对输入的原始图像进行处理，将输出的图像与原始图像进行对比，直到训练设备120输出的图像与原始图像的差值小于一定的阈值，从而完成目标模型/规则101的训练。
上述目标模型/规则101能够用于实现本申请实施例的图像处理方法。本申请实施例中的目标模型/规则101具体可以为神经网络。需要说明的是，在实际的应用中，所述数据库130中维护的训练数据不一定都来自于数据采集设备160的采集，也有可能是从其他设备接收得到的。另外需要说明的是，训练设备120也不一定完全基于数据库130维护的训练数据进行目标模型/规则101的训练，也有可能从云端或其他地方获取训练数据进行模型训练，上述描述不应该作为对本申请实施例的限定。
根据训练设备120训练得到的目标模型/规则101可以应用于不同的系统或设备中，如应用于图2所示的执行设备110，所述执行设备110可以是终端，如手机终端，平板电脑，笔记本电脑，增强现实(augmented reality，AR)AR/虚拟现实(virtual reality，VR)，车载终端等，还可以是服务器或者云端等。在图2中，执行设备110配置输入/输出(input/output，I/O)接口112，用于与外部设备进行数据交互，用户可以通过客户设备140向I/O接口112输入数据，所述输入数据在本申请实施例中可以包括：客户设备输入的待处理图像。
预处理模块113和预处理模块114用于根据I/O接口112接收到的输入数据(如待 处理图像)进行预处理，在本申请实施例中，也可以没有预处理模块113和预处理模块114(也可以只有其中的一个预处理模块)，而直接采用计算模块111对输入数据进行处理。
在执行设备110对输入数据进行预处理，或者在执行设备110的计算模块111执行计算等相关的处理过程中，执行设备110可以调用数据存储系统150中的数据、代码等以用于相应的处理，也可以将相应处理得到的数据、指令等存入数据存储系统150中。
最后，I/O接口112将处理结果，如上述得到的图像的分类结果返回给客户设备140，从而提供给用户。
值得说明的是，训练设备120可以针对不同的目标或称不同的任务，基于不同的训练数据生成相应的目标模型/规则101，该相应的目标模型/规则101即可以用于实现上述目标或完成上述任务，从而为用户提供所需的结果。
在图2中所示情况下，用户可以手动给定输入数据，该手动给定可以通过I/O接口112提供的界面进行操作。另一种情况下，客户设备140可以自动地向I/O接口112发送输入数据，如果要求客户设备140自动发送输入数据需要获得用户的授权，则用户可以在客户设备140中设置相应权限。用户可以在客户设备140查看执行设备110输出的结果，具体的呈现形式可以是显示、声音、动作等具体方式。客户设备140也可以作为数据采集端，采集如图所示输入I/O接口112的输入数据及输出I/O接口112的输出结果作为新的样本数据，并存入数据库130。当然，也可以不经过客户设备140进行采集，而是由I/O接口112直接将如图所示输入I/O接口112的输入数据及输出I/O接口112的输出结果，作为新的样本数据存入数据库130。
值得注意的是，图2仅是本申请实施例提供的一种系统架构的示意图，图中所示设备、器件、模块等之间的位置关系不构成任何限制，例如，在图2中，数据存储系统150相对执行设备110是外部存储器，在其它情况下，也可以将数据存储系统150置于执行设备110中。
如图2所示，根据训练设备120训练得到目标模型/规则101，该目标模型/规则101在本申请实施例中可以是本申请中的神经网络，具体的，本申请实施例构建的神经网络可以为CNN，深度卷积神经网络(deep convolutional neural networks,DCNN)，循环神经网络(recurrent neural network，RNNS)等等。
由于CNN是一种非常常见的神经网络，下面结合图3重点对CNN的结构进行详细的介绍。如上文的基础概念介绍所述，卷积神经网络是一种带有卷积结构的深度神经网络，是一种深度学习(deep learning)架构，深度学习架构是指通过机器学习的算法，在不同的抽象层级上进行多个层次的学习。作为一种深度学习架构，CNN是一种前馈(feed-forward)人工神经网络，该前馈人工神经网络中的各个神经元可以对输入其中的图像作出响应。
本申请实施例的图像处理方法具体采用的神经网络的结构可以如图3所示。在图3中，卷积神经网络(CNN)200可以包括输入层210，卷积层/池化层220(其中池化层为可选的)，以及神经网络层230。其中，输入层210可以获取待处理图像，并将获取到的待处理图像交由卷积层/池化层220以及后面的神经网络层230进行处理，可以得到图像的处理结果。下面对图3中的CNN 200中内部的层结构进行详细的介绍。
卷积层/池化层220：
卷积层：
如图3所示卷积层/池化层220可以包括如示例221-226层，举例来说：在一种实现中，221层为卷积层，222层为池化层，223层为卷积层，224层为池化层，225为卷积层，226为池化层；在另一种实现方式中，221、222为卷积层，223为池化层，224、225为卷积层，226为池化层。即卷积层的输出可以作为随后的池化层的输入，也可以作为另一个卷积层的输入以继续进行卷积操作。
下面将以卷积层221为例，介绍一层卷积层的内部工作原理。
卷积层221可以包括很多个卷积算子，卷积算子也称为核，其在图像处理中的作用相当于一个从输入图像矩阵中提取特定信息的过滤器，卷积算子本质上可以是一个权重矩阵，这个权重矩阵通常被预先定义，在对图像进行卷积操作的过程中，权重矩阵通常在输入图像上沿着水平方向一个像素接着一个像素(或两个像素接着两个像素……这取决于步长stride的取值)的进行处理，从而完成从图像中提取特定特征的工作。该权重矩阵的大小应该与图像的大小相关，需要注意的是，权重矩阵的纵深维度(depth dimension)和输入图像的纵深维度是相同的，在进行卷积运算的过程中，权重矩阵会延伸到输入图像的整个深度。因此，和一个单一的权重矩阵进行卷积会产生一个单一纵深维度的卷积化输出，但是大多数情况下不使用单一权重矩阵，而是应用多个尺寸(行×列)相同的权重矩阵，即多个同型矩阵。每个权重矩阵的输出被堆叠起来形成卷积图像的纵深维度，这里的维度可以理解为由上面所述的“多个”来决定。不同的权重矩阵可以用来提取图像中不同的特征，例如一个权重矩阵用来提取图像边缘信息，另一个权重矩阵用来提取图像的特定颜色，又一个权重矩阵用来对图像中不需要的噪点进行模糊化等。该多个权重矩阵尺寸(行×列)相同，经过该多个尺寸相同的权重矩阵提取后的卷积特征图的尺寸也相同，再将提取到的多个尺寸相同的卷积特征图合并形成卷积运算的输出。
这些权重矩阵中的权重值在实际应用中需要经过大量的训练得到，通过训练得到的权重值形成的各个权重矩阵可以用来从输入图像中提取信息，从而使得卷积神经网络200进行正确的预测。
当卷积神经网络200有多个卷积层的时候，初始的卷积层(例如221)往往提取较多的一般特征，该一般特征也可以称之为低级别的特征；随着卷积神经网络200深度的加深，越往后的卷积层(例如226)提取到的特征越来越复杂，比如高级别的语义之类的特征，语义越高的特征越适用于待解决的问题。
池化层：
由于常常需要减少训练参数的数量，因此卷积层之后常常需要周期性的引入池化层，在如图3中220所示例的221-226各层，可以是一层卷积层后面跟一层池化层，也可以是多层卷积层后面接一层或多层池化层。在图像处理过程中，池化层的唯一目的就是减少图像的空间大小。池化层可以包括平均池化算子和/或最大池化算子，以用于对输入图像进行采样得到较小尺寸的图像。平均池化算子可以在特定范围内对图像中的像素值进行计算产生平均值作为平均池化的结果。最大池化算子可以在特定范围内取该范围内值最大的像素作为最大池化的结果。另外，就像卷积层中用权重矩阵的大小应该与图像尺寸相关一样，池化层中的运算符也应该与图像的大小相关。通过池化层处理后输出的图像尺寸可以小于输入池化层的图像的尺寸，池化层输出的图像中每个像素点表示输入池化层的图像的对应 子区域的平均值或最大值。
神经网络层230：
在经过卷积层/池化层220的处理后，卷积神经网络200还不足以输出所需要的输出信息。因为如前所述，卷积层/池化层220只会提取特征，并减少输入图像带来的参数。然而为了生成最终的输出信息(所需要的类信息或其他相关信息)，卷积神经网络200需要利用神经网络层230来生成一个或者一组所需要的类的数量的输出。因此，在神经网络层230中可以包括多层隐含层(如图3所示的231、232至23n)以及输出层240，该多层隐含层中所包含的参数可以根据具体的任务类型的相关训练数据进行预先训练得到，例如该任务类型可以包括图像识别，图像分类，图像超分辨率重建等等。
在神经网络层230中的多层隐含层之后，也就是整个卷积神经网络200的最后层为输出层240，该输出层240具有类似分类交叉熵的损失函数，具体用于计算预测误差，一旦整个卷积神经网络200的前向传播(如图3由210至240方向的传播为前向传播)完成，反向传播(如图3由240至210方向的传播为反向传播)就会开始更新前面提到的各层的权重值以及偏差，以减少卷积神经网络200的损失，及卷积神经网络200通过输出层输出的结果和理想结果之间的误差。
本申请实施例的图像处理方法具体采用的神经网络的结构可以如图4所示。在图4中，卷积神经网络(CNN)200可以包括输入层110，卷积层/池化层120(其中池化层为可选的)，以及神经网络层130。与图3相比，图4中的卷积层/池化层120中的多个卷积层/池化层并行，将分别提取的特征均输入给全神经网络层130进行处理。
需要说明的是，图3和图4所示的卷积神经网络仅作为一种本申请实施例的图像处理方法的两种可能的卷积神经网络的示例，在具体的应用中，本申请实施例的图像处理方法所采用的卷积神经网络还可以以其他网络模型的形式存在。
另外，采用本申请实施例的神经网络结构的搜索方法得到的卷积神经网络的结构可以如图3和图4中的卷积神经网络结构所示。
图5为本申请实施例提供的一种芯片的硬件结构，该芯片包括神经网络处理器50。该芯片可以被设置在如图1所示的执行设备110中，用以完成计算模块111的计算工作。该芯片也可以被设置在如图1所示的训练设备120中，用以完成训练设备120的训练工作并输出目标模型/规则101。如图3和图4所示的卷积神经网络中各层的算法均可在如图5所示的芯片中得以实现。
神经网络处理器NPU 50作为协处理器挂载到主中央处理器(central processing unit，CPU)(host CPU)上，由主CPU分配任务。NPU的核心部分为运算电路503，控制器504控制运算电路503提取存储器(权重存储器或输入存储器)中的数据并进行运算。
在一些实现中，运算电路503内部包括多个处理单元(process engine,PE)。在一些实现中，运算电路503是二维脉动阵列。运算电路503还可以是一维脉动阵列或者能够执行例如乘法和加法这样的数学运算的其它电子线路。在一些实现中，运算电路503是通用的矩阵处理器。
举例来说，假设有输入矩阵A，权重矩阵B，输出矩阵C。运算电路从权重存储器502中取矩阵B相应的数据，并缓存在运算电路中每一个PE上。运算电路从输入存储器501中取矩阵A数据与矩阵B进行矩阵运算，得到的矩阵的部分结果或最终结果，保存在 累加器(accumulator)508中。
向量计算单元507可以对运算电路的输出做进一步处理，如向量乘，向量加，指数运算，对数运算，大小比较等等。例如，向量计算单元507可以用于神经网络中非卷积/非FC层的网络计算，如池化(pooling)，批归一化(batch normalization)，局部响应归一化(local response normalization)等。
在一些实现种，向量计算单元能507将经处理的输出的向量存储到统一缓存器506。例如，向量计算单元507可以将非线性函数应用到运算电路503的输出，例如累加值的向量，用以生成激活值。在一些实现中，向量计算单元507生成归一化的值、合并值，或二者均有。在一些实现中，处理过的输出的向量能够用作到运算电路503的激活输入，例如用于在神经网络中的后续层中的使用。
统一存储器506用于存放输入数据以及输出数据。
权重数据直接通过存储单元访问控制器505(direct memory access controller，DMAC)将外部存储器中的输入数据搬运到输入存储器501和/或统一存储器506、将外部存储器中的权重数据存入权重存储器502，以及将统一存储器506中的数据存入外部存储器。
总线接口单元(bus interface unit，BIU)510，用于通过总线实现主CPU、DMAC和取指存储器509之间进行交互。
与控制器504连接的取指存储器(instruction fetch buffer)509，用于存储控制器504使用的指令；
控制器504，用于调用指存储器509中缓存的指令，实现控制该运算加速器的工作过程。
一般地，统一存储器506，输入存储器501，权重存储器502以及取指存储器509均为片上(On-Chip)存储器，外部存储器为该NPU外部的存储器，该外部存储器可以为双倍数据率同步动态随机存储器(double data rate synchronous dynamic random access memory，简称DDR SDRAM)、高带宽存储器(high bandwidth memory，HBM)或其他可读可写的存储器。
其中，图3和图4所示的卷积神经网络中各层的运算可以由运算电路503或向量计算单元507执行。
上文中介绍的图2中的执行设备110能够执行本申请实施例的图像处理方法的各个步骤，图3和图4所示的CNN模型和图5所示的芯片也可以用于执行本申请实施例的图像处理方法的各个步骤。下面结合附图对本申请实施例的构建神经网络的方法和本申请实施例的图像处理方法进行详细的介绍。
如图6所示，本申请实施例提供了一种系统架构300。该系统架构包括本地设备301、本地设备302以及执行设备210和数据存储系统250，其中，本地设备301和本地设备302通过通信网络与执行设备210连接。
执行设备210可以由一个或多个服务器实现。可选的，执行设备210可以与其它计算设备配合使用，例如：数据存储器、路由器、负载均衡器等设备。执行设备210可以布置在一个物理站点上，或者分布在多个物理站点上。执行设备210可以使用数据存储系统250中的数据，或者调用数据存储系统250中的程序代码来实现本申请实施例的构建神经网络的方法。
具体地，执行设备210可以执行以下过程：根据目标神经网络的应用需求，构建搜索空间；对所述搜索空间中的多个元素进行聚类，得到聚类结果；并根据所述聚类结果，从所述多个元素指示的多个网络结构中选择满足预设要求的目标网络结构；根据所述目标网络结构搭建所述目标神经网络。
通过上述过程执行设备210能够搭建成一个目标神经网络，该目标神经网络可以用于图像分类或者进行图像处理等等。
用户可以操作各自的用户设备(例如本地设备301和本地设备302)与执行设备210进行交互。每个本地设备可以表示任何计算设备，例如个人计算机、计算机工作站、智能手机、平板电脑、智能摄像头、智能汽车或其他类型蜂窝电话、媒体消费设备、可穿戴设备、机顶盒、游戏机等。
每个用户的本地设备可以通过任何通信机制/通信标准的通信网络与执行设备210进行交互，通信网络可以是广域网、局域网、点对点连接等方式，或它们的任意组合。
在一种实现方式中，本地设备301、本地设备302从执行设备210获取到目标神经网络的相关参数，将目标神经网络部署在本地设备301、本地设备302上，利用该目标神经网络进行图像分类或者图像处理等等。
在另一种实现中，执行设备210上可以直接部署目标神经网络，执行设备210通过从本地设备301和本地设备302获取待处理图像，并根据目标神经网络对待处理图像进行分类或者其他类型的图像处理。
上述执行设备210也可以为云端设备，此时，执行设备210可以部署在云端；或者，上述执行设备210也可以为终端设备，此时，执行设备210可以部署在用户终端侧，本申请实施例对此并不限定。
下面先结合图7对本申请实施例的构建神经网络的方法700进行详细的介绍。图7所示的方法可以由构建神经网络的装置来执行，该构建神经网络的装置可以是移动终端，电脑、服务器等运算能力足以用来构建神经网络的装置。
图7所示的方法包括步骤710、720及730，下面分别对这些步骤进行详细的描述。
S710，根据目标神经网络的应用需求，构建搜索空间。
其中，所述搜索空间可以包括M个元素，所述M个元素可以用于指示M个网络结构，所述M个元素中的每一个元素可以包括对应的所述网络结构中的阶段(stage)的个数、所述阶段中的块(block)的个数和/或所述块的通道数，M为正整数。
可选地，所述M个元素中的每一个元素包括对应的所述网络结构是指，所述M个元素中的每一个元素指示的所述网络结构。
相应地，对于所述M个元素中的每一个元素，所述每一个元素包括该元素指示的所述网络结构中的阶段的个数、所述阶段中的块的个数和/或所述块的通道数。
上述目标神经网络可以是深度神经网络。为了便于说明，在本申请实施中，以卷积神经网络为例进行描述。
在本申请实施例中，所述网络结构可以包括卷积神经网络中的部分阶段，例如，所述网络结构可以指卷积神经网络中，用户希望调整的部分(或者称为待调整的部分)，因此，所述网络结构也可以称为待调整的网络结构。
需要说明的是，在本申请实施例中，网络结构精度可以是指由该网络结构构建的神 经网络的精度(或准确率)。
上述目标神经网络可以由至少一个所述网络结构构建而成，所述网络结构可以包括一个或多个阶段(stage)，每个阶段中可以包括至少一个块(block)。
其中，块(block)可以由卷积神经网络中的基本原子单元组成，这些基本原子单元包括：卷积层、池化层、全连接层及非线性激活层等。
在卷积神经网络中，特征通常都是以三维形式(长、宽及深度)存在的，可以将一个特征看成是多个二维特征的叠加，其中，特征的每一个二维特征可以称为特征图。或者，特征的一个特征图(二维特征)也可以称为特征的一个通道。
特征图的长和宽也可以称为特征图的分辨率。
可选地，不同阶段中的块的个数可以不同。
类似地，不同阶段处理的输入特征图的分辨率和输出特征图的分辨率也可以不同。
可选地，不同块的通道数可以不同。应理解，块的通道数也可以称为块的宽度，本申请实施例对此并不限定。
类似地，不同块处理的输入特征图的分辨率和输出特征图的分辨率也可以不同。
在本申请实施例中，所述元素还可以包括所述阶段的输入特征图的分辨率、所述阶段的输出特征图的分辨率、所述块的输入特征图的分辨率和/或所述块的输出特征图的分辨率。
随着所述元素包括的这些(卷积神经网络的)参数不同，会直接影响该卷积神经网络的效果和速度等，因此，在本申请实施例中，可以基于目标神经网络的应用需求，针对这些参数构建搜索空间，并在该搜索空间选择满足预设要求的目标网络结构。
在本申请实施例中，所述目标神经网络的应用需求可以包括所述目标神经网络的运行速度、所述目标神经网络的参数量或所述目标神经网络的结构要求，其中，所述结构要求包括所述目标神经网络结构中的阶段数、每个阶段的块数和/或每个所述块的通道数。
可选地，可以基于所述目标神经网络的运行速度和/或所述目标神经网络的参数量，构建所述搜索空间。所述搜索空间可以包括满足目标神经的应用需求的、所有可能的网络结构。
在本申请实施例中，根据所述目标神经网络的运行速度和/或所述目标神经网络的参数量构建搜索空间，可以在构建搜索空间的过程中就筛选掉一些低性能的网络结构，能够提高构建神经网络的效率。
可选地，所述目标神经网络的应用需求还可以包括所述网络结构的结构要求。
可选地，所述网络结构的结构要求可以包括所述网络结构中的阶段的个数要求。这里的阶段的个数要求可以阶段的个数的取值范围，例如，所述网络结构中允许的阶段的个数为2个至5个之间；或者，所述阶段的个数要求也可以是阶段的具体个数，例如，所述网络结构中的阶段的个数为4个。
类似地，所述网络结构的结构要求还可以包括每个阶段中的块的个数要求、和/或所述块的通道数要求。
此时，可以基于所述目标神经网络的运行速度、所述目标神经网络的参数量、所述网络结构中的阶段的个数要求、每个阶段中的块的个数要求和/或所述块的通道数要求， 构建所述搜索空间。所述搜索空间可以包括满足目标神经的应用需求的、所有可能的网络结构。
在本申请实施中，可以基于现有的神经网络构建搜索空间，并在该搜索空间选择满足预设要求的目标网络结构。
例如，本申请实施例中，可以基于现有的ResNets构建搜索空间，并在该搜索空间选择满足预设要求的目标网络结构。
可选地，ResNets可以根据输出特征图的分辨率将神经网络分为六个阶段，其中，第一个阶段仅包含一个卷积操作，最后一个阶段基于全连接操作进行回归概率。第二至第五个阶段均由一个或多个残差块(residual block)堆叠而成，残差块的结构可以如图8所示。
其中，ResNets的网络结构和残差块的具体描述可以参考现有技术，本申请实施例中不再赘述。
例如，本申请实施例中，可以将上述ResNets的第二至第五个阶段作为待调整的网络结构，基于该网络结构构建搜索空间，并规定每个阶段中的块的通道数相同。
需要说明的是，本申请实施例中的阶段的个数和块的通道数仅为示例而非限定，在实际中，并不限定待调整的网络结构中的阶段个数为4，也不限定每个阶段中的块的通道数相同，所述网络结构的结构要求可以根据具体的应用需求确定，本申请实施例对此并不限定。
作为示例而非限定，在确定待调整的网络结构的阶段的个数为4(即ResNets的第二至第五个阶段)后，实际上只需要调整这4个阶段中的块的个数及块的通道数，可以使用一个八元组对网络结构进行编码。为方便描述，可以将该八元组称为网络连接方式编码(block connection style code，BCSC)，具体可以将一个网络结构进行如下编码：
BCSC＝{D 2,D 3,D 4,D 5,C 2,C 3,C 4,C 5}
其中，D 2表示第二个阶段中的块的个数，D 3表示第三个阶段中的块的个数，D 4表示第四个阶段中的块的个数，D 5表示第五个阶段中的块的个数，C 2表示第二个阶段中的块的通道数，C 3表示第二个阶段中的块的通道数，C 4表示第二个阶段中的块的通道数，C 5表示第二个阶段中的块的通道数。
应理解，上述BCSC(八元组)的编码方式仅为示例而非限定，本申请实施例也可以采用现有的其他编码方式对网络结构进行编码。
在本申请实施例中，所述搜索空间中的M个元素可以为M个BCSC，其中，每一个BCSC可以指示一个网络结构。
在一种可能的实现方式中，可以根据目标神经网络的应用需求，确定所有可能的M个网络结构；并采用BCSC对每一个网络结构进行编码，得到M个BCSC；此时，这M个BCSC就可以作为M个元素，构成搜索空间。
在另一种可能的实现方式中，根据目标神经网络的应用需求确定所有可能的网络结构可以为N个，相应地，可以得到包括N个BCSC的初始搜索空间。此时，可以对初始搜索空间进行筛选，筛选掉其中的低性能网络结构后，得到所述包括M个元素的搜索空间。具体可以见下述对图9中的方法900的描述。
可选地，可以根据所述目标神经网络的应用需求，构建初始搜索空间，所述初始搜 索空间包括N个初始元素，所述N个初始元素用于指示N个初始网络结构，所述N个初始元素中的每一个元素包括对应的所述初始网络结构中的阶段的个数、所述阶段中的块的个数和/或所述块的通道数，N为大于或等于M的正整数；可以根据预设规则对所述N个初始元素指示的所述N个初始网络结构进行筛选，得到所述搜索空间中的所述M个元素。
其中，所述预设规则可以包括：
若所述N个初始元素中的第一初始元素指示的第一初始网络结构中的每一个阶段中的块的个数不大于所述N个初始元素中第二初始元素指示的第二初始网络结构中对应阶段中的块的个数，且所述第一初始网络结构中的每一个阶段中的每个块的通道数均不大于所述第二初始网络结构中对应阶段中的每个块的通道数，则从所述初始搜索空间中删除所述第一初始元素。
从上述预设规则可以看出，若对所述第一初始网络结构进行若干次以下操作，可以得到与所述第二初始网络结构相同的网络结构，则说明所述第二初始网络结构的精度高于所述第一初始网络结构的精度，相应地，可以从所述初始搜索空间中删除所述第一初始元素。具体的操作包括：
(1)对一个阶段中的一个块进行复制；
(2)增大一个阶段中的一个块的通道数(或宽度)。
例如，图10所示了N个网络结构，N为大于1的正整数。其中，在网络结构1中，阶段1中包括1个块，块的通道数为64，阶段2中包括2个块，每个块的通道数为128，阶段3中包括2个块，每个块的通道数为256，阶段4中包括4个块，每个块的通道数为512；在网络结构2中，阶段1中包括1个块，块的通道数为64，阶段2中包括1个块，块的通道数为128，阶段3中包括2个块，每个块的通道数为256，阶段4中包括3个块，每个块的通道数为512。
从图10中可以看出，对网络结构2中的阶段2中的一个块及阶段4中的一个块进行复制，则阶段2中包括2个通道数为128的块，阶段4中包括4个通道数为512的块。此时，复制后的网络结构2与网络结构1具有相同的网络结构，则说明网络结构2的精度低于网络结构1，可以从初始搜索空间中删除网络结构2(或者说，网络结构2对应的元素)。
或者，也可以直接比较初始搜索空间中的两个初始元素。
例如，可以直接比较所述第一初始元素的BCSC和所述第二初始元素的BCSC，在所述第一初始元素的BCSC中的每个元素，都比所述第二初始元素的BCSC中对应的元素小的情况下，则说明所述第一初始网络结构的精度低于所述第二初始网络结构的精度，相应地，可以从所述初始搜索空间中删除所述第一初始元素。
在本申请实施例中，根据上述预设规则，在不对所述初始搜索空间中的N个初始元素指示的初始网络结构进行训练的情况下，仅根据所述N个初始网络结构的结构信息就可以筛选出所述N个初始网络结构中的低性能网络结构，因此，能够提高构建神经网络的效率。
S720，根据搜索空间中未评估元素的分布关系从所述M个网络结构中选择目标网络结构。
其中，所述未评估元素的分布关系可以为所述未评估元素的聚类结果。
在本申请实施例中，根据未评估元素的分布关系，可以方便地挑选出未评估元素中具有代表性的元素，此时，根据该具有代表性的元素，能够高效地从所述M个网络结构中选择出目标网络结构，从而能够高效地构建满足性能要求的神经网络。
可选地，根据未评估元素的分布关系确定候选集，该候选集可以包括挑选出的未评估元素中具有代表性的元素。
需要说明的是，下述确定候选集的方法仅为示例而非限定，在本申请实施例中还可以通过其他方法确定候选集，本申请实施例中对根据未评估元素的分布关系确定候选集的具体方法并不限定。
例如，在本申请实施例中，可以通过下述几种方式，根据未评估元素的分布关系得到(包括未评估元素中具有代表性的元素的)候选集。
方式一：
可以构建一个(未评估元素的)候选集，使候选集内的各元素之间的距离尽可能大。这样可以使得集合内的元素在未评估元素中的分布尽可能分散，从而能够挑选出未评估元素中的具有代表性的元素。
例如，可能通过下述方法得到该候选集：
(1)、初始化各集合，其中，集合
Figure PCTCN2020116673-appb-000031
为空集，集合
Figure PCTCN2020116673-appb-000032
为所有未评估元素组成的集合，T为正整数；
(2)、从集合
Figure PCTCN2020116673-appb-000033
中随机选择一个元素β，将元素β加入集合
Figure PCTCN2020116673-appb-000034
同时，从集合
Figure PCTCN2020116673-appb-000035
移除元素β；
(3)、对于集合
Figure PCTCN2020116673-appb-000036
中的任意元素β i，计算该任意元素β i与集合
Figure PCTCN2020116673-appb-000037
中元素的最小距离，记为dis i；
(4)、选择集合{dis i|i＝1,2,3,…}中的最小值，记为dis j，将该dis j对应的集合
Figure PCTCN2020116673-appb-000038
中的元素记为元素β j；
(5)、将元素β j加入集合
Figure PCTCN2020116673-appb-000039
同时从集合
Figure PCTCN2020116673-appb-000040
移除元素β j；
(6)、如果集合
Figure PCTCN2020116673-appb-000041
的元素数量大于T，则执行步骤(7)，否则执行步骤(3)进行迭代；
(7)、将集合
Figure PCTCN2020116673-appb-000042
作为候选集，后续可以从集合
Figure PCTCN2020116673-appb-000043
中选择一个或多个元素进行评估。
方式二：
可以构建一个(未评估元素的)候选集，使候选集内的每个元素(均尽可能地)位于(该每个元素所在的)未评估元素的某个子集的中心。
例如，可能通过下述方法得到该候选集：
(1)、初始化各集合，其中，集合
Figure PCTCN2020116673-appb-000044
为空集，集合
Figure PCTCN2020116673-appb-000045
为所有未评估元素组成的集合，K为正整数；
(2)、对于集合
Figure PCTCN2020116673-appb-000046
中的任意元素β i，计算该任意元素β i在集合
Figure PCTCN2020116673-appb-000047
中的K紧邻(即集合
Figure PCTCN2020116673-appb-000048
中与元素β i最为接近的K个元素)，记为
Figure PCTCN2020116673-appb-000049
其中，i、K为正整数；
(3)、对于集合
Figure PCTCN2020116673-appb-000050
中的任意元素β i，计算该任意元素β i与
Figure PCTCN2020116673-appb-000051
中元素的最小距离，记为dis i；
(4)、选择集合{dis i|i＝1,2,3,…}中的最小值，记为dis j，将该dis j对应的集合
Figure PCTCN2020116673-appb-000052
中的元素记为元素β j；
(5)、将元素β j加入集合
Figure PCTCN2020116673-appb-000053
同时从集合
Figure PCTCN2020116673-appb-000054
移除
Figure PCTCN2020116673-appb-000055
(6)、如果集合
Figure PCTCN2020116673-appb-000056
的元素数量小于K，则执行步骤(7)，否则执行步骤(2)进行迭代；
(7)、将集合
Figure PCTCN2020116673-appb-000057
作为候选集，后续可以从集合
Figure PCTCN2020116673-appb-000058
中选择一个或多个元素进行评估。
方式三：
可以对未评估元素进行聚类，得到所述未评估元素的聚类结果(所述未评估元素的聚类结果也可以认为是所述未评估元素的分布关系)，并根据所述未评估元素的聚类结果，挑选出的未评估元素中具有代表性的元素，即候选集。
为便于理解，在下述实施例中，以通过(对未评估元素进行)聚类的方式挑选候选集为例进行描述。
可选地，可以对搜索空间中未评估元素进行聚类，得到所述未评估元素的聚类结果，然后根据所述未评估元素的聚类结果从所述M个网络结构中选择目标网络结构。
在本申请实施例中，在对所述M个元素进行聚类之前，还可以先对所述M个元素(即M个BCSC)进行标准化，具体的标准化过程可以参考现有技术，本申请对此并不限定。
可选地，可以统计所述M个BCSC在每个维度上的均值和方差。
例如，可以将每个维度的均值记为
Figure PCTCN2020116673-appb-000059
其中，
Figure PCTCN2020116673-appb-000060
表示第二个阶段中的块的个数的均值，
Figure PCTCN2020116673-appb-000061
表示第三个阶段中的块的个数的均值，
Figure PCTCN2020116673-appb-000062
表示第四个阶段中的块的个数的均值，
Figure PCTCN2020116673-appb-000063
表示第五个阶段中的块的个数的均值，
Figure PCTCN2020116673-appb-000064
表示第二个阶段中的块的通道数的均值，
Figure PCTCN2020116673-appb-000065
表示第三个阶段中的块的通道数的均值，
Figure PCTCN2020116673-appb-000066
表示第四个阶段中的块的通道数的均值，
Figure PCTCN2020116673-appb-000067
表示第五个阶段中的块的通道数的均值。
同样，可以将每个维度的方差记为
Figure PCTCN2020116673-appb-000068
其中，
Figure PCTCN2020116673-appb-000069
表示第二个阶段中的块的个数的方差，
Figure PCTCN2020116673-appb-000070
表示第三个阶段中的块的个数的方差，
Figure PCTCN2020116673-appb-000071
表示第四个阶段中的块的个数的方差，
Figure PCTCN2020116673-appb-000072
表示第五个阶段中的块的个数的方差，
Figure PCTCN2020116673-appb-000073
表示第二个阶段中的块的通道数的方差，
Figure PCTCN2020116673-appb-000074
表示第三个阶段中的块的通道数的方差，
Figure PCTCN2020116673-appb-000075
表示第四个阶段中的块的通道数的方差，
Figure PCTCN2020116673-appb-000076
表示第五个阶段中的块的通道数的方差。
对于上述每个BCSC＝{D 2,D 3,D 4,D 5,C 2,C 3,C 4,C 5}，经过标准化后可以得到
Figure PCTCN2020116673-appb-000077
其中，
Figure PCTCN2020116673-appb-000078
表示对BCSC进行标准化后的结果。
此时，可以对标准化后得到的M个
Figure PCTCN2020116673-appb-000079
进行聚类。本申请实施例对使用的聚类方法并不限定，具体的聚类方法可以参考现有技术，这里不再赘述。
例如，可以使用K-平均(K-means)算法，对标准化后得到的M个
Figure PCTCN2020116673-appb-000080
进行聚类。
可选地，可以根据所述未评估元素的分布关系，确定所述未评估元素中的K个元素，K为小于M的正整数；根据所述K个元素从所述M个网络结构中选择目标网络结构。
对所述M个元素(即M个BCSC)进行聚类，可以将所述M个元素聚成(一个或)多个簇。
相应地，所述M个元素的聚类结果可以包括对所述M个元素进行聚类后得到的(一个或)多个簇。其中，同一个簇中的元素相对于其他簇中的元素具有更强的相似性。
可选地，所述聚类结果还可以包括每个簇的中心，所述每个簇的中心可以认为是该簇中最具有代表性的元素。
例如，所述未评估元素的聚类结果可以包括K个簇，K为小于M的正整数，此时，根据所述未评估元素的分布关系确定的所述未评估元素中的K个元素可以所述K个簇的簇中心对应的K个元素。
在本申请实施例中，对所述搜索空间中的M个元素进行聚类，可以将所述M个元素指示的结构相似的网络结构聚类到一个簇中，根据聚类得到的多个簇，有助于高效地从所述M个网络结构中选择出满足预设要求的目标网络结构。
例如，所述聚类结果可以包括K个簇，此时，可以从每个簇中选择一个元素，得到上述K个元素，并对所述K个元素指示的K个网络结构进行评估，得到所述K个网络结构的评估结果。
其中，对所述K个元素指示的K个网络结构进行评估，可以是指测试所述K个网络结构的准确率(或精度)。
进一步地，所述K个元素可以是所述K个簇的中心对应的K个元素。
可选地，所述根据所述K个元素从所述M个网络结构中选择目标网络结构，可以包括：
对所述未评估元素中的K个元素指示的K个网络结构进行评估，得到已评估元素的评估结果，所述已评估元素的评估结果包括所述K个网络结构的评估结果；根据已评估元素的评估结果，从所述M个网络结构中选择目标网络结构。
在本申请实施例中，所述K个簇的中心对应的K个元素指示的K个网络结构是所述搜索空间中最具代表性的网络结构，基于所述K个簇的中心对应的K个元素指示的K个网络结构的评估结果，在所述M个网络结构中选择满足预设要求的目标网络结构，可以高效地从所述M个网络结构中选择出满足预设要求的目标网络结构，从而能够提高构建神经网络的效率。
若在所述K个元素指示的K个网络结构中，有至少一个网络结构的准确率(或精度)满足预设要求，则该网络结构就是目标网络结构，可以根据该网络结构构建神经网络。
若没有满足预设要求的网络结构，则需要继续根据已评估的网络结构的评估结果，在所述M个网络结构中进行迭代选择，直到选择出满足预设要求的目标网络结构。
例如，可以根据所述已评估元素的评估结果对第一未评估元素进行建模，得到所述第一未评估元素的模型，所述第一未评估元素包括所述搜索空间中除所述已评估元素之外的其他元素；根据所述第一未评估元素的模型，从所述M个网络结构中选择目标网络结构。
可选地，可以通过下述几种方法进行建模：
方法一：
可以根据所述已评估元素的评估结果及下述公式，对所述第一未评估元素进行建模，得到所述未评估的网络结构的模型：
Figure PCTCN2020116673-appb-000081
和/或(g(x)-l(x)) 2；
其中，x表示所述搜索空间中未评估的网络结构，y *表示精度阈值，g(x)表示精度大于精度阈值y *的网络结构x的条件概率密度函数，
Figure PCTCN2020116673-appb-000082
l(x)表示精度小于或等于精度阈值y *的网络结构x的条件概率密度函数，
Figure PCTCN2020116673-appb-000083
G *(x|x i)表示混合高斯分布，
Figure PCTCN2020116673-appb-000084
κ(x,x i)表示x与x i之间的欧式距离，κ*(x,x i)表示由κ(x,x i)组成的距离函数，
Figure PCTCN2020116673-appb-000085
x i表示所述已评估的网络结构中的第i个网络结构，y i表示所述已评估的网络结构中的第i个网络结构的精度，ω i表示G *(x|x i)对应的权重，Z表示归一化因子，σ表示G *(x|x i)的一个超参，i为正整数，e为自然对数函数的底数。
在上述模型中，G *(x|x i)对应的权重ω i可以满足：
Figure PCTCN2020116673-appb-000086
Figure PCTCN2020116673-appb-000087
且当y i≤y *时，y i越大，ω i越小，当y i＞y *时，y i越大，ω i越大；Z可以满足
Figure PCTCN2020116673-appb-000088
σ的值可以为聚类后，各个簇的中心对应的元素之间平均欧式距离的值的一半。
可选地，上述
Figure PCTCN2020116673-appb-000089
与网络结构的性能正相关，也就是说，
Figure PCTCN2020116673-appb-000090
的值越大，则
Figure PCTCN2020116673-appb-000091
对应的网络结构的性能越好。上述(g(x)-l(x)) 2与网络结构性能的不确定性正相关，也就是说(g(x)-l(x)) 2的值越大，则(g(x)-l(x)) 2对应的网络结构性能的不确定性越高。
可选地，在同时使用
Figure PCTCN2020116673-appb-000092
和(g(x)-l(x)) 2的情况下，可以根据经验设定这两个模型的比例。
例如，可以用3：1的比例，使用
Figure PCTCN2020116673-appb-000093
和(g(x)-l(x)) 2进行建模。
此时，若需要选择出4个网络结构，则可以选择出
Figure PCTCN2020116673-appb-000094
的值最大的3个网络结构，及(g(x)-l(x)) 2的值最大的1个网络结构。
若在这4个网络结构中，有至少一个网络结构的准确率(或精度)满足预设要求，则该网络结构就是目标网络结构，可以根据该网络结构构建神经网络。
若没有满足预设要求的网络结构，则重新根据已评估的网络结构(包括上述刚选择出的4个网络结构)的评估结果，从所述M个网络结构中选择目标网络结构。。
方法二：
可以根据所述已评估元素的评估结果及下述公式，对所述第一未评估元素进行建模，得到所述第一未评估元素的模型：
Figure PCTCN2020116673-appb-000095
其中，x表示所述搜索空间中的未评估元素；
Figure PCTCN2020116673-appb-000096
x i表示所述已评估元素中的第i个元素，y i表示所述已评估元素中的第i个元素的精度；τ＝max(y i)，
Figure PCTCN2020116673-appb-000097
表示期望函数；f(x)为服从高斯分布的随机变量，f(x)的均值μ(x)及f(x)的方差σ(x)与
Figure PCTCN2020116673-appb-000098
跟输入x满足下述关系：
μ(x)＝k T(K+η 2I) -1Y，
σ(x)＝1-k T(K+η 2I) -1k。
其中，n为所述已评估的网络结构的个数，Y为y i组成的向量，Y∈R n，Y i＝y i，k为κ(x,x i)组成的向量，k∈R n，k i＝κ(x,x i)，K为κ(x i,x j)组成的矩阵为，K∈R n×n，K i，j＝κ(x i,x j)，
Figure PCTCN2020116673-appb-000099
σ为一个超参，e为自然对数函数的底数；I为单位矩阵，η也为一个超参，i,j为正整数。
需要说明的是，上述建模方法仅为示例而非限定，本申请实施例中也可以通过其他方法进行建模。
进一步地，在根据搜索空间中未评估元素的分布关系从所述M个网络结构中未选择出目标网络结构的情况下，所述方法还可以包括：
根据所述第一未评估元素的分布关系与基于已评估元素构建的第一未评估元素的模型，从所述M个网络结构中选择目标网络结构。
可选地，所述根据所述第一未评估元素的分布关系与基于已评估元素构建的第一未评估元素的模型，从所述M个网络结构中选择目标网络结构，可以包括：
根据所述第一未评估元素的分布关系，确定所述第一未评估元素中的L个元素，L为小于M的正整数；根据所述L个元素与基于已评估元素构建的第一未评估元素的模型从所述M个网络结构中选择目标网络结构。
可选地，所述根据所述L个元素与基于已评估元素构建的第一未评估元素的模型从所述M个网络结构中选择目标网络结构，可以包括：
根据所述基于已评估元素构建的第一未评估元素的模型，从所述L个元素中确定Q个元素，Q为小于L的正整数；对所述Q个元素指示的Q个网络结构进行评估，得到第一已评估元素的评估结果，所述第一已评估元素的评估结果包括所述K个网络结构的评估结果和所述Q个网络结构的评估结果；根据第一已评估元素的评估结果与第二未评估元素的分布关系，从所述M个网络结构中选择目标网络结构，所述第二未评估元素包括所述搜索空间中除所述第一已评估元素之外的其他元素。
其中，所述第一未评估元素的分布关系可以为所述第一未评估元素的聚类结果，所述L个元素可以分别为所述第一未评估元素的聚类结果包括的L个簇中的元素。
可选地，所述L个元素可以分别为所述L个簇的中心对应的L个元素。
进一步地，在根据所述第一未评估元素的分布关系与基于已评估元素构建的第一未评估元素的模型，从所述M个网络结构中未选择出目标网络结构的情况下，所述方法还可以包括：
根据所述第二未评估元素的分布关系与基于第一已评估元素构建的第二未评估元素的模型，从所述M个网络结构中重新选择目标网络结构。
需要说明的是，上述根据所述第二未评估元素的分布关系与基于第一已评估元素构 建的第二未评估元素的模型，从所述M个网络结构中重新选择目标网络结构的方法，与前述根据所述第一未评估元素的分布关系与基于已评估元素构建的第一未评估元素的模型，从所述M个网络结构中选择目标网络结构的类似，这里不再赘述。
该过程会一直迭代进行，直到选择出满足预设要求的目标网络结构。
图9示出了本申请实施例提供的图像处理方法900的示意性流程图，该方法可以由能够进行图像处理的装置或设备执行，例如，该方法可以由终端设备、电脑、服务器等执行。
S910，根据目标神经网络的应用需求，构建初始搜索空间。
其中，所述初始搜索空间可以包括N个初始元素，所述N个初始元素可以用于指示N个网络结构，所述M个元素中的每一个元素可以包括对应的所述网络结构中的阶段(stage)的个数、所述阶段中的块(block)的个数和/或所述块的通道数，N为正整数。
可选地，所述目标神经网络可以是卷积神经网络。
在本申请实施例中，所述网络结构可以包括卷积神经网络中的部分阶段，例如，所述网络结构可以指卷积神经网络中，用户希望调整的部分(或者称为待调整的部分)，因此，所述网络结构也可以称为待调整的网络结构。
上述目标神经网络可以由至少一个所述网络结构构建而成，所述网络结构可以包括一个或多个阶段(stage)，每个阶段中可以包括至少一个块(block)。
S920，基于预设规则对初始搜索空间进行筛选，得到搜索空间。
其中，所述搜索空间可以包括M个元素，所述M个元素可以用于指示M个网络结构，所述元素可以包括所述网络结构中的阶段(stage)的个数、所述阶段中的块(block)的个数和/或所述块的通道数，M为小于或等于N的正整数。
可选地，所述预设规则可以包括：
若所述N个初始元素中的第一初始元素指示的第一初始网络结构中的每一个阶段中的块的个数不大于所述N个初始元素中第二初始元素指示的第二初始网络结构中对应阶段中的块的个数，且所述第一初始网络结构中的每一个阶段中的每个块的通道数均不大于所述第二初始网络结构中对应阶段中的每个块的通道数，则从所述初始搜索空间中删除所述第一初始元素。
从上述预设规则可以看出，若对所述第一初始网络结构进行若干次以下操作，可以得到与所述第二初始网络结构相同的网络结构，则说明所述第二初始网络结构的精度高于所述第一初始网络结构的精度，相应地，可以从所述初始搜索空间中删除所述第一初始元素。具体的操作包括：
(1)对一个阶段中的一个块进行复制；
(2)增大一个阶段中的一个块的通道数(或宽度)。
例如，图10所示了N个网络结构，N为大于1的正整数。其中，在网络结构1中，阶段1中包括1个块，块的通道数为64，阶段2中包括2个块，每个块的通道数为128，阶段3中包括2个块，每个块的通道数为256，阶段4中包括4个块，每个块的通道数为512；在网络结构2中，阶段1中包括1个块，块的通道数为64，阶段2中包括1个块，块的通道数为128，阶段3中包括2个块，每个块的通道数为256，阶段4中包括3个块，每个块的通道数为512。
从图10中可以看出，对网络结构2中的阶段2中的一个块及阶段4中的一个块进行 复制，则阶段2中包括2个通道数为128的块，阶段4中包括4个通道数为512的块。此时，复制后的网络结构2与网络结构1具有相同的网络结构，则说明网络结构2的精度低于网络结构1，可以从初始搜索空间中删除网络结构2(或者说，网络结构2对应的元素)。
或者，也可以直接比较初始搜索空间中的两个初始元素。
例如，可以直接比较所述第一初始元素的BCSC和所述第二初始元素的BCSC，在所述第一初始元素的BCSC中的每个元素，都比所述第二初始元素的BCSC中对应的元素小的情况下，则说明所述第一初始网络结构的精度低于所述第二初始网络结构的精度，相应地，可以从所述初始搜索空间中删除所述第一初始元素。
S930，对搜索空间中的未评估元素进行聚类，得到K个簇。
其中，同一个簇中的元素相对于其他簇中的元素具有更强的相似性。所述聚类结果还可以包括每个簇的中心，所述每个簇的中心可以认为是该簇中最具有代表性的元素。
在本申请实施例中，所述搜索空间中的未评估元素为所述搜索空间中M个元素，在对所述M个元素进行聚类之前，还可以先对所述M个元素(即M个BCSC)进行标准化，具体的标准化过程可以参考现有技术，本申请对此并不限定。
可选地，可以统计所述M个BCSC在每个维度上的均值和方差。
例如，可以将每个维度的均值记为
Figure PCTCN2020116673-appb-000100
其中，
Figure PCTCN2020116673-appb-000101
表示第二个阶段中的块的个数的均值，
Figure PCTCN2020116673-appb-000102
表示第三个阶段中的块的个数的均值，
Figure PCTCN2020116673-appb-000103
表示第四个阶段中的块的个数的均值，
Figure PCTCN2020116673-appb-000104
表示第五个阶段中的块的个数的均值，
Figure PCTCN2020116673-appb-000105
表示第二个阶段中的块的通道数的均值，
Figure PCTCN2020116673-appb-000106
表示第三个阶段中的块的通道数的均值，
Figure PCTCN2020116673-appb-000107
表示第四个阶段中的块的通道数的均值，
Figure PCTCN2020116673-appb-000108
表示第五个阶段中的块的通道数的均值。
同样，可以将每个维度的方差记为
Figure PCTCN2020116673-appb-000109
其中，
Figure PCTCN2020116673-appb-000110
表示第二个阶段中的块的个数的方差，
Figure PCTCN2020116673-appb-000111
表示第三个阶段中的块的个数的方差，
Figure PCTCN2020116673-appb-000112
表示第四个阶段中的块的个数的方差，
Figure PCTCN2020116673-appb-000113
表示第五个阶段中的块的个数的方差，
Figure PCTCN2020116673-appb-000114
表示第二个阶段中的块的通道数的方差，
Figure PCTCN2020116673-appb-000115
表示第三个阶段中的块的通道数的方差，
Figure PCTCN2020116673-appb-000116
表示第四个阶段中的块的通道数的方差，
Figure PCTCN2020116673-appb-000117
表示第五个阶段中的块的通道数的方差。
对于上述每个BCSC＝{D 2,D 3,D 4,D 5,C 2,C 3,C 4,C 5}，经过标准化后可以得到
Figure PCTCN2020116673-appb-000118
其中，
Figure PCTCN2020116673-appb-000119
表示对BCSC进行标准化后的结果。
此时，可以对标准化后得到的M个
Figure PCTCN2020116673-appb-000120
进行聚类。本申请实施例对使用的聚类方法并不限定，具体的聚类方法可以参考现有技术，这里不再赘述。
例如，可以使用K-平均(K-means)算法，对标准化后得到的M个
Figure PCTCN2020116673-appb-000121
进行聚类。
S940，对K个簇的中心对应的元素指示的K个网络结构进行评估，得到K个网络结构的评估结果。
若在所述K个簇的中心对应的元素指示的K个网络结构中，有至少一个网络结构的准确率(或精度)满足预设要求，则该网络结构就是目标网络结构，可以根据该网络结构构建神经网络。
若没有满足预设要求的网络结构，则继续执行S950。
S950，根据已评估的网络结构的评估结果，对搜索空间中未评估的网络结构进行建模，得到未评估的网络结构的模型。
可选地，可以根据所述已评估的网络结构的评估结果，对所述M个网络结构中未评估的网络结构进行建模，得到所述未评估的网络结构的模型。其中，所述已评估的网络结构可以包括上述K个簇的中心对应的元素指示的K个网络结构。
应理解，本申请实施例对具体的建模方法并不限定。关于建模方法的描述可以参见上述图7中方法700中的实施例，这里不再赘述。
S960，对搜索空间中的未评估元素进行聚类，得到P个簇。
其中，同一个簇中的元素相对于其他簇中的元素具有更强的相似性。所述聚类结果还可以包括每个簇的中心，所述每个簇的中心可以认为是该簇中最具有代表性的元素。
在本申请实施例中，在第一次迭代的过程中，所述搜索空间中的未评估元素为所述搜索空间中的M-K个元素；在第二次迭代的过程中，所述搜索空间中的未评估元素为所述搜索空间中的M-2*K个元素，在后续的迭代过程中，所述搜索空间中的未评估元素均为M个元素减去所述搜索空间中的已评估元素。
需要说明的，本申请实施例中并不限定每次迭代过程中评估相同数量的元素，上述K个元素仅为示例而非限定，也就是说，在每次迭代过程中评估的元素的数量可以不同。
S970，根据未评估的网络结构的模型，从所述P个簇中选择K个簇。
可选地，可以根据所述未评估的网络结构的模型，对所述未评估的网络结构的准确率进行评估，得到评估的准确率；然后从所述P个簇中选择K个准确率最高网络结构进行评估。
需要说明的是，若没有满足预设要求的网络结构，上述S940、S950、S960、S970可以迭代进行，即继续在所述搜索空间中进行迭代选择，直到选择出满足预设要求的目标网络结构。
在后续的迭代过程中，所述搜索空间中的未评估元素始终为除已评估元素以外的其他元素。
可选地，第一次迭代的过程中，所述搜索空间中的未评估元素为所述搜索空间中除已评估的K个元素以外的其他元素，即，此时未评估元素为M-K个元素。
后续的迭代过程与第一次迭代类似，这里不再赘述。
例如，在第一次迭代的过程中，可以按照上述S960中的方法，将所述搜索空间中的未评估元素聚类为P个簇，此时，可以根据未评估元素的模型，从P个簇中选择K个元素，并按照上述S940中的方法，对这K个元素进行评估，P、K为正整数。
若在这K个网络结构中，有至少一个网络结构的准确率(或精度)满足预设要求，则该网络结构就是目标网络结构，可以根据该网络结构构建神经网络。
若没有满足预设要求的网络结构，则继续按照上述S950中的方法，根据已经评估的2*K个网络结构的评估结果，对未评估元素进行建模，并按照上述S960、S970中的方法，根据未评估元素的模型与未评估元素的分布关系，在搜索空间中选择满足预设要求的目标神经网络。
此时，若存在至少一个网络结构的准确率(或精度)满足预设要求，则该网络结构就是目标网络结构，可以根据该网络结构构建神经网络。
若还是没有满足预设要求的网络结构，可以根据上述S940、S950、S960、S970继续进行迭代选择，直到选择出满足预设要求的目标网络结构。
图11示出了本申请实施例提供的图像处理方法1100的示意性流程图，该方法可以由能够进行图像处理的装置或设备执行，例如，该方法可以由终端设备、电脑、服务器等执行。
图11中的图像处理方法1100中使用的目标神经网络可以是通过上述图7中的方法700或图9中的方法900构建的，也可以是通过其他方法构建的，本申请实施例对此并不限定。
S1110，获取待处理图像。
其中，所述待处理图像可以是终端设备(或者电脑、服务器等其他装置或设备)通过摄像头拍摄到的图像，或者，该待处理图像还可以是从终端设备(或者电脑、服务器等其他装置或设备)内部获得的图像(例如，终端设备的相册中存储的图像，或者终端设备从云端获取的图像)，本申请实施例对此并不限定。
S1120，根据目标神经网络对所述待处理图像进行分类，得到所述待处理图像的分类结果。
其中，所述目标神经网络由目标网络结构构建而成，所述目标网络结构是通过上述图7中的方法700或图9中的方法900得到的。
应理解，上述图像处理方法所采用的目标神经网络在进行图像分类之前，还需要再根据训练图像对该目标神经网络进行训练，训练得到的目标神经网络就可以对待处理图像进行分类。
也就是说，可以采用上述图7中的方法700或图9中的方法900得到的目标网络结构构建所述目标神经网络，接下来，再根据训练图像对该目标神经网络进行训练，训练完成后就可以用该目标神经网络对待处理图像进行分类了。
本申请中，由于所述目标神经网络是采用上述图7中的方法700或图9中的方法900得到的目标网络结构构建得到的，比较符合或者贴近神经网络的应用需求，利用这样的神经网络进行图像分类，能够取得较好的图像分类效果(例如，分类结果更准确，等等)。
图12是本申请实施例提供的构建神经网络的装置的硬件结构示意图。图12所示的构建神经网络的装置3000(该装置3000具体可以是一种计算机设备)包括存储器3001、处理器3002、通信接口3003以及总线3004。其中，存储器3001、处理器3002、通信接口3003通过总线3004实现彼此之间的通信连接。
存储器3001可以是只读存储器(read only memory，ROM)，静态存储设备，动态存储设备或者随机存取存储器(random access memory，RAM)。存储器3001可以存储程序，当存储器3001中存储的程序被处理器3002执行时，处理器3002用于执行本申请实施例的构建神经网络的方法的各个步骤。
处理器3002可以采用通用的中央处理器(central processing unit，CPU)，微处理器，应用专用集成电路(application specific integrated circuit，ASIC)，图形处理器(graphics processing unit，GPU)或者一个或多个集成电路，用于执行相关程序，以实现本申请方法实施例的构建神经网络的方法。
处理器3002还可以是一种集成电路芯片，具有信号的处理能力，例如，可以是图5所 示的芯片。在实现过程中，本申请的构建神经网络的方法的各个步骤可以通过处理器3002中的硬件的集成逻辑电路或者软件形式的指令完成。
上述处理器3002还可以是通用处理器、数字信号处理器(digital signal processing，DSP)、专用集成电路(ASIC)、现成可编程门阵列(field programmable gate array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器3001，处理器3002读取存储器3001中的信息，结合其硬件完成本构建神经网络的装置中包括的单元所需执行的功能，或者执行本申请方法实施例的构建神经网络的方法。
通信接口3003使用例如但不限于收发器一类的收发装置，来实现装置3000与其他设备或通信网络之间的通信。例如，可以通过通信接口3003获取待构建的神经网络的信息以及构建神经网络过程中需要的训练数据。
总线3004可包括在装置3000各个部件(例如，存储器3001、处理器3002、通信接口3003)之间传送信息的通路。
图13是本申请实施例的图像处理装置的硬件结构示意图。图13所示的图像处理装置4000包括存储器4001、处理器4002、通信接口4003以及总线4004。其中，存储器4001、处理器4002、通信接口4003通过总线4004实现彼此之间的通信连接。
存储器4001可以是ROM，静态存储设备和RAM。存储器4001可以存储程序，当存储器4001中存储的程序被处理器4002执行时，处理器4002和通信接口4003用于执行本申请实施例的图像处理方法的各个步骤。
处理器4002可以采用通用的，CPU，微处理器，ASIC，GPU或者一个或多个集成电路，用于执行相关程序，以实现本申请实施例的图像处理装置中的单元所需执行的功能，或者执行本申请方法实施例的图像处理方法。
处理器4002还可以是一种集成电路芯片，具有信号的处理能力，例如，可以是图5所示的芯片。在实现过程中，本申请实施例的图像处理方法的各个步骤可以通过处理器4002中的硬件的集成逻辑电路或者软件形式的指令完成。
上述处理器4002还可以是通用处理器、DSP、ASIC、FPGA或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器4001，处理器4002读取存储器4001中的信息，结合其硬件完成本申请实施例的图像处理装置中包括的单元所需执行的功能，或者执行本申请方法实施例的图像处理方法。
通信接口4003使用例如但不限于收发器一类的收发装置，来实现装置4000与其他设 备或通信网络之间的通信。例如，可以通过通信接口4003获取待处理图像。
总线4004可包括在装置4000各个部件(例如，存储器4001、处理器4002、通信接口4003)之间传送信息的通路。
图14是本申请实施例的神经网络训练装置的硬件结构示意图。与上述装置3000和装置4000类似，图14所示的神经网络训练装置5000包括存储器5001、处理器5002、通信接口5003以及总线5004。其中，存储器5001、处理器5002、通信接口5003通过总线5004实现彼此之间的通信连接。
在通过图12所示的构建神经网络的装置构建得到了神经网络之后，可以通过图14所示的神经网络训练装置5000对该神经网络进行训练，训练得到的神经网络就可以用于执行本申请实施例的图像处理方法了。
具体地，图14所示的装置可以通过通信接口5003从外界获取训练数据以及待训练的神经网络，然后由处理器根据训练数据对待训练的神经网络进行训练。
应注意，尽管上述装置3000、装置4000和装置5000仅仅示出了存储器、处理器、通信接口，但是在具体实现过程中，本领域的技术人员应当理解，装置3000、装置4000和装置5000还可以包括实现正常运行所必须的其他器件。同时，根据具体需要，本领域的技术人员应当理解，装置3000、装置4000和装置5000还可包括实现其他附加功能的硬件器件。此外，本领域的技术人员应当理解，装置3000、装置4000和装置5000也可仅仅包括实现本申请实施例所必须的器件，而不必包括图12、图13和图14中所示的全部器件。
应理解，本申请实施例中的处理器可以为中央处理单元(central processing unit，CPU)，该处理器还可以是其他通用处理器、数字信号处理器(digital signal processor，DSP)、专用集成电路(application specific integrated circuit，ASIC)、现成可编程门阵列(field programmable gate array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
还应理解，本申请实施例中的存储器可以是易失性存储器或非易失性存储器，或可包括易失性和非易失性存储器两者。其中，非易失性存储器可以是只读存储器(read-only memory，ROM)、可编程只读存储器(programmable ROM，PROM)、可擦除可编程只读存储器(erasable PROM，EPROM)、电可擦除可编程只读存储器(electrically EPROM，EEPROM)或闪存。易失性存储器可以是随机存取存储器(random access memory，RAM)，其用作外部高速缓存。通过示例性但不是限制性说明，许多形式的随机存取存储器(random access memory，RAM)可用，例如静态随机存取存储器(static RAM，SRAM)、动态随机存取存储器(DRAM)、同步动态随机存取存储器(synchronous DRAM，SDRAM)、双倍数据速率同步动态随机存取存储器(double data rate SDRAM，DDR SDRAM)、增强型同步动态随机存取存储器(enhanced SDRAM，ESDRAM)、同步连接动态随机存取存储器(synchlink DRAM，SLDRAM)和直接内存总线随机存取存储器(direct rambus RAM，DR RAM)。
上述实施例，可以全部或部分地通过软件、硬件、固件或其他任意组合来实现。当使用软件实现时，上述实施例可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令或计算机程序。在计算机上加载或执行所述计算机指 令或计算机程序时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以为通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集合的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质。半导体介质可以是固态硬盘。
应理解，本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况，其中A,B可以是单数或者复数。另外，本文中字符“/”，一般表示前后关联对象是一种“或”的关系，但也可能表示的是一种“和/或”的关系，具体可参考前后文进行理解。
本申请中，“至少一个”是指一个或者多个，“多个”是指两个或两个以上。“以下至少一项(个)”或其类似表达，是指的这些项中的任意组合，包括单项(个)或复数项(个)的任意组合。例如，a,b,或c中的至少一项(个)，可以表示：a,b,c,a-b,a-c,b-c,或a-b-c，其中a,b,c可以是单个，也可以是多个。
应理解，在本申请的各种实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储 在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (38)

  1. 一种构建神经网络的方法，其特征在于，包括：
    根据目标神经网络的应用需求，构建搜索空间，所述搜索空间包括M个元素，所述M个元素用于指示M个网络结构，所述M个元素中的每一个元素包括对应的网络结构中的阶段中的块的个数和所述块的通道数，M为正整数；
    根据搜索空间中未评估元素的分布关系从所述M个网络结构中选择目标网络结构。
  2. 根据权利要求1所述的方法，其特征在于，所述目标神经网络的应用需求包括所述目标神经网络的运行速度、所述目标神经网络的参数量或所述目标神经网络的结构要求，其中，所述结构要求包括所述目标神经网络结构中的每个阶段的块的个数和每个所述块的通道数。
  3. 根据权利要求1或2所述的方法，其特征在于，所述根据目标神经网络的应用需求，构建搜索空间，包括：
    根据所述目标神经网络的应用需求，构建初始搜索空间，所述初始搜索空间包括N个初始元素，所述N个初始元素用于指示N个初始网络结构，所述N个初始元素中的每一个元素包括对应的初始网络结构中的阶段中的块的个数和所述块的通道数，N为大于或等于M的正整数；
    根据预设规则对所述N个初始元素指示的所述N个初始网络结构进行筛选，得到所述搜索空间中的所述M个元素，所述预设规则包括：
    若所述N个初始元素中的第一初始元素指示的第一初始网络结构中的每一个阶段中的块的个数不大于所述N个初始元素中第二初始元素指示的第二初始网络结构中对应阶段中的块的个数，且所述第一初始网络结构中的每一个阶段中的每个块的通道数均不大于所述第二初始网络结构中对应阶段中的每个块的通道数，则从所述初始搜索空间中删除所述第一初始元素。
  4. 根据权利要求1至3中任一项所述的方法，其特征在于，所述根据搜索空间中未评估元素的分布关系从所述M个网络结构中选择目标网络结构，包括：
    根据所述未评估元素的分布关系，确定所述未评估元素中的K个元素，K为小于M的正整数；
    根据所述K个元素从所述M个网络结构中选择目标网络结构。
  5. 根据权利要求4所述的方法，其特征在于，所述根据所述K个元素从所述M个网络结构中选择目标网络结构，包括：
    对所述未评估元素中的K个元素指示的K个网络结构进行评估，得到已评估元素的评估结果，所述已评估元素的评估结果包括所述K个网络结构的评估结果；
    根据已评估元素的评估结果，从所述M个网络结构中选择目标网络结构。
  6. 根据权利要求5所述的方法，其特征在于，所述根据已评估元素的评估结果，从所述M个网络结构中选择目标网络结构，包括：
    根据所述已评估元素的评估结果对第一未评估元素进行建模，得到所述第一未评估元素的模型，所述第一未评估元素包括所述搜索空间中除所述已评估元素之外的其他元 素；
    根据所述第一未评估元素的模型，从所述M个网络结构中选择目标网络结构。
  7. 根据权利要求6所述的方法，其特征在于，在根据搜索空间中未评估元素的分布关系从所述M个网络结构中未选择出目标网络结构的情况下，所述方法还包括：
    根据所述第一未评估元素的分布关系与基于已评估元素构建的第一未评估元素的模型，从所述M个网络结构中选择目标网络结构。
  8. 根据权利要求7所述的方法，其特征在于，所述根据所述第一未评估元素的分布关系与基于已评估元素构建的第一未评估元素的模型，从所述M个网络结构中选择目标网络结构，包括：
    根据所述第一未评估元素的分布关系，确定所述第一未评估元素中的L个元素，L为小于M的正整数；
    根据所述L个元素与基于已评估元素构建的第一未评估元素的模型从所述M个网络结构中选择目标网络结构。
  9. 根据权利要求8所述的方法，其特征在于，所述根据所述L个元素与基于已评估元素构建的第一未评估元素的模型从所述M个网络结构中选择目标网络结构，包括：
    根据所述基于已评估元素构建的第一未评估元素的模型，从所述L个元素中确定Q个元素，Q为小于L的正整数；
    对所述Q个元素指示的Q个网络结构进行评估，得到第一已评估元素的评估结果，所述第一已评估元素的评估结果包括所述K个网络结构的评估结果和所述Q个网络结构的评估结果；
    根据第一已评估元素的评估结果与第二未评估元素的分布关系，从所述M个网络结构中选择目标网络结构，所述第二未评估元素包括所述搜索空间中除所述第一已评估元素之外的其他元素。
  10. 根据权利要求8所述的方法，其特征在于，所述第一未评估元素的分布关系为所述第一未评估元素的聚类结果，所述L个元素分别为所述第一未评估元素的聚类结果包括的L个簇中的元素。
  11. 根据权利要求10所述的方法，其特征在于，所述L个元素分别为所述L个簇的中心对应的L个元素。
  12. 根据权利要求7至11中任一项所述的方法，其特征在于，在根据所述第一未评估元素的分布关系与基于已评估元素构建的第一未评估元素的模型，从所述M个网络结构中未选择出目标网络结构的情况下，所述方法还包括：
    根据所述第二未评估元素的分布关系与基于第一已评估元素构建的第二未评估元素的模型，从所述M个网络结构中重新选择目标网络结构。
  13. 根据权利要求6至12中任一项所述的方法，其特征在于，所述基于已评估元素构建的第一未评估元素的模型，包括：
    所述第一未评估元素的模型是根据已评估的网络结构的评估结果及下述公式得到的：
    Figure PCTCN2020116673-appb-100001
    和/或(g(x)-l(x)) 2；
    其中，x表示所述搜索空间中未评估的网络结构，y *表示精度阈值，g(x)表示精度大于精度阈值y *的网络结构x的条件概率密度函数，
    Figure PCTCN2020116673-appb-100002
    l(x)表示精度小于或等于精度阈值y *的网络结构x的条件概率密度函数，
    Figure PCTCN2020116673-appb-100003
    G *(x|x i)表示混合高斯分布，
    Figure PCTCN2020116673-appb-100004
    κ(x,x i)表示x与x i之间的欧式距离，κ*(x,x i)表示由κ(x,x i)组成的距离函数，
    Figure PCTCN2020116673-appb-100005
    x i表示所述已评估的网络结构中的第i个网络结构，y i表示所述已评估的网络结构中的第i个网络结构的精度，ω i表示G *(x|x i)对应的权重，Z表示归一化因子，σ表示G *(x|x i)的一个超参，i为正整数，e为自然对数函数的底数。
  14. 根据权利要求6至12中任一项所述的方法，其特征在于，所述基于已评估元素构建的第一未评估元素的模型，包括：
    所述第一未评估元素的模型是根据已评估的网络结构的评估结果及下述公式得到的：
    Figure PCTCN2020116673-appb-100006
    其中，x表示所述搜索空间中的未评估元素；
    Figure PCTCN2020116673-appb-100007
    x i表示所述已评估元素中的第i个元素，y i表示所述已评估元素中的第i个元素的精度；τ＝max(y i)，
    Figure PCTCN2020116673-appb-100008
    表示期望函数；f(x)为服从高斯分布的随机变量，f(x)的均值μ(x)及f(x)的方差σ(x)与
    Figure PCTCN2020116673-appb-100009
    跟输入x满足下述关系：
    μ(x)＝k T(K+η 2I) -1Y，
    σ(x)＝1-k T(K+η 2I) -1k，
    其中，n为所述已评估的网络结构的个数，Y为y i组成的向量，Y∈R n，Y i＝y i，k为κ(x,x i)组成的向量，k∈R n，k i＝κ(x,x i)，K为κ(x i,x j)组成的矩阵为，K∈R n×n，K i，j＝κ(x i,x j)，
    Figure PCTCN2020116673-appb-100010
    σ为一个超参，e为自然对数函数的底数；I为单位矩阵，η也为一个超参，i,j为正整数。
  15. 根据权利要求4至14中任一项所述的方法，其特征在于，所述未评估元素的分布关系为所述未评估元素的聚类结果，所述K个元素分别为所述未评估元素的聚类结果包括的K个簇中的元素。
  16. 根据权利要求15所述的方法，其特征在于，所述K个元素分别为所述K个簇的中心对应的K个元素。
  17. 一种图像处理方法，其特征在于，包括：
    获取待处理图像；
    根据目标神经网络对所述待处理图像进行分类，得到所述待处理图像的分类结果；
    其中，所述目标神经网络由目标网络结构构建而成，所述目标网络结构是通过权利要求1至16中任一项所述的方法得到的。
  18. 一种构建神经网络的装置，其特征在于，包括：
    构建单元，用于根据目标神经网络的应用需求，构建搜索空间，所述搜索空间包括M个元素，所述M个元素用于指示M个网络结构，所述M个元素中的每一个元素包括 对应的网络结构中的阶段中的块的个数和所述块的通道数，M为正整数；
    选择单元，用于根据搜索空间中未评估元素的分布关系从所述M个网络结构中选择目标网络结构。
  19. 根据权利要求18所述的装置，其特征在于，所述目标神经网络的应用需求包括所述目标神经网络的运行速度、所述目标神经网络的参数量或所述目标神经网络的结构要求，其中，所述结构要求包括所述目标神经网络结构中的每个阶段的块的个数和每个所述块的通道数。
  20. 根据权利要求18或19所述的装置，其特征在于，所述构建单元具体用于：
    根据所述目标神经网络的应用需求，构建初始搜索空间，所述初始搜索空间包括N个初始元素，所述N个初始元素用于指示N个初始网络结构，所述N个初始元素中的每一个元素包括对应的初始网络结构中的阶段中的块的个数和所述块的通道数，N为大于或等于M的正整数；
    根据预设规则对所述N个初始元素指示的所述N个初始网络结构进行筛选，得到所述搜索空间中的所述M个元素，所述预设规则包括：
    若所述N个初始元素中的第一初始元素指示的第一初始网络结构中的每一个阶段中的块的个数不大于所述N个初始元素中第二初始元素指示的第二初始网络结构中对应阶段中的块的个数，且所述第一初始网络结构中的每一个阶段中的每个块的通道数均不大于所述第二初始网络结构中对应阶段中的每个块的通道数，则从所述初始搜索空间中删除所述第一初始元素。
  21. 根据权利要求18至20中任一项所述的装置，其特征在于，所述选择单元具体用于：
    根据所述未评估元素的分布关系，确定所述未评估元素中的K个元素，K为小于M的正整数；
    根据所述K个元素从所述M个网络结构中选择目标网络结构。
  22. 根据权利要求21所述的装置，其特征在于，所述选择单元具体用于：
    对所述未评估元素中的K个元素指示的K个网络结构进行评估，得到已评估元素的评估结果，所述已评估元素的评估结果包括所述K个网络结构的评估结果；
    根据已评估元素的评估结果，从所述M个网络结构中选择目标网络结构。
  23. 根据权利要求22所述的装置，其特征在于，所述选择单元具体用于：
    根据所述已评估元素的评估结果对第一未评估元素进行建模，得到所述第一未评估元素的模型，所述第一未评估元素包括所述搜索空间中除所述已评估元素之外的其他元素；
    根据所述第一未评估元素的模型，从所述M个网络结构中选择目标网络结构。
  24. 根据权利要求23所述的装置，其特征在于，在根据搜索空间中未评估元素的分布关系从所述M个网络结构中未选择出目标网络结构的情况下，所述选择单元还用于：
    根据所述第一未评估元素的分布关系与基于已评估元素构建的第一未评估元素的模型，从所述M个网络结构中选择目标网络结构。
  25. 根据权利要求24所述的装置，其特征在于，所述选择单元具体用于：
    根据所述第一未评估元素的分布关系，确定所述第一未评估元素中的L个元素，L 为小于M的正整数；
    根据所述L个元素与基于已评估元素构建的第一未评估元素的模型从所述M个网络结构中选择目标网络结构。
  26. 根据权利要求25所述的装置，其特征在于，所述选择单元具体用于：
    根据所述基于已评估元素构建的第一未评估元素的模型，从所述L个元素中确定Q个元素，Q为小于L的正整数；
    对所述Q个元素指示的Q个网络结构进行评估，得到第一已评估元素的评估结果，所述第一已评估元素的评估结果包括所述K个网络结构的评估结果和所述Q个网络结构的评估结果；
    根据第一已评估元素的评估结果与第二未评估元素的分布关系，从所述M个网络结构中选择目标网络结构，所述第二未评估元素包括所述搜索空间中除所述第一已评估元素之外的其他元素。
  27. 根据权利要求25所述的装置，其特征在于，所述第一未评估元素的分布关系为所述第一未评估元素的聚类结果，所述L个元素分别为所述第一未评估元素的聚类结果包括的L个簇中的元素。
  28. 根据权利要求27所述的装置，其特征在于，所述L个元素分别为所述L个簇的中心对应的L个元素。
  29. 根据权利要求24至28中任一项所述的装置，其特征在于，在根据所述第一未评估元素的分布关系与基于已评估元素构建的第一未评估元素的模型，从所述M个网络结构中未选择出目标网络结构的情况下，所述选择单元还用于：
    根据所述第二未评估元素的分布关系与基于第一已评估元素构建的第二未评估元素的模型，从所述M个网络结构中重新选择目标网络结构。
  30. 根据权利要求23至29中任一项所述的装置，其特征在于，所述选择单元具体用于：
    所述第一未评估元素的模型是根据已评估的网络结构的评估结果及下述公式得到的：
    Figure PCTCN2020116673-appb-100011
    和/或(g(x)-l(x)) 2；
    其中，x表示所述搜索空间中未评估的网络结构，y *表示精度阈值，g(x)表示精度大于精度阈值y *的网络结构x的条件概率密度函数，
    Figure PCTCN2020116673-appb-100012
    l(x)表示精度小于或等于精度阈值y *的网络结构x的条件概率密度函数，
    Figure PCTCN2020116673-appb-100013
    G *(x|x i)表示混合高斯分布，
    Figure PCTCN2020116673-appb-100014
    κ(x,x i)表示x与x i之间的欧式距离，κ*(x,x i)表示由κ(x,x i)组成的距离函数，
    Figure PCTCN2020116673-appb-100015
    x i表示所述已评估的网络结构中的第i个网络结构，y i表示所述已评估的网络结构中的第i个网络结构的精度，ω i表示G *(x|x i)对应的权重，Z表示归一化因子，σ表示G *(x|x i)的一个超参，i为正整数，e为自然对数函数的底数。
  31. 根据权利要求23至29中任一项所述的装置，其特征在于，所述选择单元具体用于：
    所述第一未评估元素的模型是根据已评估的网络结构的评估结果及下述公式得到的：
    Figure PCTCN2020116673-appb-100016
    其中，x表示所述搜索空间中的未评估元素；
    Figure PCTCN2020116673-appb-100017
    x i表示所述已评估元素中的第i个元素，y i表示所述已评估元素中的第i个元素的精度；τ＝max(y i)，
    Figure PCTCN2020116673-appb-100018
    表示期望函数；f(x)为服从高斯分布的随机变量，f(x)的均值μ(x)及f(x)的方差σ(x)与
    Figure PCTCN2020116673-appb-100019
    跟输入x满足下述关系：
    μ(x)＝k T(K+η 2I) -1Y，
    σ(x)＝1-k T(K+η 2I) -1k，
    其中，n为所述已评估的网络结构的个数，Y为y i组成的向量，Y∈R n，Y i＝y i，k为κ(x,x i)组成的向量，k∈R n，k i＝κ(x,x i)，K为κ(x i,x j)组成的矩阵为，K∈R n×n，K i，j＝κ(x i,x j)，
    Figure PCTCN2020116673-appb-100020
    σ为一个超参，e为自然对数函数的底数；I为单位矩阵，η也为一个超参，i,j为正整数。
  32. 根据权利要求21至31中任一项所述的装置，其特征在于，所述未评估元素的分布关系为所述未评估元素的聚类结果，所述K个元素分别为所述未评估元素的聚类结果包括的K个簇中的元素。
  33. 根据权利要求32所述的装置，其特征在于，所述K个元素分别为所述K个簇的中心对应的K个元素。
  34. 一种图像处理装置，其特征在于，包括：
    获取单元，用于获取待处理图像；
    图像处理单元，用于根据目标神经网络对所述待处理图像进行分类，得到所述待处理图像的分类结果；
    其中，所述目标神经网络由目标网络结构构建而成，所述目标网络结构是通过权利要求1至16中任一项所述的方法得到的。
  35. 一种构建神经网络的装置，其特征在于，包括处理器和存储器，所述存储器用于存储程序指令，所述处理器用于调用所述程序指令来执行权利要求1至16中任一项所述的方法。
  36. 一种图像增强装置，其特征在于，包括处理器和存储器，所述存储器用于存储程序指令，所述处理器用于调用所述程序指令来执行权利要求17所述的方法。
  37. 一种计算机可读存储介质，其特征在于，所述计算机可读介质存储用于设备执行的程序代码，该程序代码包括用于执行如权利要求1至16或17中任一项所述的方法。
  38. 一种芯片，其特征在于，所述芯片包括处理器与数据接口，所述处理器通过所述数据接口读取存储器上存储的指令，以执行如权利要求1至16或17中任一项所述的方法。
PCT/CN2020/116673 2019-09-24 2020-09-22 构建神经网络的方法与装置、及图像处理方法与装置 WO2021057690A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP20868562.8A EP4030347A4 (en) 2019-09-24 2020-09-22 METHOD AND DEVICE FOR CONSTRUCTING NEURONAL NETWORK, AND METHOD AND DEVICE FOR IMAGE PROCESSING
US17/700,098 US20220222934A1 (en) 2019-09-24 2022-03-21 Neural network construction method and apparatus, and image processing method and apparatus

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN201910904314 2019-09-24
CN201910904314.7 2019-09-24
CN202010588884.2A CN112633460A (zh) 2019-09-24 2020-06-24 构建神经网络的方法与装置、及图像处理方法与装置
CN202010588884.2 2020-06-24

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/700,098 Continuation US20220222934A1 (en) 2019-09-24 2022-03-21 Neural network construction method and apparatus, and image processing method and apparatus

Publications (1)

Publication Number Publication Date
WO2021057690A1 true WO2021057690A1 (zh) 2021-04-01

Family

ID=75165610

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/116673 WO2021057690A1 (zh) 2019-09-24 2020-09-22 构建神经网络的方法与装置、及图像处理方法与装置

Country Status (3)

Country Link
US (1) US20220222934A1 (zh)
EP (1) EP4030347A4 (zh)
WO (1) WO2021057690A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117197487A (zh) * 2023-09-05 2023-12-08 东莞常安医院有限公司 一种免疫胶体金诊断试纸条自动识别系统

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11922314B1 (en) * 2018-11-30 2024-03-05 Ansys, Inc. Systems and methods for building dynamic reduced order physical models

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170161606A1 (en) * 2015-12-06 2017-06-08 Beijing University Of Technology Clustering method based on iterations of neural networks
CN109284820A (zh) * 2018-10-26 2019-01-29 北京图森未来科技有限公司 一种深度神经网络的结构搜索方法及装置
CN109598332A (zh) * 2018-11-14 2019-04-09 北京市商汤科技开发有限公司 神经网络生成方法及装置、电子设备和存储介质
CN110020667A (zh) * 2019-02-21 2019-07-16 广州视源电子科技股份有限公司 神经网络结构的搜索方法、系统、存储介质以及设备
CN110175671A (zh) * 2019-04-28 2019-08-27 华为技术有限公司 神经网络的构建方法、图像处理方法及装置
CN110209857A (zh) * 2019-05-23 2019-09-06 厦门市美亚柏科信息股份有限公司 基于神经网络结构搜索的车辆多属性识别方法、装置、介质

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170329892A1 (en) * 2016-05-10 2017-11-16 Accutar Biotechnology Inc. Computational method for classifying and predicting protein side chain conformations
CN108133223B (zh) * 2016-12-01 2020-06-26 富士通株式会社 确定卷积神经网络cnn模型的装置和方法
US20200097810A1 (en) * 2018-09-25 2020-03-26 Oracle International Corporation Automated window based feature generation for time-series forecasting and anomaly detection
US11604992B2 (en) * 2018-11-02 2023-03-14 Microsoft Technology Licensing, Llc Probabilistic neural network architecture generation
US11531861B2 (en) * 2018-11-06 2022-12-20 Google Llc Neural architecture search with factorized hierarchical search space

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170161606A1 (en) * 2015-12-06 2017-06-08 Beijing University Of Technology Clustering method based on iterations of neural networks
CN109284820A (zh) * 2018-10-26 2019-01-29 北京图森未来科技有限公司 一种深度神经网络的结构搜索方法及装置
CN109598332A (zh) * 2018-11-14 2019-04-09 北京市商汤科技开发有限公司 神经网络生成方法及装置、电子设备和存储介质
CN110020667A (zh) * 2019-02-21 2019-07-16 广州视源电子科技股份有限公司 神经网络结构的搜索方法、系统、存储介质以及设备
CN110175671A (zh) * 2019-04-28 2019-08-27 华为技术有限公司 神经网络的构建方法、图像处理方法及装置
CN110209857A (zh) * 2019-05-23 2019-09-06 厦门市美亚柏科信息股份有限公司 基于神经网络结构搜索的车辆多属性识别方法、装置、介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4030347A4

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117197487A (zh) * 2023-09-05 2023-12-08 东莞常安医院有限公司 一种免疫胶体金诊断试纸条自动识别系统
CN117197487B (zh) * 2023-09-05 2024-04-12 东莞常安医院有限公司 一种免疫胶体金诊断试纸条自动识别系统

Also Published As

Publication number Publication date
EP4030347A4 (en) 2022-11-16
US20220222934A1 (en) 2022-07-14
EP4030347A1 (en) 2022-07-20

Similar Documents

Publication Publication Date Title
WO2021120719A1 (zh) 神经网络模型更新方法、图像处理方法及装置
WO2020221200A1 (zh) 神经网络的构建方法、图像处理方法及装置
WO2021218517A1 (zh) 获取神经网络模型的方法、图像处理方法及装置
WO2022083536A1 (zh) 一种神经网络构建方法以及装置
WO2021057056A1 (zh) 神经网络架构搜索方法、图像处理方法、装置和存储介质
WO2021164772A1 (zh) 训练跨模态检索模型的方法、跨模态检索的方法和相关装置
WO2022052601A1 (zh) 神经网络模型的训练方法、图像处理方法及装置
WO2022001805A1 (zh) 一种神经网络蒸馏方法及装置
WO2020192736A1 (zh) 物体识别方法及装置
WO2021022521A1 (zh) 数据处理的方法、训练神经网络模型的方法及设备
WO2021008206A1 (zh) 神经网络结构的搜索方法、图像处理方法和装置
WO2021244249A1 (zh) 一种分类器的训练方法、数据处理方法、系统以及设备
WO2021018251A1 (zh) 图像分类方法及装置
WO2021018245A1 (zh) 图像分类方法及装置
WO2021164750A1 (zh) 一种卷积层量化方法及其装置
WO2021227787A1 (zh) 训练神经网络预测器的方法、图像处理方法及装置
CN110222718B (zh) 图像处理的方法及装置
WO2021136058A1 (zh) 一种处理视频的方法及装置
WO2021129668A1 (zh) 训练神经网络的方法和装置
WO2022007867A1 (zh) 神经网络的构建方法和装置
WO2022156475A1 (zh) 神经网络模型的训练方法、数据处理方法及装置
WO2024067884A1 (zh) 一种数据处理方法及相关装置
US20220222934A1 (en) Neural network construction method and apparatus, and image processing method and apparatus
WO2023125628A1 (zh) 神经网络模型优化方法、装置及计算设备
WO2022063076A1 (zh) 对抗样本的识别方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20868562

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2020868562

Country of ref document: EP

Effective date: 20220412