WO2019205604A1 - 图像处理方法、训练方法、装置、设备、介质及程序 - Google Patents

图像处理方法、训练方法、装置、设备、介质及程序 Download PDF

Info

Publication number
WO2019205604A1
WO2019205604A1 PCT/CN2018/116574 CN2018116574W WO2019205604A1 WO 2019205604 A1 WO2019205604 A1 WO 2019205604A1 CN 2018116574 W CN2018116574 W CN 2018116574W WO 2019205604 A1 WO2019205604 A1 WO 2019205604A1
Authority
WO
WIPO (PCT)
Prior art keywords
neural network
feature data
image
floating point
teacher
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2018/116574
Other languages
English (en)
French (fr)
Inventor
韦祎
秦红伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Sensetime Technology Development Co Ltd
Original Assignee
Beijing Sensetime Technology Development Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Sensetime Technology Development Co Ltd filed Critical Beijing Sensetime Technology Development Co Ltd
Priority to JP2020533824A priority Critical patent/JP2021507397A/ja
Priority to SG11201912721YA priority patent/SG11201912721YA/en
Priority to KR1020207016931A priority patent/KR102478000B1/ko
Publication of WO2019205604A1 publication Critical patent/WO2019205604A1/zh
Priority to US16/700,348 priority patent/US11334763B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G06F18/2148—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the process organisation or structure, e.g. boosting cascade
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
    • G06F18/2178—Validation; Performance evaluation; Active pattern learning techniques based on feedback of a supervisor
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0495—Quantised networks; Sparse networks; Compressed networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/096—Transfer learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks

Definitions

  • the present application relates to computer vision technology, and more particularly to an image processing method, an image processing device, a training method of a neural network, a training device for a neural network, an electronic device, a computer readable storage medium, and a computer program.
  • Neural networks for example, convolutional neural networks, etc.
  • neural networks cannot be directly applied to electronic devices such as smart mobile phones and digital cameras due to factors such as the amount of calculation and large parameter quantities.
  • Compression techniques can often reduce the size of neural networks and increase the speed of neural networks.
  • compression techniques often degrade the accuracy of neural networks.
  • Embodiments of the present application provide a technical solution for image processing and training a neural network.
  • an image processing method comprising: inputting an image to be processed into a neural network; forming discrete feature data of the image to be processed via the neural network; wherein the nerve The network is trained based on the guidance information, and the neural network is used as a student neural network during the training process, and the guidance information includes: discrete feature data formed by the teacher neural network for the image sample and the student neural network for the The difference between the discrete feature data formed by the image samples.
  • the neural network includes: a neural network having a size of no more than 2M.
  • the forming the discrete feature data of the image to be processed via the neural network comprises: forming floating point feature data of the image to be processed via the neural network, and The point feature data is quantized into discrete feature data of the image to be processed.
  • the forming the floating point feature data of the image to be processed via the neural network includes: performing floating point feature data extraction on the image to be processed via the neural network, and The extracted floating point feature data is converted into floating point feature data meeting predetermined requirements to form floating point feature data of the image to be processed.
  • the converting the extracted floating point feature data into the floating point feature data meeting the predetermined requirement comprises: converting the floating point feature data into floating point feature data having a predetermined channel number; And/or converting the floating point feature data into floating point feature data having a predetermined size.
  • the method further includes: performing, by the neural network, corresponding visual task processing on the image to be processed according to the discrete feature data of the image to be processed.
  • the guidance information further includes: a difference between a visual task processing result output by the student neural network for the image sample and the annotation information of the image sample.
  • the performing visual task processing on the image to be processed according to the discrete feature data of the image to be processed via the neural network includes: via the neural network, according to the The discrete feature data of the image to be processed is subjected to classification processing on the image to be processed; or the object detection processing is performed on the image to be processed according to the discrete feature data of the image to be processed via the neural network.
  • the guidance information further includes: a difference between a classification processing result output by the student neural network for the image sample, and a classification annotation information of the image sample; or the student neural network is directed to The difference between the object detection processing result output by the image sample and the labeling information of the detection frame of the image sample.
  • the training process of the neural network includes: inputting image samples into a student neural network and a teacher neural network, respectively; forming discrete feature data of the image samples respectively through a student neural network and a teacher neural network. And supervising learning the student neural network according to the guiding information; wherein the guiding information comprises: discrete feature data formed by the teacher neural network for the image sample, and discrete feature data formed by the student neural network for the image sample The difference between the two.
  • the training process of the neural network further includes: performing visual task processing on the image samples according to the discrete feature data of the image samples via the student neural network;
  • Supervised learning of the student neural network includes: discrete feature data formed by the teacher neural network for image samples, and differences between the student neural network and discrete feature data formed for the image samples, and visual output of the student neural network
  • the difference between the task processing result and the annotation information of the image sample is guidance information, and the student neural network is supervised and learned.
  • the teacher neural network includes: a successfully trained floating-point teacher neural network, and the successfully trained floating-point teacher neural network is configured to form floating-point feature data for an input image, and according to the Floating point feature data, visual task processing on the input image; and quantization auxiliary unit for converting floating point feature data formed by the floating point teacher neural network into discrete feature data, and providing the discrete feature data to the floating point teacher
  • the neural network causes the floating-point teacher neural network to perform visual task processing on the input image according to the discrete feature data.
  • the training process of the teacher neural network includes: inputting image samples into the successfully trained floating-point teacher neural network; and extracting floating-point features of the image samples via the successfully trained floating-point teacher neural network.
  • Data converting the floating point feature data into discrete feature data via the quantization auxiliary unit, and performing visual task processing on the image sample according to the discrete feature data via the successfully trained floating point teacher neural network; processing the result and the image with the visual task
  • the difference between the annotation information of the sample is the guidance information, and the network parameter adjustment is performed on the successfully trained floating-point teacher neural network.
  • the training process of the floating-point teacher neural network includes: inputting an image sample into a floating-point teacher neural network to be trained; and extracting the floating of the image sample via a floating-point teacher neural network to be trained. Pointing feature data, and performing visual task processing on the image sample according to the floating point feature data; using the difference between the visual task processing result and the annotation information of the image sample as guiding information, for the floating point to be trained
  • the teacher neural network conducts supervised learning.
  • a training method of a neural network is provided.
  • the neural network to be trained is used as a student neural network during training, and the training method includes: inputting image samples into the student neural network and a teacher neural network; respectively, forming discrete feature data of the image sample via a student neural network and a teacher neural network; and supervising learning the student neural network according to the guidance information; wherein the guidance information includes: a teacher neural network The difference between the discrete feature data formed by the image samples and the discrete feature data formed by the student neural network for the image samples.
  • the training process of the neural network as a student neural network further includes: performing visual task processing on the image samples according to the discrete feature data of the image samples via the student neural network;
  • Guidance information for supervised learning of the student neural network includes: discrete feature data formed by the teacher neural network for the image sample, and difference between the student neural network and the discrete feature data formed by the student neural network, and the student neural network
  • the difference between the output visual task processing result and the annotation information of the image sample is guidance information, and the student neural network is supervised and learned.
  • the teacher neural network includes: a successfully trained floating point teacher neural network, the successfully trained floating point teacher neural network is configured to form floating point feature data for an input image, and according to the Floating point feature data, visual task processing on the input image; and quantization auxiliary unit for converting floating point feature data formed by the floating point teacher neural network into discrete feature data, and providing the discrete feature data to the floating point teacher
  • the neural network causes the floating-point teacher neural network to perform visual task processing on the input image according to the discrete feature data.
  • the training process of the teacher neural network includes: inputting image samples into the successfully trained floating-point teacher neural network; and extracting floating-point features of the image samples via the successfully trained floating-point teacher neural network.
  • Data converting the floating point feature data into discrete feature data via the quantization auxiliary unit, and performing visual task processing on the image sample according to the discrete feature data via the successfully trained floating point teacher neural network; processing the result and the image with the visual task
  • the difference between the annotation information of the sample is the guidance information, and the network parameter adjustment is performed on the successfully trained floating-point teacher neural network.
  • the training process of the floating-point teacher neural network includes: inputting an image sample into a floating-point teacher neural network to be trained; and extracting the floating of the image sample via a floating-point teacher neural network to be trained. Pointing feature data, and performing visual task processing on the image sample according to the floating point feature data; using the difference between the visual task processing result and the annotation information of the image sample as guiding information, for the floating point to be trained
  • the teacher neural network conducts supervised learning.
  • an image processing apparatus comprising: an input image module for inputting an image to be processed into a neural network; and a neural network for forming discrete features of the image to be processed Data; wherein the neural network is trained based on guidance information, and the neural network is used as a student neural network during training, and the guidance information includes: discrete feature data formed by the teacher neural network for image samples The difference between the discrete feature data formed by the student neural network for the image samples.
  • the neural network includes: a neural network having a size of no more than 2M.
  • the neural network is further configured to: form floating point feature data of the image to be processed, and quantize the floating point feature data into discrete feature data of the image to be processed.
  • the neural network performs floating point feature data extraction on the image to be processed, and converts the extracted floating point feature data into floating point feature data meeting a predetermined requirement to form a The floating point feature data of the processed image is described.
  • the neural network converts the floating point feature data into floating point feature data having a predetermined number of channels; and/or the neural network converts the floating point feature data into having A predetermined size of floating point feature data.
  • the neural network is further configured to perform corresponding visual task processing on the image to be processed according to the discrete feature data of the image to be processed.
  • the guidance information further includes: a difference between a visual task processing result output by the student neural network for the image sample and the annotation information of the image sample.
  • the neural network performs classification processing on the image to be processed according to discrete feature data of the image to be processed; or the neural network is based on discrete feature data of the image to be processed. And performing object detection processing on the image to be processed.
  • the guidance information further includes: a difference between a classification processing result output by the student neural network for the image sample, and a classification annotation information of the image sample; or the student neural network is directed to The difference between the object detection processing result output by the image sample and the labeling information of the detection frame of the image sample.
  • the apparatus further includes: an input image sample module, configured to input image samples into the student neural network and the teacher neural network, respectively, to form the images respectively through the student neural network and the teacher neural network. a discrete feature data of the sample; a monitoring module, configured to perform supervised learning on the student neural network according to the guiding information; wherein the guiding information comprises: discrete feature data formed by the teacher neural network for the image sample, and the student neural The difference between the discrete feature data formed by the network for image samples.
  • the monitoring module is further configured to: The difference between the discrete feature data formed by the network for the image sample, the discrete feature data formed by the student neural network for the image sample, and the difference between the visual task processing result output by the student neural network and the annotation information of the image sample, To guide the information, the student neural network is supervised and studied.
  • the teacher neural network includes: a successfully trained floating-point teacher neural network, and the successfully trained floating-point teacher neural network is configured to form floating-point feature data for an input image, and according to the Floating point feature data, visual task processing on the input image; and quantization auxiliary unit for converting floating point feature data formed by the floating point teacher neural network into discrete feature data, and providing the discrete feature data to the floating point teacher
  • the neural network causes the floating-point teacher neural network to perform visual task processing on the input image according to the discrete feature data.
  • the apparatus further includes: a teacher training module for training the teacher neural network, the teacher training module comprising: a first unit, configured to input image samples into the successfully trained float Point the teacher neural network to extract the floating point feature data of the image sample via the successfully trained floating point teacher neural network, convert the floating point feature data into discrete feature data via the quantization auxiliary unit, and pass the successfully trained floating point teacher neural
  • the network performs visual task processing on the image samples according to the discrete feature data;
  • the first monitoring unit is configured to use the difference between the visual task processing result and the annotation information of the image sample as a guide information, and the successfully trained floating point teacher
  • the neural network performs network parameter adjustment.
  • the teacher training module further includes: a second unit, configured to input an image sample into a floating point teacher neural network to be trained to extract the image via a floating point teacher neural network to be trained. a floating point feature data of the sample, and performing visual task processing on the image sample according to the floating point feature data; and a second monitoring unit, configured to use a difference between the visual task processing result and the annotation information of the image sample as a guide Information, supervised learning of the floating point teacher neural network to be trained.
  • a training device for a neural network is provided.
  • the neural network to be trained is used as a student neural network, and the device includes: an input image sample module for inputting image samples into the student neural network separately. And a teacher neural network to separately form discrete feature data of the image sample via a student neural network and a teacher neural network; and a supervising module, configured to perform supervised learning on the student neural network according to the guiding information; wherein the guiding The information includes: a difference between the discrete feature data formed by the teacher neural network for the image samples and the discrete feature data formed by the student neural network for the image samples.
  • the monitoring module is further configured to: use a teacher neural network
  • the difference between the discrete feature data formed for the image sample, the discrete feature data formed by the student neural network for the image sample, and the difference between the visual task processing result output by the student neural network and the annotation information of the image sample is Guidance information for supervised learning of the student neural network.
  • the teacher neural network includes: a successfully trained floating point teacher neural network, the successfully trained floating point teacher neural network is configured to form floating point feature data for an input image, and according to the Floating point feature data, visual task processing on the input image; and quantization auxiliary unit for converting floating point feature data formed by the floating point teacher neural network into discrete feature data, and providing the discrete feature data to the floating point teacher
  • the neural network causes the floating-point teacher neural network to perform visual task processing on the input image according to the discrete feature data.
  • the apparatus further includes: a teacher training module for training the teacher neural network, the teacher training module comprising: a first unit, configured to input image samples into the successfully trained float Point the teacher neural network to extract the floating point feature data of the image sample via the successfully trained floating point teacher neural network, convert the floating point feature data into discrete feature data via the quantization auxiliary unit, and pass the successfully trained floating point teacher neural
  • the network performs visual task processing on the image samples according to the discrete feature data;
  • the first monitoring unit is configured to use the difference between the visual task processing result and the annotation information of the image sample as a guide information, and the successfully trained floating point teacher
  • the neural network performs network parameter adjustment.
  • the teacher training module further includes: a second unit, configured to input an image sample into a floating point teacher neural network to be trained to extract the image via a floating point teacher neural network to be trained. a floating point feature data of the sample, and performing visual task processing on the image sample according to the floating point feature data; and a second monitoring unit, configured to use a difference between the visual task processing result and the annotation information of the image sample as a guide Information, supervised learning of the floating point teacher neural network to be trained.
  • an electronic device includes: a memory for storing a computer program; a processor for executing a computer program stored in the memory, and when the computer program is executed, Implementing any of the method embodiments of the present application.
  • a computer readable storage medium having stored thereon a computer program, which when executed by a processor, implements any of the method embodiments of the present application.
  • a computer program comprising computer instructions that, when executed in a processor of a device, implement any of the method embodiments of the present application.
  • the guidance information includes: the difference between the discrete feature data formed by the teacher neural network and the discrete feature data formed by the student neural network, which is beneficial to improve the discrete feature data formed by the teacher neural network and the student neural network respectively.
  • the probability of matching is facilitated to facilitate the transfer of knowledge of the teacher neural network into the student neural network. Since the present application does not limit the network parameters of the student neural network to be fixed-point network parameters, the neural network of the present application may not be limited by a specific instruction set and a specific device.
  • the technical solution provided by the present application is beneficial to improve the accuracy of the output of the neural network on the basis of reducing the scale of the neural network and improving the running speed of the neural network, that is, improving the performance of the neural network.
  • the technical solution provided by the present application is also beneficial to improve the application range of the neural network.
  • FIG. 2 is a flow chart of another embodiment of an image processing method of the present application.
  • FIG. 3 is a flow chart of still another embodiment of an image processing method of the present application.
  • FIG. 5 is a flowchart of an embodiment of a training method of a neural network according to the present application.
  • FIG. 6 is a flow chart of another embodiment of a training method of a neural network according to the present application.
  • FIG. 7 is a flow chart of still another embodiment of a training method for a neural network according to the present application.
  • FIG. 8 is a flowchart of an embodiment of a training method of a teacher neural network according to the present application.
  • FIG. 9 is a flow chart of an embodiment of a training method for a floating-point teacher neural network of the present application.
  • FIG. 10 is a schematic structural diagram of an embodiment of an image processing apparatus according to the present application.
  • FIG. 11 is a schematic structural diagram of an embodiment of a training apparatus for a neural network according to the present application.
  • FIG. 12 is a block diagram of an exemplary apparatus that implements an embodiment of the present application.
  • Embodiments of the present application can be applied to electronic devices such as terminal devices, computer systems, and servers, which can operate with numerous other general purpose or special purpose computing system environments or configurations.
  • Examples of well-known terminal devices, computing systems, environments, and/or configurations suitable for use with electronic devices such as terminal devices, computer systems, and servers including but not limited to: personal computer systems, server computer systems, thin clients, thick Client, handheld or laptop device, microprocessor based system, set top box, programmable consumer electronics, networked personal computer, small computer system, mainframe computer system and distributed cloud computing technology environment including any of the above systems, etc. .
  • Electronic devices such as terminal devices, computer systems, and servers may be described in the general context of computer system executable instructions (such as program modules) being executed by a computer system.
  • program modules may include routines, programs, target programs, components, logic, and data structures, etc., which perform particular tasks or implement particular abstract data types.
  • the computer system/server can be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices that are linked through a communication network.
  • program modules may be located on a local or remote computing system storage medium including storage devices.
  • FIG. 1 is a flow chart of an embodiment of an image processing method of the present application. As shown in FIG. 1, the method of this embodiment includes: operating S100 and operating S110. The respective operations in Fig. 1 will be described in detail below.
  • S100 Input the image to be processed into a neural network.
  • the image to be processed in the present application may be an image such as a static picture or a photo, or a video frame in a dynamic video, for example, a video frame in a video captured by a camera.
  • the image to be processed may be a part of the original image to be processed (ie, an image block that is segmented from the original image to be processed and includes the content of the original image to be processed), and the image to be processed may also include a complete original to be processed. image.
  • the image to be processed may also be a part of the image of the original image to be processed after a certain processing, and the image to be processed may also include the complete content of the image of the original image to be processed after a certain processing.
  • This application does not limit the representation of the image to be processed.
  • the size of the image to be processed in the present application is generally related to the size requirement of the input image by the neural network.
  • the size of the image to be processed may be 256 ⁇ 256 or the like.
  • the present application may perform a scaling process corresponding to the scale of the image to be processed.
  • the size of the image to be processed and the implementation of the scaling process are not limited in this application.
  • the neural network in the present application may be a neural network that does not exceed 2M in size. That is to say, the storage space of the neural network may not exceed 2M.
  • the neural network in the present application can be referred to as a minimal neural network compared to existing neural networks having a size of several tens of megabytes or even hundreds of megabytes.
  • the neural network of the present application may include, but is not limited to, a convolutional layer, a non-linear Relu layer, a pooled layer, a fully connected layer, and the like.
  • the neural network may be a deep neural network, and the more layers the neural network contains, the deeper the network.
  • the neural network of the present application may be a convolutional neural network.
  • the network structure of the neural network of the present application may be flexibly designed according to actual visual task requirements.
  • the embodiment of the present application does not limit the network structure of the neural network; for example, the network structure of the neural network of the present application may be, but not limited to, ALexNet.
  • the network structure used by neural networks such as Deep Residual Network (ResNet) or VGGnet (Visual Geometry Group Network).
  • the neural network in the present application is trained based on the way knowledge is migrated. That is to say, the neural network in the present application is used as a student neural network during the training process.
  • the guidance information used generally includes: a difference between the discrete feature data formed by the teacher neural network for the image sample and the discrete feature data formed by the student neural network for the image sample. That is to say, the discrete feature data formed by the teacher neural network for the image samples is used as a supervised signal for training the student neural network.
  • the loss function employed to train the neural network may include an L2 loss function or the like.
  • the training process of the neural network can be referred to the following description for FIG. 5 to FIG. 7, and the description thereof will not be repeated here.
  • the discrete feature data in this application may be referred to as fixed point feature data.
  • the feature data may include at least one feature vector, at least one feature map, or other form.
  • the network parameters of the teacher neural network in the present application do not need to be fixed-point network parameters, that is, the network parameters of the teacher neural network of the present application may be floating-point network parameters, and the teacher neural network may The floating point operation is performed.
  • the feature data finally formed by the teacher neural network is discrete feature data, that is, the floating point feature data extracted by the teacher neural network for the image is quantized and processed to form discrete feature data. It can be seen that the teacher neural network of the present application can be referred to as a quantitative teacher neural network.
  • the knowledge of the quantified teacher neural network can be transferred to the student neural network, and the network parameters of the student neural network do not need to be fixed-point network parameters, that is, the present application
  • the network parameters of the student neural network can be floating-point network parameters, and the student neural network can perform floating-point operations, so that after successful training, the student neural network can be protected from the specific instruction set and specific equipment, which is beneficial to improve the application of the neural network. range.
  • the operation S100 may be performed by a processor invoking a corresponding instruction stored in a memory, or may be performed by an input image module 1000 executed by the processor.
  • the neural network in the present application may first form floating point feature data of the image to be processed, and then the neural network quantizes the floating point feature data into discrete feature data, and the quantized discrete feature data is Discrete feature data of the image to be processed formed by the neural network.
  • An optional example of the neural network forming the floating point feature data of the image to be processed in the present application may be: the neural network performs floating point feature data extraction on the image to be processed, thereby obtaining a floating point feature data (hereinafter referred to as initial Floating point feature data), in general, the initial floating point feature data does not meet the predetermined requirements specified in advance for the floating point feature data, and therefore, the neural network can convert the initial floating point feature data obtained by the neural network into a predetermined requirement.
  • the floating point feature data, the floating point feature data meeting the predetermined requirement is the floating point feature data of the image to be processed formed by the neural network.
  • the manner in which the neural network in the present application quantizes the floating point feature data into the discrete feature data may include: converting the floating point number in the feature data into a corresponding fixed point number, for example, located in the feature data.
  • the floating point numbers of the [k-0.5, k+0.5] interval are all converted to k, where k is an integer.
  • the present application does not limit the implementation of the neural network to quantize floating point feature data into discrete feature data.
  • the predetermined requirements in the present application may be flexibly designed according to actual needs.
  • the predetermined requirements may include at least one of requirements for the number of channels and requirements for the size of the feature data.
  • the predetermined requirement may specify that the feature data should have a predetermined number of channels, and the predetermined requirement may specify that the feature data should have a predetermined size, that is, the length and width of the feature data should meet the requirements.
  • the number of channels of the initial floating point feature data formed by the neural network of the present application may be set according to actual requirements.
  • the number of channels of the initial floating point feature data formed by the neural network of the present application may be VGG. (Visual Geometry Group, visual geometry group) One-third of the number of channels of the neural network, and the number of channels of the transformed discrete feature data can be the number of channels of the VGG neural network.
  • the present application can convert the initial floating point feature data into floating point feature data having a predetermined number of channels by using a technique such as a convolution layer.
  • the present application does not limit the implementation of converting initial floating point feature data to floating point feature data having a predetermined number of channels.
  • the present application may utilize techniques such as upsampling to convert the initial floating point feature data into floating point feature data having a predetermined size.
  • the upsampling can be achieved by a deconvolution layer.
  • the present application does not limit the implementation of converting initial floating point feature data to floating point feature data having a predetermined size.
  • the neural network of the present application is used not only for forming discrete feature data of an image to be processed, but also for performing corresponding visual task processing on the image to be processed according to discrete feature data of the image to be processed.
  • the visual task processing in the present application can be flexibly set according to actual needs.
  • the visual task processing can be classification processing or object detection processing. That is to say, the neural network in the present application can classify the image to be processed according to the discrete feature data of the image to be processed formed to determine the category to which the image to be processed belongs (such as a cat or a dog or a person or a vehicle).
  • the neural network in the present application may perform object detection processing on the image to be processed according to discrete feature data of the image to be processed formed thereof to determine the position and category of the detection frame in the image to be processed (for example, the diagonal of the detection frame) The coordinates of the two vertices on the top and the category of the detection frame).
  • the detection frame in this application may also be referred to as a peripheral frame or the like. This application does not limit the representation of visual task processing.
  • the guidance information used by the present application to train the neural network generally includes: visual task processing results output by the neural network and image samples. Label the differences between the information. That is to say, the discrete feature data formed by the teacher neural network for the image samples is used as a supervised signal for training the student neural network. At the same time, the annotation information of the image samples is also used as a supervised signal for training the student neural network.
  • the neural network in the present application is not only used to form discrete feature data of an image to be processed, but also used to train the image to be processed according to discrete feature data of the image to be processed.
  • the discrete feature data formed by the teacher neural network for the image samples and the classified information of the image samples are used as the supervised signals for training the student neural network. That is, the difference between the classification processing result of the student neural network output and the classification labeling information of the image sample, and the discrete feature data formed by the teacher neural network for the image sample and the discrete feature data formed by the student neural network for the image sample The differences are used as guidance for training student neural networks.
  • the neural network in the present application is used not only for forming discrete feature data of an image to be processed, but also for performing object detection processing on the image to be processed according to discrete feature data of the image to be processed, in training
  • the discrete feature data formed by the teacher neural network for the image samples and the detection frame label information of the image samples are used as the supervised signals for training the student neural network.
  • the difference between the discrete feature data formed by the teacher neural network for the image samples and the discrete feature data formed by the student neural network for the image samples is used as the guidance information for training the student neural network.
  • the operation S110 may be performed by a processor invoking a corresponding instruction stored in a memory, or may be performed by a neural network 1010 executed by the processor.
  • FIG. 2 is a flow chart of another embodiment of an image processing method of the present application. As shown in FIG. 2, the method of this embodiment includes: operation S200, operation S210, and operation S220. The respective operations in Fig. 2 will be described in detail below.
  • S200 Input the image to be processed into a neural network.
  • the neural network of the present application typically includes a neural network that does not exceed 2M in size.
  • This neural network is used as a student neural network during the training process.
  • the guidance information used generally includes: the difference between the discrete feature data formed by the teacher neural network for the image sample and the discrete feature data formed by the student neural network for the image sample, and the student neural network for The difference between the visual task processing result of the image sample processing and the annotation information of the image sample. That is to say, the discrete feature data formed by the teacher neural network for the image samples and the annotation information of the image samples are used as supervised signals for training the student neural network.
  • the loss function employed to train the neural network may include an L2 loss function or the like. For the content of the network structure and the image to be processed, etc., the description of the method in the above method embodiment is not described in detail.
  • the operation S200 may be performed by a processor invoking a corresponding instruction stored in a memory, or may be performed by an input image module 1000 executed by the processor.
  • the neural network may first form floating point feature data of the image to be processed.
  • the neural network performs floating point feature data extraction on the image to be processed, and extracts the floating point feature.
  • the data is converted into floating point feature data that meets predetermined requirements (for example, converting floating point feature data into floating point feature data having a predetermined number of channels, and, for example, converting floating point feature data into floating point feature data having a predetermined size) , thereby forming floating point feature data of the image to be processed.
  • the neural network quantizes the floating point feature data into discrete feature data of the image to be processed, such as converting the floating point number in the floating point feature data into a corresponding fixed point number.
  • S220 Perform a corresponding visual task processing on the image to be processed according to the discrete feature data of the image to be processed via the neural network.
  • the visual task processing can be flexibly set according to the actual requirements.
  • For the content of the operation refer to the related description of S110 in the foregoing method implementation manner, which is not described in detail herein.
  • the operations S210 and S220 may be performed by a processor invoking a corresponding instruction stored in a memory, or may be performed by a neural network 1010 executed by the processor.
  • FIG. 3 is a flow chart of still another embodiment of an image processing method of the present application. As shown in FIG. 3, the embodiment method includes: an operation S300, an operation S310, and an operation S320. The respective operations in Fig. 3 will be described in detail below.
  • S300 Input the image to be processed into a neural network.
  • the neural network of the present application typically includes a neural network that does not exceed 2M in size.
  • This neural network is used as a student neural network during the training process.
  • the guidance information used generally includes: the difference between the discrete feature data formed by the teacher neural network for the image sample and the discrete feature data formed by the student neural network for the image sample, and the student neural network for The difference between the classification processing result of the image sample and the classification labeling information of the image sample. That is to say, the discrete feature data formed by the teacher neural network for the image samples and the classified annotation information of the image samples are used as the supervised signals for training the student neural network.
  • the loss function employed to train the neural network may include an L2 loss function or the like. For the content of the network structure and the image to be processed, etc., the description of the method in the above method embodiment is not described in detail.
  • the operation S300 may be performed by a processor invoking a corresponding instruction stored in a memory, or may be performed by an input image module 1000 executed by the processor.
  • the neural network may first form floating point feature data of the image to be processed.
  • the neural network performs floating point feature data extraction on the image to be processed, and extracts the floating point feature.
  • the data is converted into floating point feature data that meets predetermined requirements (for example, converting floating point feature data into floating point feature data having a predetermined number of channels, and, for example, converting floating point feature data into floating point feature data having a predetermined size) , thereby forming floating point feature data of the image to be processed.
  • the neural network quantizes the floating point feature data into discrete feature data of the image to be processed, such as converting the floating point number in the floating point feature data into a corresponding fixed point number.
  • S320 Perform, by using a neural network, classification processing on the image to be processed according to discrete feature data of the image to be processed.
  • the number of categories processed by this classification and the content of the categories can be flexibly set according to actual needs.
  • the operations S310 and S320 may be performed by a processor invoking a corresponding instruction stored in a memory, or may be performed by a neural network 1010 that is executed by the processor.
  • FIG. 4 is a flow chart of still another embodiment of an image processing method of the present application. As shown in FIG. 4, the embodiment method includes an operation S400, an operation S410, and an operation S420. The respective operations in Fig. 4 will be described in detail below.
  • the neural network of the present application typically includes a neural network that does not exceed 2M in size.
  • This neural network is used as a student neural network during the training process.
  • the guidance information used generally includes: the difference between the discrete feature data formed by the teacher neural network for the image sample and the discrete feature data formed by the student neural network for the image sample, and the student neural network for The image sample performs a difference between the object detection processing result of the object detection processing and the detection frame labeling information of the image sample (such as the position labeling information of the detection frame and the category labeling information of the detection frame, etc.).
  • the discrete feature data formed by the teacher neural network for the image samples and the detection frame labeling information of the image samples are used as the supervised signals for training the student neural network.
  • the loss function employed to train the neural network may include an L2 loss function or the like.
  • the operation S400 may be performed by a processor invoking a corresponding instruction stored in a memory, or may be performed by an input image module 1000 executed by the processor.
  • the neural network may first form floating point feature data of the image to be processed.
  • the neural network performs floating point feature data extraction on the image to be processed, and extracts the floating point feature.
  • the data is converted into floating point feature data that meets predetermined requirements (for example, converting floating point feature data into floating point feature data having a predetermined number of channels, and, for example, converting floating point feature data into floating point feature data having a predetermined size) , thereby forming floating point feature data of the image to be processed.
  • the neural network type quantizes the floating point feature data into discrete feature data of the image to be processed, such as converting the floating point number in the floating point feature data into a corresponding fixed point number.
  • S420 Perform object detection processing on the image to be processed according to discrete feature data of the image to be processed via a neural network.
  • the number of detection frame categories involved in the object detection processing and the content of the detection frame category can be flexibly set according to actual needs.
  • the operations S410 and S420 may be performed by a processor invoking a corresponding instruction stored in a memory, or may be performed by a neural network 1010 executed by the processor.
  • FIG. 5 is a flow chart of one embodiment of a training method for a neural network of the present application.
  • the neural network to be trained is used as a student neural network during the training process.
  • the embodiment method includes: operation S500, operation S510, and operation S520. The respective operations in Fig. 5 will be described in detail below.
  • the present application can obtain image samples from a training data set.
  • the training data set in the present application includes a plurality of image samples for training a neural network.
  • each image sample is provided with annotation information related to a corresponding visual task.
  • the annotation information of the image sample may include: classification annotation information of the image sample.
  • the annotation information of the image sample may also include: detection frame position label information of the image sample, and detection frame classification label information.
  • the present application can read one or more image samples from the training data set at a time according to a random reading manner or a sequential reading manner according to an image sample arrangement order.
  • the size of the image samples provided to the student neural network and the teacher neural network by the present application is generally related to the size requirements of the input image of the neural network, such as the size of the image samples may be 256 ⁇ 256 or the like.
  • the present application may perform scaling processing on the read image samples such that each image sample provided to the student neural network and the teacher neural network has a predetermined size, respectively.
  • the present application does not limit the size of the image sample and the implementation of the scaling process.
  • the network parameters of the teacher neural network in the present application do not need to be fixed-point network parameters, that is, the network parameters of the teacher neural network of the present application may be floating-point network parameters, and the teacher neural network may The floating point operation is performed.
  • the feature data finally formed by the teacher neural network is discrete feature data, that is, the floating point feature data extracted by the teacher neural network for the image sample is quantized to form discrete feature data.
  • the present application can train the student neural network by using the teacher neural network forming the discrete feature data, so that the knowledge of the teacher neural network can be transferred to the student neural network, and the network parameters of the student neural network do not need to be fixed-point network parameters.
  • the network parameters of the student neural network of the present application may be floating-point network parameters, and the student neural network may perform floating-point operations, so that after successful training, the student neural network may not be limited by a specific instruction set and a specific device. It is beneficial to improve the application range of neural network.
  • the floating point feature data obtained by floating point operation can not only make the discrete feature data maintain good accuracy after being converted into discrete feature data, but also beneficial to teachers.
  • the discrete feature data output by the neural network matches the discrete feature data output by the student neural network. Therefore, it is beneficial to migrate the knowledge of the teacher neural network to the student neural network, which is beneficial to improve the performance of the neural network.
  • S510 Form discrete feature data of the image samples respectively through the student neural network and the teacher neural network.
  • the student neural network may first form floating point feature data of the image sample.
  • the student neural network performs floating point feature data extraction on the image sample, and extracts the extracted floating point feature data. Converting to floating point feature data meeting predetermined requirements (for example, converting floating point feature data into floating point feature data having a predetermined number of channels, and, for example, converting floating point feature data into floating point feature data having a predetermined size), Thereby the floating point feature data of the image sample is formed.
  • the student neural network quantizes the floating point feature data into discrete feature data of the image sample, such as converting the floating point number in the floating point feature data into a corresponding fixed point number.
  • the teacher neural network may first form floating point feature data of the image sample.
  • the teacher neural network performs floating point feature data extraction on the image sample, and converts the extracted floating point feature data into a predetermined schedule.
  • Required floating point feature data for example, converting floating point feature data into floating point feature data having a predetermined number of channels, and for example, converting floating point feature data into floating point feature data having a predetermined size
  • the teacher neural network quantizes the floating point feature data into discrete feature data of the image sample, such as converting the floating point number in the floating point feature data into a corresponding fixed point number.
  • the operations S500 and S510 may be performed by a processor invoking a corresponding instruction stored in a memory, or may be performed by an input image sample module 1020 that is executed by the processor.
  • the guidance information of the present application generally includes: a difference between the discrete feature data formed by the teacher neural network for the image samples and the discrete feature data formed by the student neural network for the image samples.
  • the guidance information of the present application also generally includes: a difference between a corresponding visual task processing result output by the student neural network and the annotation information of the image sample. That is to say, the discrete feature data formed by the teacher neural network for the image samples is used as a supervised signal for training the student neural network.
  • the annotation information of the image samples is also used as a supervised signal for training the student neural network.
  • the present application can reduce the difference between the discrete feature data formed by the teacher neural network for the image sample, the discrete feature data formed by the student neural network for the image sample, and the visual task processing result and the image sample of the student neural network output.
  • the student neural network is supervised and learned by using the corresponding loss function (such as the L2 loss function).
  • the training process ends when the training for the student neural network reaches a predetermined iteration condition.
  • the predetermined iteration condition in the present application may include: a difference between the discrete feature data output by the student neural network and the discrete feature data output by the teacher neural network, and between the visual task processing result output by the student neural network and the annotation information of the image sample. The differences meet the predetermined difference requirements. In the case that the difference satisfies the predetermined difference requirement, the successful training of the student neural network is completed.
  • the predetermined iteration conditions in the present application may also include training the student neural network, the number of image samples used reaches a predetermined number of requirements, and the like.
  • the operation S520 may be performed by a processor invoking a corresponding instruction stored in a memory, or may be performed by a supervisory module 1030 executed by the processor.
  • FIG. 6 is a flow chart of another embodiment of a training method for a neural network of the present application.
  • the neural network to be trained is used as a student neural network during the training process.
  • the method of this embodiment includes: operation S600, operation S610, operation S620, and operation S630. The respective operations in Fig. 6 will be described in detail below.
  • S610 Form discrete feature data of the image samples respectively through the student neural network and the teacher neural network.
  • the operations S600 and S610 may be performed by a processor invoking a corresponding instruction stored in a memory, or may be performed by an input image sample module 1020 that is executed by the processor.
  • S620 Perform classification processing on the image samples according to the discrete feature data of the image samples formed by the student neural network via the student neural network.
  • the present application may reduce the difference between discrete feature data formed by the teacher neural network for image samples, discrete feature data formed by the student neural network for image samples, and reduce the output of the student neural network.
  • the student neural network is supervised and learned by using the corresponding loss function (such as the L2 loss function).
  • the training process ends when the training for the student neural network reaches a predetermined iteration condition.
  • the predetermined iteration condition in the present application may include: a difference between the discrete feature data output by the student neural network and the discrete feature data output by the teacher neural network, and between the classification processing result output by the student neural network and the classification annotation information of the image sample. The differences meet the predetermined difference requirements. In the case that the difference satisfies the predetermined difference requirement, the successful training of the student neural network is completed.
  • the predetermined iteration conditions in the present application may also include training the student neural network, the number of image samples used reaches a predetermined number of requirements, and the like.
  • the student neural network has not been successfully trained.
  • Successfully trained student neural networks can be used to form discrete feature data for the image to be processed as well as classification processing.
  • the operations S620 and S630 may be performed by a processor invoking a corresponding instruction stored in a memory, or may be performed by a supervisory module 1030 executed by the processor.
  • FIG. 7 is a flow chart of still another embodiment of a training method for a neural network of the present application.
  • the neural network to be trained is used as a student neural network during the training process.
  • the method of this embodiment includes: operation S700, operation S710, operation S720, and operation S730. The respective operations in Fig. 7 will be described in detail below.
  • S710 Form discrete feature data of the image samples respectively through the student neural network and the teacher neural network.
  • S510 For the content of the operation, refer to the related description of S510 in the foregoing method implementation manner, which is not described in detail herein.
  • the operations S700 and S710 may be performed by a processor invoking a corresponding instruction stored in a memory, or may be performed by an input image sample module 1020 that is executed by the processor.
  • S720 Perform object detection processing on the image sample according to the discrete feature data of the image sample formed by the student neural network via the student neural network.
  • the object detection processing result output by the student neural network may include: detecting position information of the frame (such as coordinates of two vertices located on a diagonal line of the detection frame), a category of the detection frame, and the like.
  • the present application can reduce the difference between the discrete feature data formed by the teacher neural network for the image sample, the discrete feature data formed by the student neural network for the image sample, and the object detection processing result and the image sample of the student neural network output.
  • the student's neural network is supervised and learned by using the corresponding loss function (such as the L2 loss function).
  • the detection frame labeling information of the image sample of the present application may include: position labeling information of the detecting frame (such as coordinate labeling of two vertices located on a diagonal line of the detecting frame), and category labeling of the detecting frame.
  • the training process ends when the training for the student neural network reaches a predetermined iteration condition.
  • the predetermined iteration condition in the present application may include: a difference between the discrete feature data output by the student neural network and the discrete feature data output by the teacher neural network, and the object detection processing result output by the student neural network and the detection frame labeling information of the image sample. The differences between the two meet the predetermined difference requirements. In the case that the difference satisfies the predetermined difference requirement, the successful training of the student neural network is completed.
  • the predetermined iteration conditions in the present application may also include training the student neural network, the number of image samples used reaches a predetermined number of requirements, and the like.
  • the student neural network has not been successfully trained.
  • Successfully trained student neural networks can be used to form discrete feature data for the image to be processed as well as object detection processing.
  • the teacher neural network for knowledge migration in the present application mainly includes two parts, some of which are: a successfully trained floating-point teacher neural network, and the successfully trained floating-point teacher neural network is usually used.
  • a successfully trained floating-point teacher neural network Forming floating point feature data for the input image, and performing corresponding visual task processing (for example, classification processing or object detection processing, etc.) on the input image according to the floating point feature data; wherein another part is: quantization auxiliary unit, quantization
  • the auxiliary unit is mainly used for converting floating point feature data outputted by the floating point teacher neural network into discrete feature data, and providing the discrete feature data to the floating point teacher neural network.
  • the input to the quantization auxiliary unit is the output of the corresponding layer (such as the Nth layer) in the floating point teacher neural network
  • the output of the quantization auxiliary unit is the input of the corresponding layer (such as the N+1th layer) in the floating point teacher neural network.
  • the quantization auxiliary unit in the present application can be regarded as a layer inserted in the floating-point teacher neural network.
  • the training process for the teacher neural network is actually a fine-tuning process for the network parameters of the floating-point teacher neural network.
  • the operations S720 and S730 may be performed by a processor invoking a corresponding instruction stored in a memory, or may be performed by a supervisory module 1030 executed by the processor.
  • FIG. 1 A flowchart of one embodiment of the training process of the teacher neural network of the present application is shown in FIG.
  • the training process includes an operation S800, an operation S810, and an operation S820.
  • the respective operations in Fig. 8 will be described in detail below.
  • the successfully trained floating-point teacher neural network is a neural network that is successfully trained in advance using image samples for implementing corresponding visual tasks.
  • An embodiment of the training process of the floating point teacher neural network is shown in FIG.
  • Extract floating point feature data of the input image sample via the successfully trained floating point teacher neural network convert the floating point feature data into discrete feature data via the quantization auxiliary unit, and pass the successfully trained floating point teacher neural network.
  • the visual processing of the image samples is performed based on the discrete feature data of the image samples.
  • the successfully trained floating-point teacher neural network performs classification processing or object detection processing on the image samples based on the discrete feature data of the image samples.
  • the operations S800 and S810 may be performed by a processor invoking a corresponding instruction stored in a memory, or may be performed by a first unit of the teacher training module 1040 being executed by the processor.
  • the operation may also be considered as a guide information for the difference between the visual task processing result and the annotation information of the image sample, and the supervised learning of the successfully trained floating-point teacher neural network.
  • the present application can utilize a corresponding loss function (for example, an L2 loss function) for the purpose of reducing the difference between the visual task processing result output by the successfully trained floating point teacher neural network based on the discrete feature data and the annotation information of the image sample.
  • Fine-tuning the network parameters of the successfully trained floating-point teacher neural network Fine-tuning the network parameters of the successfully trained floating-point teacher neural network.
  • the convergence condition of the network parameter fine-tuning generally includes: the difference between the visual task processing result output by the successfully trained floating-point teacher neural network according to the discrete feature data and the annotation information of the image sample, and the predetermined difference requirement is satisfied.
  • the operation S820 may be performed by a processor invoking a corresponding instruction stored in the memory, or may be performed by a first supervisory unit of the teacher training module 1040 that is executed by the processor.
  • the visual task processing result output by the floating-point teacher neural network ie, the teacher neural network
  • the discrete feature data formed by the neural network of the present application can be made to facilitate the accurate completion of visual task processing.
  • FIG. 9 A flowchart of one embodiment of the training process of the floating-point teacher neural network of the present application is shown in FIG.
  • the training process includes: operation S900, operation S910, and operation S920.
  • the respective operations in Fig. 9 will be described in detail below.
  • S900 Input an image sample into a floating-point teacher neural network to be trained.
  • the network parameters of the floating-point teacher neural network to be trained are usually floating-point network parameters, and the floating-point teacher neural network is usually a convolutional neural network or the like.
  • the floating-point teacher neural network to be trained performs classification processing or object detection processing on the image samples according to the floating-point feature data.
  • the operations S900 and S910 may be performed by a processor invoking a corresponding instruction stored in the memory or by a second unit of the teacher training module 1040 being executed by the processor.
  • the annotation information of the image sample may be classification label information of the image sample or detection frame label information of the image sample.
  • the present application can reduce the difference between the visual task processing result outputted by the floating point teacher neural network to be trained according to the floating point feature data and the annotation information of the image sample, and utilize the corresponding loss function (for example, the L2 loss function). Etc.), supervised learning of the floating-point teacher neural network to be trained.
  • the predetermined iteration condition of the training generally includes: a difference between the visual task processing result outputted by the floating point teacher neural network to be trained according to the floating point feature data and the annotation information of the image sample, and the predetermined difference requirement is satisfied.
  • the operation S920 may be performed by a processor invoking a corresponding instruction stored in the memory, or may be performed by a second supervisory unit of the teacher training module 1040 that is executed by the processor.
  • any of the methods provided by the embodiments of the present application may be performed by any suitable device having data processing capabilities, including but not limited to: a terminal device, a server, and the like.
  • any of the methods provided by the embodiments of the present application may be executed by a processor, such as a processor, by executing a corresponding instruction stored in a memory to perform any one of the methods mentioned in the embodiments of the present application. This will not be repeated below.
  • the foregoing programs may be stored in a computer readable storage medium, and the program is executed when executed.
  • the operation of the above method embodiment is included; and the foregoing storage medium includes various media that can store program codes, such as a ROM, a RAM, a magnetic disk, or an optical disk.
  • FIG. 10 is a schematic structural diagram of an embodiment of an image processing apparatus according to the present application.
  • the image processing apparatus shown in FIG. 10 includes an input image module 1000 and a neural network 1010.
  • the device may further include: an input image sample module 1020, a monitoring module 1030, and a teacher training module 1040.
  • the input image module 1000 is for inputting an image to be processed into the neural network 1010.
  • the neural network 1010 is used to form discrete feature data for the image to be processed.
  • the neural network in this application is based on training of guidance information, and the neural network is used as a student neural network during training.
  • the guidance information for training the neural network usually includes: discrete features formed by the teacher neural network for image samples. The difference between the data and the discrete feature data formed by the student neural network for the image samples.
  • the neural network of the present application includes a neural network that does not exceed 2M in size.
  • the neural network of the present application is further used to: form floating point feature data of an image to be processed, and quantize the floating point feature data into discrete feature data of the image to be processed.
  • the neural network performs floating point feature data extraction on the image to be processed, and converts the extracted floating point feature data into floating point feature data meeting predetermined requirements to form floating point feature data of the image to be processed.
  • the neural network may convert the floating point feature data to floating point feature data having a predetermined number of channels; in another alternative example, the neural network may convert the floating point feature data to a float having a predetermined size Point feature data.
  • the neural network may continue processing the discrete feature data of the image to be processed, for example, according to the discrete feature data of the image to be processed, and the corresponding image task processing is performed on the image to be processed.
  • the guidance information for training the neural network further includes: a difference between the visual task processing result output by the student neural network for the image sample and the annotation information of the image sample.
  • the visual task processing that the neural network continues to perform may be: classification processing, object detection processing, and the like. That is to say, the neural network can classify the image to be processed according to the discrete feature data of the image to be processed; the neural network can also perform object detection processing on the image to be processed according to the discrete feature data of the image to be processed.
  • the guidance information for training the neural network may further include: a classification processing result output by the student neural network for the image sample, and the image The difference between the classification information of the sample.
  • the knowledge information for training the neural network may further include: an object detection processing result output by the student neural network for the image sample, The difference between the labeling information and the detection frame of the image sample.
  • FIG. 11 is a schematic structural diagram of an embodiment of a training apparatus for a neural network according to the present application.
  • the training device shown in FIG. 11 includes an input image sample module 1020, a supervision module 1030, and a teacher training module 1040.
  • the input image sample module 1020 is configured to input image samples into the student neural network and the teacher neural network, respectively, to form discrete feature data of the image samples respectively through the student neural network and the teacher neural network.
  • the monitoring module 1030 is configured to perform supervised learning on the student neural network according to the guidance information.
  • the guidance information includes: the difference between the discrete feature data formed by the teacher neural network for the image sample and the discrete feature data formed by the student neural network for the image sample.
  • the monitoring module 1030 is further configured to use the teacher neural network to form discrete feature data for the image samples. , the difference between the discrete feature data formed by the student neural network for the image sample, and the difference between the visual task processing result output by the student neural network and the annotation information of the image sample, for guiding information, supervising the student neural network .
  • the teacher neural network in the present application includes a successfully trained floating point teacher neural network and a quantization auxiliary unit.
  • the successfully trained floating point teacher neural network is configured to form floating point feature data for the input image, and perform visual task processing on the input image according to the floating point feature data.
  • the quantization auxiliary unit is configured to convert the floating point feature data formed by the floating point teacher neural network into discrete feature data, and provide the discrete feature data to the floating point teacher neural network, so that the floating point teacher neural network according to the discrete feature data , visual task processing of the input image.
  • the teacher training module 1040 in the present application is for training a teacher neural network
  • the teacher training module 1040 comprising: a first unit and a first supervisory unit.
  • the first unit is configured to input the image sample into the successfully trained floating point teacher neural network to extract the floating point feature data of the image sample via the successfully trained floating point teacher neural network, and to implement the floating point feature data via the quantization auxiliary unit.
  • the data is converted into discrete feature data, and the image sample is processed by the visual task according to the discrete feature data via the successfully trained floating point teacher neural network.
  • the first supervising unit is configured to adjust the network parameters of the successfully trained floating-point teacher neural network by using the difference between the visual task processing result and the annotation information of the image sample as the guiding information.
  • the teacher training module 1040 can further include: a second unit and a second supervisory unit.
  • the second unit is configured to input image samples into the floating-point teacher neural network to be trained to extract floating-point feature data of the image samples via the floating-point teacher neural network to be trained, and perform image samples according to the floating-point feature data.
  • Visual task processing The second supervising unit is configured to supervise and learn the floating point teacher neural network to be trained by using the difference between the visual task processing result and the annotation information of the image sample as the guiding information.
  • FIG. 12 illustrates an exemplary device 1200 suitable for implementing the present application, which may be a control system/electronic system configured in a car, a mobile terminal (eg, a smart mobile phone, etc.), a personal computer (PC, eg, a desktop computer Or a notebook computer, etc.), a tablet computer, a server, and the like.
  • a mobile terminal eg, a smart mobile phone, etc.
  • PC personal computer
  • tablet computer eg, a tablet computer, a server, and the like.
  • device 1200 includes one or more processors, communication units, etc., which may be: one or more central processing units (CPUs) 1201, and/or one or more utilized An image processor (GPU) 1213 or the like that performs image processing by a neural network, and the processor may be loaded into the random access memory (RAM) 1203 according to executable instructions stored in the read only memory (ROM) 1202 or from the storage portion 1208. Execute instructions to perform various appropriate actions and processes.
  • the communication unit 1212 may include, but is not limited to, a network card, which may include, but is not limited to, an IB (Infiniband) network card.
  • the processor can communicate with the read only memory 1202 and/or the random access memory 1203 to execute executable instructions, connect to the communication portion 1212 via the bus 1204, and communicate with other target devices via the communication portion 1212, thereby completing the corresponding in the present application. operating.
  • the RAM 1203 various programs and data required for the operation of the device can also be stored.
  • the CPU 1201, the ROM 1202, and the RAM 1203 are connected to each other through a bus 1204.
  • ROM 1202 is an optional module.
  • the RAM 1203 stores executable instructions or writes executable instructions to the ROM 1202 at runtime, the executable instructions causing the central processing unit 1201 to perform the operations included in the object segmentation method described above.
  • An input/output (I/O) interface 1205 is also coupled to bus 1204.
  • the communication unit 1212 may be integrated, or may be configured to have a plurality of sub-modules (for example, a plurality of IB network cards) and be respectively connected to the bus.
  • the following components are connected to the I/O interface 1205: an input portion 1206 including a keyboard, a mouse, etc.; an output portion 1207 including a cathode ray tube (CRT), a liquid crystal display (LCD), and the like, and a speaker; a storage portion 1208 including a hard disk or the like And a communication portion 1209 including a network interface card such as a LAN card, a modem, and the like.
  • the communication section 1209 performs communication processing via a network such as the Internet.
  • Driver 1210 is also coupled to I/O interface 1205 as needed.
  • a removable medium 1211 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory or the like is mounted on the drive 1210 as needed so that a computer program read therefrom is installed in the storage portion 1208 as needed.
  • FIG. 12 is only an optional implementation manner.
  • the number and type of components in the foregoing FIG. 12 may be selected, deleted, added, or replaced according to actual needs;
  • separate implementations such as separate settings or integrated settings can also be used.
  • the GPU and the CPU can be separated, and the GPU can be integrated on the CPU, and the communication unit can be separately configured or integrated.
  • the CPU or GPU etc.
  • embodiments of the present application include a computer program product comprising tangibly embodied on a machine readable medium.
  • a computer program comprising program code for performing the operations illustrated by the flowcharts, the program code comprising instructions corresponding to performing the operations in the methods provided herein.
  • the computer program can be downloaded and installed from the network via the communication portion 1209, and/or installed from the removable medium 1211.
  • the computer program is executed by the central processing unit (CPU) 1201, the instructions for realizing the above-described respective operations described in the present application are executed.
  • the embodiment of the present application further provides a computer program product for storing computer readable instructions, the instructions being executed to cause a computer to perform the operations described in any of the above embodiments.
  • Image processing method or training method of neural network the instructions being executed to cause a computer to perform the operations described in any of the above embodiments.
  • the computer program product can be implemented by means of hardware, software or a combination thereof.
  • the computer program product is embodied as a computer storage medium, and in another alternative example, the computer program product is embodied as a software product, such as a Software Development Kit (SDK) or the like.
  • SDK Software Development Kit
  • the embodiment of the present application further provides another image processing method and a training method for a neural network, and corresponding devices and electronic devices, computer storage media, computer programs, and computer program products,
  • the method includes: the first device transmitting an image processing indication to the second device or training a neural network indication, the indication causing the second device to perform the image processing method or the training method of the neural network in any of the above possible embodiments;
  • the device receives an image processing result or a neural network training result transmitted by the second device.
  • the image processing indication or training neural network indication may be a call instruction
  • the first device may instruct the second device to perform an image processing operation or to train a neural network operation by means of a call, and accordingly, in response to receiving the call Instructed, the second device may perform the operations and/or processes in any of the above described image processing methods or training methods of the neural network.
  • the methods and apparatus, electronic devices, and computer readable storage media of the present application are possible in many ways.
  • the methods and apparatus, electronic devices, and computer readable storage media of the present application can be implemented in software, hardware, firmware, or any combination of software, hardware, or firmware.
  • the above sequence of operations for the method is for illustrative purposes only, and the operation of the method of the present application is not limited to the order described above unless otherwise specifically stated.
  • the present application can also be implemented as a program recorded in a recording medium, the program including machine readable instructions for implementing the method according to the present application.
  • the present application also covers a recording medium storing a program for executing the method according to the present application.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Multimedia (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

一种图像处理方法、训练方法、装置、设备、介质及程序，其中的图像处理方法包括：将待处理图像输入神经网络(S100)；经由所述神经网络形成所述待处理图像的离散特征数据(S110)；其中，所述神经网络是基于指导信息训练而成的，且所述神经网络在训练过程中，被作为学生神经网络，所述指导信息包括：教师神经网络针对图像样本形成的离散特征数据与学生神经网络针对所述图像样本形成的离散特征数据之间的差异。有利于在缩小神经网络的规模，并提升神经网络的运行速度的基础上，提高神经网络输出结果的准确性，即有利于提高神经网络的性能。

Description

图像处理方法、训练方法、装置、设备、介质及程序
本申请要求在2018年04月25日提交中国专利局、申请号为CN 201810380453.X、发明名称为“图像处理方法、神经网络的训练方法、装置、设备及介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机视觉技术，尤其是涉及一种图像处理方法、图像处理装置、神经网络的训练方法、神经网络的训练装置、电子设备、计算机可读存储介质及计算机程序。
背景技术
神经网络(例如，卷积神经网络等)，在很多计算机视觉任务中，得到了较为广泛的应用。
目前，一些神经网络由于其计算量以及参数量较大等因素，而无法直接应用在智能移动电话以及数码相机等电子设备中。压缩技术通常可以缩小神经网络的规模，并提升神经网络的运行速度，然而，压缩技术通常会使神经网络的准确性变差。
如何在缩小神经网络的规模，提升神经网络的运行速度的基础上，保证神经网络具有一定的准确性，是一个值得关注的技术问题。
发明内容
本申请实施方式提供一种图像处理以及训练神经网络的技术方案。
根据本申请实施方式其中一个方面，提供一种图像处理方法，所述方法包括：将待处理图像输入神经网络；经由所述神经网络形成所述待处理图像的离散特征数据；其中，所述神经网络是基于指导信息训练而成的，且所述神经网络在训练过程中，被作为学生神经网络，所述指导信息包括：教师神经网络针对图像样本形成的离散特征数据与学生神经网络针对所述图像样本形成的离散特征数据之间的差异。
在本申请一实施方式中，所述神经网络包括：大小不超过2M的神经网络。
在本申请又一实施方式中，所述经由所述神经网络形成所述待处理图像的离散特征数据包括：经由所述神经网络形成所述待处理图像的浮点特征数据，并将所述浮点特征数据量化为所述待处理图像的离散特征数据。
在本申请再一实施方式中，所述经由所述神经网络形成所述待处理图像的浮点特征数据包括：经由所述神经网络针对所述待处理图像，进行浮点特征数据提取，并将提取出的浮点特征数据转换为符合预定要求的浮点特征数据，以形成所述待处理图像的浮点特征数据。
在本申请再一实施方式中，所述将提取出的浮点特征数据转换为符合预定要求的浮点特征数据包括：将所述浮点特征数据转换为具有预定通道数的浮点特征数据；和/或者，将所述浮点特征数据转换为具有预定大小的浮点特征数据。
在本申请再一实施方式中，所述方法还包括：经由所述神经网络，根据所述待处理图像的离散特征数据，对所述待处理图像进行相应视觉任务处理。
在本申请再一实施方式中，所述指导信息还包括：学生神经网络针对图像样本输出的视觉任务处理结果与图像样本的标注信息之间的差异。
在本申请再一实施方式中，所述经由所述神经网络，根据所述待处理图像的离散特征数据，对所述待处理图像进行相应视觉任务处理包括：经由所述神经网络，根据所述待处理图像的离散特征数据，对所述待处理图像进行分类处理；或者，经由所述神经网络，根据所述待处理图像的离散特征数据，对所述待处理图像进行物体检测处理。
在本申请再一实施方式中，所述指导信息还包括：所述学生神经网络针对图像样本输出的分类处理结果，与图像样本的分类标注信息之间的差异；或，所述学生神经网络针对图像样本输出的物体检测处理结果，与图像样本的检测框标注信息之间的差异。
在本申请再一实施方式中，所述神经网络的训练过程包括：将图像样本分别输入学生神经网络以及教师神经网络；经由学生神经网络以及教师神经网络，分别形成所述图像样本的离散特征数据；根据指导信息，对所述学生神经网络进行监督学习；其中，所述指导信息包括：教师神经网络针对图像样本形成的离散特征数据，与所述学生神经网络针对图像样本形成的离散特征数据之间的差异。
在本申请再一实施方式中，所述神经网络的训练过程还包括：经由所述学生神经网络，根据所述图像样本的离散特征数据，对图像样本进行视觉任务处理；所述根据指导信息，对所述学生神经网络进行监督学习包括：以教师神经网络针对图像样本形成的离散特征数据，与所述学生神经网络针对图像样本形成的离散特征数据之间的差异，以及学生神经网络输出的视觉任务处理结果与图像样本的标注信息之间的差异，为指导信息，对所述学生神经网络进行监督学习。
在本申请再一实施方式中，所述教师神经网络包括：已成功训练的浮点教师神经网络，该已成功训练的浮点教师神经网络用于针对输入图像形成浮点特征数据，并根据该浮点特征数据，对输入图像进行视觉任务处理；以及，量化辅助单元，用于将浮点教师神经网络形成的浮点特征数据转换为离散特征数据，并将该离散特征数据提供给浮点教师神经网络，使浮点教师神经网络根据该离散特征数据，对输入图像进行视觉任务处理。
在本申请再一实施方式中，所述教师神经网络的训练过程包括：将图像样本输入已成功训练的浮点教师神经网络；经由已成功训练的浮点教师神经网络提取图像样本的浮点特征数据，经由量化辅助单元将浮点特征数据转换为离散特征数据，并经由已成功训练的浮点教师神经网络根据所述离散特征数据，对图像样本进行视觉任务处理；以视觉任务处理结果与图像样本的标注信息之间的差异为指导信息，对已成功训练的浮点教师神经网络进行网络参数调整。
在本申请再一实施方式中，所述浮点教师神经网络的训练过程包括：将图像样本输入待训练的浮点教师神经网络；经由待训练的浮点教师神经网络提取所述图像样本的浮点特征数据，并根据所述浮点特征数据，对图像样本进行视觉任务处理；以所述视觉任务处理结果与图像样本的标注信息之间的差异为指导信息，对所述待训练的浮点教师神经网络进行监督学习。
根据本申请实施方式其中另一方面，提供一种神经网络的训练方法，待训练的神经网络在训练过程中，被作为学生神经网络，所述训练方法包括：将图像样本分别输入学生神经网络以及教师神经网络；经由学生神经网络以及教师神经网络，分别形成所述图像样本的离散特征数据；根据指导信息，对所述学生神经网络进行监督学习；其中，所述指导信息包括：教师神经网络针对图像样本形成的离散特征数据，与所述学生神经网络针对图像样本形成的离散特征数据之间的差异。
在本申请一实施方式中，所述神经网络作为学生神经网络的训练过程还包括：经由所述学生神经网络，根据所述图像样本的离散特征数据，对图像样本进行视觉任务处理；所述根据指导信息，对所述学生神经网络进行监督学习包括：以教师神经网络针对图像样本形成的离散特征数据，与所述学生神经网络针对图像样本形成的离散特征数据之间的差异，以及学生神经网络输出的视觉任务处理结果与图像样本的标注信息之间的差异，为指导信息，对所述学生神经网络进行监督学习。
在本申请又一实施方式中，所述教师神经网络包括：已成功训练的浮点教师神经网络，该已成功训练的浮点教师神经网络用于针对输入图像形成浮点特征数据，并根据该浮点特征数据，对输入图像进行视觉任务处理；以及，量化辅助单元，用于将浮点教师神经网络形成的浮点特征数据转换为离散特征数据，并将该离散特征数据提供给浮点教师神经网络，使浮点教师神经网络根据该离散特征数据，对输入图像进行视觉任务处理。
在本申请再一实施方式中，所述教师神经网络的训练过程包括：将图像样本输入已成功训练的浮点教师神经网络；经由已成功训练的浮点教师神经网络提取图像样本的浮点特征数据，经由量化辅助单元将浮点特征数据转换为离散特征数据，并经由已成功训练的浮点教师神经网络根据所述离散特征数据，对图像样本进行视觉任务处理；以视觉任务处理结果与图像样本的标注信息之间的差异为指导信息，对已成功训练的浮点教师神经网络进行网络参数调整。
在本申请再一实施方式中，所述浮点教师神经网络的训练过程包括：将图像样本输入待训练的浮点教师神经网络；经由待训练的浮点教师神经网络提取所述图像样本的浮点特征数据，并根据所述浮点特征数据，对图像样本进行视觉任务处理；以所述视觉任务处理结果与图像样本的标注信息之间的差异为指导信息，对所述待训练的浮点教师神经网络进行监督学习。
根据本申请实施方式其中又一方面，提供一种图像处理装置，所述装置包括：输入图像模块，用于将待处理图像输入神经网络；神经网络，用于形成所述待处理图像的离散特征数据；其中，所述神经网络是基于指导信息训练而成的，且所述神经网络在训练过程中，被作为学生神经网络，所述指导信息包括：教师神经网络针对图像样本形成的离散特征数据与学生神经网络针对所述图像样本形成的离散特征数据之间的差异。
在本申请一实施方式中，所述神经网络包括：大小不超过2M的神经网络。
在本申请又一实施方式中，所述神经网络进一步用于：形成所述待处理图像的浮点特征数据，并将 所述浮点特征数据量化为所述待处理图像的离散特征数据。
在本申请再一实施方式中，所述神经网络针对所述待处理图像，进行浮点特征数据提取，并将提取出的浮点特征数据转换为符合预定要求的浮点特征数据，以形成所述待处理图像的浮点特征数据。
在本申请再一实施方式中，所述神经网络将所述浮点特征数据转换为具有预定通道数的浮点特征数据；和/或，所述神经网络将所述浮点特征数据转换为具有预定大小的浮点特征数据。
在本申请再一实施方式中，所述神经网络还用于：根据所述待处理图像的离散特征数据，对所述待处理图像进行相应视觉任务处理。
在本申请再一实施方式中，所述指导信息还包括：学生神经网络针对图像样本输出的视觉任务处理结果与图像样本的标注信息之间的差异。
在本申请再一实施方式中，所述神经网络根据所述待处理图像的离散特征数据，对所述待处理图像进行分类处理；或者，所述神经网络根据所述待处理图像的离散特征数据，对所述待处理图像进行物体检测处理。
在本申请再一实施方式中，所述指导信息还包括：所述学生神经网络针对图像样本输出的分类处理结果，与图像样本的分类标注信息之间的差异；或，所述学生神经网络针对图像样本输出的物体检测处理结果，与图像样本的检测框标注信息之间的差异。
在本申请再一实施方式中，所述装置还包括：输入图像样本模块，用于将图像样本分别输入学生神经网络以及教师神经网络，以经由学生神经网络以及教师神经网络，分别形成所述图像样本的离散特征数据；监督模块，用于根据指导信息，对所述学生神经网络进行监督学习；其中，所述指导信息包括：教师神经网络针对图像样本形成的离散特征数据，与所述学生神经网络针对图像样本形成的离散特征数据之间的差异。
在本申请再一实施方式中，在所述学生神经网络用于，根据所述图像样本的离散特征数据，对图像样本进行视觉任务处理的情况下，所述监督模块进一步用于：以教师神经网络针对图像样本形成的离散特征数据，与所述学生神经网络针对图像样本形成的离散特征数据之间的差异，以及学生神经网络输出的视觉任务处理结果与图像样本的标注信息之间的差异，为指导信息，对所述学生神经网络进行监督学习。
在本申请再一实施方式中，所述教师神经网络包括：已成功训练的浮点教师神经网络，该已成功训练的浮点教师神经网络用于针对输入图像形成浮点特征数据，并根据该浮点特征数据，对输入图像进行视觉任务处理；以及，量化辅助单元，用于将浮点教师神经网络形成的浮点特征数据转换为离散特征数据，并将该离散特征数据提供给浮点教师神经网络，使浮点教师神经网络根据该离散特征数据，对输入图像进行视觉任务处理。
在本申请再一实施方式中，所述装置还包括：用于训练所述教师神经网络的教师训练模块，所述教师训练模块包括：第一单元，用于将图像样本输入已成功训练的浮点教师神经网络，以经由已成功训练的浮点教师神经网络提取图像样本的浮点特征数据，经由量化辅助单元将浮点特征数据转换为离散特征数据，并经由已成功训练的浮点教师神经网络根据所述离散特征数据，对图像样本进行视觉任务处理；第一监督单元，用于以视觉任务处理结果与图像样本的标注信息之间的差异为指导信息，对已成功训练的浮点教师神经网络进行网络参数调整。
在本申请再一实施方式中，所述教师训练模块还包括：第二单元，用于将图像样本输入待训练的浮点教师神经网络，以经由待训练的浮点教师神经网络提取所述图像样本的浮点特征数据，并根据所述浮点特征数据，对图像样本进行视觉任务处理；第二监督单元，用于以所述视觉任务处理结果与图像样本的标注信息之间的差异为指导信息，对所述待训练的浮点教师神经网络进行监督学习。
根据本申请实施方式其中再一方面，提供一种神经网络的训练装置，待训练的神经网络被作为学生神经网络，所述装置包括：输入图像样本模块，用于将图像样本分别输入学生神经网络以及教师神经网络，以经由学生神经网络以及教师神经网络，分别形成所述图像样本的离散特征数据；监督模块，用于根据指导信息，对所述学生神经网络进行监督学习；其中，所述指导信息包括：教师神经网络针对图像样本形成的离散特征数据，与所述学生神经网络针对图像样本形成的离散特征数据之间的差异。
在本申请一实施方式中，在所述学生神经网络用于，根据所述图像样本的离散特征数据，对图像样本进行视觉任务处理的情况下，所述监督模块进一步用于：以教师神经网络针对图像样本形成的离散特征数据，与所述学生神经网络针对图像样本形成的离散特征数据之间的差异，以及学生神经网络输出的视觉任务处理结果与图像样本的标注信息之间的差异，为指导信息，对所述学生神经网络进行监督学习。
在本申请又一实施方式中，所述教师神经网络包括：已成功训练的浮点教师神经网络，该已成功训 练的浮点教师神经网络用于针对输入图像形成浮点特征数据，并根据该浮点特征数据，对输入图像进行视觉任务处理；以及，量化辅助单元，用于将浮点教师神经网络形成的浮点特征数据转换为离散特征数据，并将该离散特征数据提供给浮点教师神经网络，使浮点教师神经网络根据该离散特征数据，对输入图像进行视觉任务处理。
在本申请再一实施方式中，所述装置还包括：用于训练所述教师神经网络的教师训练模块，所述教师训练模块包括：第一单元，用于将图像样本输入已成功训练的浮点教师神经网络，以经由已成功训练的浮点教师神经网络提取图像样本的浮点特征数据，经由量化辅助单元将浮点特征数据转换为离散特征数据，并经由已成功训练的浮点教师神经网络根据所述离散特征数据，对图像样本进行视觉任务处理；第一监督单元，用于以视觉任务处理结果与图像样本的标注信息之间的差异为指导信息，对已成功训练的浮点教师神经网络进行网络参数调整。
在本申请再一实施方式中，所述教师训练模块还包括：第二单元，用于将图像样本输入待训练的浮点教师神经网络，以经由待训练的浮点教师神经网络提取所述图像样本的浮点特征数据，并根据所述浮点特征数据，对图像样本进行视觉任务处理；第二监督单元，用于以所述视觉任务处理结果与图像样本的标注信息之间的差异为指导信息，对所述待训练的浮点教师神经网络进行监督学习。
根据本申请实施方式其中再一方面，提供一种电子设备，包括：存储器，用于存储计算机程序；处理器，用于执行所述存储器中存储的计算机程序，且所述计算机程序被执行时，实现本申请任一方法实施方式。
根据本申请实施方式其中再一个方面，提供一种计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时，实现本申请任一方法实施方式。
根据本申请实施方式其中再一个方面，提供一种计算机程序，包括计算机指令，当所述计算机指令在设备的处理器中运行时，实现本申请任一方法实施方式。
基于本申请提供的图像处理方法、图像处理装置、神经网络训练方法、神经网络训练装置、电子设备、计算机可读存储介质及计算机程序，通过在训练神经网络的过程中，将待训练的神经网络作为学生神经网络，并使指导信息包括：教师神经网络形成的离散特征数据与学生神经网络形成的离散特征数据之间的差异，有利于提高教师神经网络和学生神经网络分别形成的离散特征数据的匹配概率，从而有利于促使教师神经网络的知识迁移到学生神经网络中。由于本申请并不限制学生神经网络的网络参数必须为定点网络参数，因此，本申请的神经网络可以不受到特定指令集以及特定设备的限制。由此可知，本申请提供的技术方案有利于在缩小神经网络的规模，并提升神经网络的运行速度的基础上，提高神经网络输出结果的准确性，即有利于提高神经网络的性能。另外，本申请提供的技术方案也有利于提高神经网络的适用范围。
下面通过附图和实施方式，对本申请的技术方案做进一步的详细描述。
附图说明
构成说明书的一部分的附图描述了本申请的实施方式，并且连同描述一起用于解释本申请的原理。
参照附图，根据下面的详细描述，可以更加清楚地理解本申请，其中：
图1为本申请的图像处理方法一个实施例的流程图；
图2为本申请的图像处理方法另一个实施例的流程图；
图3为本申请的图像处理方法又一个实施例的流程图；
图4为本申请的图像处理方法再一个实施例的流程图；
图5为本申请的神经网络的训练方法一个实施例的流程图；
图6为本申请的神经网络的训练方法另一个实施例的流程图；
图7为本申请的神经网络的训练方法又一个实施例的流程图；
图8为本申请的教师神经网络的训练方法一个实施例的流程图；
图9为本申请的浮点教师神经网络的训练方法一个实施例的流程图；
图10为本申请的图像处理装置一个实施例的结构示意图；
图11为本申请的神经网络的训练装置一个实施例的结构示意图；
图12为实现本申请实施例的一示例性设备的框图。
具体实施方式
现在将参照附图来详细描述本申请的各种示例性实施例。应注意到：除非另外说明，否则在这些实 施例中阐述的部件和操作的相对布置、数字表达式和数值不限制本申请的范围。
同时，应当明白，为了便于描述，附图中所示出的各个部分的尺寸并不是按照实际的比例关系绘制的。
以下对至少一个示例性实施例的描述实际上仅仅是说明性的，决不作为对本申请及其应用或使用的任何限制。
对于相关领域普通技术人员已知的技术、方法以及设备可能不作详细讨论，但在适当情况下，所述技术、方法和设备应当被视为说明书的一部分。
应当注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步讨论。
本申请实施例可以应用于终端设备、计算机系统及服务器等电子设备，其可与众多其它通用或者专用的计算系统环境或者配置一起操作。适于与终端设备、计算机系统以及服务器等电子设备一起使用的众所周知的终端设备、计算系统、环境和/或配置的例子，包括但不限于：个人计算机系统、服务器计算机系统、瘦客户机、厚客户机、手持或膝上设备、基于微处理器的系统、机顶盒、可编程消费电子产品、网络个人电脑、小型计算机系统﹑大型计算机系统和包括上述任何系统的分布式云计算技术环境，等等。
终端设备、计算机系统以及服务器等电子设备可以在由计算机系统执行的计算机系统可执行指令(诸如程序模块)的一般语境下描述。通常，程序模块可以包括例程、程序、目标程序、组件、逻辑以及数据结构等等，它们执行特定的任务或者实现特定的抽象数据类型。计算机系统/服务器可以在分布式云计算环境中实施，分布式云计算环境中，任务是由通过通信网络链接的远程处理设备执行的。在分布式云计算环境中，程序模块可以位于包括存储设备的本地或远程计算系统存储介质上。
示例性实施例
图1为本申请图像处理方法一个实施例的流程图。如图1所示，该实施例方法包括：操作S100以及操作S110。下面对图1中的各操作分别进行详细说明。
S100、将待处理图像输入神经网络。
在一个可选示例中，本申请中的待处理图像可以为呈现静态的图片或照片等图像，也可以为呈现动态的视频中的视频帧，例如，摄像装置所摄取到的视频中的视频帧等。该待处理图像可以是待处理原始图像中的一部分(即从待处理原始图像中切分出的、包括待处理原始图像部分内容的图像块)，该待处理图像也可以包括完整的待处理原始图像。当然，该待处理图像也可以是待处理原始图像在进行了一定处理之后的图像中的一部分，该待处理图像也可以包括待处理原始图像在进行了一定处理之后的图像的完整内容。本申请不限制待处理图像的表现形式。
在一个可选示例中，本申请中的待处理图像的大小通常与神经网络对输入图像的尺寸要求相关，例如，待处理图像的大小可以为256×256等。为了获得具有预定大小的待处理图像，本申请可以对待处理图像进行相应比例的缩放处理。本申请对待处理图像的大小及缩放处理的实现方式不作限制。
在一个可选示例中，本申请中的神经网络可以为大小不超过2M的神经网络。也就是说，该神经网络的存储空间可以不超过2M。相较于现有的大小为几十兆，甚至上百兆的神经网络而言，本申请中的神经网络可以称为极小神经网络。
在一个可选示例中，本申请的神经网络可以包括但不限于卷积层、非线性Relu层、池化层以及全连接层等。该神经网络可以为深度神经网络，该神经网络所包含的层数越多，则网络越深。本申请的神经网络可以为卷积神经网络。本申请的神经网络的网络结构可以根据实际视觉任务需求，而灵活设计，本申请实施例并不限制该神经网络的网络结构；例如，本申请的神经网络的网络结构可以采用但不限于ALexNet、深度残差网络(Deep Residual Network，ResNet)或VGGnet(Visual Geometry Group Network，视觉几何组网络)等神经网络所采用的网络结构。
在一个可选示例中，本申请中的神经网络是基于知识迁移的方式，训练而成的。也就是说，本申请中的神经网络在训练过程中，被作为学生神经网络。另外，在训练该神经网络的过程中，所使用的指导信息通常包括：教师神经网络针对图像样本形成的离散特征数据与学生神经网络针对图像样本形成的离散特征数据之间的差异。也就是说，教师神经网络针对图像样本形成的离散特征数据，被作为训练学生神经网络的监督信号。训练神经网络所采用的损失函数可以包括L2损失函数等。该神经网络的训练过程可以参见下述针对图5至图7的描述，在此不再重复说明。另外需要说明的是，本申请中的离散特征数据可以称为定点特征数据。在本申请中，特征数据可以包括至少一个特征向量、至少一个特征图或其他形式。
在一个可选示例中，本申请中的教师神经网络的网络参数并不需要一定为定点网络参数，也就是说，本申请的教师神经网络的网络参数可以为浮点网络参数，教师神经网络可以进行浮点运算，然而，教师神经网络最终形成的特征数据为离散特征数据，即教师神经网络针对图像提取出的浮点特征数据在被量化处理后，形成离散特征数据。由此可知，本申请的教师神经网络可以称为量化教师神经网络。利用量化教师神经网络来训练学生神经网络，可以使量化教师神经网络的知识迁移到学生神经网络中，且学生神经网络的网络参数也并不需要一定为定点网络参数，也就是说，本申请的学生神经网络的网络参数可以为浮点网络参数，学生神经网络可以进行浮点运算，从而学生神经网络在成功训练后，可以不受到特定指令集以及特定设备的限制，有利于提高神经网络的适用范围。
在一个可选示例中，该操作S100可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的输入图像模块1000执行。
S110、经由神经网络形成待处理图像的离散特征数据。
在一个可选示例中，本申请中的神经网络可以先形成待处理图像的浮点特征数据，然后，该神经网络将该浮点特征数据量化为离散特征数据，量化后的离散特征数据即为神经网络所形成的待处理图像的离散特征数据。本申请中的神经网络形成待处理图像的浮点特征数据的一个可选例子可以为：神经网络针对待处理图像，进行浮点特征数据提取，从而获得一浮点特征数据(下述称为初始浮点特征数据)，通常情况下，该初始浮点特征数据并不符合预先针对浮点特征数据规定的预定要求，因此，神经网络可以将其获得的初始浮点特征数据转换为符合预定要求的浮点特征数据，该符合预定要求的浮点特征数据即为神经网络形成的待处理图像的浮点特征数据。
在一个可选示例中，本申请中的神经网络将浮点特征数据量化为离散特征数据的方式可以包括：将特征数据中的浮点数转换为相应的定点数，例如，将特征数据中的位于[k-0.5，k+0.5]区间的浮点数都转换为k，其中的k为整数。本申请不限制神经网络将浮点特征数据量化为离散特征数据的实现方式。
在一个可选示例中，本申请中的预定要求可以根据实际需求灵活设计，例如，预定要求可以包括：针对通道数的要求以及针对特征数据的大小的要求等中的至少一个。可选的，预定要求可以规定特征数据应具有预定通道数，且预定要求可以规定特征数据应具有预定大小，即特征数据的长和宽应符合要求。
在一个可选示例中，本申请的神经网络所形成的初始浮点特征数据的通道数可以根据实际需求设置，例如，本申请的神经网络所形成的初始浮点特征数据的通道数可以为VGG(Visual Geometry Group，视觉几何组)神经网络的通道数的三十二分之一，而转换后的离散特征数据的通道数可以为VGG神经网络的通道数。本申请可以利用卷积层等技术，将初始浮点特征数据转换为具有预定通道数的浮点特征数据。本申请不限制将初始浮点特征数据转换为具有预定通道数的浮点特征数据的实现方式。
在一个可选示例中，本申请可以利用上采样等技术，将初始浮点特征数据转换为具有预定大小的浮点特征数据。其中的上采样可以通过反卷积层来实现。本申请不限制将初始浮点特征数据转换为具有预定大小的浮点特征数据的实现方式。
在一个可选示例中，本申请的神经网络不仅用于形成待处理图像的离散特征数据，还用于根据待处理图像的离散特征数据，对待处理图像进行相应视觉任务处理。本申请中的视觉任务处理可以根据实际需求灵活设置，例如，视觉任务处理可以为分类处理或者物体检测处理等。也就是说，本申请中的神经网络可以根据其形成的待处理图像的离散特征数据，对待处理图像进行分类处理，以确定待处理图像所属的类别(如猫或者狗或者人或者车辆等类别)；本申请中的神经网络可以根据其形成的待处理图像的离散特征数据，对待处理图像进行物体检测处理，以确定待处理图像中的检测框的位置以及类别(例如，检测框的对角线上的两个顶点的坐标以及检测框的类别)。本申请中的检测框也可以称为外接框等。本申请不限制视觉任务处理的表现形式。
在一个可选示例中，在本申请的神经网络对待处理图像进行相应视觉任务处理的情况下，本申请训练神经网络所使用的指导信息通常包括：神经网络输出的视觉任务处理结果与图像样本的标注信息之间的差异。也就是说，教师神经网络针对图像样本形成的离散特征数据，被作为训练学生神经网络的监督信号，与此同时，图像样本的标注信息同样被作为训练学生神经网络的监督信号。
在一个可选示例中，在本申请的神经网络不仅用于形成待处理图像的离散特征数据，还用于根据待处理图像的离散特征数据，对待处理图像进行分类处理的情况下，在训练神经网络的过程中，教师神经网络针对图像样本形成的离散特征数据以及图像样本的分类标注信息，均被作为训练学生神经网络的监督信号。也就是说，学生神经网络输出的分类处理结果与图像样本的分类标注信息之间的差异，以及教师神经网络针对图像样本形成的离散特征数据与学生神经网络针对图像样本形成的离散特征数据之间的差异，均被作为训练学生神经网络的指导信息。
在一个可选示例中，在本申请的神经网络不仅用于形成待处理图像的离散特征数据，还用于根据待处理图像的离散特征数据，对待处理图像进行物体检测处理的情况下，在训练神经网络的过程中，教师神经网络针对图像样本形成的离散特征数据以及图像样本的检测框标注信息，均被作为训练学生神经网络的监督信号。也就是说，学生神经网络输出的物体检测处理结果(如检测框的位置以及分类信息)与图像样本的检测框标注信息(如检测框的位置标注信息以及分类标注信息)之间的差异，以及教师神经网络针对图像样本形成的离散特征数据与学生神经网络针对图像样本形成的离散特征数据之间的差异，均被作为训练学生神经网络的指导信息。
在一个可选示例中，该操作S110可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的神经网络1010执行。
图2为本申请图像处理方法另一个实施例的流程图。如图2所示，该实施例方法包括：操作S200、操作S210以及操作S220。下面对图2中的各操作分别进行详细说明。
S200、将待处理图像输入神经网络。
在一个可选示例中，本申请的神经网络通常包括：大小不超过2M的神经网络。该神经网络在训练过程中，被作为学生神经网络。在训练该神经网络的过程中，所使用的指导信息通常包括：教师神经网络针对图像样本形成的离散特征数据与学生神经网络针对图像样本形成的离散特征数据之间的差异，以及学生神经网络针对图像样本进行视觉任务处理的视觉任务处理结果与图像样本的标注信息之间的差异。也就是说，教师神经网络针对图像样本形成的离散特征数据以及图像样本的标注信息，被作为训练学生神经网络的监督信号。训练神经网络所采用的损失函数可以包括L2损失函数等。该操作所涉及到的网络结构以及待处理图像等的内容可以参见上述方法实施方式中针对S100的描述，在此不再详细说明。
在一个可选示例中，该操作S200可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的输入图像模块1000执行。
S210、经由神经网络形成待处理图像的离散特征数据。
在一个可选示例中，该神经网络可以先形成该待处理图像的浮点特征数据，可选的，该神经网络针对待处理图像，进行浮点特征数据提取，并将提取出的浮点特征数据转换为符合预定要求的浮点特征数据(例如，将浮点特征数据转换为具有预定通道数的浮点特征数据，再例如，将浮点特征数据转换为具有预定大小的浮点特征数据)，从而形成待处理图像的浮点特征数据。神经网络在形成待处理图像的浮点特征数据之后，再将该浮点特征数据量化为待处理图像的离散特征数据，如将浮点特征数据中的浮点数转换为相应的定点数等。该操作的内容可以参见上述方法实施方式中针对S110的相关描述，在此不再详细说明。
S220、经由神经网络，根据待处理图像的离散特征数据，对待处理图像进行相应视觉任务处理。该视觉任务处理可以根据实际需求灵活设置，该操作的内容可以参见上述方法实施方式中针对S110的相关描述，在此不再详细说明。
在一个可选示例中，该操作S210和S220可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的神经网络1010执行。
图3为本申请图像处理方法又一个实施例的流程图。如图3所示，该实施例方法包括：操作S300、操作S310以及操作S320。下面对图3中的各操作分别进行详细说明。
S300、将待处理图像输入神经网络。
在一个可选示例中，本申请的神经网络通常包括：大小不超过2M的神经网络。该神经网络在训练过程中，被作为学生神经网络。在训练该神经网络的过程中，所使用的指导信息通常包括：教师神经网络针对图像样本形成的离散特征数据与学生神经网络针对图像样本形成的离散特征数据之间的差异，以及学生神经网络针对图像样本进行分类处理的分类处理结果与图像样本的分类标注信息之间的差异。也就是说，教师神经网络针对图像样本形成的离散特征数据以及图像样本的分类标注信息，被作为训练学生神经网络的监督信号。训练神经网络所采用的损失函数可以包括L2损失函数等。该操作所涉及到的网络结构以及待处理图像等的内容可以参见上述方法实施方式中针对S100的描述，在此不再详细说明。
在一个可选示例中，该操作S300可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的输入图像模块1000执行。
S310、经由神经网络形成待处理图像的离散特征数据。
在一个可选示例中，该神经网络可以先形成该待处理图像的浮点特征数据，可选的，该神经网络针对待处理图像，进行浮点特征数据提取，并将提取出的浮点特征数据转换为符合预定要求的浮点特征数 据(例如，将浮点特征数据转换为具有预定通道数的浮点特征数据，再例如，将浮点特征数据转换为具有预定大小的浮点特征数据)，从而形成待处理图像的浮点特征数据。神经网络在形成待处理图像的浮点特征数据之后，再将该浮点特征数据量化为待处理图像的离散特征数据，如将浮点特征数据中的浮点数转换为相应的定点数等。该操作的内容可以参见上述方法实施方式中针对S110的相关描述，在此不再详细说明。
S320、经由神经网络，根据待处理图像的离散特征数据，对待处理图像进行分类处理。该分类处理的类别数量以及类别的内容可以根据实际需求灵活设置。
在一个可选示例中，该操作S310和S320可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的神经网络1010执行。
图4为本申请图像处理方法再一个实施例的流程图。如图4所示，该实施例方法包括：操作S400、操作S410以及操作S420。下面对图4中的各操作分别进行详细说明。
S400、将待处理图像输入神经网络。
在一个可选示例中，本申请的神经网络通常包括：大小不超过2M的神经网络。该神经网络在训练过程中，被作为学生神经网络。在训练该神经网络的过程中，所使用的指导信息通常包括：教师神经网络针对图像样本形成的离散特征数据与学生神经网络针对图像样本形成的离散特征数据之间的差异，以及学生神经网络针对图像样本进行物体检测处理的物体检测处理结果与图像样本的检测框标注信息(如检测框的位置标注信息和检测框的类别标注信息等)之间的差异。也就是说，教师神经网络针对图像样本形成的离散特征数据以及图像样本的检测框标注信息，均被作为训练学生神经网络的监督信号。训练神经网络所采用的损失函数可以包括L2损失函数等。该操作所涉及到的网络结构以及待处理图像等的内容可以参见上述方法实施方式中针对S100的描述，在此不再详细说明。
在一个可选示例中，该操作S400可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的输入图像模块1000执行。
S410、经由神经网络形成待处理图像的离散特征数据。
在一个可选示例中，该神经网络可以先形成该待处理图像的浮点特征数据，可选的，该神经网络针对待处理图像，进行浮点特征数据提取，并将提取出的浮点特征数据转换为符合预定要求的浮点特征数据(例如，将浮点特征数据转换为具有预定通道数的浮点特征数据，再例如，将浮点特征数据转换为具有预定大小的浮点特征数据)，从而形成待处理图像的浮点特征数据。神经网络型在形成待处理图像的浮点特征数据之后，再将该浮点特征数据量化为待处理图像的离散特征数据，如将浮点特征数据中的浮点数转换为相应的定点数等。该操作的内容可以参见上述方法实施方式中针对S110的相关描述，在此不再详细说明。
S420、经由神经网络，根据待处理图像的离散特征数据，对待处理图像进行物体检测处理。该物体检测处理所涉及到的检测框类别数量以及检测框类别的内容可以根据实际需求灵活设置。
在一个可选示例中，该操作S410和S420可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的神经网络1010执行。
图5为本申请的神经网络的训练方法的一个实施例的流程图。待训练的神经网络在训练过程中，被作为学生神经网络。如图5所示，该实施例方法包括：操作S500、操作S510以及操作S520。下面对图5中的各操作分别进行详细说明。
S500、将图像样本分别输入学生神经网络以及教师神经网络。
在一个可选示例中，本申请可以从训练数据集中获取图像样本。本申请中的训练数据集中包括多条用于训练神经网络的图像样本，通常情况下，每一个图像样本均设置有与相应视觉任务相关的标注信息。例如，图像样本的标注信息可以包括：图像样本的分类标注信息。再例如，图像样本的标注信息也可以包括：图像样本的检测框位置标注信息以及检测框分类标注信息等。本申请可以按照随机读取方式或者按照图像样本排列次序顺序读取方式，一次从训练数据集中读取一个或者多个图像样本。
在一个可选示例中，本申请提供给学生神经网络以及教师神经网络的图像样本的大小通常与神经网络对输入图像的尺寸要求相关，如图像样本的大小可以为256×256等。为了获得具有预定大小的图像样本，本申请可以对读取出的图像样本进行缩放处理，从而使提供给学生神经网络以及教师神经网络的每一个图像样本分别具有预定大小。本申请对图像样本的大小以及缩放处理的实现方式不作限制。
在一个可选示例中，本申请中的教师神经网络的网络参数并不需要一定为定点网络参数，也就是说，本申请的教师神经网络的网络参数可以为浮点网络参数，教师神经网络可以进行浮点运算，然而，教师神经网络最终形成的特征数据为离散特征数据，即教师神经网络针对图像样本提取出的浮点特征数据在 被量化处理后，形成离散特征数据。本申请通过利用形成离散特征数据的教师神经网络来训练学生神经网络，可以使教师神经网络的知识迁移到学生神经网络中，且学生神经网络的网络参数也并不需要一定为定点网络参数，也就是说，本申请的学生神经网络的网络参数可以为浮点网络参数，学生神经网络可以进行浮点运算，从而学生神经网络在成功训练后，一方面可以不受到特定指令集以及特定设备的限制，有利于提高神经网络的适用范围，另一方面，浮点运算获得的浮点特征数据在通过量化转换为离散特征数据后，不但可以使离散特征数据保持较好的准确性，而且有利于教师神经网络输出的离散特征数据与学生神经网络输出的离散特征数据的匹配，因此，有利于将教师神经网络的知识较好的迁移到学生神经网络中，从而有利于提高神经网络的性能。
S510、经由学生神经网络以及教师神经网络，分别形成图像样本的离散特征数据。
在一个可选示例中，学生神经网络可以先形成该图像样本的浮点特征数据，可选的，该学生神经网络针对图像样本，进行浮点特征数据提取，并将提取出的浮点特征数据转换为符合预定要求的浮点特征数据(例如，将浮点特征数据转换为具有预定通道数的浮点特征数据，再例如，将浮点特征数据转换为具有预定大小的浮点特征数据)，从而形成图像样本的浮点特征数据。学生神经网络在形成图像样本的浮点特征数据之后，再将该浮点特征数据量化为图像样本的离散特征数据，如将浮点特征数据中的浮点数转换为相应的定点数等。同样的，教师神经网络可以先形成该图像样本的浮点特征数据，可选的，该教师神经网络针对图像样本，进行浮点特征数据提取，并将提取出的浮点特征数据转换为符合预定要求的浮点特征数据(例如，将浮点特征数据转换为具有预定通道数的浮点特征数据，再例如，将浮点特征数据转换为具有预定大小的浮点特征数据)，从而形成图像样本的浮点特征数据。教师神经网络在形成图像样本的浮点特征数据后，再将该浮点特征数据量化为图像样本的离散特征数据，如将浮点特征数据中的浮点数转换为相应的定点数等。
在一个可选示例中，该操作S500和S510可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的输入图像样本模块1020执行。
S520、根据指导信息，对学生神经网络进行监督学习。
在一个可选示例中，本申请的指导信息通常包括：教师神经网络针对图像样本形成的离散特征数据，与学生神经网络针对图像样本形成的离散特征数据之间的差异。另外，本申请的指导信息还通常包括：学生神经网络输出的相应视觉任务处理结果与图像样本的标注信息之间的差异。也就是说，教师神经网络针对图像样本形成的离散特征数据，被作为训练学生神经网络的监督信号，与此同时，图像样本的标注信息同样被作为训练学生神经网络的监督信号。本申请可以以减小教师神经网络针对图像样本形成的离散特征数据，与学生神经网络针对图像样本形成的离散特征数据之间的差异，以及减小学生神经网络输出的视觉任务处理结果与图像样本的标注信息之间的差异为目的，利用相应的损失函数(如L2损失函数)，对学生神经网络进行监督学习。
在一个可选示例中，在针对学生神经网络的训练达到预定迭代条件时，本次训练过程结束。本申请中的预定迭代条件可以包括：学生神经网络输出的离散特征数据与教师神经网络输出的离散特征数据之间的差异，以及学生神经网络输出的视觉任务处理结果与图像样本的标注信息之间的差异均满足预定差异要求。在差异满足该预定差异要求的情况下，本次对学生神经网络成功训练完成。本申请中的预定迭代条件也可以包括：对学生神经网络进行训练，所使用的图像样本的数量达到预定数量要求等。在使用的图像样本的数量达到预定数量要求，然而，差异未满足预定差异要求的情况下，本次对学生神经网络并未训练成功。成功训练完成的学生神经网络可以用于形成待处理图像的离散特征数据以及视觉任务处理。
在一个可选示例中，该操作S520可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的监督模块1030执行。
图6为本申请的神经网络的训练方法另一个实施例的流程图。待训练的神经网络在训练过程中，被作为学生神经网络。如图6所示，该实施例方法包括：操作S600、操作S610、操作S620以及操作S630。下面对图6中的各操作分别进行详细说明。
S600、将图像样本分别输入学生神经网络以及教师神经网络。该操作的内容可以参见上述方法实施方式中针对S500的相关描述，在此不再详细说明。
S610、经由学生神经网络以及教师神经网络，分别形成图像样本的离散特征数据。该操作的内容可以参见上述方法实施方式中针对S510的相关描述，在此不再详细说明。
在一个可选示例中，该操作S600和S610可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的输入图像样本模块1020执行。
S620、经由学生神经网络，根据学生神经网络形成的图像样本的离散特征数据，对图像样本进行分类处理。
S630、以教师神经网络针对图像样本形成的离散特征数据与学生神经网络针对图像样本形成的离散特征数据之间的差异、以及学生神经网络输出的分类处理结果与图像样本的分类标注信息之间的差异，为指导信息，对学生神经网络进行监督学习。
在一个可选示例中，本申请可以以减小教师神经网络针对图像样本形成的离散特征数据，与学生神经网络针对图像样本形成的离散特征数据之间的差异，以及减小学生神经网络输出的分类处理结果与图像样本的分类标注信息之间的差异为目的，利用相应的损失函数(如L2损失函数)，对学生神经网络进行监督学习。
在一个可选示例中，在针对学生神经网络的训练达到预定迭代条件时，本次训练过程结束。本申请中的预定迭代条件可以包括：学生神经网络输出的离散特征数据与教师神经网络输出的离散特征数据之间的差异，以及学生神经网络输出的分类处理结果与图像样本的分类标注信息之间的差异均满足预定差异要求。在差异满足该预定差异要求的情况下，本次对学生神经网络成功训练完成。本申请中的预定迭代条件也可以包括：对学生神经网络进行训练，所使用的图像样本的数量达到预定数量要求等。在使用的图像样本的数量达到预定数量要求，而差异未满足预定差异要求的情况下，本次对学生神经网络并未训练成功。成功训练完成的学生神经网络可以用于形成待处理图像的离散特征数据以及分类处理。
在一个可选示例中，该操作S620和S630可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的监督模块1030执行。
图7为本申请的神经网络的训练方法再一个实施例的流程图。待训练的神经网络在训练过程中，被作为学生神经网络。如图7所示，该实施例方法包括：操作S700、操作S710、操作S720以及操作S730。下面对图7中的各操作分别进行详细说明。
S700、将图像样本分别输入学生神经网络以及教师神经网络。该操作的内容可以参见上述方法实施方式中针对S500的相关描述，在此不再详细说明。
S710、经由学生神经网络以及教师神经网络，分别形成图像样本的离散特征数据。该操作的内容可以参见上述方法实施方式中针对S510的相关描述，在此不再详细说明。
在一个可选示例中，该操作S700和S710可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的输入图像样本模块1020执行。
S720、经由学生神经网络，根据学生神经网络形成的图像样本的离散特征数据，对图像样本进行物体检测处理。
S730、以教师神经网络针对图像样本形成的离散特征数据与学生神经网络针对图像样本形成的离散特征数据之间的差异、以及学生神经网络输出的物体检测处理结果与图像样本的检测框标注信息之间的差异，为指导信息，对学生神经网络进行监督学习。
在一个可选示例中，学生神经网络所输出的物体检测处理结果可以包括：检测框的位置信息(如位于检测框的对角线上的两个顶点的坐标)以及检测框的类别等。本申请可以以减小教师神经网络针对图像样本形成的离散特征数据，与学生神经网络针对图像样本形成的离散特征数据之间的差异，以及减小学生神经网络输出的物体检测处理结果与图像样本的检测框标注信息之间的差异为目的，利用相应的损失函数(如L2损失函数)，对学生神经网络进行监督学习。本申请的图像样本的检测框标注信息可以包括：检测框的位置标注信息(如位于检测框的对角线上的两个顶点的坐标标注)以及检测框的类别标注等。
在一个可选示例中，在针对学生神经网络的训练达到预定迭代条件时，本次训练过程结束。本申请中的预定迭代条件可以包括：学生神经网络输出的离散特征数据与教师神经网络输出的离散特征数据之间的差异，以及学生神经网络输出的物体检测处理结果与图像样本的检测框标注信息之间的差异均满足预定差异要求。在差异满足该预定差异要求的情况下，本次对学生神经网络成功训练完成。本申请中的预定迭代条件也可以包括：对学生神经网络进行训练，所使用的图像样本的数量达到预定数量要求等。在使用的图像样本的数量达到预定数量要求，而差异未满足预定差异要求的情况下，本次对学生神经网络并未训练成功。成功训练完成的学生神经网络可以用于形成待处理图像的离散特征数据以及物体检测处理。
在一个可选示例中，本申请中的用于知识迁移的教师神经网络主要包括两部分，其中一部分为：已成功训练的浮点教师神经网络，该已成功训练的浮点教师神经网络通常用于针对输入图像形成浮点特征数据，并根据该浮点特征数据，对输入图像进行相应的视觉任务处理(例如，分类处理或者物体检测处 理等)；其中的另一部分为：量化辅助单元，量化辅助单元主要用于将浮点教师神经网络输出的浮点特征数据转换为离散特征数据，并将离散特征数据提供给浮点教师神经网络。量化辅助单元的输入是浮点教师神经网络中的相应层(如第N层)的输出，量化辅助单元的输出是浮点教师神经网络中的相应层(如第N+1层)的输入。由此可知，本申请中的量化辅助单元可以看作是在浮点教师神经网络中插入的一层。对教师神经网络的训练过程其实是对浮点教师神经网络的网络参数的微调过程。
在一个可选示例中，该操作S720和S730可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的监督模块1030执行。
本申请的教师神经网络的训练过程的一个实施例的流程图，如图8所示。
图8中，该训练过程包括：操作S800、操作S810以及操作S820。下面对图8中的各操作分别进行详细说明。
S800、将图像样本输入已成功训练的浮点教师神经网络。该已成功训练的浮点教师神经网络是预先利用图像样本成功训练的用于实现相应的视觉任务的神经网络。该浮点教师神经网络的训练过程的一个实施例如图9所示。
S810、经由已成功训练的浮点教师神经网络，提取输入的图像样本的浮点特征数据，经由量化辅助单元将浮点特征数据转换为离散特征数据，并经由已成功训练的浮点教师神经网络根据图像样本的离散特征数据，对图像样本进行视觉任务处理。例如，已成功训练的浮点教师神经网络根据图像样本的离散特征数据，对图像样本进行分类处理或者物体检测处理等。
在一个可选示例中，该操作S800和S810可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的教师训练模块1040的第一单元执行。
S820、以视觉任务处理结果与图像样本的标注信息之间的差异为指导信息，对已成功训练的浮点教师神经网络进行网络参数调整(即网络参数的微调)。
在一个可选示例中，该操作也可以认为是，以视觉任务处理结果与图像样本的标注信息之间的差异为指导信息，对已成功训练的浮点教师神经网络进行监督学习。本申请可以以减小已成功训练的浮点教师神经网络根据离散特征数据而输出的视觉任务处理结果与图像样本的标注信息之间的差异为目的，利用相应的损失函数(例如，L2损失函数等)，对已成功训练的浮点教师神经网络的网络参数进行微调。网络参数微调的收敛条件通常包括：已成功训练的浮点教师神经网络根据离散特征数据而输出的视觉任务处理结果与图像样本的标注信息之间的差异，满足预定差异要求。
在一个可选示例中，该操作S820可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的教师训练模块1040的第一监督单元执行。
本申请通过对已成功训练的浮点教师神经网络进行网络参数微调，可以使浮点教师神经网络(即教师神经网络)根据离散特征数据而输出的视觉任务处理结果更准确，从而通过知识迁移，可以使本申请的神经网络所形成的离散特征数据更有利于准确的完成视觉任务处理。
本申请的浮点教师神经网络的训练过程的一个实施例的流程图，如图9所示。图9中，该训练过程包括：操作S900、操作S910以及操作S920。下面对图9中的各操作分别进行详细说明。
S900、将图像样本输入待训练的浮点教师神经网络。该待训练的浮点教师神经网络的网络参数通常为浮点网络参数，该浮点教师神经网络通常为卷积神经网络等。
S910、经由待训练的浮点教师神经网络提取图像样本的浮点特征数据，并根据浮点特征数据，对图像样本进行视觉任务处理。例如，待训练的浮点教师神经网络根据浮点特征数据，对图像样本进行分类处理或者物体检测处理等。
在一个可选示例中，该操作S900和S910可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的教师训练模块1040的第二单元执行。
S920、以视觉任务处理结果与图像样本的标注信息之间的差异为指导信息，对待训练的浮点教师神经网络进行监督学习。
在一个可选示例中，图像样本的标注信息可以为图像样本的分类标注信息或者图像样本的检测框标注信息等。本申请可以以减小待训练的浮点教师神经网络根据浮点特征数据而输出的视觉任务处理结果与图像样本的标注信息之间的差异为目的，利用相应的损失函数(例如，L2损失函数等)，对待训练的浮点教师神经网络进行监督学习。训练的预定迭代条件通常包括：待训练的浮点教师神经网络根据浮点特征数据而输出的视觉任务处理结果与图像样本的标注信息之间的差异，满足预定差异要求。
在一个可选示例中，该操作S920可以由处理器调用存储器存储的相应指令执行，也可以由被处理器运行的教师训练模块1040的第二监督单元执行。
本申请实施例提供的任一种方法可以由任意适当的具有数据处理能力的设备执行，包括但不限于：终端设备和服务器等。或者，本申请实施例提供的任一种方法可以由处理器执行，如处理器通过调用存储器存储的相应指令来执行本申请实施例提及的任一种方法。下文不再赘述。
本领域普通技术人员可以理解：实现上述方法实施例的全部或部分操作可以通过程序指令相关的硬件来完成，前述的程序可以存储于一计算机可读取存储介质中，该程序在执行时，执行包括上述方法实施例的操作；而前述的存储介质包括：ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。
图10为本申请的图像处理装置一个实施例的结构示意图。图10所示的图像处理装置包括：输入图像模块1000和神经网络1010。可选的，该装置还可以包括：输入图像样本模块1020、监督模块1030、教师训练模块1040。
输入图像模块1000用于将待处理图像输入神经网络1010。
神经网络1010用于形成待处理图像的离散特征数据。
本申请中的神经网络是基于指导信息训练而成的，且神经网络在训练过程中，被作为学生神经网络，用于训练神经网络的指导信息通常包括：教师神经网络针对图像样本形成的离散特征数据与学生神经网络针对图像样本形成的离散特征数据之间的差异。
在一个可选示例中，本申请的神经网络包括：大小不超过2M的神经网络。
在一个可选示例中，本申请的神经网络进一步用于：形成待处理图像的浮点特征数据，并将浮点特征数据量化为待处理图像的离散特征数据。例如，神经网络针对待处理图像，进行浮点特征数据提取，并将提取出的浮点特征数据转换为符合预定要求的浮点特征数据，以形成待处理图像的浮点特征数据。
在一个可选示例中，神经网络可以将浮点特征数据转换为具有预定通道数的浮点特征数据；在另一个可选示例中，神经网络可以将浮点特征数据转换为具有预定大小的浮点特征数据。
在一个可选示例中，神经网络还可以对待处理图像的离散特征数据继续进行处理，例如，根据待处理图像的离散特征数据，对待处理图像进行相应视觉任务处理。在神经网络还需要继续执行视觉任务处理的情况下，用于训练该神经网络的指导信息还包括：学生神经网络针对图像样本输出的视觉任务处理结果与图像样本的标注信息之间的差异。
在一个可选示例中，神经网络继续执行的视觉任务处理可以为：分类处理，也可以为物体检测处理等。也就是说，神经网络可以根据待处理图像的离散特征数据，对待处理图像进行分类处理；神经网络也可以根据待处理图像的离散特征数据，对待处理图像进行物体检测处理。
在神经网络需要根据待处理图像的离散特征数据，对待处理图像进行分类处理的情况下，用于训练该神经网络的指导信息还可以包括：学生神经网络针对图像样本输出的分类处理结果，与图像样本的分类标注信息之间的差异。
在神经网络需要根据待处理图像的离散特征数据，对待处理图像进行物体检测处理的情况下，用于训练该神经网络的知道信息还可以包括：学生神经网络针对图像样本输出的物体检测处理结果，与图像样本的检测框标注信息之间的差异。
输入图像模块1000以及神经网络1010所执行的操作，可以参见上述方法实施方式中针对图1至图4的描述。而输入图像样本模块1020、监督模块1030以及教师训练模块1040所执行的操作可以参见下述针对图11的描述。在此均不再重复说明。
图11为本申请的神经网络的训练装置一个实施例的结构示意图。图11所示的训练装置包括：输入图像样本模块1020、监督模块1030以及教师训练模块1040。
输入图像样本模块1020用于将图像样本分别输入学生神经网络以及教师神经网络，以经由学生神经网络以及教师神经网络，分别形成图像样本的离散特征数据。
监督模块1030用于根据指导信息对学生神经网络进行监督学习。其中的指导信息包括：教师神经网络针对图像样本形成的离散特征数据，与学生神经网络针对图像样本形成的离散特征数据之间的差异。
在一个可选示例中，在学生神经网络用于根据图像样本的离散特征数据，对图像样本进行视觉任务处理的情况下，监督模块1030进一步用于以教师神经网络针对图像样本形成的离散特征数据，与学生神经网络针对图像样本形成的离散特征数据之间的差异，以及学生神经网络输出的视觉任务处理结果与图像样本的标注信息之间的差异，为指导信息，对学生神经网络进行监督学习。
在一个可选示例中，本申请中的教师神经网络包括：已成功训练的浮点教师神经网络以及量化辅助单元。其中的已成功训练的浮点教师神经网络用于针对输入图像形成浮点特征数据，并根据该浮点特征数据，对输入图像进行视觉任务处理。其中的量化辅助单元用于将浮点教师神经网络形成的浮点特征数 据转换为离散特征数据，并将该离散特征数据提供给浮点教师神经网络，使浮点教师神经网络根据该离散特征数据，对输入图像进行视觉任务处理。
在一个可选示例中，本申请中的教师训练模块1040用于训练教师神经网络，该教师训练模块1040包括：第一单元和第一监督单元。其中的第一单元用于将图像样本输入已成功训练的浮点教师神经网络，以经由已成功训练的浮点教师神经网络提取图像样本的浮点特征数据，经由量化辅助单元将浮点特征数据转换为离散特征数据，并经由已成功训练的浮点教师神经网络根据离散特征数据，对图像样本进行视觉任务处理。其中的第一监督单元用于以视觉任务处理结果与图像样本的标注信息之间的差异为指导信息，对已成功训练的浮点教师神经网络进行网络参数调整。
在一个可选示例中，该教师训练模块1040还可以包括：第二单元以及第二监督单元。其中的第二单元用于将图像样本输入待训练的浮点教师神经网络，以经由待训练的浮点教师神经网络提取图像样本的浮点特征数据，并根据浮点特征数据，对图像样本进行视觉任务处理。其中的第二监督单元用于以视觉任务处理结果与图像样本的标注信息之间的差异为指导信息，对待训练的浮点教师神经网络进行监督学习。
输入图像样本模块1020、监督模块1030以及教师训练模块1040所执行的操作，可以参见上述方法实施方式中针对图5至图9的描述。在此不再重复说明。
示例性设备
图12示出了适于实现本申请的示例性设备1200，设备1200可以是汽车中配置的控制系统/电子系统、移动终端(例如，智能移动电话等)、个人计算机(PC，例如，台式计算机或者笔记型计算机等)、平板电脑以及服务器等。图12中，设备1200包括一个或者多个处理器、通信部等，所述一个或者多个处理器可以为：一个或者多个中央处理单元(CPU)1201，和/或，一个或者多个利用神经网络进行图像处理的图像处理器(GPU)1213等，处理器可以根据存储在只读存储器(ROM)1202中的可执行指令或者从存储部分1208加载到随机访问存储器(RAM)1203中的可执行指令而执行各种适当的动作和处理。通信部1212可以包括但不限于网卡，所述网卡可以包括但不限于IB(Infiniband)网卡。处理器可与只读存储器1202和/或随机访问存储器1203中通信以执行可执行指令，通过总线1204与通信部1212相连、并经通信部1212与其他目标设备通信，从而完成本申请中的相应操作。
上述各指令所执行的操作可以参见上述方法实施例中的相关描述，在此不再详细说明。此外，在RAM 1203中，还可以存储有装置操作所需的各种程序以及数据。CPU1201、ROM1202以及RAM1203通过总线1204彼此相连。
在有RAM1203的情况下，ROM1202为可选模块。RAM1203存储可执行指令，或在运行时向ROM1202中写入可执行指令，可执行指令使中央处理单元1201执行上述物体分割方法所包括的操作。输入/输出(I/O)接口1205也连接至总线1204。通信部1212可以集成设置，也可以设置为具有多个子模块(例如，多个IB网卡)，并分别与总线连接。
以下部件连接至I/O接口1205：包括键盘、鼠标等的输入部分1206；包括诸如阴极射线管(CRT)、液晶显示器(LCD)等以及扬声器等的输出部分1207；包括硬盘等的存储部分1208；以及包括诸如LAN卡、调制解调器等的网络接口卡的通信部分1209。通信部分1209经由诸如因特网的网络执行通信处理。驱动器1210也根据需要连接至I/O接口1205。可拆卸介质1211，诸如磁盘、光盘、磁光盘、半导体存储器等等，根据需要安装在驱动器1210上，以便于从其上读出的计算机程序根据需要被安装在存储部分1208中。
需要特别说明的是，如图12所示的架构仅为一种可选实现方式，在实践过程中，可根据实际需要对上述图12的部件数量和类型进行选择、删减、增加或替换；在不同功能部件设置上，也可采用分离设置或集成设置等实现方式，例如，GPU和CPU可分离设置，再如理，可将GPU集成在CPU上，通信部可分离设置，也可集成设置在CPU或GPU上等。这些可替换的实施方式均落入本申请的保护范围。
特别地，根据本申请的实施方式，下文参考流程图描述的过程可以被实现为计算机软件程序，例如，本申请实施方式包括一种计算机程序产品，其包含有形地包含在机器可读介质上的计算机程序，计算机程序包含用于执行流程图所示的操作的程序代码，程序代码可包括对应执行本申请提供的方法中的操作对应的指令。
在这样的实施方式中，该计算机程序可以通过通信部分1209从网络上被下载及安装，和/或从可拆卸介质1211被安装。在该计算机程序被中央处理单元(CPU)1201执行时，执行本申请中记载的实现上述相应操作的指令。
在一个或多个可选实施方式中，本申请实施例还提供了一种计算机程序程序产品，用于存储计算机 可读指令，所述指令被执行时使得计算机执行上述任意实施例中所述的图像处理方法或者神经网络的训练方法。
该计算机程序产品可以通过硬件、软件或其结合的方式实现。在一个可选例子中，所述计算机程序产品体现为计算机存储介质，在另一个可选例子中，所述计算机程序产品体现为软件产品，例如软件开发包(Software Development Kit，SDK)等等。
在一个或多个可选实施方式中，本申请实施例还提供了另一种图像处理方法和神经网络的训练方法及其对应的装置和电子设备、计算机存储介质、计算机程序以及计算机程序产品，其中的方法包括：第一装置向第二装置发送图像处理指示或者训练神经网络指示，该指示使得第二装置执行上述任一可能的实施例中的图像处理方法或者神经网络的训练方法；第一装置接收第二装置发送的图像处理结果或者神经网络训练结果。
在一些实施例中，该图像处理指示或者训练神经网络指示可以为调用指令，第一装置可以通过调用的方式指示第二装置执行图像处理操作或者训练神经网络操作，相应地，响应于接收到调用指令，第二装置可以执行上述图像处理方法或者神经网络的训练方法中的任意实施例中的操作和/或流程。
应理解，本申请实施例中的“第一”、“第二”等术语仅仅是为了区分，而不应理解成对本申请实施例的限定。还应理解，在本申请中，“多个”可以指两个或两个以上，“至少一个”可以指一个、两个或两个以上。还应理解，对于本申请中提及的任一部件、数据或结构，在没有明确限定或者在前后文给出相反启示的情况下，一般可以理解为一个或多个。还应理解，本申请对各个实施例的描述着重强调各个实施例之间的不同之处，其相同或相似之处可以相互参考，为了简洁，不再一一赘述。
可能以许多方式来实现本申请的方法和装置、电子设备以及计算机可读存储介质。例如，可通过软件、硬件、固件或者软件、硬件、固件的任何组合来实现本申请的方法和装置、电子设备以及计算机可读存储介质。用于方法的操作的上述顺序仅是为了进行说明，本申请的方法的操作不限于以上描述的顺序，除非以其它方式特别说明。此外，在一些实施方式中，还可将本申请实施为记录在记录介质中的程序，这些程序包括用于实现根据本申请的方法的机器可读指令。因而，本申请还覆盖存储用于执行根据本申请的方法的程序的记录介质。
本申请的描述，是为了示例和描述起见而给出的，而并不是无遗漏的或者将本申请限于所申请的形式。很多修改和变化对于本领域的普通技术人员而言，是显然的。选择和描述实施方式是为了更好说明本申请的原理以及实际应用，并且使本领域的普通技术人员能够理解本申请实施例可以从而设计适于特定用途的带有各种修改的各种实施方式。

Claims (41)

  1. 一种图像处理方法，其特征在于，包括：
    将待处理图像输入神经网络；
    经由所述神经网络形成所述待处理图像的离散特征数据；
    其中，所述神经网络是基于指导信息训练而成的，且所述神经网络在训练过程中，被作为学生神经网络，所述指导信息包括：教师神经网络针对图像样本形成的离散特征数据与所述学生神经网络针对所述图像样本形成的离散特征数据之间的差异。
  2. 根据权利要求1所述的方法，其特征在于，所述神经网络包括：大小不超过2M的神经网络。
  3. 根据权利要求1至2中任一项所述的方法，其特征在于，所述经由所述神经网络形成所述待处理图像的离散特征数据包括：
    经由所述神经网络形成所述待处理图像的浮点特征数据，并将所述浮点特征数据量化为所述待处理图像的离散特征数据。
  4. 根据权利要求3所述的方法，其特征在于，所述经由所述神经网络形成所述待处理图像的浮点特征数据包括：
    经由所述神经网络针对所述待处理图像，进行浮点特征数据提取，并将提取出的浮点特征数据转换为符合预定要求的浮点特征数据，以形成所述待处理图像的浮点特征数据。
  5. 根据权利要求4所述的方法，其特征在于，所述将提取出的浮点特征数据转换为符合预定要求的浮点特征数据包括：
    将所述浮点特征数据转换为具有预定通道数的浮点特征数据；和/或者
    将所述浮点特征数据转换为具有预定大小的浮点特征数据。
  6. 根据权利要求1至5中任一项所述的方法，其特征在于，所述方法还包括：
    经由所述神经网络，根据所述待处理图像的离散特征数据，对所述待处理图像进行相应视觉任务处理。
  7. 根据权利要求6所述的方法，其特征在于，所述指导信息还包括：学生神经网络针对图像样本输出的视觉任务处理结果与图像样本的标注信息之间的差异。
  8. 根据权利要求6所述的方法，其特征在于，所述经由所述神经网络，根据所述待处理图像的离散特征数据，对所述待处理图像进行相应视觉任务处理包括：
    经由所述神经网络，根据所述待处理图像的离散特征数据，对所述待处理图像进行分类处理；或者经由所述神经网络，根据所述待处理图像的离散特征数据，对所述待处理图像进行物体检测处理。
  9. 根据权利要求8所述的方法，其特征在于，所述指导信息还包括：
    所述学生神经网络针对图像样本输出的分类处理结果，与图像样本的分类标注信息之间的差异；或所述学生神经网络针对图像样本输出的物体检测处理结果，与图像样本的检测框标注信息之间的差异。
  10. 根据权利要求1至9中任一项所述的方法，其特征在于，所述神经网络的训练过程包括：
    将图像样本分别输入学生神经网络以及教师神经网络；
    经由学生神经网络以及教师神经网络，分别形成所述图像样本的离散特征数据；
    根据指导信息，对所述学生神经网络进行监督学习；
    其中，所述指导信息包括：教师神经网络针对图像样本形成的离散特征数据与所述学生神经网络针对图像样本形成的离散特征数据之间的差异。
  11. 根据权利要求10所述的方法，其特征在于，所述神经网络的训练过程还包括：
    经由所述学生神经网络，根据所述图像样本的离散特征数据，对图像样本进行视觉任务处理；
    所述根据指导信息，对所述学生神经网络进行监督学习包括：
    以教师神经网络针对图像样本形成的离散特征数据与所述学生神经网络针对图像样本形成的离散特征数据之间的差异，以及学生神经网络输出的视觉任务处理结果与图像样本的标注信息之间的差异，为指导信息，对所述学生神经网络进行监督学习。
  12. 根据权利要求1至11中任一项所述的方法，其特征在于，所述教师神经网络包括：
    已成功训练的浮点教师神经网络，该已成功训练的浮点教师神经网络用于针对输入图像形成浮点特征数据，并根据该浮点特征数据，对输入图像进行视觉任务处理；以及
    量化辅助单元，用于将浮点教师神经网络形成的浮点特征数据转换为离散特征数据，并将该离散特征数据提供给浮点教师神经网络，使浮点教师神经网络根据该离散特征数据，对输入图像进行视觉任务 处理。
  13. 根据权利要求12所述的方法，其特征在于，所述教师神经网络的训练过程包括：
    将图像样本输入已成功训练的浮点教师神经网络；
    经由已成功训练的浮点教师神经网络提取图像样本的浮点特征数据，经由量化辅助单元将浮点特征数据转换为离散特征数据，并经由已成功训练的浮点教师神经网络根据所述离散特征数据，对图像样本进行视觉任务处理；
    以视觉任务处理结果与图像样本的标注信息之间的差异为指导信息，对已成功训练的浮点教师神经网络进行网络参数调整。
  14. 根据权利要求12至13中任一项所述的方法，其特征在于，所述浮点教师神经网络的训练过程包括：
    将图像样本输入待训练的浮点教师神经网络；
    经由待训练的浮点教师神经网络提取所述图像样本的浮点特征数据，并根据所述浮点特征数据，对图像样本进行视觉任务处理；
    以所述视觉任务处理结果与图像样本的标注信息之间的差异为指导信息，对所述待训练的浮点教师神经网络进行监督学习。
  15. 一种神经网络的训练方法，其特征在于，待训练的神经网络在训练过程中，被作为学生神经网络，所述训练方法包括：
    将图像样本分别输入学生神经网络以及教师神经网络；
    经由学生神经网络以及教师神经网络，分别形成所述图像样本的离散特征数据；
    根据指导信息，对所述学生神经网络进行监督学习；
    其中，所述指导信息包括：教师神经网络针对图像样本形成的离散特征数据与所述学生神经网络针对图像样本形成的离散特征数据之间的差异。
  16. 根据权利要求15所述的方法，其特征在于，所述神经网络作为学生神经网络的训练过程还包括：
    经由所述学生神经网络，根据所述图像样本的离散特征数据，对图像样本进行视觉任务处理；
    所述根据指导信息，对所述学生神经网络进行监督学习包括：
    以教师神经网络针对图像样本形成的离散特征数据，与所述学生神经网络针对图像样本形成的离散特征数据之间的差异，以及学生神经网络输出的视觉任务处理结果与图像样本的标注信息之间的差异，为指导信息，对所述学生神经网络进行监督学习。
  17. 根据权利要求15至16中任一项所述的方法，其特征在于，所述教师神经网络包括：
    已成功训练的浮点教师神经网络，该已成功训练的浮点教师神经网络用于针对输入图像形成浮点特征数据，并根据该浮点特征数据，对输入图像进行视觉任务处理；以及
    量化辅助单元，用于将浮点教师神经网络形成的浮点特征数据转换为离散特征数据，并将该离散特征数据提供给浮点教师神经网络，使浮点教师神经网络根据该离散特征数据，对输入图像进行视觉任务处理。
  18. 根据权利要求17所述的方法，其特征在于，所述教师神经网络的训练过程包括：
    将图像样本输入已成功训练的浮点教师神经网络；
    经由已成功训练的浮点教师神经网络提取图像样本的浮点特征数据，经由量化辅助单元将浮点特征数据转换为离散特征数据，并经由已成功训练的浮点教师神经网络根据所述离散特征数据，对图像样本进行视觉任务处理；
    以视觉任务处理结果与图像样本的标注信息之间的差异为指导信息，对已成功训练的浮点教师神经网络进行网络参数调整。
  19. 根据权利要求17至18中任一项所述的方法，其特征在于，所述浮点教师神经网络的训练过程包括：
    将图像样本输入待训练的浮点教师神经网络；
    经由待训练的浮点教师神经网络提取所述图像样本的浮点特征数据，并根据所述浮点特征数据，对图像样本进行视觉任务处理；
    以所述视觉任务处理结果与图像样本的标注信息之间的差异为指导信息，对所述待训练的浮点教师神经网络进行监督学习。
  20. 一种图像处理装置，其特征在于，包括：
    输入图像模块，用于将待处理图像输入神经网络；
    神经网络，用于形成所述待处理图像的离散特征数据；
    其中，所述神经网络是基于指导信息训练而成的，且所述神经网络在训练过程中，被作为学生神经网络，所述指导信息包括：教师神经网络针对图像样本形成的离散特征数据与所述学生神经网络针对所述图像样本形成的离散特征数据之间的差异。
  21. 根据权利要求20所述的装置，其特征在于，所述神经网络包括：大小不超过2M的神经网络。
  22. 根据权利要求20至21中任一项所述的装置，其特征在于，所述神经网络进一步用于：
    形成所述待处理图像的浮点特征数据，并将所述浮点特征数据量化为所述待处理图像的离散特征数据。
  23. 根据权利要求22所述的装置，其特征在于，所述神经网络针对所述待处理图像，进行浮点特征数据提取，并将提取出的浮点特征数据转换为符合预定要求的浮点特征数据，以形成所述待处理图像的浮点特征数据。
  24. 根据权利要求23所述的装置，其特征在于：
    所述神经网络将所述浮点特征数据转换为具有预定通道数的浮点特征数据；和/或者
    所述神经网络将所述浮点特征数据转换为具有预定大小的浮点特征数据。
  25. 根据权利要求20至24中任一项所述的装置，其特征在于，所述神经网络还用于：根据所述待处理图像的离散特征数据，对所述待处理图像进行相应视觉任务处理。
  26. 根据权利要求25所述的装置，其特征在于，所述指导信息还包括：学生神经网络针对图像样本输出的视觉任务处理结果与图像样本的标注信息之间的差异。
  27. 根据权利要求25所述的装置，其特征在于：
    所述神经网络根据所述待处理图像的离散特征数据，对所述待处理图像进行分类处理；或者
    所述神经网络根据所述待处理图像的离散特征数据，对所述待处理图像进行物体检测处理。
  28. 根据权利要求27所述的装置，其特征在于，所述指导信息还包括：
    所述学生神经网络针对图像样本输出的分类处理结果，与图像样本的分类标注信息之间的差异；或所述学生神经网络针对图像样本输出的物体检测处理结果，与图像样本的检测框标注信息之间的差异。
  29. 根据权利要求20至28中任一项所述的装置，其特征在于，所述装置还包括：
    输入图像样本模块，用于将图像样本分别输入学生神经网络以及教师神经网络，以经由学生神经网络以及教师神经网络，分别形成所述图像样本的离散特征数据；
    监督模块，用于根据指导信息，对所述学生神经网络进行监督学习；
    其中，所述指导信息包括：教师神经网络针对图像样本形成的离散特征数据，与所述学生神经网络针对图像样本形成的离散特征数据之间的差异。
  30. 根据权利要求29所述的装置，其特征在于，在所述学生神经网络用于，根据所述图像样本的离散特征数据，对图像样本进行视觉任务处理的情况下，
    所述监督模块进一步用于：
    以教师神经网络针对图像样本形成的离散特征数据，与所述学生神经网络针对图像样本形成的离散特征数据之间的差异，以及学生神经网络输出的视觉任务处理结果与图像样本的标注信息之间的差异，为指导信息，对所述学生神经网络进行监督学习。
  31. 根据权利要求20至30中任一项所述的装置，其特征在于，所述教师神经网络包括：
    已成功训练的浮点教师神经网络，该已成功训练的浮点教师神经网络用于针对输入图像形成浮点特征数据，并根据该浮点特征数据，对输入图像进行视觉任务处理；以及
    量化辅助单元，用于将浮点教师神经网络形成的浮点特征数据转换为离散特征数据，并将该离散特征数据提供给浮点教师神经网络，使浮点教师神经网络根据该离散特征数据，对输入图像进行视觉任务处理。
  32. 根据权利要求31所述的装置，其特征在于，所述装置还包括：用于训练所述教师神经网络的教师训练模块，所述教师训练模块包括：
    第一单元，用于将图像样本输入已成功训练的浮点教师神经网络，以经由已成功训练的浮点教师神经网络提取图像样本的浮点特征数据，经由量化辅助单元将浮点特征数据转换为离散特征数据，并经由已成功训练的浮点教师神经网络根据所述离散特征数据，对图像样本进行视觉任务处理；
    第一监督单元，用于以视觉任务处理结果与图像样本的标注信息之间的差异为指导信息，对已成功 训练的浮点教师神经网络进行网络参数调整。
  33. 根据权利要求32所述的装置，其特征在于，所述教师训练模块还包括：
    第二单元，用于将图像样本输入待训练的浮点教师神经网络，以经由待训练的浮点教师神经网络提取所述图像样本的浮点特征数据，并根据所述浮点特征数据，对图像样本进行视觉任务处理；
    第二监督单元，用于以所述视觉任务处理结果与图像样本的标注信息之间的差异为指导信息，对所述待训练的浮点教师神经网络进行监督学习。
  34. 一种神经网络的训练装置，其特征在于，待训练的神经网络被作为学生神经网络，所述装置包括：
    输入图像样本模块，用于将图像样本分别输入学生神经网络以及教师神经网络，以经由学生神经网络以及教师神经网络，分别形成所述图像样本的离散特征数据；
    监督模块，用于根据指导信息，对所述学生神经网络进行监督学习；
    其中，所述指导信息包括：教师神经网络针对图像样本形成的离散特征数据与所述学生神经网络针对图像样本形成的离散特征数据之间的差异。
  35. 根据权利要求34所述的装置，其特征在于，在所述学生神经网络用于，根据所述图像样本的离散特征数据，对图像样本进行视觉任务处理的情况下，
    所述监督模块进一步用于：
    以教师神经网络针对图像样本形成的离散特征数据，与所述学生神经网络针对图像样本形成的离散特征数据之间的差异，以及学生神经网络输出的视觉任务处理结果与图像样本的标注信息之间的差异，为指导信息，对所述学生神经网络进行监督学习。
  36. 根据权利要求34至35中任一项所述的装置，其特征在于，所述教师神经网络包括：
    已成功训练的浮点教师神经网络，该已成功训练的浮点教师神经网络用于针对输入图像形成浮点特征数据，并根据该浮点特征数据，对输入图像进行视觉任务处理；以及
    量化辅助单元，用于将浮点教师神经网络形成的浮点特征数据转换为离散特征数据，并将该离散特征数据提供给浮点教师神经网络，使浮点教师神经网络根据该离散特征数据，对输入图像进行视觉任务处理。
  37. 根据权利要求36所述的装置，其特征在于，所述装置还包括：用于训练所述教师神经网络的教师训练模块，所述教师训练模块包括：
    第一单元，用于将图像样本输入已成功训练的浮点教师神经网络，以经由已成功训练的浮点教师神经网络提取图像样本的浮点特征数据，经由量化辅助单元将浮点特征数据转换为离散特征数据，并经由已成功训练的浮点教师神经网络根据所述离散特征数据，对图像样本进行视觉任务处理；
    第一监督单元，用于以视觉任务处理结果与图像样本的标注信息之间的差异为指导信息，对已成功训练的浮点教师神经网络进行网络参数调整。
  38. 根据权利要求37所述的装置，其特征在于，所述教师训练模块还包括：
    第二单元，用于将图像样本输入待训练的浮点教师神经网络，以经由待训练的浮点教师神经网络提取所述图像样本的浮点特征数据，并根据所述浮点特征数据，对图像样本进行视觉任务处理；
    第二监督单元，用于以所述视觉任务处理结果与图像样本的标注信息之间的差异为指导信息，对所述待训练的浮点教师神经网络进行监督学习。
  39. 一种电子设备，其特征在于，包括：
    存储器，用于存储计算机程序；
    处理器，用于执行所述存储器中存储的计算机程序，且所述计算机程序被执行时，实现上述权利要求1-19中任一项所述的方法。
  40. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，该计算机程序被处理器执行时，实现上述权利要求1-19中任一项所述的方法。
  41. 一种计算机程序，包括计算机指令，其特征在于，当所述计算机指令在设备的处理器中运行时，实现上述权利要求1-19中任一项所述的方法。
PCT/CN2018/116574 2018-04-25 2018-11-21 图像处理方法、训练方法、装置、设备、介质及程序 Ceased WO2019205604A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
JP2020533824A JP2021507397A (ja) 2018-04-25 2018-11-21 画像処理方法、訓練方法、装置、機器、媒体及びプログラム
SG11201912721YA SG11201912721YA (en) 2018-04-25 2018-11-21 Image processing methods, training methods, apparatuses, devices, media, and programs
KR1020207016931A KR102478000B1 (ko) 2018-04-25 2018-11-21 이미지 처리 방법, 훈련 방법, 장치, 기기, 매체 및 프로그램
US16/700,348 US11334763B2 (en) 2018-04-25 2019-12-02 Image processing methods, training methods, apparatuses, devices, media, and programs

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810380453.X 2018-04-25
CN201810380453.XA CN108830288A (zh) 2018-04-25 2018-04-25 图像处理方法、神经网络的训练方法、装置、设备及介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/700,348 Continuation US11334763B2 (en) 2018-04-25 2019-12-02 Image processing methods, training methods, apparatuses, devices, media, and programs

Publications (1)

Publication Number Publication Date
WO2019205604A1 true WO2019205604A1 (zh) 2019-10-31

Family

ID=64155627

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/116574 Ceased WO2019205604A1 (zh) 2018-04-25 2018-11-21 图像处理方法、训练方法、装置、设备、介质及程序

Country Status (6)

Country Link
US (1) US11334763B2 (zh)
JP (1) JP2021507397A (zh)
KR (1) KR102478000B1 (zh)
CN (1) CN108830288A (zh)
SG (1) SG11201912721YA (zh)
WO (1) WO2019205604A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111027551A (zh) * 2019-12-17 2020-04-17 腾讯科技（深圳）有限公司 图像处理方法、设备和介质
CN111144417A (zh) * 2019-12-27 2020-05-12 创新奇智(重庆)科技有限公司 基于教师学生网络的智能货柜小目标检测方法及检测系统
CN111340964A (zh) * 2020-03-05 2020-06-26 长春中国光学科学技术馆 一种基于迁移学习的3d模型图像的构建方法
CN112801298A (zh) * 2021-01-20 2021-05-14 北京百度网讯科技有限公司 异常样本检测方法、装置、设备和存储介质

Families Citing this family (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018033137A1 (zh) * 2016-08-19 2018-02-22 北京市商汤科技开发有限公司 在视频图像中展示业务对象的方法、装置和电子设备
CN108830288A (zh) * 2018-04-25 2018-11-16 北京市商汤科技开发有限公司 图像处理方法、神经网络的训练方法、装置、设备及介质
CN110598840B (zh) * 2018-06-13 2023-04-18 富士通株式会社 知识迁移方法、信息处理设备以及存储介质
CN109583576B (zh) * 2018-12-17 2020-11-06 上海联影智能医疗科技有限公司 一种医学图像处理装置及方法
WO2020125806A1 (en) 2018-12-17 2020-06-25 Shanghai United Imaging Intelligence Co., Ltd. Systems and methods for image segmentation
CN111383156B (zh) * 2018-12-29 2022-08-02 北京市商汤科技开发有限公司 图像处理方法、装置、智能驾驶系统和车载运算平台
CN111383157B (zh) * 2018-12-29 2023-04-14 北京市商汤科技开发有限公司 图像处理方法、装置、车载运算平台、电子设备及系统
CN109800821A (zh) * 2019-01-31 2019-05-24 北京市商汤科技开发有限公司 训练神经网络的方法、图像处理方法、装置、设备和介质
CN110009052B (zh) * 2019-04-11 2022-11-18 腾讯科技（深圳）有限公司 一种图像识别的方法、图像识别模型训练的方法及装置
CN110363297A (zh) * 2019-07-05 2019-10-22 上海商汤临港智能科技有限公司 神经网络训练及图像处理方法、装置、设备和介质
CN112784978A (zh) * 2019-11-08 2021-05-11 佳能株式会社 训练神经网络的方法、装置和系统及存储指令的存储介质
CN113052768B (zh) * 2019-12-27 2024-03-19 武汉Tcl集团工业研究院有限公司 一种处理图像的方法、终端及计算机可读存储介质
CN113128661A (zh) * 2020-01-15 2021-07-16 富士通株式会社 信息处理装置和信息处理方法
CN111382870A (zh) * 2020-03-06 2020-07-07 商汤集团有限公司 训练神经网络的方法以及装置
WO2021182748A1 (ko) 2020-03-10 2021-09-16 삼성전자주식회사 전자 장치 및 그 제어 방법
CN111488476B (zh) * 2020-04-03 2023-06-27 北京爱芯科技有限公司 图像推送方法、模型训练方法及对应装置
CN111598793A (zh) * 2020-04-24 2020-08-28 云南电网有限责任公司电力科学研究院 一种用于输电线路图像去雾的方法、系统及存储介质
CN113627456B (zh) * 2020-05-09 2024-03-19 杭州海康威视数字技术股份有限公司 生成基准数据的方法和装置
CN111724306B (zh) * 2020-06-19 2022-07-08 福州大学 一种基于卷积神经网络的图像缩小方法及系统
CN111967597B (zh) * 2020-08-18 2024-12-13 上海商汤临港智能科技有限公司 神经网络训练及图像分类方法、装置、存储介质、设备
CN111767711B (zh) * 2020-09-02 2020-12-08 之江实验室 基于知识蒸馏的预训练语言模型的压缩方法及平台
CN112001364A (zh) * 2020-09-22 2020-11-27 上海商汤临港智能科技有限公司 图像识别方法及装置、电子设备和存储介质
CN112819050B (zh) * 2021-01-22 2023-10-27 北京市商汤科技开发有限公司 知识蒸馏和图像处理方法、装置、电子设备和存储介质
CN113065638B (zh) * 2021-02-27 2025-03-11 华为技术有限公司 一种神经网络压缩方法及其相关设备
CN113052868B (zh) * 2021-03-11 2023-07-04 奥比中光科技集团股份有限公司 一种抠图模型训练、图像抠图的方法及装置
CN114037056B (zh) * 2021-11-05 2025-08-19 北京市商汤科技开发有限公司 一种生成神经网络的方法、装置、计算机设备和存储介质
CN115082690B (zh) * 2022-07-12 2023-03-28 北京百度网讯科技有限公司 目标识别方法、目标识别模型训练方法及装置
CN115511693A (zh) * 2022-08-22 2022-12-23 阿里巴巴（中国）有限公司 一种神经网络模型处理方法和装置
US20240290332A1 (en) * 2023-02-28 2024-08-29 Qualcomm Incorporated Knowledge distillation from non-streaming to streaming encoder
CN116310388B (zh) * 2023-03-27 2025-11-28 安谋科技（中国）有限公司 图像处理方法、可读介质和电子设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106355248A (zh) * 2016-08-26 2017-01-25 深圳先进技术研究院 一种深度卷积神经网络训练方法及装置
US20170076224A1 (en) * 2015-09-15 2017-03-16 International Business Machines Corporation Learning of classification model
CN107247989A (zh) * 2017-06-15 2017-10-13 北京图森未来科技有限公司 一种神经网络训练方法及装置
CN107358293A (zh) * 2017-06-15 2017-11-17 北京图森未来科技有限公司 一种神经网络训练方法及装置
CN108830288A (zh) * 2018-04-25 2018-11-16 北京市商汤科技开发有限公司 图像处理方法、神经网络的训练方法、装置、设备及介质

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6364387B2 (ja) * 2015-08-17 2018-07-25 日本電信電話株式会社 特徴量生成装置、方法、及びプログラム
KR102492318B1 (ko) * 2015-09-18 2023-01-26 삼성전자주식회사 모델 학습 방법 및 장치, 및 데이터 인식 방법
US10755172B2 (en) * 2016-06-22 2020-08-25 Massachusetts Institute Of Technology Secure training of multi-party deep neural network
WO2018126213A1 (en) * 2016-12-30 2018-07-05 Google Llc Multi-task learning using knowledge distillation
US20180268292A1 (en) * 2017-03-17 2018-09-20 Nec Laboratories America, Inc. Learning efficient object detection models with knowledge distillation
US11195093B2 (en) * 2017-05-18 2021-12-07 Samsung Electronics Co., Ltd Apparatus and method for student-teacher transfer learning network using knowledge bridge
KR102570278B1 (ko) * 2017-07-31 2023-08-24 삼성전자주식회사 교사 모델로부터 학생 모델을 트레이닝하는데 사용되는 학습 데이터를 생성하는 장치 및 방법
US10885900B2 (en) * 2017-08-11 2021-01-05 Microsoft Technology Licensing, Llc Domain adaptation in speech recognition via teacher-student learning
CN107578453B (zh) * 2017-10-18 2019-11-01 北京旷视科技有限公司 压缩图像处理方法、装置、电子设备及计算机可读介质
CN107743235B (zh) * 2017-10-27 2019-09-27 厦门美图之家科技有限公司 图像处理方法、装置及电子设备
WO2019167883A1 (ja) * 2018-02-28 2019-09-06 富士フイルム株式会社 機械学習装置および方法
US10643602B2 (en) * 2018-03-16 2020-05-05 Microsoft Technology Licensing, Llc Adversarial teacher-student learning for unsupervised domain adaptation
KR102896236B1 (ko) * 2019-05-09 2025-12-08 삼성전자주식회사 모델 학습 방법 및 장치

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170076224A1 (en) * 2015-09-15 2017-03-16 International Business Machines Corporation Learning of classification model
CN106355248A (zh) * 2016-08-26 2017-01-25 深圳先进技术研究院 一种深度卷积神经网络训练方法及装置
CN107247989A (zh) * 2017-06-15 2017-10-13 北京图森未来科技有限公司 一种神经网络训练方法及装置
CN107358293A (zh) * 2017-06-15 2017-11-17 北京图森未来科技有限公司 一种神经网络训练方法及装置
CN108830288A (zh) * 2018-04-25 2018-11-16 北京市商汤科技开发有限公司 图像处理方法、神经网络的训练方法、装置、设备及介质

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111027551A (zh) * 2019-12-17 2020-04-17 腾讯科技（深圳）有限公司 图像处理方法、设备和介质
CN111027551B (zh) * 2019-12-17 2023-07-07 腾讯科技（深圳）有限公司 图像处理方法、设备和介质
CN111144417A (zh) * 2019-12-27 2020-05-12 创新奇智(重庆)科技有限公司 基于教师学生网络的智能货柜小目标检测方法及检测系统
CN111340964A (zh) * 2020-03-05 2020-06-26 长春中国光学科学技术馆 一种基于迁移学习的3d模型图像的构建方法
CN111340964B (zh) * 2020-03-05 2023-03-24 长春中国光学科学技术馆 一种基于迁移学习的3d模型图像的构建方法
CN112801298A (zh) * 2021-01-20 2021-05-14 北京百度网讯科技有限公司 异常样本检测方法、装置、设备和存储介质
CN112801298B (zh) * 2021-01-20 2023-09-01 北京百度网讯科技有限公司 异常样本检测方法、装置、设备和存储介质

Also Published As

Publication number Publication date
SG11201912721YA (en) 2020-01-30
JP2021507397A (ja) 2021-02-22
KR20200087807A (ko) 2020-07-21
CN108830288A (zh) 2018-11-16
US11334763B2 (en) 2022-05-17
US20200104642A1 (en) 2020-04-02
KR102478000B1 (ko) 2022-12-15

Similar Documents

Publication Publication Date Title
US11334763B2 (en) Image processing methods, training methods, apparatuses, devices, media, and programs
CN109816589B (zh) 用于生成漫画风格转换模型的方法和装置
US11062453B2 (en) Method and system for scene parsing and storage medium
US12288383B2 (en) Using training images and scaled training images to train an image segmentation model
WO2019129032A1 (zh) 遥感图像识别方法、装置、存储介质以及电子设备
CN108154222B (zh) 深度神经网络训练方法和系统、电子设备
WO2018121737A1 (zh) 关键点预测、网络训练及图像处理方法和装置、电子设备
CN108280451B (zh) 语义分割及网络训练方法和装置、设备、介质
CN107609506B (zh) 用于生成图像的方法和装置
WO2023078070A1 (zh) 一种字符识别方法、装置、设备、介质及产品
US11164004B2 (en) Keyframe scheduling method and apparatus, electronic device, program and medium
CN116468970B (zh) 模型训练方法、图像处理方法、装置、设备及介质
CN112766284B (zh) 图像识别方法和装置、存储介质和电子设备
WO2022156434A1 (zh) 用于生成文本的方法和装置
WO2023005386A1 (zh) 模型训练方法和装置
WO2022194130A1 (zh) 字符位置修正方法、装置、电子设备和存储介质
CN113962737B (zh) 目标识别模型训练方法和装置、目标识别方法和装置
WO2022012179A1 (zh) 生成特征提取网络的方法、装置、设备和计算机可读介质
CN113139816B (zh) 信息处理方法、装置、电子设备和存储介质
CN114494784A (zh) 深度学习模型的训练方法、图像处理方法和对象识别方法
CN108154153A (zh) 场景分析方法和系统、电子设备
CN108229680A (zh) 神经网络系统、遥感图像识别方法、装置、设备及介质
CN111444807A (zh) 目标检测方法、装置、电子设备和计算机可读介质
CN114067196A (zh) 生成图像场景信息的方法及装置
CN113688270A (zh) 图像处理方法、装置、终端设备和计算机介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18917006

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 20207016931

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2020533824

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18917006

Country of ref document: EP

Kind code of ref document: A1