WO2019223594A1 - Procédé et dispositif de traitement de modèle de réseau neuronal, procédé de traitement d'images, et terminal mobile - Google Patents

Procédé et dispositif de traitement de modèle de réseau neuronal, procédé de traitement d'images, et terminal mobile Download PDF

Info

Publication number
WO2019223594A1
WO2019223594A1 PCT/CN2019/087177 CN2019087177W WO2019223594A1 WO 2019223594 A1 WO2019223594 A1 WO 2019223594A1 CN 2019087177 W CN2019087177 W CN 2019087177W WO 2019223594 A1 WO2019223594 A1 WO 2019223594A1
Authority
WO
WIPO (PCT)
Prior art keywords
layer
weight value
quantization
value
neural network
Prior art date
Application number
PCT/CN2019/087177
Other languages
English (en)
Chinese (zh)
Inventor
陈岩
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Publication of WO2019223594A1 publication Critical patent/WO2019223594A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • the present application relates to the field of computer applications, and in particular, to a method, a device, an image processing method, a mobile terminal, and a computer-readable storage medium for processing a neural network model.
  • Neural network is a machine learning technology that simulates the human brain to enable artificial intelligence.
  • neural network technology With the continuous development of neural network technology, more and more algorithms use deep learning neural network methods to classify and detect images.
  • the traditional neural network model has a large amount of data and a complicated model, which is suitable for a server and not suitable for a mobile terminal.
  • the embodiments of the present application provide a method, a device, an image processing method, a mobile terminal, and a computer-readable storage medium for processing a neural network model, which can be applied to a mobile terminal.
  • a neural network model processing method includes:
  • An image processing method includes:
  • each weight value of each layer of the neural network model is a corresponding quantized value obtained by performing quantization processing according to a quantization step size and an offset of each layer, and the quantization step
  • the length and offset are obtained according to the maximum weight value, the minimum weight value, and the first quantization interval of the weight value of each layer;
  • a neural network model processing device includes:
  • An acquisition module for acquiring a weight value set of each layer in the neural network model
  • a parameter determining module configured to obtain a maximum weight value and a minimum weight value in the weight value set of each layer, and obtain the weight value set of each layer according to the maximum weight value, the minimum weight value, and the first quantization interval; Quantizing step size and offset; and
  • a quantization module is configured to perform quantization processing on each weight value of the corresponding layer according to the quantization step and offset of the weight value set of each layer to obtain a quantization value corresponding to each weight value of the corresponding layer.
  • An image processing device includes:
  • An input module is configured to input an image to be processed into a neural network model, and each weight value of each layer of the neural network model is a corresponding quantized value obtained by performing quantization processing according to a quantization step size and an offset of each layer. , The quantization step size and the offset are obtained according to the maximum weight value, the minimum weight value, and the first quantization interval of the weight value of each layer; and
  • a detection module configured to perform detection processing on the image to be processed according to a quantized value corresponding to the weight value.
  • a mobile terminal includes a memory and a processor, and the computer program is stored in the memory, and when the computer program is executed by the processor, causes the processor to perform the operations of the neural network model processing method, or The operations of the image processing method are performed.
  • a computer-readable storage medium has stored thereon a computer program that, when executed by a processor, implements operations of the neural network model processing method, or implements operations of the image processing method.
  • the method and device for processing a neural network model in the embodiments of the present application, a mobile terminal, and a computer-readable storage medium obtain the corresponding quantization step size and offset according to the weight value of each layer and the first quantization interval in the neural network model.
  • the quantization step and the offset quantize the weight value, which reduces the amount of calculation and can be applied to a mobile terminal.
  • FIG. 1 is a flowchart of a neural network model processing method according to an embodiment
  • FIG. 2 is a schematic diagram of a simple structure of a neural network model in an embodiment
  • FIG. 3 is a flowchart of a method for processing a neural network model according to an embodiment
  • FIG. 4 is a flowchart of a method for processing a neural network model according to an embodiment
  • FIG. 5 is a flowchart of an image processing method according to an embodiment
  • FIG. 6 is a structural block diagram of a neural network model processing apparatus in an embodiment
  • FIG. 7 is a structural block diagram of an image processing apparatus according to an embodiment
  • FIG. 8A is a schematic diagram of an internal structure of a mobile terminal according to an embodiment
  • 8B is a schematic diagram of an internal structure of a server in an embodiment
  • FIG. 9 is a schematic diagram of an image processing circuit in one embodiment.
  • FIG. 1 is a flowchart of a neural network model processing method in an embodiment. As shown in FIG. 1, a neural network model processing method includes operations 102 to 106. among them:
  • the neural network model is a model that simulates the processing of data recognition, classification, and detection by human neurons.
  • a neural network model generally includes an input layer, a hidden layer, and an output layer; the input layer is used to receive data input; the hidden layer is used to process the received data; the output layer is used to output the final result of data processing.
  • the hidden layer can be one or more layers of neural networks.
  • the nodes of the input layer are connected to the hidden layer, and the nodes of the hidden layer are connected to the output layer. Weighted values are configured on each connection line between nodes. There are multiple weight values for each layer of neural network.
  • FIG. 2 is a schematic diagram of a simple structure of a neural network model in an embodiment.
  • the nodes of the input layer are a 1 , a 2 , a 3
  • the nodes of the hidden layer are b 1 , b 2
  • the nodes of the output layer are z 1 , z 2.
  • the weight of the first layer is
  • the weight of the second layer is g (x) is an activation function, and the activation function is a non-linear function, for example, it may be a symbol function, a smoothing function, or a ReLU function.
  • the neural network model may be a convolutional neural network model.
  • the model structure and weight values of the neural network model can be stored in a file.
  • the mobile terminal or server reads the neural network model file to obtain the model structure and the weight value set of each layer.
  • the weight value set of each layer is used to record all the weight values of each layer.
  • Operation 104 Obtain the maximum weight value and the minimum weight value in the weight value set of each layer, and obtain the quantization step size and offset of the weight value set of each layer according to the maximum weight value, the minimum weight value, and the first quantization interval.
  • the processor in the mobile terminal or the processor in the server may obtain the maximum weight value max and the minimum weight value min in the weight value set of each layer, and may read the first quantization interval.
  • the first quantization interval is determined according to the processing capability of the processor of the mobile terminal. If the processor can process 8 bits, the first quantization interval is [0,255].
  • the processor in the mobile terminal or the processor in the server can obtain the quantization step and offset of the weight value set of each layer according to the maximum weight value max, the minimum weight value min, and the maximum value of the first quantization interval of each layer .
  • the difference between the minimum weight value and the maximum weight value of each layer is divided by the maximum value of the first quantization interval to obtain the corresponding quantization step size.
  • the first quantization interval is [0,255].
  • Operation 106 Perform quantization processing on each weight value of the corresponding layer according to the quantization step and offset of the weight value set of each layer to obtain a quantization value corresponding to each weight value of the corresponding layer.
  • the processor in the mobile terminal or the processor in the server may perform quantization processing on each weight value of the corresponding layer according to the quantization step size and offset of the weight value set of each layer to obtain the quantization value corresponding to each weight value.
  • the neural network model in the above embodiment obtains the corresponding quantization step size and offset according to the weight value of each layer in the model and the first quantization interval, and quantizes the weight value according to the quantization step size and offset. For mobile terminals.
  • performing quantization processing on each weight value of the corresponding layer according to the quantization step size and offset of the weight value set of each layer to obtain the quantization value corresponding to each weight value of the corresponding layer includes: Divide each weight value of each layer by the quantization step size of the corresponding layer weight value set, and then subtract the offset of the corresponding layer weight value set to obtain the quantization value corresponding to each weight value of the corresponding layer.
  • the calculated quantization value corresponding to each weight value of the corresponding layer may be rounded to obtain an integer.
  • the first quantization interval is [0,255]
  • the weight value is [100,200]
  • the above neural network processing method further includes:
  • Operation 302 Acquire an image, input the image to the neural network model for identification, and record the output value of each activation layer of the neural network model.
  • the image may be any image.
  • the image may be selected from a network, or taken by a mobile terminal.
  • the layer where the activation function is located can be called the activation layer.
  • the output value of the activation layer refers to the output value of the activation function.
  • the image is input to the neural network model, a forward inference is performed, and the output value of each activation layer is recorded.
  • the output value of the active layer can be a pixel value.
  • Operation 304 Obtain the maximum output value and the minimum output value of the output values of each activation layer, and obtain the quantization step size and offset of the output value of each activation layer according to the maximum output value, the minimum output value, and the second quantization interval .
  • the processor in the mobile terminal or the processor in the server obtains a maximum output value max and a minimum output value min among the output values of each active layer.
  • the second quantization interval is determined according to the capability of the processor of the mobile terminal.
  • the second quantization interval and the first quantization interval may be the same or different.
  • the first quantization interval and the second quantization interval are the same. For example, if the processor can process 8 bits, the first quantization interval is [0,255].
  • the processor in the mobile terminal or the processor in the server can obtain a quantization of the output value of each activation layer according to the maximum output value max, the minimum output value min, and the maximum value of the second quantization interval among the output values of each activation layer. Step size and offset.
  • the difference between the minimum output value and the maximum output value of each active layer is divided by the maximum value of the second quantization interval to obtain the corresponding quantization step size. Divide the minimum output value of each active layer by the corresponding quantization step size to get the corresponding offset.
  • the second quantization interval is [0,255].
  • Operation 306 Record the quantized step size and offset of the output value of each active layer.
  • quantizing the output value of each activation layer in the neural network model can make the output value suitable for processing by the mobile terminal.
  • the method for processing a neural network model further includes:
  • Operation 402 Acquire multiple frames of images, and record the output value of each activation layer after each frame of images is input to the neural network model for identification.
  • each frame of the multi-frame image may be any image.
  • Each frame of images can be selected from the network, or can be taken by a mobile terminal.
  • the layer where the activation function is located can be called the activation layer.
  • the output value of the activation layer refers to the output value of the activation function.
  • the image is input to the neural network model, a forward inference is performed, and the output value of each activation layer is recorded.
  • the output value of the active layer can be a pixel value.
  • Operation 404 Obtain the maximum output value and the minimum output value of the output values of each active layer corresponding to each frame of image, and obtain the output of each active layer according to the maximum output value, the minimum output value, and the second quantization interval. The quantized step size and offset of the value.
  • the processor in the mobile terminal or the processor in the server obtains a maximum output value max and a minimum output value min among the output values of each active layer.
  • the second quantization interval is determined according to the capability of the processor of the mobile terminal.
  • the second quantization interval and the first quantization interval may be the same or different.
  • the first quantization interval and the second quantization interval are the same. For example, if the processor can process 8 bits, the first quantization interval is [0,255].
  • the processor in the mobile terminal or the processor in the server inputs each frame of the image into the neural network model in order to obtain the output value of each activation layer. According to the maximum output value max and the minimum output of the output value of each activation layer The value min and the maximum value of the second quantization interval can obtain the quantization step size and offset of the output value of each active layer.
  • the processor in the mobile terminal or the processor in the server subtracts the difference between the maximum output value of each active layer and the minimum output value, and then divides it by the maximum value of the second quantization interval to obtain the corresponding quantization step size. Divide the minimum output value of each active layer by the corresponding quantization step size to get the corresponding offset.
  • the second quantization interval is [0,255].
  • Operation 406 Determine an average maximum output value, an average minimum output value, an average quantization step size, and an average offset of each activation layer.
  • each frame of N frames of images is input to the neural network model, and each activation layer will have a set of maximum output value max, minimum output value min, quantization step delta of output value, and offset offset.
  • each active layer has N sets of maximum output value max, minimum output value min, quantization step delta of output value, and offset offset. Take the average of the maximum output value max, minimum output value min, output step quantization step delta, and offset offset of each active layer to obtain the average maximum output value, average minimum output value, Average quantization step size and average offset.
  • the neural network model processing method further includes: performing enhanced processing on the training data, and inputting the enhanced data into the neural network model for training.
  • the enhanced processing includes rotation, scaling, cropping, random brightness change, and contrast At least one of random changes.
  • rotation refers to randomly adjusting the direction of an image.
  • Zooming refers to reducing or enlarging an image.
  • Cropping refers to cropping an image.
  • Brightness refers to adjusting the brightness of the image.
  • Contrast refers to adjusting the contrast of objects in the image.
  • Rotation, scaling, cropping, brightness, and contrast adjustments are all randomly adjusted to increase the diversity of training samples, so that the training samples cover multiple situations, and can be identified accurately during subsequent testing or recognition processes. Improved the robustness of the neural network model to various input directions. If the portrait in the image is not upright, if the portrait is not upright in the sample generated by the rotation, it can be identified that the portrait is not upright in the image.
  • the method for processing a neural network model further includes: reducing a size of an input image to be processed.
  • the size of the image to be input can be reduced to a preset size. For example, a 300 * 300 image is reduced to a size of 256 * 256.
  • the amount of calculation can be reduced by reducing the size of the image to be input.
  • the method for processing a neural network model further includes: reducing a detection output type of the neural network model.
  • the output type of the detection model in the neural network model may be reduced from a first number to a second number.
  • the first quantity may be 10 categories
  • the second quantity may be 5 categories.
  • a neural network model processing method includes:
  • the processor obtains a set of weight values for each layer in the neural network model.
  • the model structure and the weight value set of each layer can be obtained by reading the file of the neural network model.
  • the processor obtains the maximum weight value and the minimum weight value in the weight value set of each layer, and obtains the quantization step size and offset of the weight value set of each layer according to the maximum weight value, the minimum weight value, and the first quantization interval.
  • the difference between the minimum weight value and the maximum weight value is subtracted, and then divided by the maximum value of the first quantization interval to obtain the quantization step size, and the minimum weight value is divided by the quantization step size to obtain the offset.
  • the processor performs quantization processing on each weight value of the corresponding layer according to the quantization step size and offset of the weight value set of each layer to obtain a quantization value corresponding to each weight value of the corresponding layer. Divide each weight value by the quantization step size, and then subtract the offset to get the corresponding quantization value.
  • the processor obtains an image, inputs the image to the neural network model for identification, records the output value of each activation layer of the neural network model, and obtains a maximum output value and a minimum output among the output values of each activation layer. Value, obtain the quantization step size and offset of the output value of each active layer according to the maximum output value, the minimum output value, and the second quantization interval, and record the quantization step size and offset of the output value of each active layer.
  • the processor obtains multiple frames of images, records the output value of each activation layer after each frame of image is input to the neural network model for recognition, and obtains the maximum output value among the output values of each activation layer corresponding to each frame of image And the minimum output value, obtaining the quantization step and offset of the output value of each active layer according to the maximum output value, the minimum output value, and the second quantization interval, and determining an average maximum output value of each active layer, Average minimum output value, average quantization step size, and average offset.
  • the processor performs enhancement processing on the training data, and inputs the enhanced data into the neural network model for training.
  • the enhancement processing includes at least one of rotation, scaling, cropping, random brightness change, and random contrast change.
  • the processor performs a reduction process on the size of the input image to be processed.
  • the processor reduces the detection output type of the neural network model.
  • An embodiment of the present application further provides an image processing method.
  • an image processing method includes operations 502 and 504.
  • Operation 502 Input an image to be processed into a neural network model, and each weight value of each layer of the neural network model is a corresponding quantized value obtained by performing quantization processing according to a quantization step size and an offset of each layer.
  • the quantization step size and offset are obtained according to the maximum weight value, the minimum weight value, and the first quantization interval of the weight value of each layer.
  • the first quantization interval is determined according to the processing capability of the processor of the mobile terminal. If the processor can process 8 bits, the first quantization interval is [0,255].
  • the mobile terminal inputs the image to be processed into a neural network model for detection processing. The difference between the minimum weight value and the maximum weight value of each layer is divided by the maximum value of the first quantization interval to obtain the corresponding quantization step size. Divide the minimum weight value of each layer by the corresponding quantization step to obtain the corresponding offset.
  • Operation 504 Perform detection processing on the image to be processed according to the quantized value corresponding to the weight value.
  • the processor in the mobile terminal or the processor in the server may perform quantization processing on each weight value of the corresponding layer according to the quantization step size and offset of the weight value set of each layer to obtain the quantization value corresponding to each weight value.
  • the image processing method in the above embodiment identifies the to-be-processed image input to the neural network model according to the quantized weight value. Since the quantization of the weight value is suitable for a mobile terminal, the image can be detected on the mobile terminal through the neural network model. .
  • the above image processing method further includes: performing a quantization process on the pixel value of the image to be processed at each activation layer of the neural network model using a corresponding quantization step size and offset of the output value.
  • the processor in the mobile terminal or the processor in the server obtains the pixel value of each pixel point of the image to be processed, divides the pixel value by the quantization step size, and then subtracts the offset to obtain the corresponding quantized value.
  • the quantized value is processed in a processor.
  • the processor obtains the summation value according to the quantization value corresponding to each pixel value plus the offset, and then multiplies the summation value by the quantization value to obtain the corresponding pixel value.
  • the image processing method further includes: reducing the size of the image to be processed.
  • the processor in the mobile terminal or the processor in the server reduces the size of the image to be processed to a predetermined image size.
  • FIG. 6 is a structural block diagram of a neural network model processing apparatus in an embodiment.
  • a neural network model processing apparatus includes an acquisition module 610, a parameter determination module 620, and a quantization module 630. among them:
  • the obtaining module 610 is configured to obtain a weight value set of each layer in the neural network model.
  • the parameter determining module 620 is configured to obtain a maximum weight value and a minimum weight value in the weight value set of each layer, and obtain the weight value set of each layer according to the maximum weight value, the minimum weight value, and the first quantization interval. Quantize step size and offset.
  • the quantization module 630 is configured to perform quantization processing on each weight value of the corresponding layer according to the quantization step and offset of the weight value set of each layer to obtain a quantization value corresponding to each weight value of the corresponding layer.
  • the quantization module 630 is further configured to divide the weight value of each layer by the quantization step size of the corresponding layer weight value set, and then subtract the offset of the corresponding layer weight value set to obtain The quantization value corresponding to each weight value of the corresponding layer.
  • the neural network model processing apparatus further includes a storage module 640.
  • the obtaining module 610 is further configured to obtain an image, input the image to the neural network model for identification, and record an output value of each activation layer of the neural network model.
  • the parameter determining module 620 is further configured to obtain a maximum output value and a minimum output value among the output values of each activation layer, and obtain a quantization step size of the output value of each activation layer according to the maximum output value, the minimum output value, and the second quantization interval. And offset.
  • the storage module 640 is configured to record a quantization step size and an offset of an output value of each active layer.
  • the obtaining module 610 is further configured to obtain multiple frames of images, and record the output values of each active layer after each frame of images is input to the neural network model for identification.
  • the parameter determination module 620 obtains the maximum output value and the minimum output value among the output values of each active layer corresponding to each frame of image, and obtains the The quantization step size and offset of the output value determine the average maximum output value, average minimum output value, average quantization step size, and average offset of each active layer.
  • the aforementioned neural network model processing apparatus further includes an enhancement module.
  • the enhancement module is used to perform enhancement processing on the training data, and input the enhanced data into the neural network model for training.
  • the enhancement processing includes at least one of rotation, scaling, cropping, random brightness change, and random contrast change.
  • the aforementioned neural network model processing apparatus further includes a reduction module.
  • the reduction module is used to reduce the size of the input image to be processed.
  • the aforementioned neural network model processing apparatus further includes a configuration module.
  • the configuration module is used to reduce the detection output type of the neural network model.
  • FIG. 7 is a structural block diagram of an image processing apparatus in an embodiment. As shown in FIG. 7, an image processing apparatus includes an input module 710 and a detection module 720. among them:
  • the input module 710 is configured to input an image to be processed into a neural network model, and each weight value of each layer of the neural network model is a corresponding quantized value obtained by performing quantization processing according to a quantization step size and an offset of each layer.
  • the quantization step and offset are obtained according to the maximum weight value, the minimum weight value, and the first quantization interval of the weight value of each layer.
  • the detection module 720 is configured to perform detection processing on the image to be processed according to a quantized value corresponding to the weight value.
  • the detection module 720 is further configured to perform a quantization process on the pixel value of the image to be processed by using a corresponding quantization step size and offset of the output value at each activation layer of the neural network model.
  • the detection module 720 is further configured to reduce the size of the image to be processed.
  • each module in the above-mentioned neural network processing device or image processing device is only for illustration. In other embodiments, the neural network processing device or image processing device may be divided into different modules as required to complete the above-mentioned neural network processing. All or part of the functions of the device or image processing device.
  • An embodiment of the present application further provides a mobile terminal.
  • the mobile terminal includes a memory and a processor.
  • a computer program is stored in the memory.
  • the processor causes the processor to perform operations of the neural network model processing method.
  • the mobile terminal includes a memory and a processor.
  • a computer program is stored in the memory, and when the computer program is executed by the processor, the processor causes the processor to perform operations of the image processing method.
  • An embodiment of the present application further provides a computer-readable storage medium.
  • a computer-readable storage medium stores a computer program thereon, and when the computer program is executed by a processor, the operation of the neural network model processing method is implemented.
  • An embodiment of the present application further provides a computer-readable storage medium.
  • a computer-readable storage medium stores a computer program thereon, and when the computer program is executed by a processor, the operations of the image processing method are implemented.
  • FIG. 8A is a schematic diagram of an internal structure of a mobile terminal according to an embodiment.
  • the mobile terminal includes a processor, a memory, and a network interface connected through a system bus.
  • the processor is used to provide computing and control capabilities to support the operation of the entire mobile terminal.
  • the memory is used to store data, programs, and the like. At least one computer program is stored on the memory, and the computer program can be executed by a processor to implement the wireless network communication method applicable to the mobile terminal provided in the embodiments of the present application.
  • the memory may include a non-volatile storage medium and an internal memory.
  • the non-volatile storage medium stores an operating system and a computer program.
  • the computer program can be executed by a processor to implement a neural network model processing method or an image processing method provided by each of the following embodiments.
  • the internal memory provides a cached operating environment for operating system computer programs in a non-volatile storage medium.
  • the network interface may be an Ethernet card or a wireless network card, and is used to communicate with an external mobile terminal.
  • the mobile terminal may be a mobile phone, a tablet computer, or a personal digital assistant or a wearable device.
  • FIG. 8B is a schematic diagram of an internal structure of a server (or a cloud, etc.) in an embodiment.
  • the server includes a processor, a nonvolatile storage medium, an internal memory, and a network interface connected through a system bus.
  • the processor is used to provide computing and control capabilities to support the operation of the entire mobile terminal.
  • the memory is used to store data, programs, and the like. At least one computer program is stored on the memory, and the computer program can be executed by a processor to implement the wireless network communication method applicable to the mobile terminal provided in the embodiments of the present application.
  • the memory may include a non-volatile storage medium and an internal memory.
  • the non-volatile storage medium stores an operating system and a computer program.
  • the computer program may be executed by a processor to implement a neural network processing method or an image processing method provided by each of the following embodiments.
  • the internal memory provides a cached operating environment for operating system computer programs in a non-volatile storage medium.
  • the network interface may be an Ethernet card or a wireless network card, and is used to communicate with an external mobile terminal.
  • the server can be implemented by an independent server or a server cluster composed of multiple servers. Those skilled in the art can understand that the structure shown in FIG. 8B is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the server to which the solution of the present application is applied.
  • the specific server may include a server. More or fewer components are shown in the figure, or some components are combined, or have different component arrangements.
  • each module in the neural network model processing apparatus or the image processing apparatus provided in the embodiments of the present application may be in the form of a computer program.
  • the computer program can be run on a mobile terminal or server.
  • the program module constituted by the computer program can be stored in a memory of a mobile terminal or a server.
  • the computer program is executed by a processor, the operations of the method described in the embodiments of the present application are implemented.
  • a computer program product containing instructions that, when run on a computer, causes the computer to execute a neural network model processing method or an image processing method.
  • An embodiment of the present application further provides a mobile terminal.
  • the above mobile terminal includes an image processing circuit, and the image processing circuit may be implemented by using hardware and / or software components, and may include various processing units that define an ISP (Image Signal Processing) pipeline.
  • FIG. 9 is a schematic diagram of an image processing circuit in one embodiment. As shown in FIG. 9, for ease of description, only aspects of the image processing technology related to the embodiments of the present application are shown.
  • the image processing circuit includes an ISP processor 940 and a control logic 950.
  • the image data captured by the imaging device 910 is first processed by the ISP processor 940, which analyzes the image data to capture image statistical information that can be used to determine and / or one or more control parameters of the imaging device 910.
  • the imaging device 910 may include a camera having one or more lenses 912 and an image sensor 914.
  • the image sensor 914 may include a color filter array (such as a Bayer filter).
  • the image sensor 914 may obtain light intensity and wavelength information captured with each imaging pixel of the image sensor 914, and provide a set of raw data that may be processed by the ISP processor 940. Image data.
  • the sensor 920 may provide parameters (such as image stabilization parameters) of the acquired image processing to the ISP processor 940 based on the interface type of the sensor 920.
  • the sensor 920 interface may use a SMIA (Standard Mobile Imaging Architecture) interface, other serial or parallel camera interfaces, or a combination of the foregoing interfaces.
  • SMIA Standard Mobile Imaging Architecture
  • the image sensor 914 may also send the original image data to the sensor 920.
  • the sensor 920 may provide the original image data to the ISP processor 940 based on the interface type of the sensor 920, or the sensor 920 stores the original image data in the image memory 930.
  • the ISP processor 940 processes the original image data pixel by pixel in a variety of formats.
  • each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and the ISP processor 940 may perform one or more image processing operations on the original image data and collect statistical information about the image data.
  • the image processing operations may be performed with the same or different bit depth accuracy.
  • the ISP processor 940 may also receive image data from the image memory 930.
  • the sensor 920 interface sends the original image data to the image memory 930, and the original image data in the image memory 930 is then provided to the ISP processor 940 for processing.
  • the image memory 930 may be a part of a memory device, a storage device, or a separate dedicated memory in a mobile terminal, and may include a DMA (Direct Memory Access) feature.
  • DMA Direct Memory Access
  • the ISP processor 940 may perform one or more image processing operations, such as time-domain filtering.
  • the processed image data may be sent to the image memory 930 for further processing before being displayed.
  • the ISP processor 940 receives processing data from the image memory 930 and performs image data processing on the processing data in the original domain and in the RGB and YCbCr color spaces.
  • the image data processed by the ISP processor 940 may be output to the display 970 for viewing by the user and / or further processed by a graphics engine or a GPU (Graphics Processing Unit).
  • the output of the ISP processor 940 can also be sent to the image memory 930, and the display 970 can read image data from the image memory 930.
  • the image memory 930 may be configured to implement one or more frame buffers.
  • the output of the ISP processor 940 may be sent to an encoder / decoder 960 to encode / decode image data.
  • the encoded image data can be saved and decompressed before being displayed on the display 970 device.
  • the encoder / decoder 960 may be implemented by a CPU or a GPU or a coprocessor.
  • the statistical data determined by the ISP processor 940 may be sent to the control logic 950 unit.
  • the statistical data may include image information of the image sensor 914 such as auto exposure, auto white balance, auto focus, flicker detection, black level compensation, and lens 912 shading correction.
  • the control logic 950 may include a processor and / or a microcontroller that executes one or more routines (such as firmware). The one or more routines may determine the control parameters of the imaging device 910 and the ISP processing according to the received statistical data. Parameters of the controller 940.
  • control parameters of the imaging device 910 may include sensor 920 control parameters (such as gain, integration time for exposure control, image stabilization parameters, etc.), camera flash control parameters, lens 912 control parameters (such as focus distance for focusing or zooming), or these A combination of parameters.
  • ISP control parameters may include gain levels and color correction matrices for automatic white balance and color adjustment (eg, during RGB processing), and lens 912 shading correction parameters.
  • the ISP processor 940 in FIG. 9 may be used to implement the above-mentioned image processing method and neural network model processing method.
  • Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory can include random access memory (RAM), which is used as external cache memory.
  • RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDR, SDRAM), enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
  • SRAM static RAM
  • DRAM dynamic RAM
  • SDRAM synchronous DRAM
  • DDR dual data rate SDRAM
  • SDRAM enhanced SDRAM
  • SLDRAM synchronous Link (Synchlink) DRAM
  • SLDRAM synchronous Link (Synchlink) DRAM
  • Rambus direct RAM
  • DRAM direct memory bus dynamic RAM
  • RDRAM memory bus dynamic RAM

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

L'invention concerne un procédé de traitement de modèle de réseau neuronal, comportant les étapes consistant à: obtenir une valeur de poids maximum et une valeur de poids minimum de chaque couche d'un ensemble de valeurs de poids dans un modèle de réseau neuronal, puis les combiner avec un premier intervalle de quantification pour obtenir une longueur de pas de quantification et un décalage de chaque couche de l'ensemble de valeurs de poids; et effectuer un traitement de quantification sur chaque valeur de poids d'une couche correspondante selon la longueur de pas de quantification et le décalage de chaque couche de l'ensemble de valeurs de poids pour obtenir une valeur quantifiée correspondant à chaque valeur de poids de la couche correspondante. Au moyen d'une quantification de valeurs de poids, le procédé est applicable à un terminal mobile.
PCT/CN2019/087177 2018-05-21 2019-05-16 Procédé et dispositif de traitement de modèle de réseau neuronal, procédé de traitement d'images, et terminal mobile WO2019223594A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810488132.1A CN108805265B (zh) 2018-05-21 2018-05-21 神经网络模型处理方法和装置、图像处理方法、移动终端
CN201810488132.1 2018-05-21

Publications (1)

Publication Number Publication Date
WO2019223594A1 true WO2019223594A1 (fr) 2019-11-28

Family

ID=64091237

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/087177 WO2019223594A1 (fr) 2018-05-21 2019-05-16 Procédé et dispositif de traitement de modèle de réseau neuronal, procédé de traitement d'images, et terminal mobile

Country Status (2)

Country Link
CN (1) CN108805265B (fr)
WO (1) WO2019223594A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113468935A (zh) * 2020-05-08 2021-10-01 上海齐感电子信息科技有限公司 人脸识别方法
CN113850374A (zh) * 2021-10-14 2021-12-28 安谋科技（中国）有限公司 神经网络模型的量化方法、电子设备及介质
CN116561050A (zh) * 2023-04-07 2023-08-08 清华大学 面向rram存算一体芯片的细粒度映射方法及装置

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108805265B (zh) * 2018-05-21 2021-03-30 Oppo广东移动通信有限公司 神经网络模型处理方法和装置、图像处理方法、移动终端
CN109862208B (zh) * 2019-03-19 2021-07-02 深圳市商汤科技有限公司 视频处理方法、装置、计算机存储介质以及终端设备
US11166022B2 (en) * 2019-06-04 2021-11-02 Google Llc Quantization constrained neural image coding
CN110309911B (zh) * 2019-07-05 2021-01-05 安徽寒武纪信息科技有限公司 神经网络模型验证方法、装置、计算机设备和存储介质
CN111368972B (zh) * 2020-02-21 2023-11-10 华为技术有限公司 一种卷积层量化方法及其装置
CN112149797B (zh) * 2020-08-18 2023-01-03 Oppo（重庆）智能科技有限公司 神经网络结构优化方法和装置、电子设备
TWI762055B (zh) * 2020-11-30 2022-04-21 鴻海精密工業股份有限公司 卷積神經網路、運算優化方法、裝置、電子設備及介質

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107256422A (zh) * 2017-06-06 2017-10-17 上海兆芯集成电路有限公司 数据量化方法及装置
CN107403221A (zh) * 2016-05-03 2017-11-28 想象技术有限公司 卷积神经网络的硬件实现
CN107636697A (zh) * 2015-05-08 2018-01-26 高通股份有限公司 基于浮点神经网络量化的定点神经网络
CN108805265A (zh) * 2018-05-21 2018-11-13 Oppo广东移动通信有限公司 神经网络模型处理方法和装置、图像处理方法、移动终端

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0435163B1 (fr) * 1989-12-25 1998-03-25 Mitsubishi Denki Kabushiki Kaisha Appareil de codage
CN107657316B (zh) * 2016-08-12 2020-04-07 北京深鉴智能科技有限公司 通用处理器与神经网络处理器的协同系统设计
CN107644254A (zh) * 2017-09-09 2018-01-30 复旦大学 一种卷积神经网络权重参数量化训练方法及系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107636697A (zh) * 2015-05-08 2018-01-26 高通股份有限公司 基于浮点神经网络量化的定点神经网络
CN107403221A (zh) * 2016-05-03 2017-11-28 想象技术有限公司 卷积神经网络的硬件实现
CN107256422A (zh) * 2017-06-06 2017-10-17 上海兆芯集成电路有限公司 数据量化方法及装置
CN108805265A (zh) * 2018-05-21 2018-11-13 Oppo广东移动通信有限公司 神经网络模型处理方法和装置、图像处理方法、移动终端

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113468935A (zh) * 2020-05-08 2021-10-01 上海齐感电子信息科技有限公司 人脸识别方法
CN113468935B (zh) * 2020-05-08 2024-04-02 上海齐感电子信息科技有限公司 人脸识别方法
CN113850374A (zh) * 2021-10-14 2021-12-28 安谋科技（中国）有限公司 神经网络模型的量化方法、电子设备及介质
CN116561050A (zh) * 2023-04-07 2023-08-08 清华大学 面向rram存算一体芯片的细粒度映射方法及装置

Also Published As

Publication number Publication date
CN108805265B (zh) 2021-03-30
CN108805265A (zh) 2018-11-13

Similar Documents

Publication Publication Date Title
WO2019223594A1 (fr) Procédé et dispositif de traitement de modèle de réseau neuronal, procédé de traitement d'images, et terminal mobile
WO2019233393A1 (fr) Procédé et appareil de traitement d'image, support de stockage et dispositif électronique
WO2019233394A1 (fr) Procédé et appareil de traitement d'image, support de stockage et dispositif électronique
CN110428366B (zh) 图像处理方法和装置、电子设备、计算机可读存储介质
EP3477931B1 (fr) Procédé et dispositif de traitement d'images, support d'informations lisible et dispositif électronique
US10997696B2 (en) Image processing method, apparatus and device
JP7027537B2 (ja) 画像処理方法および装置、電子機器、ならびにコンピュータ可読記憶媒体
WO2021022983A1 (fr) Appareil et procédé de traitement d'images, dispositif électronique et support d'enregistrement lisible par ordinateur
WO2020001197A1 (fr) Procédé de traitement d'images, dispositif électronique et support de stockage lisible par ordinateur
CN107886484B (zh) 美颜方法、装置、计算机可读存储介质和电子设备
CN110536068B (zh) 对焦方法和装置、电子设备、计算机可读存储介质
EP3480784B1 (fr) Procédé et dispositif de traitement d'images
US11127117B2 (en) Information processing method, information processing apparatus, and recording medium
CN111028137B (zh) 图像处理方法、装置、电子设备和计算机可读存储介质
CN108717530B (zh) 图像处理方法、装置、计算机可读存储介质和电子设备
CN108804658B (zh) 图像处理方法和装置、存储介质、电子设备
CN107993209B (zh) 图像处理方法、装置、计算机可读存储介质和电子设备
CN107395991B (zh) 图像合成方法、装置、计算机可读存储介质和计算机设备
WO2019037739A1 (fr) Procédé d'acquisition de paramètre de traitement d'image, support de stockage lisible et dispositif informatique
WO2019223513A1 (fr) Procédé de reconnaissance d'image, dispositif électronique et support de stockage
CN108769523B (zh) 图像处理方法和装置、电子设备、计算机可读存储介质
CN108093182A (zh) 图像处理方法和装置、电子设备、计算机可读存储介质
CN110956679B (zh) 图像处理方法和装置、电子设备、计算机可读存储介质
CN109559352B (zh) 摄像头标定方法、装置、电子设备和计算机可读存储介质
WO2020034776A1 (fr) Procédé et appareil de traitement d'image, dispositif terminal, et support d'informations lisible par ordinateur

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19806482

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19806482

Country of ref document: EP

Kind code of ref document: A1