CN114004731B - Image processing method and device based on convolutional neural network and related equipment - Google Patents
Image processing method and device based on convolutional neural network and related equipment Download PDFInfo
- Publication number
- CN114004731B CN114004731B CN202111166369.6A CN202111166369A CN114004731B CN 114004731 B CN114004731 B CN 114004731B CN 202111166369 A CN202111166369 A CN 202111166369A CN 114004731 B CN114004731 B CN 114004731B
- Authority
- CN
- China
- Prior art keywords
- image data
- convolution calculation
- target image
- convolution
- calculation unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013527 convolutional neural network Methods 0.000 title claims abstract description 56
- 238000003672 processing method Methods 0.000 title claims abstract description 27
- 238000004364 calculation method Methods 0.000 claims abstract description 199
- 238000012545 processing Methods 0.000 claims abstract description 46
- 238000000034 method Methods 0.000 claims abstract description 28
- 230000015654 memory Effects 0.000 claims description 33
- 238000004590 computer program Methods 0.000 claims description 10
- 238000006243 chemical reaction Methods 0.000 claims description 9
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 238000013528 artificial neural network Methods 0.000 description 27
- 238000010586 diagram Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000010349 pulsation Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44505—Configuring for program initiating, e.g. using registry, configuration files
- G06F9/4451—User profiles; Roaming
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Neurology (AREA)
- Image Analysis (AREA)
Abstract
The application discloses an image processing method based on a convolutional neural network, which is applied to a convolutional calculation unit controller and comprises the steps of obtaining target image data; reading configuration information corresponding to the target image data; determining convolution calculation parameters of each convolution calculation unit according to the configuration information; transmitting the convolution calculation parameters to corresponding convolution calculation units so that the convolution calculation units can carry out convolution calculation on the target image data by utilizing the convolution calculation parameters; the image processing method based on the convolutional neural network can effectively reduce the resource consumption in the convolutional neural network processing process. The application also discloses an image processing device based on the convolutional neural network, a convolutional calculation unit controller and a computer readable storage medium, which have the beneficial effects.
Description
Technical Field
The application relates to the field of artificial neural networks, in particular to an image processing method based on a convolutional neural network, and further relates to an image processing device, a convolutional calculation unit controller and a computer readable storage medium based on the convolutional neural network.
Background
The convolutional neural network (CNN, convolutional Neural Network) is one of artificial neural networks and is widely applied to the fields of image classification, target recognition, behavior recognition, voice recognition, natural language processing, document classification and the like. In recent years, with the increase of computing power of computers and the development of CNN structures, recognition accuracy of CNN networks is greatly improved, but at the same time, depth of CNNs is also increased, network structures are more complex, and computing quantities are also increased, so heterogeneous computing devices such as GPUs (graphics processors, graphics Processing Unit), FPGAs (field programmable gate arrays, field Programmable Gate Array) and the like are required to accelerate CNN reasoning computation.
In the related art, the implementation of the CNN reasoning accelerator generally takes a convolution computing unit as a computing core, and has both a computing function and other logic control functions; or the convolution computing unit is mainly used for the computing function, but also bears part of the logic control function. It can be seen that the prior art generally implements part of the control logic function in the convolution calculation unit to implement control over the convolution calculation. However, since the inference accelerator generally adopts a vectorized computing manner, the control logic is relatively complex; furthermore, the inference accelerator typically consists of a systolic array of multiple convolution computation units, the cost of the logic resources in the convolution computation units being multiplied.
Therefore, how to effectively reduce the resource consumption in the convolutional neural network processing process is a problem to be solved urgently by those skilled in the art.
Disclosure of Invention
The application aims to provide an image processing method based on a convolutional neural network, which can effectively reduce the resource consumption in the process of processing the convolutional neural network; another object of the present application is to provide an image processing apparatus, a convolutional calculation unit controller, and a computer readable storage medium based on a convolutional neural network, which all have the above-mentioned advantageous effects.
In a first aspect, the present application provides an image processing method based on a convolutional neural network, applied to a convolutional calculation unit controller, including:
acquiring target image data;
reading configuration information corresponding to the target image data;
determining convolution calculation parameters of each convolution calculation unit according to the configuration information;
and transmitting the convolution calculation parameters to corresponding convolution calculation units so that the convolution calculation units can carry out convolution calculation on the target image data by utilizing the convolution calculation parameters.
Preferably, the reading the configuration information corresponding to the target image data includes:
reading configuration information corresponding to the target image data from a configuration file; the configuration file is obtained by converting a preset model file by a model conversion tool.
Preferably, the configuration file includes a weight file and a model description header file, and the determining the convolution calculation parameters of each convolution calculation unit according to the configuration information includes:
determining weight data corresponding to each convolution computing unit according to the configuration information in the weight file;
and determining control signals, weight data addresses and target image data base addresses corresponding to the convolution computing units according to the configuration information in the model description header file.
Preferably, the issuing each of the convolution calculation parameters to each of the convolution calculation units so that each of the convolution calculation units performs convolution calculation on the target image data using the convolution calculation parameters includes:
calculating and obtaining a target image data offset address corresponding to the corresponding convolution calculation unit according to the control signal;
calculating according to the target image data base address and the target image data offset address to obtain a target image data reading address;
reading the target image data according to the target image data reading address to obtain reading data;
and sending the read data and the convolution calculation parameters to corresponding convolution calculation units so that the convolution calculation units carry out convolution calculation on the read data by utilizing the convolution calculation parameters.
Preferably, the convolution calculating unit performs convolution calculation on the target image data using the convolution calculation parameters, including:
and each convolution computing unit carries out convolution computing processing on the target image data by adopting a daisy chain technology.
Preferably, the convolutional neural network-based image processing method further comprises:
after all convolution calculation units process the target image data, obtaining output image data;
and storing the output image data into a preset memory.
Preferably, the acquiring the target image data includes:
the target image data is obtained by reading by an input image reader.
In a second aspect, the present application also discloses an image processing device based on a convolutional neural network, which is applied to a convolutional calculation unit controller, and includes:
the image acquisition module is used for acquiring target image data;
the configuration reading module is used for reading configuration information corresponding to the target image data;
the parameter determining module is used for determining convolution calculation parameters of each convolution calculation unit according to the configuration information;
and the convolution calculation module is used for transmitting the convolution calculation parameters to a corresponding convolution calculation unit so that the convolution calculation unit carries out convolution calculation on the target image data by utilizing the convolution calculation parameters.
In a third aspect, the present application also discloses a convolution computing unit controller, including:
a memory for storing a computer program;
a processor for implementing the steps of any one of the convolutional neural network-based image processing methods described above when executing the computer program.
In a fourth aspect, the present application also discloses a computer readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps of any of the convolutional neural network-based image processing methods described above.
The application provides an image processing method based on a convolutional neural network, which is applied to a convolutional calculation unit controller and comprises the steps of obtaining target image data; reading configuration information corresponding to the target image data; determining convolution calculation parameters of each convolution calculation unit according to the configuration information; and transmitting the convolution calculation parameters to corresponding convolution calculation units so that the convolution calculation units can carry out convolution calculation on the target image data by utilizing the convolution calculation parameters.
Therefore, in the image processing method based on the convolutional neural network, the convolutional calculation unit controller acquires the target image data and the corresponding configuration information thereof, so that the determination of the convolutional calculation parameters of each convolutional calculation unit is realized, and finally, each parameter is issued to the corresponding convolutional calculation unit to realize the convolutional calculation processing of the target image data.
The image processing device, the convolution computing unit controller and the computer readable storage medium based on the convolution neural network provided by the application have the beneficial effects and are not repeated herein.
Drawings
In order to more clearly illustrate the technical solutions in the prior art and the embodiments of the present application, the following will briefly describe the drawings that need to be used in the description of the prior art and the embodiments of the present application. Of course, the following drawings related to embodiments of the present application are only a part of embodiments of the present application, and it will be obvious to those skilled in the art that other drawings can be obtained from the provided drawings without any inventive effort, and the obtained other drawings also fall within the scope of the present application.
FIG. 1 is a schematic flow chart of an image processing method based on convolutional neural network;
FIG. 2 is a schematic diagram of an image processing system based on convolutional neural network according to the present application;
fig. 3 is a schematic structural diagram of an image processing device based on a convolutional neural network according to the present application;
fig. 4 is a schematic structural diagram of a convolution calculating unit controller according to the present application.
Detailed Description
The core of the application is to provide an image processing method based on a convolutional neural network, which can effectively reduce the resource consumption in the convolutional neural network processing process; another core of the present application is to provide an image processing apparatus, a convolution neural network, a convolution computing unit controller, and a computer readable storage medium, which also have the above-mentioned advantageous effects.
In order to more clearly and completely describe the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application. It will be apparent that the described embodiments are only some, but not all, embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The embodiment of the application provides an image processing method based on a convolutional neural network.
Referring to fig. 1, fig. 1 is a flow chart of an image processing method based on a convolutional neural network, provided by the application, the image processing method based on the convolutional neural network can be applied to a convolutional calculation unit controller, and includes:
s101: acquiring target image data;
it should be noted that, the image processing method based on the convolutional neural network provided by the application is realized based on the convolutional calculation unit controller, the convolutional calculation unit controller is arranged on each neural network layer, and in each neural network layer, the convolutional calculation unit controller is connected with the convolutional calculation unit, wherein the convolutional calculation unit controller is used for realizing a logic control function, and the convolutional calculation unit is used for realizing a convolutional calculation function, so that the image processing of the current neural network layer is realized.
On the basis, the step aims at realizing acquisition of target image data, wherein the target image data is the image data which needs to be processed by the current neural network layer, the acquired target image data of the first layer neural network is the image data input by the convolutional neural network input layer, and the acquired target image data of each layer of neural network after the first layer of neural network is the image data output by the previous layer of neural network, and is generally a vectorization feature map.
As a preferred embodiment, the acquiring the target image data may include: the target image data is read by an input image reader.
The present preferred embodiment provides a method of acquiring target image data, i.e., acquired by reading by an input image reader. Specifically, an input image reader may be provided to be connected to the convolution computing unit controller, and target image data output from the upper layer neural network may be read by the input image reader so that the convolution computing unit controller obtains the target image data from the input image reader. Further, the convolution computing unit controller may also save the target image data to its own memory, i.e., a feature map memory, after obtaining it.
S102: reading configuration information corresponding to the target image data;
the step aims to read configuration information, namely configuration information corresponding to the target image, wherein the configuration information comprises various parameter information which corresponds to each convolution computing unit and is used for carrying out convolution computing processing on the target image data, so that each convolution computing unit can carry out convolution computing processing on the target image data according to the parameter information, and image processing is realized. The configuration information may be pre-stored in a corresponding storage space, so that the convolution computing unit controller may directly call the configuration information, and it is understood that the configuration information is set by a technician according to the computing requirement and the model requirement, and the configuration information is not unique, and the present application is not limited to this.
As a preferred embodiment, the configuration information corresponding to the read target image data may include: reading configuration information corresponding to the obtained target image data from the configuration file; the configuration file is obtained by converting a preset model file by a model conversion tool.
The preferred embodiment provides a method for acquiring configuration information, namely, acquiring the configuration information by reading a configuration file. In particular, the configuration information may be stored in a corresponding configuration file, and thus the convolution computing unit controller may obtain the corresponding configuration information by reading the configuration file. As described above, the configuration information is set by a technician according to the computing requirement and the model requirement, and the configuration files preset by the technician are model files, that is, the preset model files, on the basis of which the configuration files can be converted by using a model conversion tool to obtain the converted configuration files, so that the convolution computing unit controller can read the configuration information. The specific type of the model conversion tool does not affect the implementation of the technical scheme, and the technical personnel can selectively set the model conversion tool according to actual requirements, so that the application is not limited to the implementation.
S103: determining convolution calculation parameters of each convolution calculation unit according to the configuration information;
the step aims to acquire the convolution calculation parameters corresponding to the convolution calculation units, and after configuration information is acquired, the convolution calculation parameters corresponding to the convolution calculation units in the current neural network layer can be acquired from the configuration information, and the convolution calculation parameters are used for carrying out convolution calculation on target image data.
As a preferred embodiment, the configuration file may include a weight file and a model description header file, and determining the convolution calculation parameters of each convolution calculation unit according to the configuration information may include: determining weight data corresponding to each convolution computing unit according to the configuration information in the weight file; and determining control signals, weight data addresses and target image data base addresses corresponding to the convolution calculation units according to the configuration information in the model description header file.
The preferred embodiment provides a realization method for determining convolution calculation parameters corresponding to each convolution calculation unit from configuration information. First, the configuration file may include a weight file for storing weight data required for each convolution computing unit and a model description file for storing control signals required for each convolution computing unit, weight data addresses, and base addresses of target image data. Furthermore, after the convolution calculation unit controller obtains the two configuration files, the convolution calculation unit controller can obtain the various parameters through analyzing the configuration files. It is conceivable that each convolution calculation parameter stored in the configuration file is parameter information required by the corresponding convolution calculation unit when performing convolution calculation.
S104: and transmitting the convolution calculation parameters to the corresponding convolution calculation units so that the convolution calculation units carry out convolution calculation on the target image data by utilizing the convolution calculation parameters.
The step aims at realizing the convolution calculation of the target image data, specifically, after the convolution calculation unit controller obtains the convolution calculation parameters corresponding to each convolution calculation unit based on the configuration information, each convolution calculation parameter can be issued to the corresponding convolution calculation unit, and the convolution calculation unit carries out the convolution calculation on the target image data by utilizing the corresponding convolution calculation parameters, so that after all the convolution calculation units complete the convolution calculation, the image processing of the current neural network layer can be completed. On the basis, the output image obtained by the current neural network layer is sent to the next neural network layer to continue processing until all the neural network layers are processed, and a final output image is obtained, so that the image processing based on the convolutional neural network is completed.
As a preferred embodiment, the foregoing issuing of each convolution calculation parameter to each convolution calculation unit so that each convolution calculation unit performs convolution calculation on the target image data using the convolution calculation parameter may include: calculating according to the control signal to obtain a target image data offset address corresponding to the corresponding convolution calculation unit; calculating to obtain a target image data reading address according to the target image data base address and the target image data offset address; reading the target image data according to the target image data reading address to obtain read data; and sending the read data and the convolution calculation parameters to the corresponding convolution calculation units so that the convolution calculation units carry out convolution calculation on the read data by utilizing the convolution calculation parameters.
As described above, the convolution computing unit controller may store the target image data in its own memory, that is, the feature map memory, after obtaining the target image data, and then, when processing the target image data, it is necessary to acquire the target image data from the feature map memory. In the specific implementation process, firstly, calculating and obtaining the offset address of the target image data required by the corresponding convolution calculation unit according to the control signal, and adding and calculating the offset address and the base address of the obtained target image data to obtain the actual reading address of the target image data; further, reading image data corresponding to the actual reading address from a feature map memory to obtain reading data; and finally, sending the read data and the convolution calculation parameters to a corresponding convolution calculation unit so that the convolution calculation unit can carry out convolution calculation on the read data by utilizing the convolution calculation parameters, and realizing the convolution calculation processing of the convolution calculation unit on the target image data. On the basis, each subsequent convolution calculation unit performs the above processing, and the convolution calculation processing of the current neural network layer can be realized.
As a preferred embodiment, the above convolution calculating unit performs convolution calculation on the target image data using a convolution calculation parameter, and may include: each convolution computing unit carries out convolution computing processing on the target image data by adopting a daisy chain technology.
The preferred embodiment provides a realization method for image processing by each convolution computing unit, namely, the realization method is realized based on a daisy chain technology. Specifically, all convolution computing units in each neural network layer are connected with each other in a daisy chain mode to form a pulsation array, and a convolution computing unit controller is only connected with the first convolution computing unit, so that when image processing is carried out, each convolution computing unit sequentially carries out convolution computation until the last convolution computing unit finishes computation, and the image processing of the current neural network layer is completed.
As a preferred embodiment, the convolutional neural network-based image processing method may further include: after all convolution computing units process the target image data, obtaining output image data; the output image data is stored to a preset memory.
As described above, the convolution computing unit controller may also save the target image data to its own memory, i.e., the feature map memory, after it has been obtained. On the basis, when all convolution calculation units in the current neural network layer process the target data and obtain output image data, the output image data can be stored in a corresponding memory, namely the preset memory, so that the next neural network layer can read the output image data from the preset memory to continue image processing. Of course, the target image data and the output image data may be stored in the same memory or may be stored in different memories, which is not limited in the present application.
Therefore, in the image processing method based on the convolutional neural network, the convolutional calculation unit controller acquires the target image data and the corresponding configuration information thereof, so that the determination of the convolutional calculation parameters of each convolutional calculation unit is realized, and finally, each parameter is issued to the corresponding convolutional calculation unit to realize the convolutional calculation processing of the target image data.
Based on the above embodiments, the embodiment of the present application provides another image processing method based on a convolutional neural network.
Referring to fig. 2, fig. 2 is a schematic structural diagram of an image processing system based on a convolutional neural network according to the present application, wherein a convolutional calculation unit controller is composed of a sequence generator and a dump, and n_vector convolutional calculation units are connected together.
Before the neural network computation starts, a model conversion tool is used to convert model files under a typical framework such as Tensorflow, pytorch into weight files and model description header files for use by a sequence generator.
The memory comprises a feature map memory, wherein the feature map memory is used for storing input feature map data and output feature map data calculated by the neural network of the layer. The weight reader reads the weight data required by the N_VECTOR convolution computing units each time, so that the partial weight data can be read by the dump.
Specifically, before convolution calculation starts, the dump device stores the input picture data read by the input picture reader into the feature map memory; after the convolution calculation starts, the sequence generator calculates a control signal, a weight data address and a base address of the vectorized feature map data required by the convolution calculation unit in the present clock period based on the model description header file, and outputs the control signal, the weight data address and the base address to the dump. Further, the transfer device calculates and obtains the offset address of the vectorization feature map data required in the clock period according to the received various data, adds the base address and the offset address to obtain the final address of the vectorization feature map data, takes out the data of the corresponding address in the feature map memory, and sends the data, the control signal, the first part weight data and the weight data address required by the current calculation into the first convolution calculation unit. The dump device reads the next part of weight data pre-read by the weight reader synchronously.
Further, after the first convolution computing unit takes out the weight data and the feature map data required by the unit and stores them in the weight memory and the feature map memory of the unit, the rest weight data and the feature map data are input into the next convolution computing unit, and so on, all the convolution computing units are processed as the first convolution computing unit until the last convolution computing unit obtains the weight data and the feature map data.
Further, each convolution computing unit, after obtaining the first set of weight data and feature map data, caches it in the on-chip memory of the unit, thereby performing convolution computation using the weight data and feature map data obtained in the previous computing period at the beginning of the next computing period, and synchronously reading in the weight data and feature map data required in the next computing period through the daisy chain.
And finally, after calculation for a plurality of clock cycles, the convolution calculation units obtain output data, the output data of each convolution calculation unit is sent to other subsequent modules to be subjected to activation, pooling and other processing, and then the output characteristic diagram data calculated by the neural network of the layer is sent to a memory and stored in a characteristic diagram memory for calculation of the neural network of the next layer.
Therefore, in the image processing method based on the convolutional neural network provided by the embodiment of the application, in the image processing process based on the convolutional neural network, the convolutional calculation unit controller acquires the target image data and the corresponding configuration information thereof, so that the determination of the convolutional calculation parameters of each convolutional calculation unit is realized, and finally, each parameter is issued to the corresponding convolutional calculation unit to realize the convolutional calculation processing of the target image data.
In order to solve the above technical problems, the present application further provides an image processing apparatus based on a convolutional neural network, please refer to fig. 3, fig. 3 is a schematic structural diagram of the image processing apparatus based on a convolutional neural network, the image processing apparatus based on a convolutional neural network may be applied to a convolutional computing unit controller, and the image processing apparatus includes:
an image acquisition module 1 for acquiring target image data;
a configuration reading module 2, configured to read configuration information corresponding to the target image data;
a parameter determining module 3, configured to determine a convolution calculation parameter of each convolution calculation unit according to the configuration information;
the convolution calculating module 4 is configured to issue the convolution calculating parameters to a corresponding convolution calculating unit, so that the convolution calculating unit performs convolution calculation on the target image data using the convolution calculating parameters.
Therefore, in the image processing device based on the convolutional neural network provided by the embodiment of the application, in the image processing process based on the convolutional neural network, the convolutional calculation unit controller acquires the target image data and the corresponding configuration information thereof, so that the determination of the convolutional calculation parameters of each convolutional calculation unit is realized, and finally, each parameter is issued to the corresponding convolutional calculation unit to realize the convolutional calculation processing of the target image data.
As a preferred embodiment, the configuration reading module 2 may be specifically configured to read configuration information corresponding to the obtained target image data from a configuration file; the configuration file is obtained by converting a preset model file by a model conversion tool.
As a preferred embodiment, the configuration file may include a weight file and a model description header file, and the parameter determining module 3 may be specifically configured to determine weight data corresponding to each convolution computing unit according to configuration information in the weight file; and determining control signals, weight data addresses and target image data base addresses corresponding to the convolution calculation units according to the configuration information in the model description header file.
As a preferred embodiment, the convolution calculating module 4 may be specifically configured to calculate and obtain, according to the control signal, the target image data offset address corresponding to the corresponding convolution calculating unit; calculating to obtain a target image data reading address according to the target image data base address and the target image data offset address; reading the target image data according to the target image data reading address to obtain read data; and sending the read data and the convolution calculation parameters to the corresponding convolution calculation units so that the convolution calculation units carry out convolution calculation on the read data by utilizing the convolution calculation parameters.
As a preferred embodiment, the image processing apparatus based on the convolutional neural network may further include an image storage module for acquiring output image data after all the convolutional calculation units process the target image data; the output image data is stored to a preset memory.
As a preferred embodiment, the above-described image acquisition module 1 may be specifically used for obtaining target image data by reading by an input image reader.
For the description of the device provided by the present application, please refer to the above method embodiment, and the description of the present application is omitted herein.
In order to solve the above technical problems, the present application further provides a convolution calculating unit controller, please refer to fig. 4, fig. 4 is a schematic structural diagram of the convolution calculating unit controller provided by the present application, the convolution calculating unit controller may include:
a memory 10 for storing a computer program;
the processor 20 is configured to execute the computer program to implement any of the steps of the convolutional neural network-based image processing method described above.
For the description of the apparatus provided by the present application, please refer to the above method embodiment, and the description of the present application is omitted herein.
To solve the above-mentioned problems, the present application also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, can implement the steps of any of the above-mentioned convolutional neural network-based image processing methods.
The computer readable storage medium may include: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
For the description of the computer-readable storage medium provided by the present application, refer to the above method embodiments, and the disclosure is not repeated here.
In the description, each embodiment is described in a progressive manner, and each embodiment is mainly described by the differences from other embodiments, so that the same similar parts among the embodiments are mutually referred. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The technical scheme provided by the application is described in detail. The principles and embodiments of the present application have been described herein with reference to specific examples, the description of which is intended only to facilitate an understanding of the method of the present application and its core ideas. It should be noted that it will be apparent to those skilled in the art that the present application may be modified and practiced without departing from the spirit of the present application.
Claims (7)
1. An image processing method based on a convolutional neural network, which is applied to a convolutional calculation unit controller, the method comprising:
acquiring target image data;
reading configuration information corresponding to the target image data from a configuration file; the configuration file is obtained by converting a preset model file by a model conversion tool; the configuration file comprises a weight file and a model description header file;
determining convolution calculation parameters of each convolution calculation unit according to the configuration information;
issuing each convolution calculation parameter to a corresponding convolution calculation unit so that each convolution calculation unit carries out convolution calculation on the target image data by using each convolution calculation parameter;
wherein the determining the convolution calculation parameters of each convolution calculation unit according to the configuration information includes: determining weight data corresponding to each convolution computing unit according to the configuration information in the weight file; determining control signals, weight data addresses and target image data base addresses corresponding to the convolution computing units according to configuration information in the model description header file;
the issuing of each convolution calculation parameter to a corresponding convolution calculation unit, so that each convolution calculation unit performs convolution calculation on the target image data by using each convolution calculation parameter, includes: calculating and obtaining a target image data offset address corresponding to the corresponding convolution calculation unit according to the control signal; calculating according to the target image data base address and the target image data offset address to obtain a target image data reading address; reading the target image data according to the target image data reading address to obtain reading data; and sending the read data and the convolution calculation parameters to corresponding convolution calculation units, so that each convolution calculation unit carries out convolution calculation on the read data by using each convolution calculation parameter.
2. The method according to claim 1, wherein the convolution computing unit performs convolution computation on the target image data using the convolution computation parameters, comprising:
and each convolution computing unit carries out convolution computing processing on the target image data by adopting a daisy chain technology.
3. The method as recited in claim 1, further comprising:
after all convolution calculation units process the target image data, obtaining output image data;
and storing the output image data into a preset memory.
4. The method of claim 1, wherein the acquiring the target image data comprises:
the target image data is obtained by reading by an input image reader.
5. An image processing apparatus based on a convolutional neural network, applied to a convolutional calculation unit controller, comprising:
the image acquisition module is used for acquiring target image data;
the configuration reading module is used for reading and obtaining configuration information corresponding to the target image data from the configuration file; the configuration file is obtained by converting a preset model file by a model conversion tool; the configuration file comprises a weight file and a model description header file;
the parameter determining module is used for determining convolution calculation parameters of each convolution calculation unit according to the configuration information;
the convolution calculation module is used for transmitting each convolution calculation parameter to a corresponding convolution calculation unit so that each convolution calculation unit carries out convolution calculation on the target image data by using each convolution calculation parameter;
the parameter determining module is specifically configured to determine weight data corresponding to each convolution computing unit according to configuration information in the weight file; determining control signals, weight data addresses and target image data base addresses corresponding to the convolution computing units according to configuration information in the model description header file;
the convolution calculation module is specifically used for calculating and obtaining a target image data offset address corresponding to the corresponding convolution calculation unit according to the control signal; calculating according to the target image data base address and the target image data offset address to obtain a target image data reading address; reading the target image data according to the target image data reading address to obtain reading data; and sending the read data and the convolution calculation parameters to corresponding convolution calculation units, so that each convolution calculation unit carries out convolution calculation on the read data by using each convolution calculation parameter.
6. A convolution computing unit controller, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the convolutional neural network-based image processing method according to any one of claims 1 to 4 when executing the computer program.
7. A computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the steps of the convolutional neural network-based image processing method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111166369.6A CN114004731B (en) | 2021-09-30 | 2021-09-30 | Image processing method and device based on convolutional neural network and related equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111166369.6A CN114004731B (en) | 2021-09-30 | 2021-09-30 | Image processing method and device based on convolutional neural network and related equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114004731A CN114004731A (en) | 2022-02-01 |
| CN114004731B true CN114004731B (en) | 2023-11-07 |
Family
ID=79922240
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111166369.6A Active CN114004731B (en) | 2021-09-30 | 2021-09-30 | Image processing method and device based on convolutional neural network and related equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114004731B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106529517A (en) * | 2016-12-30 | 2017-03-22 | 北京旷视科技有限公司 | Image processing method and image processing device |
| CN108701015A (en) * | 2017-11-30 | 2018-10-23 | 深圳市大疆创新科技有限公司 | For the arithmetic unit of neural network, chip, equipment and correlation technique |
| CN108710941A (en) * | 2018-04-11 | 2018-10-26 | 杭州菲数科技有限公司 | The hard acceleration method and device of neural network model for electronic equipment |
| CN109886395A (en) * | 2019-03-06 | 2019-06-14 | 上海熠知电子科技有限公司 | A kind of method for reading data towards multi-core image processing convolutional neural networks |
| WO2019134298A1 (en) * | 2018-01-02 | 2019-07-11 | 武汉斗鱼网络科技有限公司 | Image processing method, appapatus, electronic device and storage medium |
| CN111815510A (en) * | 2020-09-11 | 2020-10-23 | 平安国际智慧城市科技股份有限公司 | Image processing method based on improved convolutional neural network model and related equipment |
| CN112580660A (en) * | 2020-11-17 | 2021-03-30 | 上海闻泰信息技术有限公司 | Image processing method, image processing device, computer equipment and readable storage medium |
-
2021
- 2021-09-30 CN CN202111166369.6A patent/CN114004731B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106529517A (en) * | 2016-12-30 | 2017-03-22 | 北京旷视科技有限公司 | Image processing method and image processing device |
| CN108701015A (en) * | 2017-11-30 | 2018-10-23 | 深圳市大疆创新科技有限公司 | For the arithmetic unit of neural network, chip, equipment and correlation technique |
| WO2019134298A1 (en) * | 2018-01-02 | 2019-07-11 | 武汉斗鱼网络科技有限公司 | Image processing method, appapatus, electronic device and storage medium |
| CN108710941A (en) * | 2018-04-11 | 2018-10-26 | 杭州菲数科技有限公司 | The hard acceleration method and device of neural network model for electronic equipment |
| CN109886395A (en) * | 2019-03-06 | 2019-06-14 | 上海熠知电子科技有限公司 | A kind of method for reading data towards multi-core image processing convolutional neural networks |
| CN111815510A (en) * | 2020-09-11 | 2020-10-23 | 平安国际智慧城市科技股份有限公司 | Image processing method based on improved convolutional neural network model and related equipment |
| CN112580660A (en) * | 2020-11-17 | 2021-03-30 | 上海闻泰信息技术有限公司 | Image processing method, image processing device, computer equipment and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114004731A (en) | 2022-02-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110390385B (en) | BNRP-based configurable parallel general convolutional neural network accelerator | |
| CN111144561B (en) | Neural network model determining method and device | |
| CN112418392A (en) | Neural network construction method and device | |
| CN111091278B (en) | Edge detection model construction method and device for mechanical equipment anomaly detection | |
| CN112163601B (en) | Image classification method, system, computer device and storage medium | |
| CN110991630A (en) | Convolutional neural network processor for edge calculation | |
| CN108171328B (en) | Neural network processor and convolution operation method executed by same | |
| CN111797992A (en) | Machine learning optimization method and device | |
| CN113505883A (en) | Neural network training method and device | |
| CN111967608A (en) | Data processing method, device, equipment and storage medium | |
| JP2022502724A (en) | Methods, equipment, and related products for processing data | |
| CN117131974A (en) | Model training method, power prediction method and equipment | |
| CN111831354A (en) | Data precision configuration method, device, chip array, equipment and medium | |
| US20200050924A1 (en) | Data Processing Method and Apparatus for Neural Network | |
| WO2022111002A1 (en) | Method and apparatus for training neural network, and computer readable storage medium | |
| CN111160049A (en) | Text translation method, device, machine translation system and storage medium | |
| CN114004731B (en) | Image processing method and device based on convolutional neural network and related equipment | |
| CN115238883A (en) | Neural network model training method, device, equipment and storage medium | |
| JP2022117866A (en) | Neural network generation apparatus, neural network computing apparatus, edge device, neural network control method, and software generation program | |
| CN108376283B (en) | Pooling device and pooling method for neural network | |
| CN112183744A (en) | Neural network pruning method and device | |
| CN116739154A (en) | Fault prediction method and related equipment thereof | |
| CN115456149B (en) | Impulse neural network accelerator learning method, device, terminal and storage medium | |
| CN109542513B (en) | Convolutional neural network instruction data storage system and method | |
| CN116227599A (en) | Inference model optimization method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |