CN110874813A - Image processing method, device and equipment and readable storage medium - Google Patents
Image processing method, device and equipment and readable storage medium Download PDFInfo
- Publication number
- CN110874813A CN110874813A CN202010048762.4A CN202010048762A CN110874813A CN 110874813 A CN110874813 A CN 110874813A CN 202010048762 A CN202010048762 A CN 202010048762A CN 110874813 A CN110874813 A CN 110874813A
- Authority
- CN
- China
- Prior art keywords
- weight parameter
- output
- layer
- image processing
- output characteristic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/60—Memory management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
Abstract
The invention discloses an image processing method, which comprises the following steps: acquiring image input characteristics of an associated layer, and simultaneously acquiring a first weight parameter and a second weight parameter from an external memory, wherein the first weight parameter corresponds to the associated layer, and the second weight parameter corresponds to an independent layer; calculating a first output feature using the image input feature and the first weight parameter; calculating a second output characteristic corresponding to the first output characteristic by using the first output characteristic and the second weight parameter, and storing the second output characteristic into an external memory; the method adopts a multi-layer continuous calculation mode, and calculates a plurality of network layers for each channel during each calculation, thereby reducing the data exchange times of the memory and the external memory, reducing the time consumption of data transmission and further improving the processing speed of image processing; in addition, the invention also provides an image processing device, an image processing device and a computer readable storage medium, which also have the beneficial effects.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an image processing method, an image processing apparatus, an image processing device, and a computer-readable storage medium.
Background
The Digital Signal Processor (DSP) is helpful to solve the problem of slow inference speed of the neural network model in the embedded device. The digital signal processor is a processor composed of large-scale or ultra-large scale integrated circuit chips and used for completing certain signal processing tasks, and adopts a Harvard structure with separated programs and data, and the design allows the next instruction to be read in advance when the current instruction is executed, so that the image processing has higher execution efficiency.
However, the internal storage space of the dsp is usually very small (e.g. only 116 KB), and when the image input features and weights of the first layer network occupy the memory of the dsp, the remaining memory space often cannot put down the values of all output channels of the first layer. Therefore, the output value of each channel needs to be saved to the external storage after the first layer network finishes calculating the output value of each channel, until all the output channels of the first layer network finish calculating, namely, each layer network is calculated in sequence. Therefore, when the digital signal processor with limited internal storage space is used for image processing, frequent memory and external memory exchange is brought by sequentially executing the network layer, and the problem of serious time consumption of data transmission is caused.
Therefore, how to solve the problems of serious data transmission time consumption and low operation speed when a digital signal processor with small internal storage space is used for image processing is a technical problem to be solved by the technical personnel in the field.
Disclosure of Invention
In view of the above, the present invention provides an image processing method, an image processing apparatus, an image processing device, and a computer readable storage medium, which solve the problems of a digital signal processor with a small internal storage space that the data transmission consumes a lot of time and the operation speed is low when the image processing is performed.
In order to solve the above technical problem, the present invention provides an image processing method, including:
acquiring image input characteristics of an associated layer, and simultaneously acquiring a first weight parameter and a second weight parameter from an external memory, wherein the first weight parameter corresponds to the associated layer, and the second weight parameter corresponds to an independent layer;
calculating a first output feature using the image input feature and the first weight parameter;
calculating a second output characteristic corresponding to the first output characteristic by using the first output characteristic and the second weight parameter, and storing the second output characteristic into the external memory;
wherein the process of obtaining the second weight parameter from the external memory includes:
obtaining a residual space value of an internal memory and a volume value of each second weight parameter;
determining a current second weight parameter according to a preset calculation sequence, and updating the residual space value by using a current volume value corresponding to the current second weight parameter;
judging whether the residual space numerical value is larger than a preset threshold value or not;
and if so, acquiring the current second weight parameter from the external memory and storing the current second weight parameter into the internal memory.
Optionally, when the number of the second weighting parameters is greater than one, the calculating a second output characteristic corresponding to the first output characteristic by using the first output characteristic and the second weighting parameters includes:
determining a preset calculation sequence;
and calculating the first output characteristics according to the preset calculation sequence by using the second weight parameters to obtain the second output characteristics.
Optionally, the calculating a first output feature using the image input feature and the first weight parameter includes:
determining target image input features in the image input features according to a preset convolution sequence;
and performing convolution calculation by using the target image input characteristic and the first weight parameter to obtain the first output characteristic.
Optionally, the storing the second output characteristic to the external memory includes:
quantizing the second output characteristic to obtain a second character characteristic;
saving the second character characteristic to the external memory.
Optionally, after the calculating a first output feature by using the image input feature and the first weight parameter, before the calculating a second output feature corresponding to the first output feature by using the first output feature and the second weight parameter, the method further includes:
and carrying out inverse quantization processing on the first output characteristic.
Optionally, the independent layers include linear rectification layers with parameters, batch normalization layers, depth convolution layers, pooling layers.
The present invention also provides an image processing apparatus comprising:
the acquisition module is used for acquiring the image input characteristics of the associated layer and simultaneously acquiring a first weight parameter and a second weight parameter from an external memory, wherein the first weight parameter corresponds to the associated layer, and the second weight parameter corresponds to the independent layer;
a calculation module for calculating a first output feature using the image input feature and the first weight parameter;
the output module is used for calculating a second output characteristic corresponding to the first output characteristic by using the first output characteristic and the second weight parameter and storing the second output characteristic to the external memory;
wherein, the obtaining module includes:
a volume value acquiring unit for acquiring the remaining space value of the internal memory and the volume value of each second weight parameter;
the updating unit is used for determining a current second weight parameter according to a preset calculating sequence and updating the remaining space numerical value by using a current volume numerical value corresponding to the current second weight parameter;
the judging unit is used for judging whether the residual space value is larger than a preset threshold value or not;
and the first storage unit is used for acquiring the current second weight parameter from the external memory and storing the current second weight parameter into the internal memory if the residual space value is larger than the preset threshold.
The present invention also provides an image processing apparatus comprising a memory and a processor, wherein:
the memory is used for storing a computer program;
the processor is configured to execute the computer program to implement the image processing method.
The invention also provides a computer-readable storage medium for storing a computer program, wherein the computer program realizes the image processing method described above when executed by a processor.
The image processing method provided by the invention obtains the image input characteristics of the associated layer, and simultaneously obtains a first weight parameter and a second weight parameter from an external memory, wherein the first weight parameter corresponds to the associated layer, and the second weight parameter corresponds to the independent layer. A first output feature is calculated using the image input feature and the first weight parameter. And calculating a second output characteristic corresponding to the first output characteristic by using the first output characteristic and the second weight parameter, and storing the second output characteristic into the external memory. Wherein the process of obtaining the second weight parameter from the external memory includes: and acquiring the residual space value of the internal memory and the volume value of each second weight parameter. And determining a current second weight parameter according to a preset calculation sequence, and updating the residual space numerical value by using a current volume numerical value corresponding to the current second weight parameter. And judging whether the residual space numerical value is larger than a preset threshold value or not. And if so, acquiring the current second weight parameter from the external memory and storing the current second weight parameter into the internal memory.
Therefore, the method not only obtains the first weight parameter when calculating the associated layer, but also obtains the second weight parameter according to the remaining space value of the internal memory and the volume value of the second weight. Because the second weight parameter corresponds to the independent layer, when the first output feature corresponding to a certain channel is calculated by using the first weight parameter and the image input feature, the second output feature corresponding to the certain channel can be calculated by using the second weight parameter and the first output feature, that is, the subsequent independent layer calculation is performed on the first output feature, so that the continuous multilayer calculation on the channel is realized, and the obtained second output feature is stored in the external memory. The method adopts a multi-layer continuous calculation mode, and calculates a plurality of network layers during each calculation, thereby reducing the data exchange between the internal memory and the external memory, reducing the time consumption of data transmission, further improving the processing speed of image processing, and solving the problems of serious time consumption of data transmission and lower operation speed when a digital signal processor is used for image processing.
In addition, the invention also provides an image processing device, an image processing device and a computer readable storage medium, which also have the beneficial effects.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a flowchart of an image processing method according to an embodiment of the present invention;
fig. 2 is a flowchart of a second method for obtaining weight parameters according to an embodiment of the present invention;
FIG. 3 is a flowchart of a second output characteristic storage method according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a bottleneck structure according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a flowchart illustrating an image processing method according to an embodiment of the invention. The method comprises the following steps:
s101: the image input characteristics of the associated layer are acquired, and a first weight parameter and a second weight parameter are acquired simultaneously from an external memory, the first weight parameter corresponding to the associated layer, and the second weight parameter corresponding to the independent layer.
In the prior art, since the internal storage space of the digital signal processor is small, after the image input features and the first weight parameters corresponding to the association layer are stored in the internal storage space, all the first output features corresponding to the association layer cannot be stored in the internal storage space, and therefore, only all the first output features can be stored in the external memory for subsequent calculation. This results in frequent data reading and storage, increasing the time consumed for data transmission, and thus increasing the overall computation time of the convolutional neural network.
The image processing method provided by the invention is executed by a digital signal processor. It should be noted that, in the embodiment of the present invention, the association layer is a network layer in which the computation of the output features is related to more than one input features, that is, a network layer associated with the computation of each channel, such as a common convolutional layer; the independent layer is a network layer in which the calculation of the output features is related to only one corresponding input feature, namely, a network layer in which the calculation of each channel is independent from each other, such as a pooling layer.
Further, since each output channel in the depth convolution layer is associated with only one input channel, i.e. the channels of the depth convolution layer are independent of each other, the depth convolution layer can be determined to be an independent layer.
When it is determined that the calculation is to be performed for the associated layer, for example, when an instruction to perform calculation for a certain convolution layer is detected, or when it is determined that the calculation is to be performed for a certain associated layer according to a stored preset calculation order, the image input feature of the associated layer is acquired. The present embodiment does not limit the specific form and number of the image input features, and may be, for example, a vector form, and the number may be 16. The image input features may be retrieved from an external memory; or the image input characteristics sent by the preset port can be obtained through the preset port. After the image input feature is obtained, the first weight parameter and the second weight parameter are obtained from the external memory at the same time. It should be noted that the first weight parameter is a group, the second weight parameter may be one or more groups, and the specific content in each group of weight parameters is not limited in this embodiment. The first weight parameter corresponds to the association layer and is used for calculating the output characteristics of the association layer; and the independent layer corresponding to the second weight parameter is used for calculating the output characteristics of the independent layer behind the associated layer. The specific obtaining process of the weight parameter is not limited in this embodiment, and for example, the first weight parameter and the second weight parameter may be obtained by reading a configuration file; or a corresponding table of calculation order and weight parameter may be stored in an external memory, and the corresponding weight parameter may be obtained by determining the corresponding calculation position and calculation order.
It should be noted that the specific number of the second weight parameters may be determined according to actual situations, for example, the specific number may be determined according to the space size of the internal memory, the size of each weight parameter, and the size of the image input feature, and the specific determination method is not limited in this embodiment. Because the second weight parameter is obtained, subsequent independent layers can be continuously calculated after the calculation of the associated layer is carried out, and the times of data writing and reading are reduced.
Further, in order to increase the image processing speed, the second weight parameters are acquired from the external memory as much as possible, and in this embodiment, it is preferable that the acquired second weight parameters are determined by using the remaining space value of the internal memory and the volume value of each second weight. Referring to fig. 2, fig. 2 is a flowchart of a second method for obtaining weight parameters according to an embodiment of the present invention, including:
s201: the remaining space value of the internal memory and the volume value of each second weight parameter are obtained.
The internal memory is an internal storage space, and is used for storing the first weight parameter, the second weight parameter and the image input feature required by the calculation, and also used for storing partial calculation results, such as intermediate features generated in the calculation process when a plurality of second weight parameters are provided. The remaining space value of the internal memory and the volume value of each second weight parameter may be acquired before acquiring the second weight parameter. It should be noted that, the image input features and the first weight parameters of the associated layer are already stored in the internal memory, and therefore how many second weight parameters can be obtained is determined by the remaining space value and the volume size of the second weight parameters. The volume value of the second weight parameter may be its specific number of characters, or may be the size of the storage space occupied by it.
S202: and determining a current second weight parameter according to a preset calculation sequence, and updating the remaining space value by using a current volume value corresponding to the current second weight parameter.
The preset calculation sequence is used for determining the calculation sequence of the network layer, and because the first weight parameter is already determined, the current second weight parameter can be determined according to the preset calculation sequence, and the current second weight parameter is the weight parameter corresponding to the first independent layer behind the associated layer. And after the current second weight parameter is determined, updating the residual space data by using the current volume value corresponding to the current second weight parameter. Specifically, the residual space value is subtracted from the current volume value corresponding to the current second weight parameter, so that the residual space value can be updated.
S203: and judging whether the residual space value is larger than a preset threshold value or not.
The preset threshold is used to represent a minimum remaining space value in the internal memory, and the specific size of the preset threshold can be set according to practical situations, for example, the specific size can be set as the size of the second output characteristic. And after the residual space value is updated, judging whether the residual space value is larger than a preset threshold value. If the remaining space value is not greater than the preset threshold, it means that if the current second weight parameter is stored in the internal memory, the remaining space of the internal memory is too small to complete the normal operation, so the process may proceed to step S205, i.e., no operation is performed.
S204: and acquiring the current second weight parameter from the external memory and storing the current second weight parameter into the internal memory.
When the remaining space value is greater than the preset threshold, it indicates that the internal memory still has enough remaining space to complete normal operation after the current second weight parameter is stored in the internal memory, so that the current second weight parameter is obtained from the external memory and stored in the internal memory. And after the current second weight parameter is stored in the internal memory, the current second weight parameter can be updated, and whether the updated current second weight parameter can be stored in the internal memory is judged.
By applying the second weight parameter obtaining method provided by the embodiment of the invention, the second weight parameters capable of being placed in the internal memory can be determined according to the size of the remaining space value of the internal memory and the volume value of each second weight parameter, and the second weight parameters are placed in the internal memory as much as possible, so that the network layers as many as possible can be calculated in one calculation, the exchange times of data between the internal memory and the external memory are reduced, the time consumed by data transmission is reduced, and the processing speed of image processing is further improved.
S102: a first output feature is calculated using the image input feature and the first weight parameter.
After the image input feature and the first weight parameter are obtained, the first output feature is calculated by using the image input feature and the first weight parameter, the first output feature is the output feature of the association layer, and the embodiment of which channel is the output feature is not limited. The calculation method of the first output feature is related to the specific type of the associated layer, for example, when the associated layer is a common convolutional layer, the calculation method of the first output feature is convolution. The calculation of the first output feature requires a plurality of image input features, and the embodiment is not limited to which image input features are specifically required.
Further, when the associated layer is a convolution layer, in order to ensure that the order of the first output features is correct, the convolution order may be preset. When the first output feature is calculated, the target image input feature may be determined in each image input feature according to a preset convolution sequence, and convolution calculation is performed by using the target image input feature and the first weight parameter to obtain the first output feature. The first output characteristic is calculated according to the preset convolution sequence, and the calculation sequence of the first output characteristic can be determined so as to determine the corresponding input position of the first output characteristic obtained by current calculation on the independent layer, thereby ensuring the accuracy of subsequent calculation.
S103: and calculating a second output characteristic corresponding to the first output characteristic by using the first output characteristic and the second weight parameter, and storing the second output characteristic into an external memory.
And after the first output characteristic is obtained, calculating a corresponding second output characteristic by using the first output characteristic and the second weight parameter. It should be noted that the second output feature is an output feature obtained by calculating all the second weight parameters, that is, an output feature corresponding to the first output feature after calculation in one or more independent layers, and in a specific calculation process, the output feature of a previous independent layer is used as the input feature of a next independent layer.
Further, it should be noted that, when the number of the second weight parameters is greater than one, in order to prevent a calculation error between the independent layers, a preset calculation order may be determined, and the first image output feature is calculated according to the preset calculation order by using each second weight parameter, so as to obtain the second output feature. For example, the predetermined calculation order may be layer a-layer B-layer C-layer D-layer E, where layer a corresponds to the first weighting parameter a1, layer B corresponds to the second weighting parameter B1, layer C corresponds to the second weighting parameter C1, layer D corresponds to the second weighting parameter D1, and layer E corresponds to the second weighting parameter E1. And obtaining a first output characteristic after the calculation of the layer A is finished, determining that the layer B needs to be calculated according to a preset calculation sequence, and calculating by using the second weight parameter B1 and the first output characteristic. And obtaining a first intermediate feature after the B layer is calculated, determining that the C layer needs to be calculated according to a preset calculation sequence, and calculating by using the second weight parameter C1 and the first intermediate feature. And obtaining a second intermediate feature after the C layer is calculated, determining that the D layer needs to be calculated according to a preset calculation sequence, and calculating by using a second weight parameter D1 and the second intermediate feature. And obtaining a third intermediate characteristic after the D layer is calculated, determining that the E layer needs to be calculated according to a preset calculation sequence, and calculating by using the second weight parameter E1 and the third intermediate characteristic to obtain a second output characteristic.
After the second output characteristic is obtained, the second output characteristic is output to an external memory for storage so as to carry out subsequent calculation, namely the second output characteristic is used as the image input characteristic of other related layers; when the second output characteristic is an image processing result or an image recognition result, the second output characteristic may be output through the target port after the second output characteristic is stored in the external memory.
By applying the image processing method provided by the embodiment of the invention, not only the first weight parameter is obtained when the associated layer is calculated, but also the second weight parameter is obtained, and as the second weight parameter corresponds to the independent layer, when the first output characteristic corresponding to a certain channel is calculated by using the first weight parameter and the image input characteristic, the second output characteristic corresponding to the certain channel can be calculated by using the second weight parameter and the first output characteristic, namely, the subsequent independent layer calculation is carried out on the first output characteristic, so that the continuous multi-layer calculation of the channel is realized, and the obtained second output characteristic is stored in the external memory. The method adopts a multi-layer continuous calculation mode, and calculates a plurality of network layers during each calculation, thereby reducing the data exchange times of the internal memory and the external memory, reducing the time consumption of data transmission, further improving the processing speed of image processing, and solving the problems of serious data transmission time consumption and lower operation speed when a digital signal processor with small internal storage space is used for image processing.
Based on the above-described embodiment of the invention, a specific second output characteristic storage method will be described in the embodiment of the invention. Referring to fig. 3, fig. 3 is a flowchart of a second output characteristic storing method according to an embodiment of the present invention, including:
s301: and carrying out quantization processing on the second output characteristic to obtain a second character characteristic.
The quantization process may also be referred to as a quantization operation, by which floating point type data may be converted into character type data, for example 32 bit floating point type data may be converted into 8 bit character type data. Because the processing speed of the data of the character type is faster than the processing speed and the storage speed of the data of the floating point type, the second output characteristic is quantized to obtain the second character characteristic, so that the storage speed of the second output characteristic is accelerated.
S302: and saving the second character characteristic to an external memory.
After the second character feature is obtained, the second character feature is stored in an external memory, and it should be noted that because the calculation of the convolution layer is the most time-consuming, quantization operation is performed before convolution operation, so that the convolution operation is changed from floating point type operation to character type operation, and the effect of accelerating the convolution operation is realized by using a digital signal processor to operate the characteristic that character type data is faster than floating point type data. In order to reduce the influence of quantization operation on the image processing result, it is necessary to perform inverse quantization operation after the convolution is finished to ensure the accuracy of the final image processing result.
Specifically, when the image input feature is character type data with 8 bits, the first output feature obtained through convolution layer calculation is integer type data with 32 bits, and before subsequent calculation is performed by using the first output feature, for example, before calculation of a linear rectification layer or a batch normalization layer with parameters is performed, inverse quantization processing may be performed on the first output feature, that is, the first output feature is modified into a floating point type, so as to ensure that a subsequent network layer calculates a correct output feature.
By applying the image processing method provided by the embodiment of the invention, the second output characteristic is subjected to quantization processing, and the second output characteristic is converted into the second character characteristic for storage, so that the data reading and writing speed is increased, the time consumption of data transmission is reduced, and the processing speed of image processing is further increased.
Based on the above embodiments of the present invention, a method for processing a bottleneck structure will be specifically described in the embodiments of the present invention. Referring to fig. 4, fig. 4 is a schematic structural diagram of a bottleneck structure according to an embodiment of the present invention. It can be seen that the bottleneck structure is a convolution-deep convolution-convolution structure. Other network layers, such as pooling layers, may also be present in the convolutional layers and the deep convolutional layers. Since the amount of computation is large for convolutional layers and depth convolutional layers, a digital signal processor is generally called to perform the computation when performing the computation on the convolutional layers or the depth convolutional layers. Taking the numbers of output channels of three convolutions of the bottleneck structure as positive integers c1, c2 and c3, respectively, as an example, when the conventional method is applied to a digital signal processor with a small internal storage space, the output channel of each convolution layer needs to be timely saved to an external memory, that is, c1+ c2+ c3 data writing operations are required in total. Because the depth convolutional layer is an independent layer, namely the calculation of one output channel is only related to one input channel, the image processing method provided by the invention can be used for storing the weight parameters of the depth convolutional layer as second weight parameters into the internal memory when calculating the first convolutional layer, and the first convolutional layer immediately calculates one output channel of the depth convolutional layer by using the second weight parameters every time the first convolutional layer calculates one output channel, so that the digital signal processor only needs to store the output channel of the depth convolutional layer to the external memory and does not need to store the output channel of the first convolutional layer, therefore, the calculation of the whole bottleneck structure only needs c2+ c3 times of data writing operation, the time consumption of data transmission is greatly reduced, and the processing speed of image processing is further improved.
In the following, the image processing apparatus provided by the embodiment of the present invention is described, and the image processing apparatus described below and the image processing method described above may be referred to correspondingly.
Referring to fig. 5, fig. 5 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention, including:
an obtaining module 510, configured to obtain an image input feature of an associated layer, and obtain a first weight parameter and a second weight parameter from an external memory at the same time, where the first weight parameter corresponds to the associated layer, and the second weight parameter corresponds to an independent layer;
a calculating module 520 for calculating a first output feature using the image input feature and the first weight parameter;
a storage module 530, configured to calculate a second output characteristic corresponding to the first output characteristic by using the first output characteristic and the second weight parameter, and store the second output characteristic in an external memory;
wherein, the obtaining module 510 includes:
a volume value acquiring unit for acquiring the remaining space value of the internal memory and the volume value of each second weight parameter;
the updating unit is used for determining a current second weight parameter according to a preset calculating sequence and updating the remaining space numerical value by using a current volume numerical value corresponding to the current second weight parameter;
the judging unit is used for judging whether the residual space value is larger than a preset threshold value or not;
and the first storage unit is used for acquiring the current second weight parameter from the external memory and storing the current second weight parameter into the internal memory if the residual space value is larger than the preset threshold.
Optionally, the storage module 530 includes:
an order determination unit for determining a preset calculation order;
and the first calculating unit is used for calculating the first output characteristics according to a preset calculating sequence by utilizing each second weight parameter to obtain second output characteristics.
Optionally, the calculation module 520 includes:
the determining unit is used for determining target image input characteristics in the image input characteristics according to a preset convolution sequence;
and the second calculating unit is used for performing convolution calculation by using the target image input characteristic and the first weight parameter to obtain a first output characteristic.
Optionally, the storage module 530 includes:
the quantization unit is used for performing quantization processing on the second output characteristic to obtain a second character characteristic;
and the second storage unit is used for saving the second character characteristics to the external memory.
Optionally, the method further comprises:
and the inverse quantization module is used for carrying out inverse quantization processing on the first output characteristic.
Optionally, the independent layers include parametric linear rectifier layers, batch normalization layers, depth convolution layers, and pooling layers.
The following describes an image processing apparatus provided by an embodiment of the present invention, and the image processing apparatus described below and the image processing method described above may be referred to in correspondence with each other.
Referring to fig. 6, fig. 6 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention, the image processing apparatus includes a memory and a processor, where:
a memory 610 for storing a computer program;
a processor 620 for executing the computer program to implement the image processing method described above.
In the following, the computer-readable storage medium provided by the embodiment of the present invention is introduced, and the computer-readable storage medium described below and the image processing method described above may be referred to correspondingly.
The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the image processing method described above.
The computer-readable storage medium may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant part can be referred to the method part for description.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
Finally, it should also be noted that, herein, relationships such as first and second, etc., are intended only to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
The image processing method, the image processing apparatus, the image processing device and the computer readable storage medium provided by the present invention are described in detail above, and the principle and the implementation of the present invention are explained herein by applying specific examples, and the description of the above embodiments is only used to help understanding the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (9)
1. An image processing method, comprising:
acquiring image input characteristics of an associated layer, and simultaneously acquiring a first weight parameter and a second weight parameter from an external memory, wherein the first weight parameter corresponds to the associated layer, and the second weight parameter corresponds to an independent layer;
calculating a first output feature using the image input feature and the first weight parameter;
calculating a second output characteristic corresponding to the first output characteristic by using the first output characteristic and the second weight parameter, and storing the second output characteristic into the external memory;
wherein the process of obtaining the second weight parameter from the external memory includes:
obtaining a residual space value of an internal memory and a volume value of each second weight parameter;
determining a current second weight parameter according to a preset calculation sequence, and updating the residual space value by using a current volume value corresponding to the current second weight parameter;
judging whether the residual space numerical value is larger than a preset threshold value or not;
and if so, acquiring the current second weight parameter from the external memory and storing the current second weight parameter into the internal memory.
2. The image processing method according to claim 1, wherein when the number of the second weighting parameters is greater than one, the calculating a second output feature corresponding to the first output feature using the first output feature and the second weighting parameters includes:
determining a preset calculation sequence;
and calculating the first output characteristics according to the preset calculation sequence by using the second weight parameters to obtain the second output characteristics.
3. The method of claim 1, wherein the computing the first output feature using the image input feature and the first weight parameter comprises:
determining target image input features in the image input features according to a preset convolution sequence;
and performing convolution calculation by using the target image input characteristic and the first weight parameter to obtain the first output characteristic.
4. The image processing method of claim 1, wherein the storing the second output characteristic to the external memory comprises:
quantizing the second output characteristic to obtain a second character characteristic;
saving the second character characteristic to the external memory.
5. The image processing method according to claim 4, wherein after said calculating a first output feature using said image input feature and said first weight parameter, before said calculating a second output feature corresponding to said first output feature using said first output feature and said second weight parameter, further comprises:
and carrying out inverse quantization processing on the first output characteristic.
6. The image processing method of claim 1, wherein the independent layers comprise a parametric linear rectifying layer, a batch normalization layer, a depth convolution layer, and a pooling layer.
7. An image processing apparatus characterized by comprising:
the acquisition module is used for acquiring the image input characteristics of the associated layer and simultaneously acquiring a first weight parameter and a second weight parameter from an external memory, wherein the first weight parameter corresponds to the associated layer, and the second weight parameter corresponds to the independent layer;
a calculation module for calculating a first output feature using the image input feature and the first weight parameter;
the output module is used for calculating a second output characteristic corresponding to the first output characteristic by using the first output characteristic and the second weight parameter and storing the second output characteristic to the external memory;
wherein, the obtaining module includes:
a volume value acquiring unit for acquiring the remaining space value of the internal memory and the volume value of each second weight parameter;
the updating unit is used for determining a current second weight parameter according to a preset calculating sequence and updating the remaining space numerical value by using a current volume numerical value corresponding to the current second weight parameter;
the judging unit is used for judging whether the residual space value is larger than a preset threshold value or not;
and the first storage unit is used for acquiring the current second weight parameter from the external memory and storing the current second weight parameter into the internal memory if the residual space value is larger than the preset threshold.
8. An image processing apparatus comprising a memory and a processor, wherein:
the memory is used for storing a computer program;
the processor for executing the computer program to implement the image processing method according to any one of claims 1 to 6.
9. A computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the image processing method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010048762.4A CN110874813B (en) | 2020-01-16 | 2020-01-16 | Image processing method, device and equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010048762.4A CN110874813B (en) | 2020-01-16 | 2020-01-16 | Image processing method, device and equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110874813A true CN110874813A (en) | 2020-03-10 |
| CN110874813B CN110874813B (en) | 2020-05-05 |
Family
ID=69718413
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010048762.4A Active CN110874813B (en) | 2020-01-16 | 2020-01-16 | Image processing method, device and equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110874813B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6947916B2 (en) * | 2001-12-21 | 2005-09-20 | Quicksilver Technology, Inc. | IC for universal computing with near zero programming complexity |
| CN106203619A (en) * | 2015-05-29 | 2016-12-07 | 三星电子株式会社 | Data-optimized neutral net traversal |
| CN108090496A (en) * | 2017-12-22 | 2018-05-29 | 银河水滴科技（北京）有限公司 | The method and apparatus of image procossing based on convolutional neural networks |
| CN109086879A (en) * | 2018-07-05 | 2018-12-25 | 东南大学 | A kind of implementation method of the dense Connection Neural Network based on FPGA |
| CN109409514A (en) * | 2018-11-02 | 2019-03-01 | 广州市百果园信息技术有限公司 | Fixed-point calculation method, apparatus, equipment and the storage medium of convolutional neural networks |
-
2020
- 2020-01-16 CN CN202010048762.4A patent/CN110874813B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6947916B2 (en) * | 2001-12-21 | 2005-09-20 | Quicksilver Technology, Inc. | IC for universal computing with near zero programming complexity |
| CN106203619A (en) * | 2015-05-29 | 2016-12-07 | 三星电子株式会社 | Data-optimized neutral net traversal |
| CN108090496A (en) * | 2017-12-22 | 2018-05-29 | 银河水滴科技（北京）有限公司 | The method and apparatus of image procossing based on convolutional neural networks |
| CN109086879A (en) * | 2018-07-05 | 2018-12-25 | 东南大学 | A kind of implementation method of the dense Connection Neural Network based on FPGA |
| CN109409514A (en) * | 2018-11-02 | 2019-03-01 | 广州市百果园信息技术有限公司 | Fixed-point calculation method, apparatus, equipment and the storage medium of convolutional neural networks |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110874813B (en) | 2020-05-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109840589B (en) | Method and device for operating convolutional neural network on FPGA | |
| CN107340993B (en) | Arithmetic device and method | |
| CN110008952B (en) | Target identification method and device | |
| CN110929865A (en) | Network quantification method, service processing method and related product | |
| CN111091184B (en) | Deep neural network quantification method and device, electronic equipment and medium | |
| CN110647974A (en) | Network layer operation method and device in deep neural network | |
| WO2022028577A1 (en) | Processing mode determining method, and data processing method | |
| CN109284761B (en) | Image feature extraction method, device and equipment and readable storage medium | |
| CN112825199B (en) | Collision detection method, device, equipment and storage medium | |
| CN110874813B (en) | Image processing method, device and equipment and readable storage medium | |
| CN110751400B (en) | Risk assessment method and device | |
| CN112183744A (en) | Neural network pruning method and device | |
| CN116884398A (en) | Speech recognition method, device, equipment and medium | |
| US20230058500A1 (en) | Method and machine learning system to perform quantization of neural network | |
| CN114139678A (en) | Convolutional neural network quantization method and device, electronic equipment and storage medium | |
| CN114581879A (en) | Image recognition method, image recognition device, electronic equipment and storage medium | |
| US20220207346A1 (en) | Data processing method and device used in neural network | |
| CN116090511A (en) | Preprocessing method and acceleration method, acceleration system and medium of convolutional neural network | |
| CN113408724A (en) | Model compression method and device | |
| CN112613604A (en) | Neural network quantification method and device | |
| CN111767204A (en) | Overflow risk detection method, device and equipment | |
| CN111507195A (en) | Iris segmentation neural network model training method, iris segmentation method and device | |
| CN114596498B (en) | Assignment method and system for geochemical sampling blind area and storage medium | |
| CN113535407B (en) | Optimization method, system, equipment and storage medium of server | |
| CN108875919B (en) | Data processing apparatus, data processing method, and storage medium product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20230406 Address after: 528000 No. 102, 103, 104, 105, Block E, Reserved Building, No. 12, West Lecong Avenue, Ge'an Village, Lecong Town, Shunde District, Foshan City, Guangdong Province (residence declaration) Patentee after: FOSHAN JUNLAN INTELLIGENT TECHNOLOGY CO.,LTD. Address before: 410000 58 Zhongchuang 508-2, 328 Xiaoxiang Middle Road, Yuelu District, Changsha City, Hunan Province Patentee before: HUNAN G.MAX INTELLIGENT TECHNOLOGY CO.,LTD. |