CN110309837A - Data processing method and image processing method based on convolutional neural networks characteristic pattern - Google Patents
Data processing method and image processing method based on convolutional neural networks characteristic pattern Download PDFInfo
- Publication number
- CN110309837A CN110309837A CN201910603672.4A CN201910603672A CN110309837A CN 110309837 A CN110309837 A CN 110309837A CN 201910603672 A CN201910603672 A CN 201910603672A CN 110309837 A CN110309837 A CN 110309837A
- Authority
- CN
- China
- Prior art keywords
- characteristic pattern
- convolution
- processing method
- rule
- rearrangement
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/457—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by analysing connectivity, e.g. edge linking, connected component analysis or slices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/32—Indexing scheme for image data processing or generation, in general involving image mosaicing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The present invention relates to provide data processing method, device and the image processing method of a kind of characteristic pattern based on convolutional neural networks.Include: to obtain data to be processed, the first convolution algorithm is carried out to the data to be processed, obtains primitive character figure；In the length, width dimensions of the primitive character figure, the element of the primitive character figure is rearranged according to the first rule and Second Rule respectively；Region division is carried out to the characteristic pattern after the rearrangement, obtains N number of feature subgraph；N number of feature subgraph is spliced in channel dimension；The convolution results that convolution algorithm is reset are carried out according to spliced rearrangement tensor structure；The convolution results of the rearrangement are restored, by each element reduction in the convolution results of the rearrangement at the arrangement mode of the length and width dimension of primitive character figure, obtain resetting treated characteristic；It resets treated characteristic according to described and carries out data processing.It forms new convolutional coding structure and improves network performance.
Description
Technical field
The present invention relates to depth learning technology field, in particular to a kind of based on convolutional neural networks characteristic pattern
Data processing method, device and image processing method.
Background technique
As computer calculates the growth of power and continuously improving for data processing method, nerual network technique, especially depth
The application of study becomes research hotspot in recent years, and commercially achieves huge success.Wherein, especially to be based on depth
The computer vision and image processing algorithm of habit, it is technically the most mature, it is most wide using landing.Convolutional neural networks
The Zhuo of (convolutional neural networks, CNN) with its network structure in image real time transfer and pattern-recognition
It more shows and becomes in computer vision task and be most widely used and one of network the most successful.
When selection carries out image processing tasks using CNN, it usually needs the index of three aspects below considering: precision,
Simulation velocity and memory consumption.Above-mentioned performance indicator is directly related with model structure, and different CNN networks can be to these performances
Index is weighed.Wherein, with the reinforcement of mobile terminal processing capacity, the CNN model based on mobile terminal apply from without to
Have, and demand is growing day by day.For the application scenarios of mobile terminal, simulation velocity and memory consumption two indices are more seen
Weight.
In order to be suitable for the various computer visions and image processing application of mobile terminal, some CNN were proposed in recent years
The improved technical solution of master-plan can accelerate the time of CNN simulation run simultaneously under the premise of not losing too many precision
Memory consumption is reduced, to adapt to light-weighted application.These light-weighted technical solutions include: to separate convolution using depth
The MobileNets of technology；Use the XNOR-Net of binary system convolution (i.e. only there are two types of values for convolution kernel: -1 or 1)；Remove CNN
The part-structure of model is to reduce the Network Pruning (network beta pruning) etc. of simulation run time and memory consumption.
In particular, the ShuffleNet proposed by Sun Jian et al., uses point-by-point grouping convolution (pointwise group
Convolution it) resets (channel shuffle) technology with channel and greatly reduces calculating cost, while network model
Precision is better than Mobile Nets.However, presently relevant channel reordering technique, is the rearrangement carried out in channel dimension, it is right
Bottleneck has been had reached in terms of the raising of calculated performance, has been difficult to further increase.New thinking is needed to carry out the model knot to CNN
Structure is further improved.
Summary of the invention
The present invention is directed to solve at least some of the technical problems in related technologies, provide a kind of based on volume
The data processing method of product neural network characteristics figure comprising:
Data to be processed are obtained, the first convolution algorithm is carried out to the data to be processed, obtains primitive character figure；
In the length, width dimensions of the primitive character figure, respectively according to the first rule and Second Rule by the original
The element of beginning characteristic pattern is rearranged, the characteristic pattern after being reset；
Region division is carried out to the characteristic pattern after the rearrangement, obtains N number of feature subgraph, N is natural number；
N number of feature subgraph is spliced in channel dimension according to third rule, obtains the increased rearrangement of port number
Tensor structure；
The convolution results that convolution algorithm is reset are carried out according to the rearrangement tensor structure；
According to first rule and Second Rule and third rule, the convolution results of the rearrangement are restored,
By each element reduction in the convolution results of the rearrangement at the arrangement mode of the length and width dimension of primitive character figure, weight is obtained
Row's treated characteristic；
It resets treated characteristic according to described and carries out data processing.
In some embodiments, described in the length, width dimensions of the primitive character figure, respectively according to the first rule
The element of the primitive character figure is rearranged with Second Rule, in the characteristic pattern step after being reset, described
One rule includes: the principle in the length dimension of the primitive character figure, according to coordinate about natural number Mh congruence, by element
It is divided into Mh group, and is arranged as unit of the Mh group element.
In some embodiments, the integral multiple that the natural number Mh is 2 will be first in the length dimension of primitive character figure
Element is divided at least one odd number group and at least one even number set, and the odd number group and the even number set are staggered.
In some embodiments, described in the length, width dimensions of the primitive character figure, respectively according to the first rule
The element of the primitive character figure is rearranged with Second Rule, in the characteristic pattern step after being reset, described
Two rules include: the principle in the width dimensions of the primitive character figure, according to coordinate about natural number Mw congruence, by element
It is divided into Mw group, and is arranged as unit of the Mw group element.
In some embodiments, the integral multiple that the natural number Mw is 2 will be first in the width dimensions of primitive character figure
Element is divided at least one odd number group and at least one even number set, and the odd number group and the even number set are staggered.
In some embodiments, region division is carried out to the characteristic pattern after the rearrangement, obtaining N number of feature subgraph includes:
Characteristic pattern after rearrangement is subjected to equal area partition, obtains N number of feature subgraph of area equation.
In some embodiments, region division is carried out to the characteristic pattern after the rearrangement, obtaining N number of feature subgraph includes:
Determine that the characteristic pattern after the rearrangement carries out the mode of region division, Yi Jite according to first rule and the Second Rule
Levy the quantity N of subgraph.
In some embodiments, described that the convolution results that convolution algorithm is reset are carried out according to the rearrangement tensor structure
It include: the feature that each feature subgraph is extracted with depth separation convolution；To the N number of institute extracted through depth separation convolution
The feature for stating feature subgraph does the convolution results that the point convolution that convolution kernel is 1 is reset according to every N number of point.
In some embodiments, the feature for extracting each feature subgraph with depth separation convolution includes: to use
The depth that convolution kernel is 3*3 separates the feature that convolution extracts each feature subgraph.
In order to achieve the above object, the embodiment of first aspect present invention additionally provides one kind based on convolutional neural networks characteristic pattern
Data processing equipment comprising:
Data acquisition module carries out the first convolution algorithm to the data to be processed for obtaining data to be processed,
Obtain primitive character figure；
Characteristic pattern reordering module, for length, the width dimensions in the primitive character figure, respectively according to the first rule
The element of the primitive character figure is rearranged with Second Rule, the characteristic pattern after being reset；
Characteristic pattern division module obtains N number of feature subgraph, N for carrying out region division to the characteristic pattern after the rearrangement
For natural number；
Channel splicing module is obtained for splicing N number of feature subgraph in channel dimension according to third rule
The increased rearrangement tensor structure of port number；
Convolution algorithm module, for carrying out the convolution results that convolution algorithm is reset according to the rearrangement tensor structure；
Construction recovery module, for regular according to first rule and Second Rule and third, to the rearrangement
Convolution results are restored, by each element reduction in the convolution results of the rearrangement at the length and width of corresponding primitive character figure
Dimension puts in order, and obtains resetting treated characteristic；
Data processing module, for resetting treated characteristic according to described and carrying out data processing.
Using the data processing method or device of the invention based on convolutional neural networks characteristic pattern, by characteristic pattern
Reordering operations are carried out, new convolutional coding structure is formed and replaces traditional convolutional coding structure, and then recognition effect can be improved；And it is not improving
Network performance is improved under the premise of neural computing amount (FLOPs).In addition, most of all, not using method of the invention
It needs to rewrite convolution kernel, only simple characteristic pattern reordering operations need to be done to tensor (tensor), so that network structure of the invention
Can not only be applied in field of image recognition, and have good generalization ability, can rapidly be applied to it is other it is any
In some convolutional neural networks structures.
In order to achieve the above object, the embodiment of second aspect of the present invention provides at a kind of image based on convolutional neural networks
Reason method, in the convolutional neural networks, including at least one characteristic pattern resets convolutional layer, and the characteristic pattern resets convolutional layer packet
At least one is included for realizing the data processing method of the characteristic pattern described in first aspect present invention based on convolutional neural networks
Computing unit.
In some embodiments, the structure of the convolutional neural networks is the deep learning network of rearrangement network structure, and
The characteristic pattern processing method and the rearrangement network channel reset use in conjunction.
Image processing method according to the present invention based on convolutional neural networks not only can carry out weight in channel dimension
Row can also form new convolutional coding structure and replace traditional convolutional coding structure, and then can improve knowledge in the enterprising rearrangement operation of characteristic pattern
Other effect；And network performance is improved under the premise of not improving neural computing amount (FLOPs).In addition, most of all,
It does not need to rewrite convolution kernel using method of the invention, only need to do simple characteristic pattern reordering operations to tensor (tensor), make
Obtaining network structure of the invention can not only be applied in field of image recognition, but also have good generalization ability, Ke Yixun
Speed is applied to other any applications.
In order to achieve the above object, the embodiment of third aspect present invention provides a kind of non-transitory computer-readable storage medium
Matter is stored thereon with computer program, when the computer program is executed by processor, realizes institute according to a first aspect of the present invention
The data processing method for the characteristic pattern based on convolutional neural networks stated, or realize the base described according to a second aspect of the present invention
In the image processing method of convolutional neural networks.
In order to achieve the above object, the embodiment of fourth aspect present invention provides a kind of calculating equipment, including memory, processing
Device and storage on a memory and the computer program that can run on a processor, when the processor executes described program, reality
The now data processing method of the characteristic pattern based on convolutional neural networks according to a first aspect of the present invention, or realize basis
Based on the image processing method of convolutional neural networks described in second aspect of the present invention.
Non-transitorycomputer readable storage medium according to the present invention and calculate equipment, have and according to the present invention first
The data processing method of the characteristic pattern based on convolutional neural networks of aspect and second aspect based on convolutional neural networks
Image processing method has similar beneficial effect, and details are not described herein.
Detailed description of the invention
Fig. 1 is the operation principle schematic diagram of ShuffleNet；
Fig. 2 is the schematic diagram of tensor data structure (n, c, h, w)；
Fig. 3 is the flow diagram according to the characteristic pattern processing method of the embodiment of the present invention；
Fig. 4 is the schematic diagram reset to the grouping of characteristic pattern length and width dimension according to the embodiment of the present invention；
Fig. 5 is the schematic diagram spliced according to the channel of the embodiment of the present invention；
Fig. 6 is to reset the schematic diagram that convolution is restored to feature subgraph according to the embodiment of the present invention；
Fig. 7 is the structural block diagram according to the characteristic pattern processing unit of the embodiment of the present invention；
Fig. 8 a is that the structural schematic diagram of computing unit is reset in the channel of ShuffleNet in the related technology；
Fig. 8 b is that the meter that use in conjunction is handled with characteristic pattern is reset according to the channel of the ShuffleNet of the embodiment of the present invention
Calculate the structural schematic diagram of unit；
Fig. 9 is the structural schematic diagram according to the calculating equipment of the embodiment of the present invention.
Specific embodiment
Detailed description is according to an embodiment of the invention, description when being related to attached drawing below with reference to accompanying drawings, unless otherwise indicated,
Same reference numerals in different attached drawings indicate the same or similar element.It is noted that institute in following exemplary embodiment
The embodiment of description does not represent all of the embodiments of the present invention.They be only with it is being described in detail in such as claims,
The example of the consistent device and method of some aspects of the disclosure, the scope of the invention is not limited to this.Reconcilable
Under the premise of, the feature in each embodiment of the present invention can be combined with each other.
In addition, term " first ", " second " are used for descriptive purposes only and cannot be understood as indicating or suggesting relative importance
Or implicitly indicate the quantity of indicated technical characteristic.Define " first " as a result, the feature of " second " can be expressed or
Implicitly include at least one this feature.In the description of the present invention, the meaning of " plurality " is at least two, such as two, three
It is a etc., unless otherwise specifically defined.
Technical solution in order to better illustrate the present invention is first simply situated between to Shuffle Net (resetting network)
It continues.Main thought at the beginning of Shuffle Net is designed is that, using grouping convolution point by point, opposite 1*1 convolution reduces and calculates complexity
Degree, meanwhile, after overcoming grouping convolution, problem only related to the channel in grouping is exported, channel is used to reset skill
Art helps the information flow between feature channel.
Fig. 1 is the operation principle schematic diagram of Shuffle Net, wherein for the convolutional layer stacked by two, each convolution
Layer is grouping convolution.In subgraph 110, the model structure of ordinary groups convolution is shown, wherein two convolutional layers have phase
Same number of packet, each output channel is only related to the input channel in its group, unrelated with the input channel of other groups, leads to
Without information exchange between road, this can seriously affect the accuracy rate of study.In subgraph 120, the input of second packet convolution is shown
Each different grouping from the first grouping convolutional layer, outputting and inputting channel realizes correlation.Subgraph 130 then shows application
(channel shuffle) technology is reset to realize the mode of relative operation in channel.Channel, which is reset, pertains only to dimension rearrangement
Transposition operates (dimension shuffle), hardly needs extra computation amount, therefore not will increase total calculating of neural network
Amount (usually with floating-point operation number --- FLOPs is measured).
Currently, channel is reset (channel shuffle) technology and is more and more answered in lightweight designs a model
With the especially application for set calculation field of force scapes of mobile terminals such as image recognition personages, for example, living things feature recognition, people
In terms of the terminal applies such as face unlock.Various improved plans are also constantly proposed, but so far, various network structures change
Into being that diversity in channel dimension, by increasing neural network disparate modules promotes recognition effect mostly.
Inventors noted that reordering technique is not applied on characteristic pattern at present.It on the one hand is that industry generally believes
It is operated on characteristic pattern and is often difficult improvement effect, be on the other hand also that there is presently no efficient processing modes to characteristic pattern
It is operated.In addition, receptive field (Receptive Field) is always one of major issue of convolutional neural networks, it is last
One layer of the prediction result energy key as much as possible for embodying entire original image information, by convolution kernel size, the parameters such as step-length
Influence, receptive field suffers from many restrictions in CNN, and in grouping convolutional coding structure, the limited problem of receptive field is particularly acute.
Current channel reordering technique there is no effective means also in this regard to improve impression caused by grouping convolution and other reasons
The limited problem of wild range.
Based on above-mentioned analysis, the invention proposes a kind of data processing method of characteristic pattern based on convolutional neural networks,
The above problem present in the relevant technologies is solved to a certain extent by the rearrangement of characteristic pattern.
Referring to fig. 2, Fig. 2 is that related fields describes a kind of tensor data structure schematic diagram, institute used in CNN network structure
Stating data structure can usually be denoted as: tensor (n, c, h, w), and wherein n indicates characteristic pattern quantity, and c indicates number of channels, h table
Show characteristic pattern height, w indicates characteristic pattern width, data format NCHW.By taking the data structure of Fig. 2 as an example, the first of characteristic pattern
A element is, for example, 000, and second element is along the direction w, i.e., 001, subsequent 002,003, followed by being exactly along the side h
To, i.e., 004,005,006,007... in this way to after 019, be 020 along the direction c, later 021,022... until 319, then
Again along the direction n.
For the ease of illustrate, the embodiment of the present invention will in conjunction with four dimensional tensor tensor (n, c, h, w) description form into
Row explanation, it will be understood by those skilled in the art that the description form of CNN is not limited thereto, the scope of the present invention is not also limited to
In the data structure form of this four dimensional vector.Using Fig. 1 as the existing Shuffle Net and its distressed structure of representative, usually
In c dimension, i.e. channel dimension is reset, and the present invention is then creatively proposed in characteristic pattern dimension, i.e. height h dimension and width
The mode that degree w dimension is reset.
The embodiment of first aspect present invention provide the characteristic pattern based on convolutional neural networks data processing method and
Device, the data processing method may include the processing method of a variety of specific data objects.For example, it may be at image data
Reason method, language data processing method, voice data processing method, knowledge data processing method etc., it is any that convolution can be used
The data type that neural network is modeled and calculated can carry out intermediate data using data processing method of the invention
Processing.
It is the stream of the data processing method according to the embodiment of the present invention based on convolutional neural networks characteristic pattern referring to Fig. 3, Fig. 3
Journey schematic diagram.The data processing method based on convolutional neural networks characteristic pattern includes step S100 to S700.
Step S100 obtains data to be processed, carries out the first convolution algorithm to the data to be processed, obtains original
Characteristic pattern.
Step S200, in the length, width dimensions of the primitive character figure, respectively according to the first rule and Second Rule
The element of the primitive character figure is rearranged, the characteristic pattern after being reset.
The main purpose of rearrangement is to make non-conterminous part in primitive character figure to increase correlation after resetting, thus
Convolution can provide more information exchange approach after rearrangement for different interchannels, therefore, all rearrangement rule that can reach this purpose
Then it is all applied to this step.
For example, in some embodiments, proposing a kind of mode for calculating simplicity, first rule can be equidistant
Grouping is reset, in the length dimension of the primitive character figure, principle according to coordinate about natural number Mh congruence, by element point
For Mh group, and arranged as unit of the Mh group element.In this way, calculation amount is seldom and in each group after grouping, it is first
Plain number is essentially identical, and error is no more than 1, convenient for operations such as the channel splicings in later period.
Preferably, the natural number Mh is chosen as 2 integral multiple, in the length dimension of primitive character figure, by element point
For at least one odd number group and at least one even number set, and the odd number group and the even number set are staggered.It is easy to operate
It is convenient, and it is conducive to have the rearrangement of good symmetry to convert.
In some embodiments, the Second Rule is equally in the width dimensions of the primitive character figure, according to seat
The principle about natural number Mw congruence is marked, element is divided into Mw group, and arranged as unit of the Mw group element.
Preferably, the natural number Mw be 2 integral multiple, in the width dimensions of primitive character figure, by element be divided into
A few odd number group and at least one even number set, and the odd number group and the even number set are staggered.
Referring to fig. 4, wherein by taking h=w=8 as an example, a kind of characteristic pattern rearranged form for showing.Wherein, according to odevity
Mode is grouped (h, w) dimension of input feature vector figure: h dimension being divided into odd number group and even number set, w dimension is also classified into surprise
Array and even number set.Characteristic pattern is subjected to equal part in length and width dimension in the way of taking the remainder to 2 that is, can be considered, using Mw
The block form of=2, Mh=2.After grouping, for the h dimension of characteristic pattern, odd number group (1,3,5,7) is come into the left side, even number
Group (2,4,6,8) comes the right；Top come by odd number group (1,3,5,7) for the w dimension of characteristic pattern, even number set (2,4,6,
8) it comes following.Obviously, shown in Fig. 4 reset is the more succinct arrangement mode example of one of realization, can also be adopted
With other packet modes and other arrangement modes.
Step S300 carries out region division to the characteristic pattern after the rearrangement, obtains N number of feature subgraph, N is natural number.
The purpose for being divided into multiple subgraphs is spliced in channel dimension, and therefore, the size between each subgraph is consistent
It will be helpful to subsequent operation.So equal area partition can be carried out the characteristic pattern after rearrangement, N number of spy of area equation is obtained
Levy subgraph.For example, carrying out etc. point dividing on the direction w and h.
It is noted that so-called homalographic includes that the sub- shape of feature is identical with the area of pictural surface and error is less than predetermined
The case where threshold value, for example, every side number of elements difference be not more than 1 or specified threshold the case where.Because, it is clear that as w, h of characteristic pattern
When quantity is not easy divided evenly, rough equal part is carried out, what is obtained is the roughly equal N number of feature subgraph of area.At this point, can pass through
The subgraph of area equation is further obtained to operations such as subgraph progress edge polishings to realize subsequent channel splicing.
Preferably, can determine that the characteristic pattern after the rearrangement carries out area according to first rule and the Second Rule
The quantity N of mode and feature subgraph that domain divides.For example, with reference to Fig. 4 reordering rule, the when resetting with characteristic pattern is selected
One 2*2 equal area partition form identical with Second Rule carries out region division to the characteristic pattern after rearrangement, referring to Fig. 5, to h ×
W does 2 × 2 division, obtains 4 feature subgraphs.The size of each feature subgraph is the 1/4 of original image at this time.In the present invention
In other embodiments, the feature subgraph of other divisible quantity can also be divided into.Again by 4 feature after the completion of dividing
Figure, which is put into c dimension, to be spliced, then number of channels becomes original 4 times, i.e. 4c.
Step S400 splices N number of feature subgraph in channel dimension according to third rule, obtains port number increasing
The rearrangement tensor structure added.
It is relatively independent since subsequent convolution operation is to carry out convolution operation respectively to each feature subgraph in this step,
Therefore special requirement is had no for third rule, N number of feature subgraph can be carried out in channel dimension in any order
Splicing.
It referring to Fig. 5, is formed at this time new tensor (n, 4c, h/2, w/2), channel dimension becomes 4 times, the length of feature subgraph
Degree and width are respectively the 1/2 of primitive character figure.In the 5 embodiment of figure 5, such as with upper left, upper right, the sequence of lower-left, bottom right
4 feature subgraphs are put into c dimension.
Step S500 carries out the convolution results that convolution algorithm is reset according to the rearrangement tensor structure.This step, can
To carry out corresponding convolution algorithm according to the model structure of CNN.
In some embodiments, in order to obtain better receptive field, the invention proposes a kind of depth to separate convolution sum point
The convolution algorithm method that convolution combines.Firstly, extracting the feature of each feature subgraph with depth separation convolution；By originally not
Adjacent element can become adjacent element by common convolution now, to expand receptive field.Then to described through depth point
Feature from N number of feature subgraph that convolution is extracted is done the point convolution that convolution kernel is 1 according to every N number of corresponding element vegetarian refreshments and is obtained
The convolution results of rearrangement, and then remain small receptive field.
To the two operations be merged and be equivalent to while proposing big and small sense by extracting different receptive fields twice
By open country, the convolutional coding structure that size receptive field all retains is formd.Solve the problems, such as that receptive field is limited to a certain extent.Pass through
The information fusion for promoting size receptive field, improves the accuracy rate of neural network.
In some embodiments, for application scenarios such as common image procossings, the depth that convolution kernel is 3*3 can be used
Separation convolution extracts the feature of each feature subgraph, to obtain preferable effect.
After carrying out the depth separation convolution of 3*3 to each feature subgraph (for example, the step-length of depth separation convolution is 1,
It is filled with 1), every 4 parts are done the point convolution (point-wise convolution) that convolution kernel is 1, then every 4 parts of same positions are corresponding
4 elements be original 4 adjacent elements, remain small receptive field.For example, the step-length of default point convolution is 1, it is filled with 0.
Tensor dimension after above-mentioned process of convolution remains unchanged, and is still tensor (n, 4c, h/2, w/2).
Following discovery of the selection of above-mentioned parameter based on following inventor for building high-efficiency network framework: (1) identical
Channel width can minimize internal storage access cost (MAC), therefore use the convolution (identical channel width) of " balance "；(2) excessively
Group convolution will increase MAC, so need to consider the cost using group convolution；(3) degree of fragmentation is reduced；(4) Element-Level is reduced
Operation.Using the point convolution of the depth separation convolution sum 1*1 of 3*3, good calculating effect has been reached according to the above design principle,
Reach the optimization balance of operation cost and network effect.
Step S600, according to first rule and Second Rule and third rule, to the convolution results of the rearrangement
It is restored, by each element reduction in the convolution results of the rearrangement at the arrangement side of the length and width dimension of primitive character figure
Formula obtains resetting treated characteristic.
This step can regard the applied in reverse to first rule and Second Rule and third rule as.First root
Reversed union operation is carried out according to the division mode of third rule, the convolution results of the rearrangement are subjected to the dimension of tensor also
Original makes h and w be returned to the dimension of primitive character figure.Further according to according to it is described first rule and Second Rule h and w dimension into
Row element resets, and restores its arrangement mode.
Referring to Fig. 6, can first by above-mentioned Fig. 5, treated that 4 feature subgraphs are reduced into a characteristic pattern, then to h dimension and w
Dimension carries out being reduced into original sequence.It is restored according to feature subgraph fractionation sequence, i.e. upper left, upper right, lower-left, bottom right
Order restoring at tensor (n, c, h, w).Original sequence (1,2,3,4,5,6,7,8) is reduced into h dimension and w dimension again
?.
Step S700 resets treated characteristic and carries out data processing according to described.
It is noted that flow chart in any process described otherwise above herein or method description it is understood that
To indicate the mould for including the steps that one or more codes for realizing custom logic function or the executable instruction of process
Block, segment or part, and the range of the preferred embodiment of the present invention includes other realization, wherein can not be by shown
Or the sequence discussed, including according to related function by it is basic and meanwhile in the way of or in the opposite order, Lai Zhihang function, this
It should be understood by those skilled in the art.
Using the data processing method of the invention based on convolutional neural networks characteristic pattern, by carrying out weight on characteristic pattern
Row's operation forms new convolutional coding structure and replaces traditional convolutional coding structure, and then can improve recognition effect；And do not improving nerve net
Network performance is improved under the premise of network calculation amount (FLOPs).Additionally, it is important that not needing to rewrite using method of the invention
Convolution kernel only need to do simple characteristic pattern reordering operations, so that network structure of the invention is not only being schemed to tensor (tensor)
As identification field can be applied, and there is good generalization ability, other any existing convolution can be applied to rapidly
In neural network structure.
To, data processing method and device according to an embodiment of the present invention can be applied to various deep learning fields,
And it is not limited to image procossing.For example, CNN demonstrates the brilliance in natural language processing (NLP) task at present, and characteristic pattern
Rearrangement be beneficial to identification word order on it is non-conterminous, it is interrelated between words and phrases especially apart from each other.Characteristic pattern is answered
For NLP, it will bring beneficial promotion to the understanding of long sentence, paragraph or even chapter to neural network.
In addition, although the data processing method of the characteristic pattern of the invention based on convolutional neural networks is particularly suitable for moving
The lightweight network of terminal, it is clear that can be used for server end to reduce calculation amount.
The present invention also provides a kind of data processing equipments 200 based on convolutional neural networks characteristic pattern, referring to Fig. 7 institute
Show, the characteristic pattern processing unit 200 includes data acquisition module 210, characteristic pattern reordering module 220, characteristic pattern division module
230, channel splicing module 240, convolution algorithm module 250, construction recovery module 260 and data processing module 270.
Data acquisition module 210 carries out the first convolution fortune for obtaining data to be processed, to the data to be processed
It calculates, obtains primitive character figure.
Characteristic pattern reordering module 220 is used for length, width dimensions in the primitive character figure, respectively according to the first rule
Then the element of the primitive character figure is rearranged with Second Rule, the characteristic pattern after being reset.
In some embodiments, first rule can include: in the length dimension of the primitive character figure, according to coordinate
About the principle of natural number Mh congruence, element is divided into Mh group, and arranged as unit of the Mh group element.The nature
Element is divided at least one odd number group and at least one in the length dimension of primitive character figure by the integral multiple that number Mh can be taken as 2
A even number set, and the odd number group and the even number set are staggered.
In some embodiments, the Second Rule can include: in the width dimensions of the primitive character figure, according to coordinate
About the principle of natural number Mw congruence, element is divided into Mw group, and arranged as unit of the Mw group element.The nature
Element is divided at least one odd number group and at least one in the width dimensions of primitive character figure by the integral multiple that number Mw can be taken as 2
A even number set, and the odd number group and the even number set are staggered.
Characteristic pattern division module 230 is used to carry out region division to the characteristic pattern after the rearrangement, obtains N number of feature
Figure, N is natural number.For example, equal area partition can be carried out the characteristic pattern after rearrangement, N number of feature subgraph of area equation is obtained.
Either, it can determine that the characteristic pattern after the rearrangement carries out the side of region division according to first rule and the Second Rule
The quantity N of formula and feature subgraph.
Channel splicing module 240 is obtained for splicing N number of feature subgraph in channel dimension according to third rule
To the increased rearrangement tensor structure of port number.
Convolution algorithm module 250 is used to carry out the convolution knot that convolution algorithm is reset according to the rearrangement tensor structure
Fruit.Wherein, the feature that each feature subgraph can be first extracted with depth separation convolution, then separates convolution through depth to described
The feature of the N number of feature subgraph extracted does what the point convolution that convolution kernel is 1 was reset according to every N number of corresponding element vegetarian refreshments
Convolution results.In some embodiments, the depth that convolution kernel is 3*3 can be used to separate the spy that convolution extracts each feature subgraph
Sign.
Construction recovery module 260 is used for according to first rule and Second Rule and third rule, to the rearrangement
Convolution results restored, by each element reduction in the convolution results of the rearrangement at corresponding primitive character figure length,
Wide dimension puts in order, and obtains resetting treated characteristic.
Data processing module 270 is used to reset treated characteristic according to described and carry out data processing.
The more specific implementation of the modules of the data processing equipment may refer at data of the invention
The description of reason method, and there is similar beneficial effect, details are not described herein.
Embodiment according to a second aspect of the present invention provides a kind of image processing method based on convolutional neural networks, with
Make full use of the good action of data processing method of the invention in terms of image procossing.In the convolutional neural networks, including
At least one characteristic pattern resets convolutional layer, and it includes at least one for realizing first party of the present invention that the characteristic pattern, which resets convolutional layer,
The computing unit of the data processing method of the characteristic pattern based on convolutional neural networks of face embodiment.
Wherein, the image processing method based on convolutional neural networks can be deep learning method, including multiple volumes
Lamination.Wherein, the characteristic pattern rearrangement can be applied among one or more of hidden layers, for increasing different interchannels
Data correlation and reduce calculation amount the purpose of.
In some embodiments, the structure of the convolutional neural networks can be the deep learning net of Shuffle Net structure
Network, and use in conjunction is reset in the channel of the characteristic pattern processing method and the Shuffle Net, to obtain preferably whole effect
Fruit.
Illustrate effect of the invention below with reference to specific example.Referring to figs. 8a and 8b, Fig. 8 a is a kind of ShuffleNet
The structural schematic diagram of the characteristic pattern computing unit of V2 (referring to document 2)；Fig. 8 b is to use characteristic pattern of the invention in second channel
The structural schematic diagram of the computing unit of calculation method.Entire neural network may include the multiple computing units stacked.
The description of pertinent literature can be joined about being described in further detail for ShuffleNet, such as:
Document 1:Xiangyu Zhang etc., Shuffle Net:An Extremely Efficient
Convolutional Neural Network for Mobile Devices, arXiv:1707.01083v2；
Document 2:Ningning Ma etc., Shuffle Net V2:Practical Guidelines for Efficient
CNN Architecture Design, arXiv:1807.11164v1.Experiment shows with of the invention based on convolutional Neural net
The data processing method of the characteristic pattern of network, replace Fig. 8 a in second channel convolution algorithm after, can get higher accuracy.With
For running ImageNet data set, Top-1 error rate (the maximum mistake of original ShuffleNet V2 shown in Fig. 8 a
Rate) it is 30.5, its Top-1 error rate is 30.2 after the replacement present invention.Under the premise of not promoting calculation amount, realize
The image recognition of higher precision.Also, it is not necessary to rewrite convolution kernel, reordering operations only are done to characteristic pattern, are shown good general
Change ability.
Image processing method according to the present invention based on convolutional neural networks not only can carry out weight in channel dimension
Row can also form new convolutional coding structure and replace traditional convolutional coding structure, and then can improve knowledge in the enterprising rearrangement operation of characteristic pattern
Other effect；And network performance is improved under the premise of not improving neural computing amount (FLOPs).In addition, most of all,
It does not need to rewrite convolution kernel using method of the invention, only need to do simple characteristic pattern reordering operations to tensor (tensor), make
Obtaining network structure of the invention can not only be applied in field of image recognition, but also have good generalization ability, Ke Yixun
Speed is applied to other any applications.
The embodiment of third aspect present invention proposes a kind of non-transitorycomputer readable storage medium, is stored thereon with
Computer program when the program is executed by processor, is realized described in embodiment according to a first aspect of the present invention based on convolution mind
The data processing method of characteristic pattern through network；Either realize according to embodiment according to a second aspect of the present invention based on
The image processing method of convolutional neural networks.
In general, one or more computer-readable for realizing can using for the computer instruction of the method for the present invention
Any combination of storage medium carry.Non-transitorycomputer readable storage medium may include any computer-readable Jie
Matter, in addition to the signal itself in temporarily propagating.
Computer readable storage medium for example may be-but not limited to-electricity, magnetic, optical, electromagnetic, infrared ray or half
System, device or the device of conductor, or any above combination.The more specific example of computer readable storage medium is (non-
The list of exhaustion) it include: the electrical connection with one or more conducting wires, portable computer diskette, hard disk, random access memory
Device (RAM), read-only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, Portable, compact magnetic
Disk read-only memory (CD-ROM), light storage device, magnetic memory device or above-mentioned any appropriate combination.In this document
In, computer readable storage medium can be any tangible medium for including or store program, which can be commanded execution
System, device or device use or in connection.
The computer for executing operation of the present invention can be write with one or more programming languages or combinations thereof
Program code, described program design language include object oriented program language-such as Java, Smalltalk, C++,
Further include conventional procedural programming language-such as " C " language or similar programming language, can especially make
With the Python for being suitable for neural computing and based on platform frameworks such as tensorflow, PyTorch.Program code can be with
It fully executes, partly execute on the user computer on the user computer, being executed as an independent software package, portion
Divide and partially executes or executed on a remote computer or server completely on the remote computer on the user computer.?
Be related in the situation of remote computer, remote computer can pass through the network of any kind --- including local area network (LAN) or
Wide area network (WAN)-be connected to subscriber computer, or, it may be connected to outer computer (such as mentioned using Internet service
It is connected for quotient by internet).
The embodiment of fourth aspect present invention provides a kind of computer program product, when in the computer program product
Instruction when being executed by processor, realize the feature based on convolutional neural networks described in embodiment according to a first aspect of the present invention
The data processing method of figure；Or it realizes according to embodiment according to a second aspect of the present invention based on convolutional neural networks
Image processing method.
The embodiment of fifth aspect present invention provides a kind of calculating equipment, including memory, processor and is stored in
On reservoir and the computer program that can run on a processor, when the processor executes described program, realize according to the present invention
The data processing method of characteristic pattern described in first aspect based on convolutional neural networks；Or it realizes according to according to the present invention the
Image processing method based on convolutional neural networks described in two aspect embodiments.
Non-transitorycomputer readable storage medium of the third to five aspects according to the present invention, computer program product and meter
Calculate equipment, be referred to the content that embodiment according to a first aspect of the present invention specifically describes and realize, and have with according to the present invention
The data processing method of the characteristic pattern based on convolutional neural networks of first aspect and second aspect based on convolutional Neural net
The image processing method of network has similar beneficial effect, and details are not described herein.
Fig. 9 shows the block diagram for being suitable for the exemplary computer device for being used to realize embodiment of the present disclosure.What Fig. 9 was shown
Calculating equipment 12 is only an example, should not function to the embodiment of the present disclosure and use scope bring any restrictions.
As shown in figure 9, calculating equipment 12 can be realized in the form of universal computing device.The component for calculating equipment 12 can wrap
Include but be not limited to: one or more processor or processing unit 16, system storage 28 connect different system component (packets
Include system storage 28 and processing unit 16) bus 18.
Bus 18 indicates one of a few class bus structures or a variety of, including memory bus or Memory Controller,
Peripheral bus, graphics acceleration port, processor or the local bus using any bus structures in a variety of bus structures.It lifts
For example, these architectures include but is not limited to industry standard architecture (Industry Standard
Architecture；Hereinafter referred to as: ISA) bus, microchannel architecture (Micro Channel Architecture；Below
Referred to as: MAC) bus, enhanced isa bus, Video Electronics Standards Association (Video Electronics Standards
Association；Hereinafter referred to as: VESA) local bus and peripheral component interconnection (Peripheral Component
Interconnection；Hereinafter referred to as: PCI) bus.
It calculates equipment 12 and typically comprises a variety of computer system readable media.These media can be and any can be counted
Calculate the usable medium that equipment 12 accesses, including volatile and non-volatile media, moveable and immovable medium.
Memory 28 may include the computer system readable media of form of volatile memory, such as random access memory
Device (Random Access Memory；Hereinafter referred to as: RAM) 30 and/or cache memory 32.Calculating equipment 12 can be into
One step includes other removable/nonremovable, volatile, nonvolatile computer readable storage mediums.Only as an example,
Storage system 34 can be used for reading and writing immovable, non-volatile magnetic media and (not show in figure, commonly referred to as " hard drive
Device ").Although being not shown in Fig. 9, the disk for reading and writing to removable non-volatile magnetic disk (such as " floppy disk ") can be provided and driven
Dynamic device, and to removable anonvolatile optical disk (such as: compact disc read-only memory (Compact Disc Read Only
Memory；Hereinafter referred to as: CD-ROM), digital multi CD-ROM (Digital Video Disc Read Only
Memory；Hereinafter referred to as: DVD-ROM) or other optical mediums) read-write CD drive.In these cases, each driving
Device can be connected by one or more data media interfaces with bus 18.Memory 28 may include that at least one program produces
Product, the program product have one group of (for example, at least one) program module, and it is each that these program modules are configured to perform the disclosure
The function of embodiment.
Program/utility 40 with one group of (at least one) program module 42 can store in such as memory 28
In, such program module 42 include but is not limited to operating system, one or more application program, other program modules and
It may include the realization of network environment in program data, each of these examples or certain combination.Program module 42 is usual
Execute the function and/or method in disclosure described embodiment.
Calculating equipment 12 can also be with one or more external equipment 14 (such as keyboard, sensing equipment, display 24 etc.)
Communication can also enable a user to the equipment interacted with the computer system/server 12 communication with one or more, and/or
With enable the computer system/server 12 with it is one or more of the other calculating equipment communicated any equipment (such as
Network interface card, modem etc.) communication.This communication can be carried out by input/output (I/O) interface 22.Also, calculating is set
Standby 12 can also pass through network adapter 20 and one or more network (such as local area network (Local Area Network；With
Lower abbreviation: LAN), wide area network (Wide Area Network；Hereinafter referred to as: WAN) and/or public network, for example, internet) it is logical
Letter.As shown, network adapter 20 is communicated by bus 18 with the other modules for calculating equipment 12.Although being noted that
It is not shown in the figure, other hardware and/or software module can be used in conjunction with equipment 12 is calculated, including but not limited to: microcode is set
Standby driver, redundant processing unit, external disk drive array, RAID system, tape drive and data backup storage system
System etc..
Processing unit 16 by the program that is stored in system storage 28 of operation, thereby executing various function application and
Data processing, such as realize the method referred in previous embodiment.
Calculating equipment of the invention can be server, can also limited calculation power terminal device, lightweight of the invention
Network structure is particularly suitable for the latter.The matrix realization of the terminal device includes but is not limited to: intelligent mobile communication terminal, nothing
Man-machine, robot, portable image processing equipment, security device etc..
Although the embodiments of the present invention has been shown and described above, it should be appreciated that above-described embodiment is example
Property, it is not construed as limiting the claims, those skilled in the art within the scope of the invention can be to above-mentioned
Embodiment is changed, modifies, replacement and variant.
Claims (14)
1. a kind of data processing method based on convolutional neural networks characteristic pattern characterized by comprising
Data to be processed are obtained, the first convolution algorithm is carried out to the data to be processed, obtains primitive character figure；
In the length, width dimensions of the primitive character figure, respectively according to the first rule and Second Rule by the original spy
The element of sign figure is rearranged, the characteristic pattern after being reset；
Region division is carried out to the characteristic pattern after the rearrangement, obtains N number of feature subgraph, N is natural number；
N number of feature subgraph is spliced in channel dimension according to third rule, obtains the increased rearrangement tensor of port number
Structure；
The convolution results that convolution algorithm is reset are carried out according to the rearrangement tensor structure；
According to first rule and Second Rule and third rule, the convolution results of the rearrangement are restored, by institute
Each element reduction in the convolution results of rearrangement is stated into the arrangement mode of the length and width dimension of primitive character figure, is obtained at rearrangement
Characteristic after reason；
It resets treated characteristic according to described and carries out data processing.
2. the data processing method according to claim 1 based on convolutional neural networks characteristic pattern, which is characterized in that described
In the length, width dimensions of the primitive character figure, respectively according to the first rule and Second Rule by the primitive character figure
Element rearranged, in the characteristic pattern step after being reset, first rule includes:
In the length dimension of the primitive character figure, element is divided into Mh by principle according to coordinate about natural number Mh congruence
Group, and arranged as unit of the Mh group element.
3. the data processing method according to claim 2 based on convolutional neural networks characteristic pattern, which is characterized in that described
Element is divided at least one odd number group and at least one in the length dimension of primitive character figure by the integral multiple that natural number Mh is 2
A even number set, and the odd number group and the even number set are staggered.
4. the data processing method according to claim 1 based on convolutional neural networks characteristic pattern, which is characterized in that described
In the length, width dimensions of the primitive character figure, respectively according to the first rule and Second Rule by the primitive character figure
Element rearranged, in the characteristic pattern step after being reset, the Second Rule includes:
In the width dimensions of the primitive character figure, element is divided into Mw by principle according to coordinate about natural number Mw congruence
Group, and arranged as unit of the Mw group element.
5. the data processing method according to claim 4 based on convolutional neural networks characteristic pattern, which is characterized in that described
Element is divided at least one odd number group and at least one in the width dimensions of primitive character figure by the integral multiple that natural number Mw is 2
A even number set, and the odd number group and the even number set are staggered.
6. the data processing method according to claim 1 based on convolutional neural networks characteristic pattern, which is characterized in that institute
It states the characteristic pattern after resetting and carries out region division, obtaining N number of feature subgraph includes:
Characteristic pattern after rearrangement is subjected to equal area partition, obtains N number of feature subgraph of area equation.
7. the data processing method according to claim 1 based on convolutional neural networks characteristic pattern, which is characterized in that institute
It states the characteristic pattern after resetting and carries out region division, obtaining N number of feature subgraph includes:
Determine that the characteristic pattern after the rearrangement carries out the mode of region division according to first rule and the Second Rule, with
And the quantity N of feature subgraph.
8. the data processing method according to claim 1 based on convolutional neural networks characteristic pattern, which is characterized in that described
Carrying out the convolution results that convolution algorithm is reset according to the rearrangement tensor structure includes:
The feature of each feature subgraph is extracted with depth separation convolution；
To the feature of the N number of feature subgraph extracted through depth separation convolution, doing convolution kernel according to every N number of point is 1
The convolution results that point convolution is reset.
9. the data processing method according to claim 8 based on convolutional neural networks characteristic pattern, which is characterized in that described
Include: with the feature that depth separation convolution extracts each feature subgraph
The feature that convolution extracts each feature subgraph is separated using the depth that convolution kernel is 3*3.
10. a kind of data processing equipment based on convolutional neural networks characteristic pattern characterized by comprising
Data acquisition module carries out the first convolution algorithm to the data to be processed, obtains for obtaining data to be processed
Primitive character figure；
Characteristic pattern reordering module, for length, the width dimensions in the primitive character figure, respectively according to the first rule and the
Two rules rearrange the element of the primitive character figure, the characteristic pattern after being reset；
Characteristic pattern division module obtains N number of feature subgraph, N is certainly for carrying out region division to the characteristic pattern after the rearrangement
So number；
Channel splicing module obtains channel for splicing N number of feature subgraph in channel dimension according to third rule
The increased rearrangement tensor structure of number；
Convolution algorithm module, for carrying out the convolution results that convolution algorithm is reset according to the rearrangement tensor structure；
Construction recovery module, for regular according to first rule and Second Rule and third, to the convolution of the rearrangement
As a result it is restored, by each element reduction in the convolution results of the rearrangement at the length and width dimension of corresponding primitive character figure
Put in order, obtain resetting treated characteristic；
Data processing module, for resetting treated characteristic according to described and carrying out data processing.
11. a kind of image processing method based on convolutional neural networks, which is characterized in that in the convolutional neural networks, including
At least one characteristic pattern resets convolutional layer, and it includes at least one for realizing claim 1-9 that the characteristic pattern, which resets convolutional layer,
In any characteristic pattern based on convolutional neural networks data processing method computing unit.
12. the image processing method according to claim 11 based on convolutional neural networks, which is characterized in that the convolution
The structure of neural network is to reset the deep learning network of network structure, and the characteristic pattern processing method and the rearrangement network
Channel reset use in conjunction.
13. a kind of non-transitorycomputer readable storage medium, is stored thereon with computer program, which is characterized in that the meter
When calculation machine program is executed by processor, realize according to claim 1 described in any one of -9 based on convolutional neural networks
The data processing method of characteristic pattern, or realize described in any one of 1-12 according to claim 1 based on convolutional Neural net
The image processing method of network.
14. a kind of calculating equipment including memory, processor and stores the calculating that can be run on a memory and on a processor
Machine program, which is characterized in that when the processor executes described program, realize according to claim 1 described in any one of -9
The characteristic pattern based on convolutional neural networks data processing method, or realize any one of 1-12 according to claim 1
The image processing method based on convolutional neural networks.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910603672.4A CN110309837B (en) | 2019-07-05 | 2019-07-05 | Data processing method and image processing method based on convolutional neural network characteristic diagram |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910603672.4A CN110309837B (en) | 2019-07-05 | 2019-07-05 | Data processing method and image processing method based on convolutional neural network characteristic diagram |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110309837A true CN110309837A (en) | 2019-10-08 |
| CN110309837B CN110309837B (en) | 2021-07-06 |
Family
ID=68078359
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910603672.4A Active CN110309837B (en) | 2019-07-05 | 2019-07-05 | Data processing method and image processing method based on convolutional neural network characteristic diagram |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110309837B (en) |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110633700A (en) * | 2019-10-21 | 2019-12-31 | 深圳市商汤科技有限公司 | Video processing method and device, electronic equipment and storage medium |
| CN110826694A (en) * | 2019-10-30 | 2020-02-21 | 瀚博半导体(上海)有限公司 | Image processing method and device based on convolutional neural network |
| CN111008924A (en) * | 2019-12-02 | 2020-04-14 | 西安交通大学深圳研究院 | Image processing method and device, electronic equipment and storage medium |
| CN111161195A (en) * | 2020-01-02 | 2020-05-15 | 重庆特斯联智慧科技股份有限公司 | Feature map processing method and device, storage medium and terminal |
| CN111210006A (en) * | 2020-01-02 | 2020-05-29 | 北京科技大学 | Multi-feature-diagram-oriented heterogeneous deep neural network structure combination method and system |
| CN111723662A (en) * | 2020-05-18 | 2020-09-29 | 南京师范大学 | Human body posture recognition method based on convolutional neural network |
| CN111784555A (en) * | 2020-06-16 | 2020-10-16 | 杭州海康威视数字技术股份有限公司 | Image processing method, device and equipment |
| CN111985618A (en) * | 2020-08-14 | 2020-11-24 | 杭州海康威视数字技术股份有限公司 | Processing method and device of 3D convolutional neural network on neural network processor |
| CN112215754A (en) * | 2020-10-26 | 2021-01-12 | 北京达佳互联信息技术有限公司 | Image amplification method, device, electronic equipment and storage medium |
| CN112766462A (en) * | 2021-01-18 | 2021-05-07 | 苏州浪潮智能科技有限公司 | Data processing method, device and computer readable storage medium |
| CN112990370A (en) * | 2021-04-26 | 2021-06-18 | 腾讯科技（深圳）有限公司 | Image data processing method and device, storage medium and electronic equipment |
| CN113011384A (en) * | 2021-04-12 | 2021-06-22 | 重庆邮电大学 | Anchor-frame-free target detection method based on lightweight convolution |
| CN113033761A (en) * | 2019-12-09 | 2021-06-25 | 中科寒武纪科技股份有限公司 | Data processing method, data processing device, computer equipment and storage medium |
| CN113221896A (en) * | 2021-05-31 | 2021-08-06 | 北京灵汐科技有限公司 | Target detection method, target detection device, neuromorphic device, and medium |
| CN113554095A (en) * | 2021-07-26 | 2021-10-26 | 湖南国科微电子股份有限公司 | Feature map processing method and device and computer equipment |
| CN113919405A (en) * | 2020-07-07 | 2022-01-11 | 华为技术有限公司 | Data processing method and device and related equipment |
| CN114115296A (en) * | 2022-01-21 | 2022-03-01 | 南京理工大学 | Intelligent inspection and early warning system and method for key area |
| CN114648107A (en) * | 2022-03-10 | 2022-06-21 | 北京宏景智驾科技有限公司 | Method and circuit for improving efficiency of calculation of neural network input image point cloud convolution layer |
| CN113033761B (en) * | 2019-12-09 | 2024-05-14 | 中科寒武纪科技股份有限公司 | Data processing method, device, computer equipment and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108256544A (en) * | 2016-12-29 | 2018-07-06 | 深圳光启合众科技有限公司 | Picture classification method and device, robot |
| US20180314937A1 (en) * | 2017-04-28 | 2018-11-01 | Microsoft Technology Licensing, Llc | Learning-based noise reduction in data produced by a network of sensors, such as one incorporated into loose-fitting clothing worn by a person |
| US20180358003A1 (en) * | 2017-06-09 | 2018-12-13 | Qualcomm Incorporated | Methods and apparatus for improving speech communication and speech interface quality using neural networks |
| CN109426858A (en) * | 2017-08-29 | 2019-03-05 | 京东方科技集团股份有限公司 | Neural network, training method, image processing method and image processing apparatus |
| CN109643396A (en) * | 2016-06-17 | 2019-04-16 | 诺基亚技术有限公司 | Construct convolutional neural networks |
| CN109934236A (en) * | 2019-01-24 | 2019-06-25 | 杰创智能科技股份有限公司 | A kind of multiple dimensioned switch target detection algorithm based on deep learning |
| CN109948794A (en) * | 2019-02-28 | 2019-06-28 | 清华大学 | Neural network structure pruning method, pruning device and electronic equipment |
-
2019
- 2019-07-05 CN CN201910603672.4A patent/CN110309837B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109643396A (en) * | 2016-06-17 | 2019-04-16 | 诺基亚技术有限公司 | Construct convolutional neural networks |
| CN108256544A (en) * | 2016-12-29 | 2018-07-06 | 深圳光启合众科技有限公司 | Picture classification method and device, robot |
| US20180314937A1 (en) * | 2017-04-28 | 2018-11-01 | Microsoft Technology Licensing, Llc | Learning-based noise reduction in data produced by a network of sensors, such as one incorporated into loose-fitting clothing worn by a person |
| US20180358003A1 (en) * | 2017-06-09 | 2018-12-13 | Qualcomm Incorporated | Methods and apparatus for improving speech communication and speech interface quality using neural networks |
| CN109426858A (en) * | 2017-08-29 | 2019-03-05 | 京东方科技集团股份有限公司 | Neural network, training method, image processing method and image processing apparatus |
| CN109934236A (en) * | 2019-01-24 | 2019-06-25 | 杰创智能科技股份有限公司 | A kind of multiple dimensioned switch target detection algorithm based on deep learning |
| CN109948794A (en) * | 2019-02-28 | 2019-06-28 | 清华大学 | Neural network structure pruning method, pruning device and electronic equipment |
Non-Patent Citations (3)
| Title |
|---|
| KAIQIANG CHEN等: ""RESIDUAL SHUFFLING CONVOLUTIONAL NEURAL NETWORKS FOR DEEP SEMANTIC IMAGE SEGMENTATION USING MULTI-MODAL DATA"", 《ISPRS ANNALS OF PHOTOGRAMMETRY, REMOTE SENSING & SPATIAL INFORMATION SCIENCES》 * |
| NINGNING MA等: ""ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design"", 《HTTPS://LINK.SPRINGER.COM/CONFERENCE/ECCV》 * |
| 袁晖等: ""基于CNN特征加权和区域整合的图像检索"", 《计算机工程与科学》 * |
Cited By (31)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110633700A (en) * | 2019-10-21 | 2019-12-31 | 深圳市商汤科技有限公司 | Video processing method and device, electronic equipment and storage medium |
| CN110826694A (en) * | 2019-10-30 | 2020-02-21 | 瀚博半导体(上海)有限公司 | Image processing method and device based on convolutional neural network |
| CN110826694B (en) * | 2019-10-30 | 2021-06-11 | 瀚博半导体(上海)有限公司 | Image processing method and device based on convolutional neural network |
| CN111008924A (en) * | 2019-12-02 | 2020-04-14 | 西安交通大学深圳研究院 | Image processing method and device, electronic equipment and storage medium |
| CN111008924B (en) * | 2019-12-02 | 2023-09-12 | 西安交通大学深圳研究院 | Image processing method and device, electronic equipment and storage medium |
| CN113033761B (en) * | 2019-12-09 | 2024-05-14 | 中科寒武纪科技股份有限公司 | Data processing method, device, computer equipment and storage medium |
| CN113033761A (en) * | 2019-12-09 | 2021-06-25 | 中科寒武纪科技股份有限公司 | Data processing method, data processing device, computer equipment and storage medium |
| CN111161195A (en) * | 2020-01-02 | 2020-05-15 | 重庆特斯联智慧科技股份有限公司 | Feature map processing method and device, storage medium and terminal |
| CN111210006A (en) * | 2020-01-02 | 2020-05-29 | 北京科技大学 | Multi-feature-diagram-oriented heterogeneous deep neural network structure combination method and system |
| CN111210006B (en) * | 2020-01-02 | 2023-07-28 | 北京科技大学 | Heterogeneous deep neural network structure combination method and system for multi-feature graph |
| CN111161195B (en) * | 2020-01-02 | 2023-10-13 | 重庆特斯联智慧科技股份有限公司 | Feature map processing method and device, storage medium and terminal |
| CN111723662A (en) * | 2020-05-18 | 2020-09-29 | 南京师范大学 | Human body posture recognition method based on convolutional neural network |
| CN111784555B (en) * | 2020-06-16 | 2023-08-25 | 杭州海康威视数字技术股份有限公司 | Image processing method, device and equipment |
| CN111784555A (en) * | 2020-06-16 | 2020-10-16 | 杭州海康威视数字技术股份有限公司 | Image processing method, device and equipment |
| CN113919405A (en) * | 2020-07-07 | 2022-01-11 | 华为技术有限公司 | Data processing method and device and related equipment |
| CN113919405B (en) * | 2020-07-07 | 2024-01-19 | 华为技术有限公司 | Data processing method and device and related equipment |
| CN111985618B (en) * | 2020-08-14 | 2024-03-05 | 杭州海康威视数字技术股份有限公司 | Processing method and device of 3D convolutional neural network on neural network processor |
| CN111985618A (en) * | 2020-08-14 | 2020-11-24 | 杭州海康威视数字技术股份有限公司 | Processing method and device of 3D convolutional neural network on neural network processor |
| CN112215754B (en) * | 2020-10-26 | 2024-01-26 | 北京达佳互联信息技术有限公司 | Image amplifying method, device, electronic equipment and storage medium |
| CN112215754A (en) * | 2020-10-26 | 2021-01-12 | 北京达佳互联信息技术有限公司 | Image amplification method, device, electronic equipment and storage medium |
| CN112766462A (en) * | 2021-01-18 | 2021-05-07 | 苏州浪潮智能科技有限公司 | Data processing method, device and computer readable storage medium |
| CN113011384A (en) * | 2021-04-12 | 2021-06-22 | 重庆邮电大学 | Anchor-frame-free target detection method based on lightweight convolution |
| CN113011384B (en) * | 2021-04-12 | 2022-11-25 | 重庆邮电大学 | Anchor-frame-free target detection method based on lightweight convolution |
| WO2022228105A1 (en) * | 2021-04-26 | 2022-11-03 | 腾讯科技（深圳）有限公司 | Processing method and apparatus for image data, storage medium, and electronic device |
| CN112990370A (en) * | 2021-04-26 | 2021-06-18 | 腾讯科技（深圳）有限公司 | Image data processing method and device, storage medium and electronic equipment |
| CN113221896A (en) * | 2021-05-31 | 2021-08-06 | 北京灵汐科技有限公司 | Target detection method, target detection device, neuromorphic device, and medium |
| CN113554095B (en) * | 2021-07-26 | 2022-08-19 | 湖南国科微电子股份有限公司 | Feature map processing method and device and computer equipment |
| CN113554095A (en) * | 2021-07-26 | 2021-10-26 | 湖南国科微电子股份有限公司 | Feature map processing method and device and computer equipment |
| CN114115296B (en) * | 2022-01-21 | 2022-05-20 | 南京理工大学 | Intelligent inspection and early warning system and method for key area |
| CN114115296A (en) * | 2022-01-21 | 2022-03-01 | 南京理工大学 | Intelligent inspection and early warning system and method for key area |
| CN114648107A (en) * | 2022-03-10 | 2022-06-21 | 北京宏景智驾科技有限公司 | Method and circuit for improving efficiency of calculation of neural network input image point cloud convolution layer |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110309837B (en) | 2021-07-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110309837A (en) | Data processing method and image processing method based on convolutional neural networks characteristic pattern | |
| CN106447034B (en) | A kind of neural network processor based on data compression, design method, chip | |
| CN108615036A (en) | A kind of natural scene text recognition method based on convolution attention network | |
| CN106529670A (en) | Neural network processor based on weight compression, design method, and chip | |
| CN107169563A (en) | Processing system and method applied to two-value weight convolutional network | |
| CN110084274A (en) | Realtime graphic semantic segmentation method and system, readable storage medium storing program for executing and terminal | |
| CN107256424A (en) | Three value weight convolutional network processing systems and method | |
| CN108228728B (en) | Parameterized thesis network node representation learning method | |
| CN109766995A (en) | The compression method and device of deep neural network | |
| CN110097172A (en) | A kind of convolutional neural networks data processing method and device based on winograd convolution algorithm | |
| CN111259154B (en) | Data processing method and device, computer equipment and storage medium | |
| CN112445876A (en) | Entity alignment method and system fusing structure, attribute and relationship information | |
| CN111860528B (en) | Image segmentation model based on improved U-Net network and training method | |
| WO2024001100A1 (en) | Method and apparatus for processing text, and device and non-volatile readable storage medium | |
| CN109784159A (en) | The processing method of scene image, apparatus and system | |
| CN113626612A (en) | Prediction method and system based on knowledge graph reasoning | |
| KR102403330B1 (en) | Technique for generating and utilizing virtual fingerprint representing text data | |
| CN112906865A (en) | Neural network architecture searching method and device, electronic equipment and storage medium | |
| CN114281968A (en) | Model training and corpus generation method, device, equipment and storage medium | |
| CN114077673A (en) | Knowledge graph construction method based on BTBC model | |
| Singh et al. | Artificial intelligence in edge devices | |
| CN114329148A (en) | Content information identification method and device, computer equipment and storage medium | |
| CN113722436A (en) | Text information extraction method and device, computer equipment and storage medium | |
| CN115082371B (en) | Image fusion method and device, mobile terminal equipment and readable storage medium | |
| CN116561272A (en) | Open domain visual language question-answering method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |