CN107610140A - Near edge detection method, device based on depth integration corrective networks - Google Patents

Near edge detection method, device based on depth integration corrective networks Download PDF

Info

Publication number
CN107610140A
CN107610140A CN201710666537.5A CN201710666537A CN107610140A CN 107610140 A CN107610140 A CN 107610140A CN 201710666537 A CN201710666537 A CN 201710666537A CN 107610140 A CN107610140 A CN 107610140A
Authority
CN
China
Prior art keywords
feature
edge detection
mrow
detection method
convolutional neural
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201710666537.5A
Other languages
Chinese (zh)
Inventor
黄凯奇
赵鑫
王裕沛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Institute of Automation of Chinese Academy of Science
Original Assignee
Institute of Automation of Chinese Academy of Science
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Institute of Automation of Chinese Academy of Science filed Critical Institute of Automation of Chinese Academy of Science
Priority to CN201710666537.5A priority Critical patent/CN107610140A/en
Publication of CN107610140A publication Critical patent/CN107610140A/en
Pending legal-status Critical Current

Links

Landscapes

  • Image Analysis (AREA)

Abstract

The present invention relates to pattern-recognition, computer vision, deep learning field, propose a kind of near edge detection method based on depth integration corrective networks, device, the problem of edge positioning is not accurate enough, the edge of detection is not fine enough in image is aimed to solve the problem that, this method includes:Step S1, by the propagated forward subnetwork of convolutional neural networks, obtain the Analysis On Multi-scale Features of input picture;Step S2, by the reverse amendment subnetwork of convolutional neural networks, the method for gradually increase feature resolution is taken to obtain the final image feature that there is equal resolution with input picture;Step S3, it is single channel by the feature passage dimensionality reduction of final image feature, edge detection results is generated by fitting function.Inventive network structure is simpler, and the image feature representation obtained more retains minutia, and Detection results are more preferable, and edge visualization result is finer.

Description

Near edge detection method, device based on depth integration corrective networks
Technical field
It is more particularly to a kind of to be repaiied based on depth integration the present invention relates to pattern-recognition, computer vision, deep learning field The near edge detection method of positive network, device.
Background technology
Have benefited from the field fast development such as depth convolutional neural networks, computer vision, artificial intelligence, machine perception.Side Edge detection is used as a basic problem in computer vision, has also obtained significant progress.Rim detection is exactly to utilize computer Image is analyzed, and then obtains the marginal information of objects in images.Rim detection is passed through frequently as instrument, aids in other visions Task.Traditional edge detection method generally relies on engineer's feature, it is easy to changed by light, object color change with And the interference that background is noisy, very robust, precision also are not difficult to user's request in practice.Based on depth convolutional Neural net The edge detection method of network overcomes drawbacks described above to a certain extent, there is provided the more preferable edge detector of robustness.
Although depth convolutional neural networks achieve success, or even some high-caliber methods specific in rim detection Reach on data set manually to the level at image detection edge, but the edge of its detection is not fine enough, it is very coarse, visual Change in effect, and still there is larger gap at the edge manually extracted.On the other hand, the existing edge based on depth convolutional neural networks Positioning of the detection method to objects in images edge is not accurate enough, can not obtain fine and accurate positioning edge detection results. Therefore, the present invention starts with from finer edge is detected, and makes full use of Image Multiscale feature, proposes that one kind is based on depth integration The near edge detection method of corrective networks, this method is by reversely correcting subnetwork, in gradually fusion propagated forward part While the Analysis On Multi-scale Features comprising different abstraction hierarchies of network, gradually increase the resolution ratio of feature, it is final to obtain and input Image has the feature representation of equal resolution.Existing hand-designed characterization method can be so made up well and is only included The deficiency of the edge detection method based on depth convolutional neural networks of propagated forward subnetwork, obtain a kind of more robust, The feature representation of abundant detailed information is included, caused edge is more fine.
The content of the invention
It has been to solve edge in image to position not accurate enough, detection to solve above mentioned problem of the prior art The problem of edge is not fine enough, an aspect of of the present present invention, it is proposed that a kind of near edge inspection based on depth integration corrective networks Survey method, comprises the following steps:
Step S1, by the propagated forward subnetwork of convolutional neural networks, obtain the Analysis On Multi-scale Features of input picture;
Step S2, by the reverse amendment subnetwork of convolutional neural networks, take the side for gradually increasing feature resolution Method obtains the final image feature for having equal resolution with input picture;It is described gradually increase feature resolution method be: While merging the Analysis On Multi-scale Features of the input picture, the resolution ratio of feature is increased by the method for sub-pixel convolution;
Step S3, it is single channel by the feature passage dimensionality reduction of final image feature, rim detection is generated by fitting function As a result.
Preferably, the Analysis On Multi-scale Features of input picture described in step S1, it is the different chis that the convolutional layer of different depth obtains The feature pyramid that characteristics of image under degree is formed.
Preferably, " take the method for gradually increase feature resolution to obtain has identical point with input picture in step S2 The final image feature of resolution ", its method are:
To all features in the feature pyramid, top-down selection adjacent feature, repeat step S21 and step S22, until obtaining the fusion feature F ' of first layer feature1
Wherein,
Step S21, give feature FnAnd Fn-1, by the less feature F of yardstickn, pass through the method increase point of sub-pixel convolution Resolution obtains feature Fαn, feature FαnYardstick and Fn-1Unanimously;Wherein feature FnFor the characteristics of image of n-th layer in Analysis On Multi-scale Features;
Step S22:By convolution, concatenation by feature FαnAnd Fn-1Merged, produce (n-1)th layer of fusion feature Fβn-1, and be F by (n-1)th layer in the feature pyramid of feature replacementβn-1
Preferably, in step S22 " by convolution, concatenation by feature FαnAnd Fn-1Merged ", its method includes:
Step S221, by convolution operation, by FαnAnd Fn-1Dimensionality reduction is carried out, the feature that furthers port number, obtains feature representation F′αnWith F 'n-1, size is respectively wn-1×hn-1×d′αnAnd wn-1×hn-1×d′n-1
Step S222, by F 'αnWith F 'n-1Splicing, generation size is wn-1×hn-1×(d′αn+d′n-1) intermediate features, then The intermediate features are used with convolution operation, generation size is wn-1×hn-1×dβn-1(n-1)th layer of fusion feature Fβn-1, and It is F by (n-1)th layer in the feature pyramid of feature replacementβn-1
Preferably, " given feature F described in step S21nAnd Fn-1, by the less feature F of yardstickn, pass through sub-pixel convolution Method increase resolution ratio obtain feature Fαn", its method is:
Step S211, by feature FnCarry out multilayer convolution operation, generation new feature F "n, wherein F "nSize be wn ×hn×(dn×r×r);Wherein FnSize be wn×hn×dn, r is characterized Fn-1And FnScale size multiple;
Step S212, to new feature F "nPixel reorganization is carried out in a fixed order, and it is rw to obtain sizen×rhn×dn's Feature Fαn
Preferably, " to new feature F " in step S212nPixel reorganization is carried out in a fixed order ", its method is:
By wn×hnD on the two-dimensional coordinate of determinationn× r × r value is divided into r2Part, then by this r2Part is laid into r × r area Domain, each point in the region contain dnIndividual passage.
Preferably, convolutional neural networks are same convolutional Neural net in convolutional neural networks and step S2 in step S1 Network, the convolutional neural networks are by merging the propagated forward subnetwork, the reversely amendment subnetwork is formed Can end-to-end training near edge detection convolutional neural networks.
Preferably, it is described can end-to-end training near edge detection convolutional neural networks, its train loss function L (w) it is:
Wherein, X represent input picture, W be propagated forward subnetwork weight, Y+Represent edge pixel point set, Y_Table Show non-edge pixels point set, β is the ratio of non-edge pixels point sum and edge pixel point sum.
Preferably, fitting function described in step S3 is Sigmoid functions.
Another aspect of the present invention, it is proposed that a kind of storage device, wherein be stored with a plurality of program, described program be suitable to by Processor is loaded and performed to realize the above-mentioned near edge detection method based on depth integration corrective networks.
The third aspect of the present invention, it is proposed that a kind of processing unit, including
Processor, it is adapted for carrying out each bar program;And
Storage device, suitable for storing a plurality of program;
Described program is above-mentioned based on the fine of depth integration corrective networks to realize suitable for being loaded by processor and being performed Edge detection method.
The present invention has advantages below compared to existing edge detection method:
(1) Analysis On Multi-scale Features of propagated forward subnetwork are merged by reversely correcting subnetwork, and reversely corrected Subnetwork increases the resolution ratio of feature when merging Analysis On Multi-scale Features, by the method for sub-pixel convolution, finally obtains and defeated Enter the feature representation of image equal resolution, the up-sampling mode than traditional bilinear interpolation more retains minutia;
(2) propagated forward subnetwork and the reversely convolutional neural networks of the U-shaped network structure of amendment subnetwork composition, Compared to the current existing rim detection network structure based on deep learning, network structure is simpler, and Detection results are more preferable, edge Visualization result is finer.
Brief description of the drawings
Fig. 1 is the schematic flow sheet of near edge detection method of the embodiment of the present invention based on depth integration corrective networks;
Fig. 2 is the structural representation of the convolutional neural networks of the near edge detection of the embodiment of the present invention;
Fig. 3 is the edge detection method result visualization legend of the embodiment of the present invention.
Embodiment
The preferred embodiment of the present invention described with reference to the accompanying drawings.It will be apparent to a skilled person that this A little embodiments are used only for explaining the technical principle of the present invention, it is not intended that limit the scope of the invention.
The present invention thought main points be:1) depth integration corrective networks network structure proposed by the present invention utilizes reversely amendment Subnetwork gradually merges the feature of the different scale of propagated forward subnetwork;2) reversely correction portion subnetting proposed by the present invention Network is when merging propagated forward subnetwork particular dimensions feature and current amendment feature, after reducing feature port number, using spelling The mode connect is merged;3) reversely amendment subnetwork proposed by the present invention is in fusion propagated forward subnetwork different scale During feature, gradually increase the resolution ratio of characteristic response, it is final to obtain and an equal amount of characteristic response of input picture;4) it is of the invention Merge above-mentioned thought devise can end-to-end training depth convolutional neural networks structure.
The near edge detection method based on depth integration corrective networks of one embodiment of the invention, as shown in figure 1, including Following steps:
Step S1, by the propagated forward subnetwork of convolutional neural networks, obtain the Analysis On Multi-scale Features of input picture;
Step S2, by the reverse amendment subnetwork of convolutional neural networks, take the side for gradually increasing feature resolution Method obtains the final image feature for having equal resolution with input picture;It is described gradually increase feature resolution method be: While merging the Analysis On Multi-scale Features of the input picture, the resolution ratio of feature is increased by the method for sub-pixel convolution;
Step S3, it is single channel by the feature passage dimensionality reduction of final image feature, rim detection is generated by fitting function As a result.
In the embodiment of the present invention, convolutional neural networks are same convolution in convolutional neural networks and step S2 in step S1 Neutral net, as shown in Fig. 2 the convolutional neural networks are by merging the propagated forward subnetwork, the reversely amendment Subnetwork formed can end-to-end training near edge detection convolutional neural networks, for vivider expression network Structure, it is properly termed as U-typed convolutional neural networks.
The " u "-shaped convolutional neural networks obtain the multiple dimensioned spy of different abstract levels by the propagated forward subnetwork Sign, then by the Analysis On Multi-scale Features reversely corrected subnetwork and gradually merge the propagated forward subnetwork, while not Disconnected increase feature resolution, it is final to obtain the feature that there is equal resolution with input picture.
It is above-mentioned can end-to-end training near edge detection convolutional neural networks, its loss function L (w) trained is as public Shown in formula (1):
Wherein, X represent input picture, W be propagated forward subnetwork weight, Y+Represent edge pixel point set, Y-Table Showing non-edge pixels point set, β is the ratio of non-edge pixels point sum and edge pixel point sum, and Pr represents conditional probability, Pr (A | B) is the conditional probability of A after known B occurs, also due to the value derived from B and the posterior probability referred to as A, yj=1 table Show that j-th of pixel is edge pixel point, yj=0 represents that j-th of pixel is non-edge pixels point.
The Analysis On Multi-scale Features of input picture in step S1, it is the difference that the convolutional layer of different depth obtains in the present embodiment The feature pyramid that characteristics of image under yardstick is formed.
In the present embodiment, " method of gradually increase feature resolution is taken to obtain and input picture tool described in step S2 Have the final image feature of equal resolution ", its method is:
To all features in the feature pyramid, top-down selection adjacent feature, repeat step S21 and step S22, until obtaining the fusion feature F ' of first layer feature1
Wherein,
Step S21, up-sampling:Given feature FnAnd Fn-1, by the less feature F of yardstickn, pass through the method for sub-pixel convolution Increase resolution ratio obtains feature Fαn, feature FαnYardstick and Fn-1Unanimously;Wherein feature FnFor n-th layer in the Analysis On Multi-scale Features Characteristics of image;
Step S22:Merge Analysis On Multi-scale Features:By convolution, concatenation by feature FαnAnd Fn-1Merged, produce the The fusion feature F of n-1 layersβn-1, and be F by (n-1)th layer in feature pyramid of feature replacementβn-1
In the present embodiment, " by convolution, concatenation by feature F in step S22αnAnd Fn-1Merged ", its method bag Include:
Step S221, by convolution operation, by FαnAnd Fn-1Dimensionality reduction is carried out, the feature that furthers port number, obtains feature representation F′αnWith F 'n-1, size is respectively wn-1×hn-1×d′αnAnd wn-1×hn-1×d′n-1
Step S222, by F 'αnWith F 'n-1Splicing, generation size is wn-1×hn-1×(d′αn+d′n-1) intermediate features, then The intermediate features are used with convolution operation, generation size is wn-1×hn-1×dβn-1(n-1)th layer of fusion feature Fβn-1, and It is F by (n-1)th layer in the feature pyramid of feature replacementβn-1
In the present embodiment, " given feature F in step S21nAnd Fn-1, by the less feature F of yardstickn, pass through sub-pixel convolution Method increase resolution ratio obtain feature Fαn", its method is:
Step S211, by feature FnCarry out multilayer convolution operation, generation new feature F "n, wherein F "nSize be wn ×hn×(dn×r×r);Wherein FnSize be wn×hn×dn, r is characterized Fn-1And FnScale size multiple;
Step S212, to new feature F "nPixel reorganization is carried out in a fixed order, and it is rw to obtain sizei×rhi×di's Feature Fαn, specific method is:By wn×hnD on the two-dimensional coordinate of determinationn× r × r value is divided into r2Part, then by this r2Part is flat R × r region is paved into, each point in the region contains dnIndividual passage.
In the present embodiment, fitting function described in step S3 is Sigmoid functions.
In order to further clearly be illustrated to the technology of the present invention details, hereafter carried respectively from forward direction part of propagation network Take Analysis On Multi-scale Features, reversely amendment subnetwork fusion Analysis On Multi-scale Features, reverse amendment subnetwork pass through in sub-pixel convolution Sampling three parts are developed in details.
1st, propagated forward subnetwork extraction Analysis On Multi-scale Features
Characteristics of image is the abstractdesription to image, can representative image the characteristics of.The quality of feature directly affects side Edge testing result.The method that traditional feature extraction typically uses hand-designed, it is difficult to the deep information of description image very well, and It is and easily poor by the interference of the conditions such as such as illumination, angle, robustness.Different from traditional-handwork design feature method, it is based on The mode of deep learning is learnt end to end, can be according to optimization aim, by error back propagation method, from a large amount of numbers Feature representation according to learning to data robust.Therefore, this method employs the mode based on deep learning, to obtain input figure As more preferable feature representation.This method is first using a series of propagated forward subnetwork knot operated comprising convolution and down-sampling Structure, obtain the feature representation under the multiple different scales of image.
Specifically, the propagated forward subnetwork structure of this method uses depth convolutional neural networks, by image A series of convolution and down-sampling operation are carried out, the convolutional layer of different depth can be obtained under different scale (as shown in Figure 2 1,1/2,1/4,1/8 of original image resolution etc. respectively) image feature representation, composition characteristic pyramid.It is represented by Formula (2):
Wherein, X represents input picture, and W is the weight of propagated forward subnetwork,Represent not With the feature of yardstick.
Wherein, more rich space characteristics being included compared with shallow-layer feature, deeper feature abstraction ability is stronger, therefore comprising more Abundant semantic feature.These different abstraction hierarchies, the multiple dimensioned basis for being characterized in subsequent characteristics amendment.Spy in the present invention It is a three-dimensional matrice R to levy expressionw×h×d, the resolution ratio of w and h Expressive Features, width and height are represented respectively, d represents special in addition Levy port number.
2nd, reversely amendment subnetwork merges Analysis On Multi-scale Features
, can be although the feature that traditional side edge detection calligraphy learning based on deep learning obtains has good presentation skills To a certain extent response diagram as the characteristics of, but still deficiency with for fine edge provides enrich detailed information.Therefore, we On propagated forward network foundation of the tradition based on deep learning method, reversely amendment subnetwork is introduced, before preferably merging To the Analysis On Multi-scale Features of part of propagation network, to obtain the preferably feature representation of balanced semantic feature and space characteristics.Meanwhile Reversely amendment subnetwork gradually increases the resolution of feature in the gradually Analysis On Multi-scale Features of fusion propagated forward subnetwork Rate, it is final to obtain the feature representation that there is equal resolution with input picture.It is represented by formula (3):
WhereinThe feature under propagated forward subnetwork current scale is represented,Represent repercussion correction portion The upper revised feature of a yardstick, f in subnetworkfuse() represents to be merged both features, fsubConv() represents to melt Feature after conjunction increases feature resolution in a manner of sub-pixel convolution.The feature representation so obtained is for producing fine side Edge is particularly important.
In embodiments of the present invention, F is usednRepresent the feature of n-layer in feature pyramid.The characteristic modification step of the present invention (the step of merging Analysis On Multi-scale Features), is top-down, i.e., since top-level feature.It is w for sizen×hn×dn's N-th layer feature, i.e. top feature Fn, use it to (n-1)th layer of feature F of amendmentn-1.Therefore, first to FnCarry out convolution and upper Sampling operation, it is w to obtain sizen-1×hn-1×dnFeature representation Fαn, wherein wn-1, hn-1It is consistent with the w of n-1 layer features, h. Although by up-sampling operation by FnScale expansion most with Fn-1Yardstick it is consistent, but due to the feature of different levels Contained feature port number is inconsistent, if directly to feature FαnAnd Fn-1Merged, can (feature passage be few to low-dimensional feature Feature) produce inhibitory action, it is difficult to due ability to express is played in the feature after fusion.In order to more preferably retain each level Feature generate new feature F ', it is necessary to pass through convolutional layerαnWith F 'n-1To express primitive character FαnAnd Fn-1.New feature and original spy The yardstick of sign is the same, and the feature port number of feature to be fused approaches, and will not produce suppression to low-dimensional feature during Fusion Features. Specifically, by convolution operation, by feature (i.e. F to be fusedαnAnd Fn-1) dimensionality reduction is carried out, the feature that furthers port number, obtain feature Express F 'αnWith F 'n-1, size is respectively wn-1×hn-1×d′αnAnd wn-1×hn-1×d′n-1
Obtain F 'αnWith F 'n-1Afterwards, it is necessary to use the F ' containing more global informationsαnInstruct F 'n-1Produce fusion feature Fβn-1。 In order to improve fusion after feature ability to express, on the basis of splicing, reuse convolution operation, feature after splicing taken out As so as to produce revised fusion feature Fβn-1.Specifically, we are first by F 'αnWith F 'n-1Splicing, producing size is wn-1×hn-1×(d′αn+d′n-1) intermediate features, then convolution operation is used to it, generation size is wn-1×hn-1×dβn-1's Fusion feature Fβn-1, the mistake with top-level feature amendment time top-level feature is so far completed by way of merging Analysis On Multi-scale Features Journey.
For n-2 layer features Fn-2, the fusion feature F of the invention implemented using n-1 layersβn-1Analysis On Multi-scale Features are carried out to it Fusion, realize to feature Fn-2Amendment.Specific steps are similar to the characteristic modification to n-1 layers, and the present invention is first by Fβn-1On Sample Fn-2Yardstick, afterwards according to the obtained F of up-samplingαn-1And Fn-2, difference convolution generation new feature F 'αn-1With F 'n-2.Most Afterwards, by F 'αn-1With F 'n-2Generate fusion feature Fβn-2
Said process is repeated, until completing to correct bottom feature F1, generation size is w1×h1×d1Feature F '1 (w1And h1Equal to the size of input picture).In general, the feature port number of edge graph is 1, therefore, we are needed to feature F′1Convolution operation is carried out, its feature passage is reduced to 1, F is obtained by Sigmoid functionsout, i.e. the rim detection knot of this method Fruit.
3rd, reversely amendment subnetwork is up-sampled by sub-pixel convolution
Edge detection method based on depth convolutional neural networks uses a series of convolution and down-sampling layer, more to be taken out The feature representation of elephant, but because a series of down-samplings operate, the resolution ratio of feature representation is gradually lowered.It is end-to-end in order to carry out Study, the edge detection method based on depth convolutional neural networks generally require by resolution ratio reduce feature be upsampled to it is defeated Enter the original resolution ratio of image.Traditional edge detection method based on depth convolutional neural networks is by way of bilinearity difference Feature representation is directly up-sampled to increase feature resolution, the top sampling method of this bilinear interpolation can lose details letter Breath, this is unfavorable to obtaining finer feature representation.In order to obtain and input image resolution identical feature representation, sheet Method takes the mode of gradually increase feature resolution.The reverse amendment subnetwork of this method is in gradually fusion propagated forward portion During the Analysis On Multi-scale Features of subnetwork, gradually increase the resolution ratio of feature, final obtain has equal resolution with input picture Feature representation.Wherein, mode of the feature resolution using sub-pixel convolution is increased.
Given size is respectively wn×hn×dnAnd wn-1×hn-1×dn-1Feature FnAnd Fn-1, the purpose of sub-pixel convolution It is to increase FnFeature resolution, it is set to be converted into and Fn-1Feature F with same scaleαn, wherein FαnSize be wn-1× hn-1×dn-1.The top sampling method for being conventionally used to increase resolution ratio is usually the wave filter of engineer, such as bilinearity or double three Secondary sampler.The wave filter of engineer uses the up-sampling mode of bilinear interpolation, it is excessively smooth to up-sample result, it is difficult to protect Spacing detailed information, and sub-pixel convolution constantly updates parameter by the study of mass data, up-sampling result retains richer Rich spatial information.
Different from other up-sampling modes based on study, sub-pixel convolution includes multilayer convolution and pixel reorganization two Point.The purpose of multilayer convolution is to be abstracted input feature vector from many levels, stronger so as to generate feature representation ability New feature.For feature FnAnd Fn-1If Fn-1Yardstick be FnR times, i.e. wn-1=r × wn, hn-1=r × hn.We pass through To FnCarry out multilayer convolution operation, generation new feature F "n.Wherein F "nSize be wn×hn×(dn×r×r).New feature F″nWith FnCompare, maintain the uniformity of yardstick, only change feature port number in proportion.Obtain new feature F "nAfterwards, according still further to Permanent order carries out pixel reorganization operation, makes F "nSize and Fn-1Match (i.e. F "nSize reaches wn-1And hn-1).For three Dimensional feature matrix F "n, have d on the two-dimensional coordinate that is determined by a certain group of w and hn× r × r value.These values are divided into r2Part, then will This r2Part is laid into r × r region, and each point in the region contains diIndividual passage.To F "nAfter tiling in a manner described, obtain Size is rwn×rhn×dnFeature Fαn.After pixel reorganization, obtain and Fn-1The feature F of yardstick matchingαn, this feature can To regard feature F asnMapping in target signature space.Sub-pixel convolution operation is used in addition to top feature by the present invention The upsampling process of all hierarchy characteristics, it is final to obtain the feature representation that there is equal resolution with input picture.
Increase feature resolution by way of sub-pixel convolution, its detailed explanation may be referred to Wenzhe Shi and exist 2016 entitled in having delivered one on CVPR《By efficient sub-pixel convolutional neural networks realize real-time single image and Video super-resolution (Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network)》Paper, be no longer developed in details herein It is bright.
Fig. 3 is the edge detection method result visualization legend of the present embodiment.First behavior image to be detected, the second row The edge detection method result designed for the standard edge testing result manually marked, the third line for the present invention.
The method that is described with reference to the embodiments described herein can use hardware, computing device the step of algorithm Software module, or the two combination are implemented.Software module can be placed in random access memory (RAM), internal memory, read-only storage (ROM), electrically programmable ROM, electrically erasable ROM, register, hard disk, moveable magnetic disc, CD-ROM or technical field In any other form of storage medium well known to interior.
Those skilled in the art should be able to recognize that, the side of each example described with reference to the embodiments described herein Method step, can be realized with electronic hardware, computer software or the combination of the two, in order to clearly demonstrate electronic hardware and The interchangeability of software, the composition and step of each example are generally described according to function in the above description.These Function is performed with electronic hardware or software mode actually, application-specific and design constraint depending on technical scheme. Those skilled in the art can realize described function using distinct methods to each specific application, but this reality Now it is not considered that beyond the scope of this invention.
The storage device of an embodiment of the present invention, wherein being stored with a plurality of program, described program is suitable to be added by processor Carry and perform to realize the above-mentioned near edge detection method based on depth integration corrective networks.
The processing unit of an embodiment of the present invention, including processor, storage device;Processor, it is adapted for carrying out each bar journey Sequence;Storage device, suitable for storing a plurality of program;Described program is above-mentioned based on depth to realize suitable for being loaded by processor and being performed The near edge detection method of degree fusion corrective networks.
Person of ordinary skill in the field can be understood that, for convenience and simplicity of description, foregoing description Storage device, the specific work process of processing unit and relevant explanation, may be referred to the corresponding process in preceding method embodiment, It will not be repeated here.
So far, combined preferred embodiment shown in the drawings describes technical scheme, still, this area Technical staff is it is easily understood that protection scope of the present invention is expressly not limited to these embodiments.Without departing from this On the premise of the principle of invention, those skilled in the art can make equivalent change or replacement to correlation technique feature, these Technical scheme after changing or replacing it is fallen within protection scope of the present invention.

Claims (11)

1. a kind of near edge detection method based on depth integration corrective networks, it is characterised in that comprise the following steps:
Step S1, by the propagated forward subnetwork of convolutional neural networks, obtain the Analysis On Multi-scale Features of input picture;
Step S2, by the reverse amendment subnetwork of convolutional neural networks, the method for gradually increase feature resolution is taken to obtain Take the final image feature that there is equal resolution with input picture;It is described gradually increase feature resolution method be:Fusion While the Analysis On Multi-scale Features of the input picture, the resolution ratio of feature is increased by the method for sub-pixel convolution;
Step S3, it is single channel by the feature passage dimensionality reduction of final image feature, edge detection results is generated by fitting function.
2. near edge detection method according to claim 1, it is characterised in that more chis of input picture described in step S1 Feature is spent, the feature pyramid being made up of the characteristics of image under different scale that the convolutional layer of different depth obtains.
3. near edge detection method according to claim 1, it is characterised in that " take gradually increase special in step S2 The method of sign resolution ratio obtains the final image feature for having equal resolution with input picture ", its method is:
To all features in the feature pyramid, top-down selection adjacent feature, repeat step S21 and step S22, Until obtaining the fusion feature F of first layer feature1′;
Wherein,
Step S21, give feature FnAnd Fn-1, by the less feature F of yardstickn, resolution ratio is increased by the method for sub-pixel convolution Obtain feature Fαn, feature FαnYardstick and Fn-1Unanimously;Wherein feature FnFor the characteristics of image of n-th layer in Analysis On Multi-scale Features;
Step S22:By convolution, concatenation by feature FαnAnd Fn-1Merged, produce (n-1)th layer of fusion feature Fβn-1, And by (n-1)th layer in the feature pyramid of feature replacement it is Fβn-1
4. near edge detection method according to claim 3, it is characterised in that " pass through convolution, splicing in step S22 Operate feature FαnAnd Fn-1Merged ", its method includes:
Step S221, by convolution operation, by FαnAnd Fn-1Dimensionality reduction is carried out, the feature that furthers port number, obtains feature representation F 'αnWith F′n-1, size is respectively wn-1×hn-1×d′αnAnd wn-1×hn-1×d′n-1
Step S222, by F 'αnWith F 'n-1Splicing, generation size is wn-1×hn-1×(d′αn+d′n-1) intermediate features, then to institute State intermediate features and use convolution operation, generation size is wn-1×hn-1×dβn-1(n-1)th layer of fusion feature Fβn-1, and by institute The feature replacement for stating (n-1)th layer in feature pyramid is Fβn-1
5. near edge detection method according to claim 3, it is characterised in that " given feature F described in step S21n And Fn-1, by the less feature F of yardstickn, resolution ratio is increased by the method for sub-pixel convolution and obtains feature Fαn", its method is:
Step S211, by feature FnCarry out multilayer convolution operation, generation new feature Fn", wherein Fn" size be wn×hn× (dn×r×r);Wherein FnSize be wn×hn×dn, r is characterized Fn-1And FnScale size multiple;
Step S212, to new feature Fn" pixel reorganization is carried out in a fixed order, it is rw to obtain sizen×rhn×dnFeature Fαn
6. near edge detection method according to claim 5, it is characterised in that " to new feature F in step S212n" press Pixel reorganization is carried out according to permanent order ", its method is:
By wn×hnD on the two-dimensional coordinate of determinationn× r × r value is divided into r2Part, then by this r2Part is laid into r × r region, Each point in the region contains dnIndividual passage.
7. according to the near edge detection method described in claim any one of 1-6, it is characterised in that convolutional Neural in step S1 Convolutional neural networks are same convolutional neural networks in network and step S2, the convolutional neural networks be by merge it is described before To part of propagation network, it is described reversely amendment subnetwork formed can end-to-end training near edge detect convolution god Through network.
8. near edge detection method according to claim 7, it is characterised in that it is described can end-to-end training fine side The convolutional neural networks of edge detection, its loss function L (w) trained are:
<mrow> <mi>L</mi> <mrow> <mo>(</mo> <mi>w</mi> <mo>)</mo> </mrow> <mo>=</mo> <mo>-</mo> <mi>&amp;beta;</mi> <munder> <mo>&amp;Sigma;</mo> <mrow> <mi>j</mi> <mo>&amp;Element;</mo> <msub> <mi>Y</mi> <mo>+</mo> </msub> </mrow> </munder> <mi>log</mi> <mi>Pr</mi> <mrow> <mo>(</mo> <msub> <mi>y</mi> <mi>j</mi> </msub> <mo>=</mo> <mn>1</mn> <mo>|</mo> <mi>X</mi> <mo>,</mo> <mi>w</mi> <mo>)</mo> </mrow> <mo>-</mo> <mrow> <mo>(</mo> <mn>1</mn> <mo>-</mo> <mi>&amp;beta;</mi> <mo>)</mo> </mrow> <munder> <mo>&amp;Sigma;</mo> <mrow> <mi>j</mi> <mo>&amp;Element;</mo> <msub> <mi>Y</mi> <mo>-</mo> </msub> </mrow> </munder> <mi>log</mi> <mi>Pr</mi> <mrow> <mo>(</mo> <msub> <mi>y</mi> <mi>j</mi> </msub> <mo>=</mo> <mn>0</mn> <mo>|</mo> <mi>X</mi> <mo>,</mo> <mi>w</mi> <mo>)</mo> </mrow> </mrow>
Wherein, X represent input picture, W be propagated forward subnetwork weight, Y+Represent edge pixel point set, Y-Represent non- Edge pixel point set, β are the ratios of non-edge pixels point sum and edge pixel point sum.
9. according to the near edge detection method described in claim any one of 1-6, it is characterised in that be fitted described in step S3 Function is Sigmoid functions.
10. a kind of storage device, wherein being stored with a plurality of program, it is characterised in that described program is suitable to by processor loading simultaneously Perform to realize the near edge detection method based on depth integration corrective networks described in claim any one of 1-9.
11. a kind of processing unit, including
Processor, it is adapted for carrying out each bar program;And
Storage device, suitable for storing a plurality of program;
Characterized in that, described program is suitable to be loaded by processor and performed to realize:
The near edge detection method based on depth integration corrective networks described in claim any one of 1-9.
CN201710666537.5A 2017-08-07 2017-08-07 Near edge detection method, device based on depth integration corrective networks Pending CN107610140A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710666537.5A CN107610140A (en) 2017-08-07 2017-08-07 Near edge detection method, device based on depth integration corrective networks

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710666537.5A CN107610140A (en) 2017-08-07 2017-08-07 Near edge detection method, device based on depth integration corrective networks

Publications (1)

Publication Number Publication Date
CN107610140A true CN107610140A (en) 2018-01-19

Family

ID=61064751

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710666537.5A Pending CN107610140A (en) 2017-08-07 2017-08-07 Near edge detection method, device based on depth integration corrective networks

Country Status (1)

Country Link
CN (1) CN107610140A (en)

Cited By (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108805889A (en) * 2018-05-07 2018-11-13 中国科学院自动化研究所 The fining conspicuousness method for segmenting objects of margin guide and system, equipment
CN108830196A (en) * 2018-05-31 2018-11-16 上海贵和软件技术有限公司 Pedestrian detection method based on feature pyramid network
CN108830790A (en) * 2018-05-16 2018-11-16 宁波大学 It is a kind of based on the fast video super resolution ratio reconstruction method for simplifying convolutional neural networks
CN108898609A (en) * 2018-06-21 2018-11-27 深圳辰视智能科技有限公司 A kind of method for detecting image edge, detection device and computer storage medium
CN109035251A (en) * 2018-06-06 2018-12-18 杭州电子科技大学 One kind being based on the decoded image outline detection method of Analysis On Multi-scale Features
CN109118504A (en) * 2018-07-26 2019-01-01 深圳辰视智能科技有限公司 A kind of method for detecting image edge neural network based, device and its equipment
CN109472801A (en) * 2018-11-22 2019-03-15 廖祥 It is a kind of for multiple dimensioned neuromorphic detection and dividing method
CN109741351A (en) * 2018-12-12 2019-05-10 中国科学院深圳先进技术研究院 A kind of classification responsive type edge detection method based on deep learning
CN109816037A (en) * 2019-01-31 2019-05-28 北京字节跳动网络技术有限公司 The method and apparatus for extracting the characteristic pattern of image
CN110033469A (en) * 2019-04-01 2019-07-19 北京科技大学 A kind of sub-pixel edge detection method and system
CN110399900A (en) * 2019-06-26 2019-11-01 腾讯科技(深圳)有限公司 Method for checking object, device, equipment and medium
WO2019227954A1 (en) * 2018-05-31 2019-12-05 京东方科技集团股份有限公司 Method and apparatus for identifying traffic light signal, and readable medium and electronic device
CN110648316A (en) * 2019-09-07 2020-01-03 创新奇智(成都)科技有限公司 Steel coil end face edge detection algorithm based on deep learning
CN110908566A (en) * 2018-09-18 2020-03-24 珠海格力电器股份有限公司 Information processing method and device
CN110987189A (en) * 2019-11-21 2020-04-10 北京都是科技有限公司 Method, system and device for detecting temperature of target object
CN111582353A (en) * 2020-04-30 2020-08-25 恒睿(重庆)人工智能技术研究院有限公司 Image feature detection method, system, device and medium
JPWO2020225879A1 (en) * 2019-05-08 2020-11-12
JPWO2020225880A1 (en) * 2019-05-08 2020-11-12
US10977798B2 (en) 2018-08-24 2021-04-13 Apple Inc. Direct thin boundary prediction
CN113132755A (en) * 2019-12-31 2021-07-16 北京大学 Extensible man-machine cooperative image coding method and coding system
CN113240584A (en) * 2021-05-11 2021-08-10 上海大学 Multitask gesture picture super-resolution method based on picture edge information
CN113538484A (en) * 2021-07-01 2021-10-22 广西科技大学 Deep-refinement multiple-information nested edge detection method
WO2022267046A1 (en) * 2021-06-25 2022-12-29 京东方科技集团股份有限公司 Un-decimated image processing method and apparatus

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105120130A (en) * 2015-09-17 2015-12-02 京东方科技集团股份有限公司 Image ascending frequency system and training method and image ascending frequency method thereof
CN106340036A (en) * 2016-08-08 2017-01-18 东南大学 Binocular stereoscopic vision-based stereo matching method
CN106898011A (en) * 2017-01-06 2017-06-27 广东工业大学 A kind of method that convolutional neural networks convolution nuclear volume is determined based on rim detection
CN106934765A (en) * 2017-03-14 2017-07-07 长沙全度影像科技有限公司 Panoramic picture fusion method based on depth convolutional neural networks Yu depth information
CN106952229A (en) * 2017-03-15 2017-07-14 桂林电子科技大学 Image super-resolution rebuilding method based on the enhanced modified convolutional network of data

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105120130A (en) * 2015-09-17 2015-12-02 京东方科技集团股份有限公司 Image ascending frequency system and training method and image ascending frequency method thereof
CN106340036A (en) * 2016-08-08 2017-01-18 东南大学 Binocular stereoscopic vision-based stereo matching method
CN106898011A (en) * 2017-01-06 2017-06-27 广东工业大学 A kind of method that convolutional neural networks convolution nuclear volume is determined based on rim detection
CN106934765A (en) * 2017-03-14 2017-07-07 长沙全度影像科技有限公司 Panoramic picture fusion method based on depth convolutional neural networks Yu depth information
CN106952229A (en) * 2017-03-15 2017-07-14 桂林电子科技大学 Image super-resolution rebuilding method based on the enhanced modified convolutional network of data

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
WENZHE SHI.ETC: "Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network", 《2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》 *
YUPEI WANG.ETC: "Deep Crisp Boundaries", 《2017 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》 *

Cited By (35)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108805889B (en) * 2018-05-07 2021-01-08 中国科学院自动化研究所 Edge-guided segmentation method, system and equipment for refined salient objects
CN108805889A (en) * 2018-05-07 2018-11-13 中国科学院自动化研究所 The fining conspicuousness method for segmenting objects of margin guide and system, equipment
CN108830790B (en) * 2018-05-16 2022-09-13 宁波大学 Rapid video super-resolution reconstruction method based on simplified convolutional neural network
CN108830790A (en) * 2018-05-16 2018-11-16 宁波大学 It is a kind of based on the fast video super resolution ratio reconstruction method for simplifying convolutional neural networks
CN108830196A (en) * 2018-05-31 2018-11-16 上海贵和软件技术有限公司 Pedestrian detection method based on feature pyramid network
US11410549B2 (en) 2018-05-31 2022-08-09 Boe Technology Group Co., Ltd. Method, device, readable medium and electronic device for identifying traffic light signal
WO2019227954A1 (en) * 2018-05-31 2019-12-05 京东方科技集团股份有限公司 Method and apparatus for identifying traffic light signal, and readable medium and electronic device
CN109035251A (en) * 2018-06-06 2018-12-18 杭州电子科技大学 One kind being based on the decoded image outline detection method of Analysis On Multi-scale Features
CN109035251B (en) * 2018-06-06 2022-05-27 杭州电子科技大学 Image contour detection method based on multi-scale feature decoding
CN108898609A (en) * 2018-06-21 2018-11-27 深圳辰视智能科技有限公司 A kind of method for detecting image edge, detection device and computer storage medium
CN109118504A (en) * 2018-07-26 2019-01-01 深圳辰视智能科技有限公司 A kind of method for detecting image edge neural network based, device and its equipment
CN109118504B (en) * 2018-07-26 2021-03-30 深圳辰视智能科技有限公司 Image edge detection method, device and equipment based on neural network
US10977798B2 (en) 2018-08-24 2021-04-13 Apple Inc. Direct thin boundary prediction
CN110908566A (en) * 2018-09-18 2020-03-24 珠海格力电器股份有限公司 Information processing method and device
CN109472801A (en) * 2018-11-22 2019-03-15 廖祥 It is a kind of for multiple dimensioned neuromorphic detection and dividing method
CN109741351A (en) * 2018-12-12 2019-05-10 中国科学院深圳先进技术研究院 A kind of classification responsive type edge detection method based on deep learning
CN109816037A (en) * 2019-01-31 2019-05-28 北京字节跳动网络技术有限公司 The method and apparatus for extracting the characteristic pattern of image
CN110033469A (en) * 2019-04-01 2019-07-19 北京科技大学 A kind of sub-pixel edge detection method and system
CN110033469B (en) * 2019-04-01 2021-08-27 北京科技大学 Sub-pixel edge detection method and system
JPWO2020225879A1 (en) * 2019-05-08 2020-11-12
JPWO2020225880A1 (en) * 2019-05-08 2020-11-12
JP7184176B2 (en) 2019-05-08 2022-12-06 日本電気株式会社 Allocation device, method and program
JP7184175B2 (en) 2019-05-08 2022-12-06 日本電気株式会社 Operation unit and operation allocation method
CN110399900A (en) * 2019-06-26 2019-11-01 腾讯科技(深圳)有限公司 Method for checking object, device, equipment and medium
CN110648316A (en) * 2019-09-07 2020-01-03 创新奇智(成都)科技有限公司 Steel coil end face edge detection algorithm based on deep learning
CN110987189A (en) * 2019-11-21 2020-04-10 北京都是科技有限公司 Method, system and device for detecting temperature of target object
CN113132755B (en) * 2019-12-31 2022-04-01 北京大学 Method and system for encoding extensible man-machine cooperative image and method for training decoder
CN113132755A (en) * 2019-12-31 2021-07-16 北京大学 Extensible man-machine cooperative image coding method and coding system
CN111582353B (en) * 2020-04-30 2022-01-21 恒睿(重庆)人工智能技术研究院有限公司 Image feature detection method, system, device and medium
CN111582353A (en) * 2020-04-30 2020-08-25 恒睿(重庆)人工智能技术研究院有限公司 Image feature detection method, system, device and medium
CN113240584A (en) * 2021-05-11 2021-08-10 上海大学 Multitask gesture picture super-resolution method based on picture edge information
CN113240584B (en) * 2021-05-11 2023-04-28 上海大学 Multitasking gesture picture super-resolution method based on picture edge information
WO2022267046A1 (en) * 2021-06-25 2022-12-29 京东方科技集团股份有限公司 Un-decimated image processing method and apparatus
CN113538484B (en) * 2021-07-01 2022-06-10 广西科技大学 Deep-refinement multiple-information nested edge detection method
CN113538484A (en) * 2021-07-01 2021-10-22 广西科技大学 Deep-refinement multiple-information nested edge detection method

Similar Documents

Publication Publication Date Title
CN107610140A (en) Near edge detection method, device based on depth integration corrective networks
CN111047516B (en) Image processing method, image processing device, computer equipment and storage medium
CN109816012A (en) A kind of multiscale target detection method of integrating context information
Köhler et al. Mask-specific inpainting with deep neural networks
US20210073982A1 (en) Medical image processing method and apparatus, electronic device, and storage medium
CN109829855A (en) A kind of super resolution ratio reconstruction method based on fusion multi-level features figure
CN110189255A (en) Method for detecting human face based on hierarchical detection
CN104574347B (en) Satellite in orbit image geometry positioning accuracy evaluation method based on multi- source Remote Sensing Data data
CN110827213A (en) Super-resolution image restoration method based on generation type countermeasure network
CN107133955B (en) A kind of collaboration conspicuousness detection method combined at many levels
CN106600538A (en) Human face super-resolution algorithm based on regional depth convolution neural network
CN109583445A (en) Character image correction processing method, device, equipment and storage medium
CN105657402A (en) Depth map recovery method
CN106067161A (en) A kind of method that image is carried out super-resolution
FR2646729A1 (en) Method and system for manipulating drafts of a symbol image in various dimensions and by various displacements of points in order to improve a numerical character figure on a frame output device
CN112116543B (en) Image restoration method, system and device based on detection type generation framework
CN113160062A (en) Infrared image target detection method, device, equipment and storage medium
CN110490232A (en) Method, apparatus, the equipment, medium of training literal line direction prediction model
CN111652864A (en) Casting defect image generation method for generating countermeasure network based on conditional expression
CN112270366B (en) Micro target detection method based on self-adaptive multi-feature fusion
CN109544694A (en) A kind of augmented reality system actual situation hybrid modeling method based on deep learning
CN110415284A (en) A kind of haplopia color image depth map preparation method and device
Li et al. Line drawing guided progressive inpainting of mural damages
CN107169498A (en) It is a kind of to merge local and global sparse image significance detection method
CN115953330B (en) Texture optimization method, device, equipment and storage medium for virtual scene image

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20180119

WD01 Invention patent application deemed withdrawn after publication