CN109522807A - Satellite image identifying system, method and electronic equipment based on self-generating feature - Google Patents

Satellite image identifying system, method and electronic equipment based on self-generating feature Download PDF

Info

Publication number
CN109522807A
CN109522807A CN201811227460.2A CN201811227460A CN109522807A CN 109522807 A CN109522807 A CN 109522807A CN 201811227460 A CN201811227460 A CN 201811227460A CN 109522807 A CN109522807 A CN 109522807A
Authority
CN
China
Prior art keywords
layer
network
feature
self
convolutional layer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201811227460.2A
Other languages
Chinese (zh)
Other versions
CN109522807B (en
Inventor
张昱航
阳文斯
叶可江
须成忠
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Institute of Advanced Technology of CAS
Original Assignee
Shenzhen Institute of Advanced Technology of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Institute of Advanced Technology of CAS filed Critical Shenzhen Institute of Advanced Technology of CAS
Priority to CN201811227460.2A priority Critical patent/CN109522807B/en
Publication of CN109522807A publication Critical patent/CN109522807A/en
Application granted granted Critical
Publication of CN109522807B publication Critical patent/CN109522807B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/44Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/30Noise filtering
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/10Terrestrial scenes
    • G06V20/13Satellite images
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/46Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/469Contour-based spatial representations, e.g. vector-coding
    • G06V10/473Contour-based spatial representations, e.g. vector-coding using gradient analysis

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Remote Sensing (AREA)
  • Astronomy & Astrophysics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

This application involves a kind of satellite image identifying system, method and electronic equipments based on self-generating feature.The system comprises: FCN network struction module: for first of VGG network full articulamentum and second full articulamentum to be become first layer convolutional layer and second layer convolutional layer respectively, construct new FCN network;Self-generating character network constructs module: for being based on GAN network struction self-generating character network;Self-generating feature calculation module: for the self-generating character network to be embedded in the FCN network, the characteristic pattern of input image is generated by the self-generating character network, send the characteristic pattern to the first layer convolutional layer and second layer convolutional layer, the first layer convolutional layer and second layer convolutional layer obtain the recognition result of the input image using the characteristic pattern.The application constantly calculates and updates newest image feature by automatically generating character network, realizes the enhancing of data characteristics and the identification of pixel for pixel.

Description

Satellite image identifying system, method and electronic equipment based on self-generating feature
Technical field
The application belongs to image recognition technology field, in particular to a kind of satellite image based on self-generating feature identifies system System, method and electronic equipment.
Background technique
With the continuous development of national economy, the utilization of China's land resources is increasingly eager, but accurately to grasp territory money The land state in source, the algorithm learnt based on traditional feature extracting method and conventional machines cannot be large-scale complex Land resources image make accurate judgement.
In satellite image identification, the object being accurately partitioned into figure is needed, such as: agricultural land, forest cover, building Object, river, road, massif etc..Its boundary of exact picture can provide accurate technical parameter for the exploration of next step.It is existing Technology utilizes traditional Principal Component Analysis, Scale invariant features transform (Scale-invariant feature Transform, SIFT) method, Haar-like characterization method etc. extract the edge feature of satellite image, to reach recognition effect. But these features can regard shallow-layer feature as under deep learning, can not capture more effective high-level characteristic, this Many useful information are just lost, cannot be used in cognitive phase.
At the same time, has 1 [Lin M, Chen Q, Yan S.Network in network [J] .arXiv of document Preprint arXiv:1312.4400,2013.] it is proved current convolutional neural networks (Convolution Neural Network, hereinafter referred CNN) last two layers full articulamentum be converted to convolutional layer be not only able to achieve reduce more parameters with Optimize network, is also able to achieve the recognition effect of pixel for pixel (Pixel to Pixel, hereinafter referred PTP), this is in satellite shadow As being had a very important significance in accurately identifying.
Another part technology has used CNN network to identify, but CNN network is more suitable for object category Identification, is not suitable for this intensive identification, that is, the identification of PTP rank, effect can be poor.And operational network occupies GPU Time is long, and precision is not high.So FCN (Fully Convolutional Network, full convolutional network) network has been used, But even FCN network can carry out dense-pixel identification, since it was there is no optimization was done to satellite image, not adapt to In the identification of satellite image.
In addition, a big difficulty that geodata faces is precisely due to satellite image requirement is directed to the identification of pixel, because wanting Accurately mark off profile, thus picture pretreatment before to carry out artificial element marking, this be it is very time consuming, Over time, this labeled picture is just very rare, and the picture marked that can really use is very limited.Many works thus The method that people in the industry utilizes traditional data to expand carries out increase-volume to existing image.But unfortunately, this data extending is brought Performance boost be very limited, or even under some scenes, the influence of data increase-volume is very little.
Summary of the invention
This application provides a kind of satellite image identifying system, method and electronic equipments based on self-generating feature, it is intended to One of above-mentioned technical problem in the prior art is solved at least to a certain extent.
To solve the above-mentioned problems, this application provides following technical solutions:
A kind of satellite image identifying system based on self-generating feature, comprising:
FCN network struction module: for becoming first of VGG network full articulamentum and second full articulamentum respectively First layer convolutional layer and second layer convolutional layer construct new FCN network;
Self-generating character network constructs module: for being based on GAN network struction self-generating character network;
Self-generating feature calculation module: for the self-generating character network to be embedded in the FCN network, by described Self-generating character network generates the characteristic pattern of input image, sends the characteristic pattern to the first layer convolutional layer and the second layer Convolutional layer, the first layer convolutional layer and second layer convolutional layer obtain the identification knot of the input image using the characteristic pattern Fruit.
The technical solution that the embodiment of the present application is taken further includes operator replacement module, and the operator replacement module is used for will The convolution kernel feature extraction operator of original first convolutional layer replaces with HOG operator in FCN network, passes through the HOG operator Carry out the Boundary characteristic extraction of the input image.
The technical solution that the embodiment of the present application is taken further include: the mode of the HOG operator extraction boundary characteristic are as follows:
Gradient in different directions is calculated first:
Gx(x, y)=H (x+1, y)-H (x-1, y)
Gy(x, y)=H (x, y+1)-H (x, y-1)
In above-mentioned formula, Gx(x, y) indicates the gradient value of horizontal direction, Gy(x, y) indicates the gradient value of vertical direction, H (x, y) indicates the gray value that the pixel represents;
Calculate the gradient magnitude of image current pixel point:
Calculate the gradient direction of current pixel point:
The technical solution that the embodiment of the present application is taken further include: the self-generating character network generates satellite image characteristic pattern Calculation are as follows: after input image, by five convolutional layers original in FCN network carry out image Boundary characteristic extraction, And boundary characteristic is transferred to self-generating character network, the self-generating character network passes through primary full convolutional layer learning characteristic, Then one single pass feature of the feature learnt formation is transferred to the second layer convolutional layer;The second layer convolutional layer Feature and first layer convolutional layer feature common transport to arbiter, minimax process is carried out in arbiter, it is described Result is fed back to self-generating character network by arbiter again, and the self-generating character network is made to be continuously generated new feature, After the iterative steps for reaching setting, the newest feature once generated is sent to the second layer convolutional layer, and be transmitted directly to Full articulamentum, the full articulamentum carry out attribute ballot according to feature, finally obtain the recognition result of input image.
The technical solution that the embodiment of the present application is taken further include: the self-generating character network is embedded into FCN network Being embedded in point includes:
On the one hand feature after layer 5 convolution sends first layer convolutional layer and second layer convolutional layer to, while by the spy Sign is transferred to feature generator, this is first insertion point;
The feature that the feature generator is formed is transferred to second layer convolutional layer, this is second insertion point;
In the case where not up to given threshold, the output result of the second layer convolutional layer is only transferred to arbiter, this Point is embedded in for third.
A kind of another technical solution that the embodiment of the present application is taken are as follows: satellite image identification side based on self-generating feature Method, comprising the following steps:
Step a: by first of VGG network full articulamentum and second full articulamentum become respectively first layer convolutional layer and Second layer convolutional layer constructs new FCN network;
Step b: it is based on GAN network struction self-generating character network;
Step c: the self-generating character network is embedded in the FCN network, raw by the self-generating character network At the characteristic pattern of input image, the characteristic pattern is sent to the first layer convolutional layer and second layer convolutional layer, described first Layer convolutional layer and second layer convolutional layer using the characteristic pattern obtain the recognition result of the input image.
The technical solution that the embodiment of the present application is taken further include: step a further include: by original first in FCN network The convolution kernel feature extraction operator of convolutional layer replaces with HOG operator, and the boundary of the input image is carried out by the HOG operator Feature extraction.
The technical solution that the embodiment of the present application is taken further include: the mode of the HOG operator extraction boundary characteristic are as follows:
Gradient in different directions is calculated first:
Gx(x, y)=H (x+1, y)-H (x-1, y)
Gy(x, y)=H (x, y+1)-H (x, y-1)
In above-mentioned formula, Gx(x, y) indicates the gradient value of horizontal direction, Gy(x, y) indicates the gradient value of vertical direction, H (x, y) indicates the gray value that the pixel represents;
Calculate the gradient magnitude of image current pixel point:
Calculate the gradient direction of current pixel point:
The technical solution that the embodiment of the present application is taken further include: in the step c, the self-generating character network is generated The calculation of satellite image characteristic pattern are as follows: after input image, image is carried out by five convolutional layers original in FCN network Boundary characteristic extraction, and boundary characteristic is transferred to self-generating character network, the self-generating character network is by primary full volume Then one single pass feature of the feature learnt formation is transferred to the second layer convolutional layer by lamination learning characteristic;Institute The feature of second layer convolutional layer and the feature common transport of first layer convolutional layer are stated to arbiter, carries out minimum most in arbiter Result is fed back to self-generating character network by bigization process, the arbiter again, makes the self-generating character network not medium well The feature of Cheng Xin sends the newest feature once generated to the second layer convolutional layer after reaching the iterative steps of setting, And it is transmitted directly to full articulamentum, the full articulamentum carries out attribute ballot according to feature, finally obtains the identification of input image As a result.
The technical solution that the embodiment of the present application is taken further include: in the step c, the self-generating character network insertion Include: to the insertion point in FCN network
On the one hand feature after layer 5 convolution sends first layer convolutional layer and second layer convolutional layer to, while by the spy Sign is transferred to feature generator, this is first insertion point;
The feature that the feature generator is formed is transferred to second layer convolutional layer, this is second insertion point;
In the case where not up to given threshold, the output result of the second layer convolutional layer is only transferred to arbiter, this Point is embedded in for third.
The another technical solution that the embodiment of the present application is taken are as follows: a kind of electronic equipment, comprising:
At least one processor;And
The memory being connect at least one described processor communication;Wherein,
The memory is stored with the instruction that can be executed by one processor, and described instruction is by described at least one It manages device to execute, so that at least one described processor is able to carry out the above-mentioned satellite image recognition methods based on self-generating feature Following operation:
Step a: by first of VGG network full articulamentum and second full articulamentum become respectively first layer convolutional layer and Second layer convolutional layer constructs new FCN network;
Step b: it is based on GAN network struction self-generating character network;
Step c: the self-generating character network is embedded in the FCN network, raw by the self-generating character network At the characteristic pattern of input image, the characteristic pattern is sent to the first layer convolutional layer and second layer convolutional layer, described first Layer convolutional layer and second layer convolutional layer using the characteristic pattern obtain the recognition result of the input image.
Compared with the existing technology, the embodiment of the present application generate beneficial effect be: the embodiment of the present application based on spontaneous At satellite image identifying system, method and the electronic equipment of feature by replacing with the basic convolution nuclear operator in FCN network Multichannel HOG operator, can four general orientation i.e. up and down diagonal line amount to generated on eight directions it is more sensitive Detection effect.The full volume specially designed and first, second layer of VGG network full articulamentum is revised as satellite image detection Product network, while a self-generating character network is added, character network will be automatically generated and be embedded into new full convolutional network, To constantly calculate and update newest image feature, tie down full convolution net without spending many useless redundant datas of generation Network.It can not only realize the enhancing of data characteristics (this is also the content of the data of covert expansion), moreover it is possible to realize pixel to picture The identification of element, to reach the requirement accurately identified.The application avoids expending a large amount of computer resource to realize data extending, energy Enough requirements for solving data redundancy and exact boundary very well, at the same also particular for appearance less in image before or from The satellite mapping not occurred optimizes, so that recognizer has more robustness.
Detailed description of the invention
Fig. 1 is the structural schematic diagram of the satellite image identifying system based on self-generating feature of the embodiment of the present application;
Fig. 2 is the FCN network frame figure of the embodiment of the present application;
Fig. 3 is the network frame figure of the satellite image identifying system based on self-generating feature of the embodiment of the present application;
Fig. 4 is the flow chart of the satellite image recognition methods based on self-generating feature of the embodiment of the present application;
Fig. 5 is the hardware device structure of the satellite image recognition methods provided by the embodiments of the present application based on self-generating feature Schematic diagram.
Specific embodiment
It is with reference to the accompanying drawings and embodiments, right in order to which the objects, technical solutions and advantages of the application are more clearly understood The application is further elaborated.It should be appreciated that specific embodiment described herein is only to explain the application, not For limiting the application.
Aiming at the problems existing in the prior art, the application generates network (Generative by confrontation Adversarial Nets, hereinafter referred GAN) [Goodfellow I, Pouget-Abadie J, Mirza M, et al.Generative adversarial nets[C]//Advances in neural information processing Systems.2014:2672-2680.] thought constructs a kind of satellite image identifying system based on self-generating feature, automatic raw The neural network that neotectonics is then utilized at the feature in satellite image, the neural network for automatically generating feature is embedded into new In full convolutional network, so that it is used to detect the classification of various ground objects in satellite image, can not only realize data characteristics Enhance (this is also the content of the data of covert expansion), moreover it is possible to realize the identification of pixel for pixel, be accurately identified with reaching It is required that.
Specifically, referring to Fig. 1, being the knot of the satellite image identifying system based on self-generating feature of the embodiment of the present application Structure schematic diagram.The satellite image identifying system based on self-generating feature of the embodiment of the present application includes operator replacement module, FCN net Network constructs module, self-generating character network building module and self-generating feature calculation module.
Operator replacement module: for by the SOBEL operator (Sobel Operator) of the first layer convolutional layer in original FCN network HOG is replaced with SIFT (Scale Invariant Feature Transform, scale invariant feature conversion) operator (Histogram of Oriented Gradient, histograms of oriented gradients) operator, remainder layer feature extraction point are constant;Its In, SOBEL operator can only calculate horizontal and vertical both direction, perform poor when in face of some non-horizontal and vertical direction; SIFT operator is primarily used to processing Scale invariant, and does not involve too many dimensional variation in satellite image identification, Only need to consider the detection of image boundary.Horizontal, vertical, four sides in 45 ° of 45 ° of north by east and north by west used herein To, because of reason symmetrical up and down, i.e. totally eight directions.Since HOG operator extends to multiple directions, this Shen The convolution kernel feature extraction operator of first layer convolutional layer is please replaced with into HOG operator, it is made to be more in line with the edge of satellite image Feature detection, can more comprehensively complete the Boundary characteristic extraction of satellite image multiple directions.
In the embodiment of the present application, the direction of HOG operator is determined by the matrix of a 3*3 size, wherein HOG operator calculates The mode of feature are as follows:
1, gradient in different directions is calculated first:
Gx(x, y)=H (x+1, y)-H (x-1, y) (1)
Gy(x, y)=H (x, y+1)-H (x, y-1) (2)
In formula (1), (2), Gx(x, y) indicates the gradient value of horizontal direction, Gy(x, y) indicates the gradient value of vertical direction, H (x, y) indicates the gray value that the pixel represents, in image processing, intensity value ranges all in (0,255), wherein 0 represent it is black Color, 255 indicate white.
(2) gradient magnitude of image current pixel point is calculated:
(3) gradient direction of current pixel point is calculated:
FCN network struction module: for using VGG16 network as bottom frame, by VGG network under the bottom frame First full articulamentum and second full articulamentum all become convolutional layer (increase newly two convolutional layers), construct new intensive The FCN network of pixel identification;Wherein, the advantage for increasing two convolutional layers newly is: if only being carried out using a convolutional layer It samples, then the result up-sampled is likely to be insufficient, is difficult to accomplish precisely to identify in the case where satellite image complexity, Therefore added one layer of connection, can more preferable characteristic feature, and then accurately indicate the attribute of last pixel.The application is real The FCN network structure for applying example is described as follows (CONV# represents the convolutional layer number of plies in table 1) shown in table 1:
Port number is the number for representing convolution kernel in table, this is that the important parameter of network is compared with existing VGG network, this Original two layers full articulamentum is all changed to convolutional layer by application, executes image up-sampling behaviour by two newly-increased convolutional layers Make, gradual perfection image.What last SOFT-MAX was represented is a kind of classification method common in image.Specifically also referring to Fig. 2, is the FCN network frame figure of the embodiment of the present application, and the dotted box portion in figure indicates that the application increases structure newly.
Self-generating character network constructs module: for being based on GAN network struction self-generating character network.Wherein, GAN network It is divided into two parts of generator and arbiter, the basic principle of GAN network are as follows: the effect of generator is from the feature extracted Study, and then many new images similar with existing image are manufactured, then send the new image of manufacture to arbiter, arbiter Judge whether image is the picture manufactured, and under the continuous game of two sides, result is fed back to generation by arbiter Device, generator constantly adjust the study of oneself further according to result, and then produce image more true to nature until arbiter of out-tricking. The application is based on GAN network, but different is not go to manufacture false picture, but construct self-generating character network by GAN, Feature more true to nature is generated, the characteristic pattern of more robust is trained, sends these characteristic patterns to up-sampling layer, up-sampling layer will More accurate satellite image can be identified using these characteristic patterns.
Self-generating character network building mode specifically includes:
In existing x, generator learns a kind of data distribution Pg, simultaneously as there are noise, definition in data distribution One noise profile function: PZ(Z) guarantee function robust.In addition original parameter θ in networkg, so defined G (z, θg) For a mapping of legacy data.Arbiter D (x) is used to indicate probability of the data from x, and training D (x) can be maximum Ability, that is, maximum probability identifies that data come from self training data set or G (x).Simultaneously but also log represented by G (1-D (G (z))) is minimum, this formula innermost layer nesting is generator, minimum to make the formula, then D (the G of internal layer (z)) must be maximum, it is such to be meant that arbiter maximization probability accurately identifies the content from generator.By above two A content combines, and obtains:
Data distribution is become into feature distribution, defining original feature is Of, it is N that generator, which generates feature,f, by formula (5) Change are as follows:
In formula (6), feature Of、NfIt is a single channel i.e. layer network feature.
Self-generating feature calculation module: for self-generating character network to be embedded into the FCN network of dense-pixel identification, Generate the satellite image identifying system based on self-generating feature.
It is the satellite image identifying system based on self-generating feature of the embodiment of the present application specifically also referring to Fig. 3 Network frame figure.The satellite image identification method of satellite image identifying system based on self-generating feature specifically: input satellite After image, firstly, (CONV1 to CONV5) carries out satellite image multiple directions by 5 convolutional layers original in FCN network Boundary characteristic extraction is then communicated to self-generating character network, and the characteristic pattern of satellite image is generated by self-generating character network, Send characteristic pattern to up-sampling layer (i.e. newly-increased NEWCONV1 and NEWCONV2), up-sampling layer is obtained using these characteristic patterns Satellite image recognition result;Wherein, self-generating character network generates the calculation of satellite image characteristic pattern are as follows: original satellite shadow The feature of picture (CONV5) after layer 5 convolution is transmitted to self-generating character network, and self-generating character network is by primary full convolution LayerThen the feature learnt is formed a single pass feature (FEATURE by learning characteristic MAP) it is transferred to newly-increased second layer convolutional layer (NEWCONV2).The feature and first layer of second layer convolutional layer (NEWCONV2) are rolled up Then the feature common transport of lamination (NEWCONV1) carries out minimax process, arbiter to arbiter in arbiter Result is fed back to self-generating character network again, it is allowed to be continuously generated the feature that can more mix the spurious with the genuine.Reaching changing for setting It rides instead of walk after number (i.e. loss function is less than given threshold), newest primary feature is sent to newly-increased second layer convolutional layer (NEWCONV2), it is then transmitted directly to full articulamentum, full articulamentum carries out attribute ballot according to feature, then obtains satellite shadow As recognition result.
In the embodiment of the present application, self-generating character network is embedded into the insertion point packet in the FCN network of dense-pixel identification It includes:
(1), on the one hand the feature after layer 5 convolution sends the newly-increased first layer convolutional layer on original path to (NEWCONV1) and second layer convolutional layer (NEWCONV2), while this feature is transferred to feature generator, this is embedding for first Access point.
(2) feature that feature generator is formed is transferred to newly-increased second layer convolutional layer (NEWCONV2), this is second Insertion point.
(3) it by the output result of second layer convolutional layer (NEWCONV2) in the case where not up to given threshold, only transmits To arbiter, this is third insertion point.
These three insertion points have taken into account the difference of every layer of feature, have also taken into account the work under arbiter and the mutual game of generator Use mechanism.
Referring to Fig. 4, being the flow chart of the satellite image recognition methods based on self-generating feature of the embodiment of the present application.This Apply embodiment the satellite image recognition methods based on self-generating feature the following steps are included:
Step 100: the SOBEL operator of first layer convolutional layer (CONV1) original in FCN network and SIFT operator are replaced For HOG operator, remainder layer feature extraction point is constant;
In step 100, SOBEL operator can only calculate horizontal and vertical both direction, in face of some non-horizontal and vertical It performs poor when direction;SIFT operator is primarily used to processing Scale invariant, and does not involve in satellite image identification Too many dimensional variation, it is only necessary to consider the detection of image boundary.Used herein horizontal, vertical, 45 ° of north by east and north 45 ° of four directions to the west, because of reason symmetrical up and down, i.e. totally eight directions.Due to HOG operator extend to it is multiple Direction, therefore the convolution kernel feature extraction operator of first layer convolutional layer is replaced with HOG operator by the application, is more in line with it and is defended The edge feature of star image detects, and can more comprehensively complete the Boundary characteristic extraction of satellite image multiple directions.
In the embodiment of the present application, the direction of HOG operator is determined by the matrix of a 3*3 size, wherein HOG operator calculates The mode of feature are as follows:
1, gradient in different directions is calculated first:
Gx(x, y)=H (x+1, y)-H (x-1, y) (1)
Gy(x, y)=H (x, y+1)-H (x, y-1) (2)
In formula (1), (2), Gx(x, y) indicates the gradient value of horizontal direction, Gy(x, y) indicates the gradient value of vertical direction, H (x, y) indicates the gray value that the pixel represents, in image processing, intensity value ranges all in (0,255), wherein 0 represent it is black Color, 255 indicate white.
(2) gradient magnitude of image current pixel point is calculated:
(3) gradient direction of current pixel point is calculated:
Step 200: using VGG16 network as bottom frame, connect first of VGG network entirely under the bottom frame Connecing layer and second full articulamentum all becomes convolutional layer (increasing two convolutional layers newly), constructs new dense-pixel identification FCN network;
In step 200, the advantage of newly-increased two convolutional layers is: if only up-sampled using a convolutional layer, The result so up-sampled is likely to be insufficient, is difficult to accomplish precisely to identify in the case where satellite image complexity, therefore Added one layer of connection, can more preferable characteristic feature, and then accurately indicate the attribute of last pixel.The embodiment of the present application FCN network structure be described as follows (CONV# represents the convolutional layer number of plies in table 1) shown in table 1:
Port number is the number for representing convolution kernel in table, this is that the important parameter of network is compared with existing VGG network, this Original two layers full articulamentum is all changed to convolutional layer by application, executes image up-sampling behaviour by two newly-increased convolutional layers Make, gradual perfection image.What last SOFT-MAX was represented is a kind of classification method common in image.
Step 300: being based on GAN network struction self-generating character network;
In step 300, GAN network is divided into two parts of generator and arbiter, the basic principle of GAN network are as follows: generates The effect of device is to learn from the feature extracted, and then manufacture many new images similar with existing image, then will manufacture New image send arbiter to, arbiter judges whether image is the picture manufactured, in the continuous game of two sides Under, result is fed back to generator by arbiter, and generator constantly adjusts the study of oneself further according to result, and then produces more It is image true to nature until arbiter of out-tricking.The application is based on GAN network, but it is different be do not go to manufacture false picture, and It is to construct self-generating character network by GAN, generates feature more true to nature, train the characteristic pattern of more robust, by these spy Sign figure sends up-sampling layer to, and up-sampling layer will identify more accurate satellite image using these characteristic patterns.
Self-generating character network building mode specifically includes:
In existing x, generator learns a kind of data distribution Pg, simultaneously as there are noise, definition in data distribution One noise profile function: PZ(Z) guarantee function robust.In addition original parameter θ in networkg, so defined G (z, θg) For a mapping of legacy data.Arbiter D (x) is used to indicate probability of the data from x, and training D (x) can be maximum Ability, that is, maximum probability identifies that data come from self training data set or G (x).Simultaneously but also log represented by G (1-D (G (z))) is minimum, this formula innermost layer nesting is generator, minimum to make the formula, then D (the G of internal layer (z)) must be maximum, it is such to be meant that arbiter maximization probability accurately identifies the content from generator.By above two A content combines, and obtains:
Data distribution is become into feature distribution, defining original feature is Of, it is N that generator, which generates feature,f, by formula (5) Change are as follows:
In formula (6), feature Of、NfIt is a single channel i.e. layer network feature.
Step 400: self-generating character network being embedded into the FCN network of dense-pixel identification, generate and be based on self-generating The satellite image identifying system of feature carries out satellite image identification by the satellite image identifying system based on self-generating feature;
In step 400, the satellite image identification method of the satellite image identifying system based on self-generating feature specifically: base In the satellite image identification method of the satellite image identifying system of self-generating feature specifically: after input satellite image, firstly, logical Crossing original 5 convolutional layers in FCN network, (CONV1 to CONV5) carries out the Boundary characteristic extraction of satellite image multiple directions, so After be transferred to self-generating character network, by self-generating character network generate satellite image characteristic pattern, characteristic pattern is sent to It up-samples layer (i.e. newly-increased NEWCONV1 and NEWCONV2), up-sampling layer obtains satellite image identification knot using these characteristic patterns Fruit;Wherein, self-generating character network generates the calculation of satellite image characteristic pattern are as follows: the feature of original satellite image is the 5th (CONV5) is transmitted to self-generating character network after layer convolution, and self-generating character network is by primary full convolutional layerThen the feature learnt is formed a single pass feature (FEATURE by learning characteristic MAP) it is transferred to newly-increased second layer convolutional layer (NEWCONV2).The feature and first layer of second layer convolutional layer (NEWCONV2) are rolled up Then the feature common transport of lamination (NEWCONV1) carries out minimax process, arbiter to arbiter in arbiter Result is fed back to self-generating character network again, it is allowed to be continuously generated the feature that can more mix the spurious with the genuine.Reaching changing for setting It rides instead of walk after number (i.e. loss function is less than given threshold), newest primary feature is sent to newly-increased second layer convolutional layer (NEWCONV2), it is then transmitted directly to full articulamentum, full articulamentum carries out attribute ballot according to feature, then obtains satellite shadow As recognition result.
Fig. 5 is the hardware device structure of the satellite image recognition methods provided by the embodiments of the present application based on self-generating feature Schematic diagram.As shown in figure 5, the equipment includes one or more processors and memory.It takes a processor as an example, the equipment It can also include: input system and output system.
Processor, memory, input system and output system can be connected by bus or other modes, in Fig. 5 with For being connected by bus.
Memory as a kind of non-transient computer readable storage medium, can be used for storing non-transient software program, it is non-temporarily State computer executable program and module.Processor passes through operation non-transient software program stored in memory, instruction And module realizes the place of above method embodiment thereby executing the various function application and data processing of electronic equipment Reason method.
Memory may include storing program area and storage data area, wherein storing program area can storage program area, extremely Application program required for a few function;It storage data area can storing data etc..In addition, memory may include that high speed is random Memory is accessed, can also include non-transient memory, a for example, at least disk memory, flush memory device or other are non- Transient state solid-state memory.In some embodiments, it includes the memory remotely located relative to processor that memory is optional, this A little remote memories can pass through network connection to processing system.The example of above-mentioned network includes but is not limited to internet, enterprise Intranet, local area network, mobile radio communication and combinations thereof.
Input system can receive the number or character information of input, and generate signal input.Output system may include showing Display screen etc. shows equipment.
One or more of module storages in the memory, are executed when by one or more of processors When, execute the following operation of any of the above-described embodiment of the method:
Step a: by first of VGG network full articulamentum and second full articulamentum become respectively first layer convolutional layer and Second layer convolutional layer constructs new FCN network;
Step b: it is based on GAN network struction self-generating character network;
Step c: the self-generating character network is embedded in the FCN network, raw by the self-generating character network At the characteristic pattern of input image, the characteristic pattern is sent to the first layer convolutional layer and second layer convolutional layer, described first Layer convolutional layer and second layer convolutional layer using the characteristic pattern obtain the recognition result of the input image.
Method provided by the embodiment of the present application can be performed in the said goods, has the corresponding functional module of execution method and has Beneficial effect.The not technical detail of detailed description in the present embodiment, reference can be made to method provided by the embodiments of the present application.
The embodiment of the present application provides a kind of non-transient (non-volatile) computer storage medium, and the computer storage is situated between Matter is stored with computer executable instructions, the executable following operation of the computer executable instructions:
Step a: by first of VGG network full articulamentum and second full articulamentum become respectively first layer convolutional layer and Second layer convolutional layer constructs new FCN network;
Step b: it is based on GAN network struction self-generating character network;
Step c: the self-generating character network is embedded in the FCN network, raw by the self-generating character network At the characteristic pattern of input image, the characteristic pattern is sent to the first layer convolutional layer and second layer convolutional layer, described first Layer convolutional layer and second layer convolutional layer using the characteristic pattern obtain the recognition result of the input image.
The embodiment of the present application provides a kind of computer program product, and the computer program product is non-temporary including being stored in Computer program on state computer readable storage medium, the computer program include program instruction, when described program instructs When being computer-executed, the computer is made to execute following operation:
Step a: by first of VGG network full articulamentum and second full articulamentum become respectively first layer convolutional layer and Second layer convolutional layer constructs new FCN network;
Step b: it is based on GAN network struction self-generating character network;
Step c: the self-generating character network is embedded in the FCN network, raw by the self-generating character network At the characteristic pattern of input image, the characteristic pattern is sent to the first layer convolutional layer and second layer convolutional layer, described first Layer convolutional layer and second layer convolutional layer using the characteristic pattern obtain the recognition result of the input image.
Satellite image identifying system, method and the electronic equipment based on self-generating feature of the embodiment of the present application pass through by Basic convolution nuclear operator in FCN network replaces with multichannel HOG operator, can be right up and down in four general orientation Linea angulata, which amounts to, generates more sensitive detection effect on eight directions.By the way that first, second layer of VGG network full articulamentum is modified The full convolutional network specially designed for satellite image detection, while a self-generating character network is added, it will automatically generate Character network is embedded into new full convolutional network, to constantly calculate and update newest image feature, generates without spending Many useless redundant datas tie down full convolutional network.It can not only realize that (this is also covert expansion for the enhancing of data characteristics The content for the data filled), moreover it is possible to the identification of pixel for pixel is realized, to reach the requirement accurately identified.The application avoids expending A large amount of computer resource realizes data extending, can solve the requirement of data redundancy and exact boundary, while also spy very well Safety pin to appearance less in image before or the never satellite mapping that occurs optimize so that recognizer has more robust Property.
The foregoing description of the disclosed embodiments makes professional and technical personnel in the field can be realized or use the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, defined herein General Principle can realize in other embodiments without departing from the spirit or scope of the application.Therefore, this Shen These embodiments shown in the application please be not intended to be limited to, and are to fit to special with principle disclosed in the present application and novelty The consistent widest scope of point.

Claims (11)

1. a kind of satellite image identifying system based on self-generating feature characterized by comprising
FCN network struction module: for first of VGG network full articulamentum and second full articulamentum to be become first respectively Layer convolutional layer and second layer convolutional layer, construct new FCN network;
Self-generating character network constructs module: for being based on GAN network struction self-generating character network;
Self-generating feature calculation module: for the self-generating character network to be embedded in the FCN network, by described spontaneous The characteristic pattern that input image is generated at character network, sends the characteristic pattern to the first layer convolutional layer and second layer convolution Layer, the first layer convolutional layer and second layer convolutional layer obtain the recognition result of the input image using the characteristic pattern.
2. the satellite image identifying system according to claim 1 based on self-generating feature, which is characterized in that further include calculating Sub- replacement module, the operator replacement module are used for the convolution kernel feature extraction of first convolutional layer original in FCN network Operator replaces with HOG operator, and the Boundary characteristic extraction of the input image is carried out by the HOG operator.
3. the satellite image identifying system according to claim 2 based on self-generating feature, which is characterized in that the HOG The mode of operator extraction boundary characteristic are as follows:
Gradient in different directions is calculated first:
Gx(x, y)=H (x+1, y)-H (x-1, y)
Gy(x, y)=H (x, y+1)-H (x, y-1)
In above-mentioned formula, Gx(x, y) indicates the gradient value of horizontal direction, Gy(x, y) indicates the gradient value of vertical direction, H (x, y) Indicate the gray value that the pixel represents;
Calculate the gradient magnitude of image current pixel point:
Calculate the gradient direction of current pixel point:
4. the satellite image identifying system according to claim 3 based on self-generating feature, which is characterized in that described spontaneous The calculation of satellite image characteristic pattern is generated at character network are as follows: after input image, pass through five volumes original in FCN network Lamination carries out the Boundary characteristic extraction of image, and boundary characteristic is transferred to self-generating character network, the self-generating feature net Then feature learn is formed a single pass feature and is transferred to described the by network by primary full convolutional layer learning characteristic Two layers of convolutional layer;The feature of the second layer convolutional layer and the feature common transport of first layer convolutional layer are differentiating to arbiter Minimax process is carried out in device, result is fed back to self-generating character network again, makes the self-generating by the arbiter Character network is continuously generated new feature, after reaching the iterative steps of setting, sends the newest feature once generated to institute Second layer convolutional layer is stated, and is transmitted directly to full articulamentum, the full articulamentum carries out attribute ballot according to feature, finally obtains The recognition result of input image.
5. the satellite image identifying system according to claim 4 based on self-generating feature, which is characterized in that described spontaneous Being embedded into the insertion point in FCN network at character network includes:
On the one hand feature after layer 5 convolution sends first layer convolutional layer and second layer convolutional layer to, while this feature being passed It is defeated by feature generator, this is first insertion point;
The feature that the feature generator is formed is transferred to second layer convolutional layer, this is second insertion point;
In the case where not up to given threshold, the output result of the second layer convolutional layer is only transferred to arbiter, this is Three insertion points.
6. a kind of satellite image recognition methods based on self-generating feature, which comprises the following steps:
Step a: first of VGG network full articulamentum and second full articulamentum are become into first layer convolutional layer and second respectively Layer convolutional layer, constructs new FCN network;
Step b: it is based on GAN network struction self-generating character network;
Step c: the self-generating character network is embedded in the FCN network, is generated by the self-generating character network defeated The characteristic pattern for entering image sends the characteristic pattern to the first layer convolutional layer and second layer convolutional layer, the first layer volume Lamination and second layer convolutional layer obtain the recognition result of the input image using the characteristic pattern.
7. the satellite image recognition methods according to claim 6 based on self-generating feature, which is characterized in that step a is also It include: that the convolution kernel feature extraction operator of first convolutional layer original in FCN network is replaced with into HOG operator, by described HOG operator carries out the Boundary characteristic extraction of the input image.
8. the satellite image recognition methods according to claim 7 based on self-generating feature, which is characterized in that the HOG The mode of operator extraction boundary characteristic are as follows:
Gradient in different directions is calculated first:
Gx(x, y)=H (x+1, y)-H (x-1, y)
Gy(x, y)=H (x, y+1)-H (x, y-1)
In above-mentioned formula, Gx(x, y) indicates the gradient value of horizontal direction, Gy(x, y) indicates the gradient value of vertical direction, H (x, y) Indicate the gray value that the pixel represents;
Calculate the gradient magnitude of image current pixel point:
Calculate the gradient direction of current pixel point:
9. the satellite image recognition methods according to claim 8 based on self-generating feature, which is characterized in that in the step In rapid c, the self-generating character network generates the calculation of satellite image characteristic pattern are as follows: after input image, passes through FCN network In original five convolutional layers carry out the Boundary characteristic extraction of image, and boundary characteristic is transferred to self-generating character network, institute Self-generating character network is stated by primary full convolutional layer learning characteristic, the feature learnt is then formed into a single pass spy Sign is transferred to the second layer convolutional layer;The feature of the second layer convolutional layer and the feature common transport of first layer convolutional layer are given Arbiter carries out minimax process in arbiter, and result is fed back to self-generating character network by the arbiter again, So that the self-generating character network is continuously generated new feature, after reaching the iterative steps of setting, is once generated newest Feature sends the second layer convolutional layer to, and is transmitted directly to full articulamentum, and the full articulamentum carries out attribute according to feature Ballot, finally obtains the recognition result of input image.
10. the satellite image recognition methods according to claim 9 based on self-generating feature, which is characterized in that described In step c, the self-generating character network is embedded into the insertion point in FCN network and includes:
On the one hand feature after layer 5 convolution sends first layer convolutional layer and second layer convolutional layer to, while this feature being passed It is defeated by feature generator, this is first insertion point;
The feature that the feature generator is formed is transferred to second layer convolutional layer, this is second insertion point;
In the case where not up to given threshold, the output result of the second layer convolutional layer is only transferred to arbiter, this is Three insertion points.
11. a kind of electronic equipment, comprising:
At least one processor;And
The memory being connect at least one described processor communication;Wherein,
The memory is stored with the instruction that can be executed by one processor, and described instruction is by least one described processor It executes, so that at least one described processor is able to carry out above-mentioned 6 to 10 described in any item satellites based on self-generating feature The following operation of image recognition method:
Step a: first of VGG network full articulamentum and second full articulamentum are become into first layer convolutional layer and second respectively Layer convolutional layer, constructs new FCN network;
Step b: it is based on GAN network struction self-generating character network;
Step c: the self-generating character network is embedded in the FCN network, is generated by the self-generating character network defeated The characteristic pattern for entering image sends the characteristic pattern to the first layer convolutional layer and second layer convolutional layer, the first layer volume Lamination and second layer convolutional layer obtain the recognition result of the input image using the characteristic pattern.
CN201811227460.2A 2018-10-22 2018-10-22 Satellite image recognition system and method based on self-generated features and electronic equipment Active CN109522807B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811227460.2A CN109522807B (en) 2018-10-22 2018-10-22 Satellite image recognition system and method based on self-generated features and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811227460.2A CN109522807B (en) 2018-10-22 2018-10-22 Satellite image recognition system and method based on self-generated features and electronic equipment

Publications (2)

Publication Number Publication Date
CN109522807A true CN109522807A (en) 2019-03-26
CN109522807B CN109522807B (en) 2020-11-27

Family

ID=65772178

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811227460.2A Active CN109522807B (en) 2018-10-22 2018-10-22 Satellite image recognition system and method based on self-generated features and electronic equipment

Country Status (1)

Country Link
CN (1) CN109522807B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110866908A (en) * 2019-11-12 2020-03-06 腾讯科技(深圳)有限公司 Image processing method, image processing apparatus, server, and storage medium
CN112132155A (en) * 2019-06-25 2020-12-25 珠海市一微半导体有限公司 SIFT algorithm image boundary expansion method and processing method based on hardware circuit
CN112446266A (en) * 2019-09-04 2021-03-05 北京君正集成电路股份有限公司 Face recognition network structure suitable for front end
CN114648706A (en) * 2022-05-23 2022-06-21 四川嘉普信工程技术咨询有限公司 Forest tree species identification method, device and equipment based on satellite remote sensing image
US11379722B2 (en) 2019-05-23 2022-07-05 Htc Corporation Method for training generative adversarial network (GAN), method for generating images by using GAN, and computer readable storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105631415A (en) * 2015-12-25 2016-06-01 中通服公众信息产业股份有限公司 Video pedestrian recognition method based on convolution neural network
CN107016677A (en) * 2017-03-24 2017-08-04 北京工业大学 A kind of cloud atlas dividing method based on FCN and CNN
CN108009472A (en) * 2017-10-25 2018-05-08 五邑大学 A kind of finger back arthrosis line recognition methods based on convolutional neural networks and Bayes classifier
CN108537807A (en) * 2017-03-02 2018-09-14 中国科学院遥感与数字地球研究所 One kind enhancing detection method based on the improved edge of No. four satellite images of high score

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105631415A (en) * 2015-12-25 2016-06-01 中通服公众信息产业股份有限公司 Video pedestrian recognition method based on convolution neural network
CN108537807A (en) * 2017-03-02 2018-09-14 中国科学院遥感与数字地球研究所 One kind enhancing detection method based on the improved edge of No. four satellite images of high score
CN107016677A (en) * 2017-03-24 2017-08-04 北京工业大学 A kind of cloud atlas dividing method based on FCN and CNN
CN108009472A (en) * 2017-10-25 2018-05-08 五邑大学 A kind of finger back arthrosis line recognition methods based on convolutional neural networks and Bayes classifier

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11379722B2 (en) 2019-05-23 2022-07-05 Htc Corporation Method for training generative adversarial network (GAN), method for generating images by using GAN, and computer readable storage medium
TWI772805B (en) * 2019-05-23 2022-08-01 宏達國際電子股份有限公司 Method for training generative adversarial network, method for generating image, and computer-readable storage medium
CN112132155A (en) * 2019-06-25 2020-12-25 珠海市一微半导体有限公司 SIFT algorithm image boundary expansion method and processing method based on hardware circuit
CN112132155B (en) * 2019-06-25 2022-06-14 珠海一微半导体股份有限公司 SIFT algorithm image boundary expansion method and processing method based on hardware circuit
CN112446266A (en) * 2019-09-04 2021-03-05 北京君正集成电路股份有限公司 Face recognition network structure suitable for front end
CN112446266B (en) * 2019-09-04 2024-03-29 北京君正集成电路股份有限公司 Face recognition network structure suitable for front end
CN110866908A (en) * 2019-11-12 2020-03-06 腾讯科技(深圳)有限公司 Image processing method, image processing apparatus, server, and storage medium
CN114648706A (en) * 2022-05-23 2022-06-21 四川嘉普信工程技术咨询有限公司 Forest tree species identification method, device and equipment based on satellite remote sensing image

Also Published As

Publication number Publication date
CN109522807B (en) 2020-11-27

Similar Documents

Publication Publication Date Title
CN109522807A (en) Satellite image identifying system, method and electronic equipment based on self-generating feature
JP6976270B2 (en) Remote determination of the amount stored in a container in a geographic area
Kong et al. A federated learning-based license plate recognition scheme for 5G-enabled internet of vehicles
US10157479B2 (en) Synthesizing training data for broad area geospatial object detection
US8379994B2 (en) Digital image analysis utilizing multiple human labels
US11507777B2 (en) Labeling techniques for a modified panoptic labeling neural network
CN103177458B (en) A kind of visible remote sensing image region of interest area detecting method based on frequency-domain analysis
CN106709486A (en) Automatic license plate identification method based on deep convolutional neural network
Zhang et al. Curriculum-style local-to-global adaptation for cross-domain remote sensing image segmentation
CN108776777A (en) The recognition methods of spatial relationship between a kind of remote sensing image object based on Faster RCNN
CN108229274A (en) Multilayer neural network model training, the method and apparatus of roadway characteristic identification
Lou et al. Preliminary investigation on single remote sensing image inpainting through a modified GAN
JP2019185787A (en) Remote determination of containers in geographical region
Khryashchev et al. Deep learning for region detection in high-resolution aerial images
Stomberg et al. Jungle-net: Using explainable machine learning to gain new insights into the appearance of wilderness in satellite imagery
Li et al. An aerial image segmentation approach based on enhanced multi-scale convolutional neural network
CN111310690A (en) Forest fire recognition method and device based on CN and three-channel capsule network
CN104915641B (en) The method that facial image light source orientation is obtained based on Android platform
Wu et al. Real-time vehicle color recognition based on yolo9000
Yu et al. Automatic segmentation of golden pomfret based on fusion of multi-head self-attention and channel-attention mechanism
CN110751163B (en) Target positioning method and device, computer readable storage medium and electronic equipment
Li et al. An improved PCB defect detector based on feature pyramid networks
CN116824485A (en) Deep learning-based small target detection method for camouflage personnel in open scene
CN116543325A (en) Unmanned aerial vehicle image-based crop artificial intelligent automatic identification method and system
Miao et al. Use of colour transformation and the geodesic method for road centreline extraction from VHR satellite images

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant