CN110992257A - Remote sensing image sensitive information automatic shielding method and device based on deep learning - Google Patents

Remote sensing image sensitive information automatic shielding method and device based on deep learning Download PDF

Info

Publication number
CN110992257A
CN110992257A CN201911329520.6A CN201911329520A CN110992257A CN 110992257 A CN110992257 A CN 110992257A CN 201911329520 A CN201911329520 A CN 201911329520A CN 110992257 A CN110992257 A CN 110992257A
Authority
CN
China
Prior art keywords
remote sensing
sensing image
sensitive information
image
sample
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201911329520.6A
Other languages
Chinese (zh)
Other versions
CN110992257B (en
Inventor
刘志强
钱晓明
韩冰
许青云
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Aerospace Titan Technology Co ltd
Original Assignee
Beijing Aerospace Titan Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Aerospace Titan Technology Co ltd filed Critical Beijing Aerospace Titan Technology Co ltd
Priority to CN201911329520.6A priority Critical patent/CN110992257B/en
Publication of CN110992257A publication Critical patent/CN110992257A/en
Application granted granted Critical
Publication of CN110992257B publication Critical patent/CN110992257B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/04Context-preserving transformations, e.g. by using an importance map
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/002D [Two Dimensional] image generation
    • G06T11/40Filling a planar surface by adding surface attributes, e.g. colour or texture
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The embodiment of the invention discloses a remote sensing image sensitive information automatic shielding method and device based on deep learning. The method comprises the following steps: acquiring a remote sensing image to be processed; determining coordinate information of sensitive information contained in the remote sensing image to be processed, and converting pixel values of a region corresponding to the coordinate information in the remote sensing image to be processed into preset pixel values to obtain a candidate remote sensing image; inputting the candidate remote sensing image into a model obtained by pre-training to obtain a target remote sensing image obtained by pixel filling of a preset pixel value area in the candidate remote sensing image; the model is obtained by training candidate sample remote sensing images obtained by converting pixel values of preset areas of the sample remote sensing images into preset pixel values according to the sample remote sensing images. By applying the scheme provided by the embodiment of the invention, sensitive information can be ensured not to be disclosed, and the image quality of the issued remote sensing image is not influenced.

Description

Remote sensing image sensitive information automatic shielding method and device based on deep learning
Technical Field
The invention relates to the technical field of image processing, in particular to a remote sensing image sensitive information automatic shielding method and device based on deep learning.
Background
At present, the resolution of remote sensing satellites is higher and higher, and sensitive information which does not meet public requirements, such as special buildings, fixed facilities and the like, may exist in images shot by the remote sensing satellites. For the remote sensing image containing the sensitive information, the remote sensing image can be released after being processed.
In known methods, sensitive information can be masked by reducing the resolution of the image. However, this method may affect other image areas that do not need to be masked, for example, the definition of other image areas may be reduced, and the quality of the distributed remote sensing image may be affected. Therefore, in order to improve the quality of the distributed remote sensing image, a method for shielding the sensitive information of the remote sensing image is urgently needed.
Disclosure of Invention
The invention provides a remote sensing image sensitive information automatic shielding method and device based on deep learning, and aims to improve the quality of a released remote sensing image. The specific technical scheme is as follows.
In a first aspect, an embodiment of the present invention provides an automatic shielding method for sensitive information of remote sensing images based on deep learning, where the method includes:
acquiring a remote sensing image to be processed;
determining coordinate information of sensitive information contained in the remote sensing image to be processed, and converting pixel values of a region corresponding to the coordinate information in the remote sensing image to be processed into preset pixel values to obtain a candidate remote sensing image;
inputting the candidate remote sensing image into a model obtained by pre-training to obtain a target remote sensing image obtained by pixel filling of a preset pixel value area in the candidate remote sensing image; the model is obtained by training candidate sample remote sensing images obtained by converting pixel values of a preset area of each sample remote sensing image into the preset pixel values according to each sample remote sensing image.
Optionally, the training process of the model includes:
constructing an initial model; the initial model comprises a full convolution fill sub-network and a recognizer;
obtaining sample remote sensing images, and converting pixel values of preset areas of the sample remote sensing images into preset pixel values to obtain candidate sample remote sensing images;
inputting each sample remote sensing image and each corresponding candidate sample remote sensing image into the initial model, filling each candidate sample remote sensing image by the full convolution filling sub-network to obtain a target sample remote sensing image, identifying each sample remote sensing image and each target sample remote sensing image by the identifier, and taking the current initial model as the model when the ratio of the number of the remote sensing images identified as the original image by the identifier to the number of the remote sensing images of the filled images is greater than a preset proportional threshold.
Optionally, the full-convolution padding sub-network includes: a convolutional layer, a pooling layer, a void convolutional layer, and a reverse convolutional layer; the step of obtaining the target sample remote sensing image after filling each candidate sample remote sensing image by the full convolution filling sub-network comprises the following steps:
and after the candidate sample remote sensing images pass through a convolution layer, a pooling layer, a cavity convolution layer and a deconvolution layer, dimension reduction is carried out through a zero-padding deconvolution layer, and the target sample remote sensing image is obtained through up-sampling of the convolution layer.
Optionally, the identifier includes a global identifier and a local identifier; the step of identifying each sample remote sensing image and each target sample remote sensing image by the identifier comprises the following steps:
aiming at each sample remote sensing image, the global recognizer recognizes the sample remote sensing image and determines the sample remote sensing image as an original image or a filling image; the local recognizer recognizes the remote sensing image of the preset area in the sample remote sensing image and determines the remote sensing image as an original image or a filling image; superposing the output results of the global recognizer and the local recognizer to be used as the recognition result of the sample remote sensing image;
aiming at each target sample remote sensing image, the global recognizer recognizes the target sample remote sensing image and determines the target sample remote sensing image as an original image or a filling image; the local recognizer recognizes the remote sensing image of the preset area in the target sample remote sensing image and determines the remote sensing image as an original image or a filling image; and superposing the output results of the global recognizer and the local recognizer to be used as the recognition result of the remote sensing image of the target sample.
Optionally, the step of determining the coordinate information of the sensitive information contained in the remote sensing image to be processed includes:
acquiring attribute information and coordinate information of each known sensitive information contained in a sensitive information base, and attribute information and coordinate information of each sensitive information contained in the remote sensing image to be processed;
detecting whether candidate sensitive information matched with the known sensitive information exists in the remote sensing image to be processed or not according to the attribute information and the coordinate information of the known sensitive information and the attribute information and the coordinate information of the sensitive information contained in the remote sensing image to be processed;
if so, determining the coordinate information of the candidate sensitive information as the coordinate information of the sensitive information contained in the remote sensing image to be processed;
if not, inputting the remote sensing image to be processed into a detection network to obtain coordinate information of sensitive information contained in the remote sensing image to be processed; the detection network is obtained by training according to the remote sensing image containing the sensitive information and the remote sensing image not containing the sensitive information.
Optionally, after the to-be-processed remote sensing image is input into a detection network to obtain coordinate information of sensitive information included in the to-be-processed remote sensing image, the method further includes:
and storing the attribute information and the coordinate information of the sensitive information to the sensitive information library.
Optionally, the preset pixel value is 0 or 255.
In a second aspect, an embodiment of the present invention provides an automatic shielding device for sensitive information of remote sensing images based on deep learning, where the device includes:
the to-be-processed image acquisition module is used for acquiring a to-be-processed remote sensing image;
the coordinate information determining module is used for determining coordinate information of sensitive information contained in the remote sensing image to be processed, and converting pixel values of a region corresponding to the coordinate information in the remote sensing image to be processed into preset pixel values to obtain a candidate remote sensing image;
the sensitive information shielding module is used for inputting the candidate remote sensing image into a model obtained by pre-training to obtain a target remote sensing image after pixel filling is carried out on a preset pixel value area in the candidate remote sensing image; the model is obtained by training candidate sample remote sensing images obtained by converting pixel values of a preset area of each sample remote sensing image into the preset pixel values according to each sample remote sensing image.
Optionally, the apparatus further comprises:
the initial model building module is used for building an initial model; the initial model comprises a full convolution fill sub-network and a recognizer;
the sample image acquisition module is used for acquiring sample remote sensing images, and converting pixel values of preset areas of the sample remote sensing images into the preset pixel values to obtain candidate sample remote sensing images;
the model training module is used for inputting each sample remote sensing image and each corresponding candidate sample remote sensing image into the initial model, the full convolution filling sub-network fills each candidate sample remote sensing image to obtain a target sample remote sensing image, the recognizer recognizes each sample remote sensing image and each target sample remote sensing image, and when the ratio of the number of the remote sensing images recognized as original images by the recognizer to the number of the remote sensing images of the filled images is larger than a preset proportion threshold value, the current initial model is used as the model.
Optionally, the full-convolution padding sub-network includes: a convolutional layer, a pooling layer, a void convolutional layer, and a reverse convolutional layer; the neural network training module is specifically configured to:
and after the candidate sample remote sensing images pass through a convolution layer, a pooling layer, a cavity convolution layer and a deconvolution layer, dimension reduction is carried out through a zero-padding deconvolution layer, and the target sample remote sensing image is obtained through up-sampling of the convolution layer.
Optionally, the identifier includes a global identifier and a local identifier; the neural network training module is specifically configured to:
aiming at each sample remote sensing image, the global recognizer recognizes the sample remote sensing image and determines the sample remote sensing image as an original image or a filling image; the local recognizer recognizes the remote sensing image of the preset area in the sample remote sensing image and determines the remote sensing image as an original image or a filling image; superposing the output results of the global recognizer and the local recognizer to be used as the recognition result of the sample remote sensing image;
aiming at each target sample remote sensing image, the global recognizer recognizes the target sample remote sensing image and determines the target sample remote sensing image as an original image or a filling image; the local recognizer recognizes the remote sensing image of the preset area in the target sample remote sensing image and determines the remote sensing image as an original image or a filling image; and superposing the output results of the global recognizer and the local recognizer to be used as the recognition result of the remote sensing image of the target sample.
Optionally, the coordinate information determining module includes:
the information acquisition submodule is used for acquiring attribute information and coordinate information of each known sensitive information contained in a sensitive information base and attribute information and coordinate information of each sensitive information contained in the remote sensing image to be processed;
the sensitive information matching sub-module is used for detecting whether candidate sensitive information matched with the known sensitive information exists in the remote sensing image to be processed or not according to the attribute information and the coordinate information of the known sensitive information and the attribute information and the coordinate information of the sensitive information contained in the remote sensing image to be processed;
the coordinate information determining submodule is used for determining the coordinate information of the candidate sensitive information as the coordinate information of the sensitive information contained in the remote sensing image to be processed when the sensitive information matching submodule detects that the candidate sensitive information matched with the known sensitive information exists in the remote sensing image to be processed;
the sensitive information detection submodule is used for inputting the remote sensing image to be processed into a detection network when the sensitive information matching submodule does not detect that candidate sensitive information matched with the known sensitive information exists in the remote sensing image to be processed, so as to obtain coordinate information of the sensitive information contained in the remote sensing image to be processed; the detection network is obtained by training according to the remote sensing image containing the sensitive information and the remote sensing image not containing the sensitive information.
Optionally, the apparatus further comprises:
and the information storage module is used for storing the attribute information and the coordinate information of the sensitive information to the sensitive information base.
Optionally, the preset pixel value is 0 or 255.
As can be seen from the above, the method and the device for automatically shielding sensitive information of a remote sensing image based on deep learning according to the embodiments of the present invention can remove the sensitive information after determining the coordinate information of the sensitive information in the remote sensing image to be processed, and fill the remote sensing image after removing the sensitive information based on a pre-trained model, that is, only fill a proper picture in the area where the sensitive information is located in the remote sensing image without performing any processing on other areas, thereby not only ensuring that the sensitive information is not disclosed, but also not affecting the image quality of the released remote sensing image. Of course, not all of the advantages described above need to be achieved at the same time in the practice of any one product or method of the invention.
The innovation points of the embodiment of the invention comprise:
1. after determining the coordinate information of the sensitive information in the remote sensing image to be processed, the sensitive information is scratched out, and the remote sensing image with the sensitive information scratched out is filled based on a model obtained by pre-training, namely, only a proper picture is filled in the area where the sensitive information is located in the remote sensing image, and no processing is performed on other areas, so that the sensitive information is not disclosed, and the image quality of the issued remote sensing image is not influenced.
2. The candidate sample remote sensing image is an image obtained by cutting out a certain region in the sample remote sensing image, the correlation between the certain region in the remote sensing image and the surrounding region image can be obtained through the sample remote sensing image and a model obtained by training the candidate sample remote sensing image, therefore, when the sensitive information of the remote sensing image is shielded, the model can fill the sensitive information region according to the image characteristics of the surrounding region of the sensitive information region after the candidate remote sensing image obtained by cutting out the sensitive information is input into the model, the complete remote sensing image is obtained, and the image quality of the released remote sensing image is ensured.
3. The attribute information and the coordinate information of the sensitive information identified by the detection network are stored in the sensitive information base, so that the coordinate information of the identified sensitive information can be directly determined according to the sensitive information base in the subsequent sensitive information shielding process, and the sensitive information shielding efficiency is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It is to be understood that the drawings in the following description are merely exemplary of some embodiments of the invention. For a person skilled in the art, without inventive effort, further figures can be obtained from these figures.
FIG. 1 is a flowchart of a method for automatically shielding sensitive information of remote sensing images based on deep learning according to an embodiment of the present invention;
FIG. 2 is another flowchart of a method for automatically shielding sensitive information of remote sensing images based on deep learning according to an embodiment of the present invention;
FIG. 3 is another flowchart of a method for automatically shielding sensitive information of remote sensing images based on deep learning according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a model training process according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an automatic shielding device for sensitive information of remote sensing images based on deep learning according to an embodiment of the present invention.
Detailed Description
The technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. It is to be understood that the described embodiments are merely a few embodiments of the invention, and not all embodiments. All other embodiments, which can be obtained by a person skilled in the art without inventive effort based on the embodiments of the present invention, are within the scope of the present invention.
It is to be noted that the terms "comprises" and "comprising" and any variations thereof in the embodiments and drawings of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.
The embodiment of the invention discloses a remote sensing image sensitive information automatic shielding method and device based on deep learning, which can improve the quality of a released remote sensing image. The following provides a detailed description of embodiments of the invention.
Fig. 1 is a schematic flow chart of a method for automatically shielding sensitive information of remote sensing images based on deep learning according to an embodiment of the present invention. The method is applied to the electronic equipment. The method specifically comprises the following steps.
S110: and acquiring a remote sensing image to be processed.
The remote sensing image to be processed can be a remote sensing image acquired by a remote sensing satellite. After the remote sensing satellite acquires the remote sensing image, the remote sensing image can be sent to the electronic equipment, and the electronic equipment can take the received remote sensing image as the remote sensing image to be processed.
S120: and determining coordinate information of sensitive information contained in the remote sensing image to be processed, and converting pixel values of a region corresponding to the coordinate information in the remote sensing image to be processed into preset pixel values to obtain a candidate remote sensing image.
The sensitive information is the sensitive information which does not meet the public requirement, in the embodiment of the invention, the sensitive information can also be called as a target, and the remote sensing image can be released only after the sensitive information is shielded. The coordinate information of the sensitive information may be longitude and latitude coordinates thereof.
It can be understood that the remote sensing images shot by the remote sensing satellite are usually marked with coordinate information of each sensitive information. In addition, in some cases, it is known which sensitive information cannot be disclosed in the remote sensing image, and in some cases, the sensitive information that cannot be disclosed in the remote sensing image is unknown.
In the embodiment of the invention, in order to accurately determine the sensitive information in the remote sensing image to be processed, a sensitive information base can be constructed in advance, and the sensitive information base can comprise attribute information and coordinate information of known sensitive information. For example, the sensitive information base may include all known attribute information such as names, colors, purposes and the like which cannot disclose the sensitive information, and longitude and latitude coordinates of each sensitive information.
In one implementation, the process of determining the coordinate information of the sensitive information contained in the remote sensing image to be processed may include the following steps.
S210: and acquiring attribute information and coordinate information of each known sensitive information contained in the sensitive information base and attribute information and coordinate information of each sensitive information contained in the remote sensing image to be processed.
Specifically, the attribute information and the coordinate information of each known sensitive information may be obtained from the corresponding storage location according to the storage location of the sensitive information base. The attribute information of each sensitive information in the remote sensing image to be processed can be obtained by carrying out sensitive information identification on the remote sensing image to be processed; or, the attribute information of each sensitive information can be searched according to the existing data and the coordinate information of each sensitive information in the remote sensing image to be processed.
S220: detecting whether candidate sensitive information matched with the known sensitive information exists in the remote sensing image to be processed or not according to the attribute information and the coordinate information of the known sensitive information and the attribute information and the coordinate information of the sensitive information contained in the remote sensing image to be processed; if yes, step S230 is performed, and if no, step S240 is performed.
The candidate sensitive information matched with the known sensitive information is the sensitive information which is the same as the attribute information and the coordinate information of the known sensitive information.
S230: and determining the coordinate information of the candidate sensitive information as the coordinate information of the sensitive information contained in the remote sensing image to be processed.
When the candidate sensitive information matched with the known sensitive information exists in the remote sensing image to be processed, the candidate sensitive information in the remote sensing image to be processed is shown to be the known sensitive information. That is to say, it is known whether the sensitive information contained in the remote sensing image to be processed can be disclosed, and all the sensitive information in the remote sensing image to be processed can be determined only by the sensitive information base.
In this case, the coordinate information of the candidate sensitive information may be directly determined as the coordinate information of the sensitive information included in the remote sensing image to be processed. That is to say, the candidate sensitive information is sensitive information that does not meet public requirements in the remote sensing image to be processed, and the coordinate information of the candidate sensitive information is coordinate information of known sensitive information that is matched with the candidate sensitive information.
S240: inputting the remote sensing image to be processed into a detection network to obtain coordinate information of sensitive information contained in the remote sensing image to be processed; the detection network is obtained by training according to the remote sensing image containing the sensitive information and the remote sensing image not containing the sensitive information.
When the candidate sensitive information matched with the known sensitive information does not exist in the remote sensing image to be processed, the fact that whether the sensitive information contained in the remote sensing image to be processed can be disclosed or not is unknown is shown, and the sensitive information in the remote sensing image to be processed cannot be determined only through the sensitive information base. In this case, the sensitive information in the remote sensing image to be processed can be detected, and the coordinate information of the sensitive information included in the remote sensing image can be determined.
In the embodiment of the invention, the detection network capable of carrying out two-classification on the sensitive information in the remote sensing image can be obtained by pre-training. That is, the network can identify the sensitive information in the input remote sensing image, and determine that each piece of sensitive information is the sensitive information which does not meet the public standard or the sensitive information which can be disclosed.
Specifically, the detection network may be obtained in advance by training based on the remote sensing image containing the sensitive information and the remote sensing image not containing the sensitive information. For example, a resnet50 network structure scene classification method can be used for performing classification training to obtain a detection network. The basis for completing the detection network training is that the identification result of the remote sensing image containing the sensitive information and the remote sensing image not containing the sensitive information is the same as the result of whether the marked remote sensing images contain the sensitive information or not.
After the remote sensing image to be processed is input into the trained detection network, the detection network can perform sliding prediction on the remote sensing image to be processed in a blocking sliding window mode, and the position corresponding to the sensitive information is detected. Specifically, the detection network can perform convolution pooling on the input image, ensure transmission of effective information in the image by connecting shallow and deep network features, and finally output a binary classification result by 50 layers of transmission training.
In one implementation, after identifying the coordinate information of the sensitive information in the remote sensing image to be processed, the attribute information and the coordinate information of the sensitive information can be stored in a sensitive information base. Therefore, in the subsequent sensitive information shielding process, the coordinate information of the identified sensitive information can be directly determined according to the sensitive information base, and the sensitive information shielding efficiency is improved.
After the coordinate information of the sensitive information is determined, the pixel value of the area corresponding to the coordinate information in the remote sensing image to be processed can be converted into a preset pixel value, and a candidate remote sensing image is obtained. The preset pixel value may be 0 or 255. That is, the sensitive information can be scratched out, and the area where the sensitive information is located can be changed into white or black.
S130: inputting the candidate remote sensing image into a model obtained by pre-training to obtain a target remote sensing image obtained by pixel filling of a preset pixel value area in the candidate remote sensing image; the model is obtained by training candidate sample remote sensing images obtained by converting pixel values of preset areas of the sample remote sensing images into preset pixel values according to the sample remote sensing images.
In the embodiment of the invention, the model can be obtained by training according to the remote sensing images of the samples and the candidate remote sensing images obtained after the pixel values of the preset area of the remote sensing images of the samples are converted into the preset pixel values. The candidate sample remote sensing image is an image obtained by cutting out a certain region in the sample remote sensing image, and the correlation between the certain region in the remote sensing image and the surrounding region image can be obtained through the sample remote sensing image and a model obtained by training the candidate sample remote sensing image.
When the sensitive information of the remote sensing image is shielded, after the candidate remote sensing image with the sensitive information removed is input into the model, the model can fill the sensitive information region according to the image characteristics of the surrounding region of the sensitive information region to obtain a target remote sensing image, namely a complete remote sensing image, after pixel filling is carried out on the preset pixel value region in the candidate remote sensing image, so that the image quality of the released remote sensing image is ensured.
As can be seen from the above, the method for automatically shielding sensitive information of remote sensing images based on deep learning provided by the embodiment of the present invention can remove the sensitive information after determining the coordinate information of the sensitive information in the remote sensing image to be processed, and fill the remote sensing image after removing the sensitive information based on the model obtained by pre-training, that is, only fill a proper picture in the area where the sensitive information is located in the remote sensing image, and do not perform any processing on other areas, thereby not only ensuring that the sensitive information is not disclosed, but also not affecting the image quality of the released remote sensing image.
As an implementation manner of the embodiment of the present invention, as shown in fig. 3, the training process of the model may include:
s310: constructing an initial model; the initial model includes a full convolution fill sub-network and a recognizer.
S320: and obtaining sample remote sensing images, and converting pixel values of preset areas of the sample remote sensing images into preset pixel values to obtain candidate sample remote sensing images.
Each sample remote sensing image may be an image acquired by a remote sensing satellite. The sample remote sensing images may include sensitive information that does not meet public requirements, and may also include sensitive information that can be public, which is not limited in the embodiments of the present invention.
The pixel value of the preset area of each sample remote sensing image is converted into the preset pixel value, namely the content of the preset area of each sample remote sensing image is removed, and an image with the preset area as a single pixel value is obtained and can be called as a candidate sample remote sensing image.
S330: inputting each sample remote sensing image and each corresponding candidate sample remote sensing image into an initial model, filling each candidate sample remote sensing image by a full-convolution filling sub-network to obtain a target sample remote sensing image, identifying each sample remote sensing image and each target sample remote sensing image by an identifier, and taking the current initial model as the model when the ratio of the number of the remote sensing images identified as the original image by the identifier to the number of the remote sensing images of the filled images is greater than a preset proportional threshold.
Specifically, the process of filling each candidate sample remote sensing image by the full convolution filling sub-network may be that after each candidate sample remote sensing image passes through the convolution layer, the pooling layer, the void convolution layer and the deconvolution layer, dimension reduction is performed by the zero-padding deconvolution layer, and the target sample remote sensing image is obtained by upsampling the convolution layer.
The process of identifying the sample remote sensing image and the target sample remote sensing image by the identifier can be that for each sample remote sensing image, the global identifier identifies the sample remote sensing image and determines the sample remote sensing image as an original image or a filling image; the local recognizer recognizes the remote sensing image of a preset area in the sample remote sensing image, namely recognizes the area with the content removed, and determines the area as an original image or a filling image; and superposing the output results of the global recognizer and the local recognizer to obtain the recognition result of the sample remote sensing image.
Aiming at each target sample remote sensing image, the global recognizer recognizes the target sample remote sensing image and determines the target sample remote sensing image as an original image or a filling image; the local recognizer recognizes the remote sensing image of a preset area in the remote sensing image of the target sample, namely recognizes the area with the content removed, and determines the area as an original image or a filling image; and superposing the output results of the global recognizer and the local recognizer to obtain the recognition result of the remote sensing image of the target sample.
In a specific implementation, as shown in fig. 4, a schematic diagram of a model training process in an embodiment of the present invention is shown.
After the sample remote sensing image 410 is obtained, the sample remote sensing image may be copied, and a 0 value is randomly filled in the copied image, so as to obtain a candidate sample remote sensing image 420. Pairs of images can then be placed into the constructed initial model for training.
Aiming at the characteristics of large map range, unfixed ground objects and the like of the high-definition remote sensing image, the sample remote sensing image and the candidate sample remote sensing image can be read into the network for training in a mode of overlapping area and unfixed window size. Wherein, the network structure adopts convolution kernels with different scales.
The model in the embodiment of the invention comprises two parts, wherein the first part is a full convolution filling sub-network 430, the input candidate sample remote sensing image is subjected to convolution, pooling and cavity convolution, dimension reduction is performed by zero padding deconvolution, and finally, a new image is formed by convolution and up-sampling, and the new image can be called as a target sample remote sensing image.
The second part is the recognizer 440, which includes a global recognizer and a local recognizer, the global recognizer inputs the size of the whole image, the local recognizer inputs the region of 0 complement, and after multi-layer convolution pooling, the outputs of the two are finally concatenated, the output result is predicted, and whether the image is an actual image (true) or a filled image (false) is determined.
The candidate sample remote sensing image is an image obtained by cutting out a certain region in the sample remote sensing image, the correlation between the certain region in the remote sensing image and the surrounding region image can be obtained through the sample remote sensing image and a model obtained by training the candidate sample remote sensing image, therefore, when the sensitive information of the remote sensing image is shielded, the model can fill the sensitive information region according to the image characteristics of the surrounding region of the sensitive information region after the candidate remote sensing image obtained by cutting out the sensitive information is input into the model, the complete remote sensing image is obtained, and the image quality of the released remote sensing image is ensured.
As shown in fig. 5, an automatic shielding device for sensitive information of remote sensing images based on deep learning according to an embodiment of the present invention may include:
a to-be-processed image obtaining module 510, configured to obtain a to-be-processed remote sensing image;
a coordinate information determining module 520, configured to determine coordinate information of sensitive information included in the remote sensing image to be processed, and convert a pixel value of a region corresponding to the coordinate information in the remote sensing image to be processed into a preset pixel value to obtain a candidate remote sensing image;
the sensitive information shielding module 530 is configured to input the candidate remote sensing image into a model obtained through pre-training, so as to obtain a target remote sensing image obtained by pixel filling of a preset pixel value region in the candidate remote sensing image; the model is obtained by training candidate sample remote sensing images obtained by converting pixel values of a preset area of each sample remote sensing image into the preset pixel values according to each sample remote sensing image.
Optionally, the apparatus further comprises:
the initial network building module is used for building an initial model; the initial model comprises a full convolution fill sub-network and a recognizer;
the sample image acquisition module is used for acquiring sample remote sensing images, and converting pixel values of preset areas of the sample remote sensing images into the preset pixel values to obtain candidate sample remote sensing images;
the model training module is used for inputting each sample remote sensing image and each corresponding candidate sample remote sensing image into the initial model, the full convolution filling sub-network fills each candidate sample remote sensing image to obtain a target sample remote sensing image, the recognizer recognizes each sample remote sensing image and each target sample remote sensing image, and when the ratio of the number of the remote sensing images recognized as original images by the recognizer to the number of the remote sensing images of the filled images is larger than a preset proportion threshold value, the current initial model is used as the model.
Optionally, the full-convolution padding sub-network includes: a convolutional layer, a pooling layer, a void convolutional layer, and a reverse convolutional layer; the neural network training module is specifically configured to:
and after the candidate sample remote sensing images pass through a convolution layer, a pooling layer, a cavity convolution layer and a deconvolution layer, dimension reduction is carried out through a zero-padding deconvolution layer, and the target sample remote sensing image is obtained through up-sampling of the convolution layer.
Optionally, the identifier includes a global identifier and a local identifier; the neural network training module is specifically configured to:
aiming at each sample remote sensing image, the global recognizer recognizes the sample remote sensing image and determines the sample remote sensing image as an original image or a filling image; the local recognizer recognizes the remote sensing image of the preset area in the sample remote sensing image and determines the remote sensing image as an original image or a filling image; superposing the output results of the global recognizer and the local recognizer to be used as the recognition result of the sample remote sensing image;
aiming at each target sample remote sensing image, the global recognizer recognizes the target sample remote sensing image and determines the target sample remote sensing image as an original image or a filling image; the local recognizer recognizes the remote sensing image of the preset area in the target sample remote sensing image and determines the remote sensing image as an original image or a filling image; and superposing the output results of the global recognizer and the local recognizer to be used as the recognition result of the remote sensing image of the target sample.
Optionally, the coordinate information determining module 520 includes:
the information acquisition submodule is used for acquiring attribute information and coordinate information of each known sensitive information contained in a sensitive information base and attribute information and coordinate information of each sensitive information contained in the remote sensing image to be processed;
the sensitive information matching sub-module is used for detecting whether candidate sensitive information matched with the known sensitive information exists in the remote sensing image to be processed or not according to the attribute information and the coordinate information of the known sensitive information and the attribute information and the coordinate information of the sensitive information contained in the remote sensing image to be processed;
the coordinate information determining submodule is used for determining the coordinate information of the candidate sensitive information as the coordinate information of the sensitive information contained in the remote sensing image to be processed when the sensitive information matching submodule detects that the candidate sensitive information matched with the known sensitive information exists in the remote sensing image to be processed;
the sensitive information detection submodule is used for inputting the remote sensing image to be processed into a detection network when the sensitive information matching submodule does not detect that candidate sensitive information matched with the known sensitive information exists in the remote sensing image to be processed, so as to obtain coordinate information of the sensitive information contained in the remote sensing image to be processed; the detection network is obtained by training according to the remote sensing image containing the sensitive information and the remote sensing image not containing the sensitive information.
Optionally, the apparatus further comprises:
and the information storage module is used for storing the attribute information and the coordinate information of the sensitive information to the sensitive information base.
Optionally, the preset pixel value is 0 or 255.
As can be seen from the above, the automatic shielding device for sensitive information of remote sensing images based on deep learning provided in the embodiments of the present invention can remove the sensitive information after determining the coordinate information of the sensitive information in the remote sensing image to be processed, and fill the remote sensing image after removing the sensitive information based on the model obtained by pre-training, that is, only fill a proper picture in the area where the sensitive information is located in the remote sensing image, and do not perform any processing on other areas, thereby not only ensuring that the sensitive information is not disclosed, but also not affecting the image quality of the remote sensing image that is released.
The above device embodiment corresponds to the method embodiment, and has the same technical effect as the method embodiment, and for the specific description, refer to the method embodiment. The device embodiment is obtained based on the method embodiment, and for specific description, reference may be made to the method embodiment section, which is not described herein again.
Those of ordinary skill in the art will understand that: the figures are merely schematic representations of one embodiment, and the blocks or flow diagrams in the figures are not necessarily required to practice the present invention.
Those of ordinary skill in the art will understand that: modules in the devices in the embodiments may be distributed in the devices in the embodiments according to the description of the embodiments, or may be located in one or more devices different from the embodiments with corresponding changes. The modules of the above embodiments may be combined into one module, or further split into multiple sub-modules.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (10)

1. A remote sensing image sensitive information automatic shielding method based on deep learning is characterized by comprising the following steps:
acquiring a remote sensing image to be processed;
determining coordinate information of sensitive information contained in the remote sensing image to be processed, and converting pixel values of a region corresponding to the coordinate information in the remote sensing image to be processed into preset pixel values to obtain a candidate remote sensing image;
inputting the candidate remote sensing image into a model obtained by pre-training to obtain a target remote sensing image obtained by pixel filling of a preset pixel value area in the candidate remote sensing image; the model is obtained by training candidate sample remote sensing images obtained by converting pixel values of a preset area of each sample remote sensing image into the preset pixel values according to each sample remote sensing image.
2. The method of claim 1, wherein the training process of the model comprises:
constructing an initial model; the initial model comprises a full convolution fill sub-network and a recognizer;
obtaining sample remote sensing images, and converting pixel values of preset areas of the sample remote sensing images into preset pixel values to obtain candidate sample remote sensing images;
inputting each sample remote sensing image and each corresponding candidate sample remote sensing image into the initial model, filling each candidate sample remote sensing image by the full convolution filling sub-network to obtain a target sample remote sensing image, identifying each sample remote sensing image and each target sample remote sensing image by the identifier, and taking the current initial model as the model when the ratio of the number of the remote sensing images identified as the original image by the identifier to the number of the remote sensing images of the filled images is greater than a preset proportional threshold.
3. The method of claim 2, wherein the fully convolved fill subnetwork comprises: a convolutional layer, a pooling layer, a void convolutional layer, and a reverse convolutional layer; the step of obtaining the target sample remote sensing image after filling each candidate sample remote sensing image by the full convolution filling sub-network comprises the following steps:
and after the candidate sample remote sensing images pass through a convolution layer, a pooling layer, a cavity convolution layer and a deconvolution layer, dimension reduction is carried out through a zero-padding deconvolution layer, and the target sample remote sensing image is obtained through up-sampling of the convolution layer.
4. The method of claim 2, wherein the recognizer comprises a global recognizer and a local recognizer; the step of identifying each sample remote sensing image and each target sample remote sensing image by the identifier comprises the following steps:
aiming at each sample remote sensing image, the global recognizer recognizes the sample remote sensing image and determines the sample remote sensing image as an original image or a filling image; the local recognizer recognizes the remote sensing image of the preset area in the sample remote sensing image and determines the remote sensing image as an original image or a filling image; superposing the output results of the global recognizer and the local recognizer to be used as the recognition result of the sample remote sensing image;
aiming at each target sample remote sensing image, the global recognizer recognizes the target sample remote sensing image and determines the target sample remote sensing image as an original image or a filling image; the local recognizer recognizes the remote sensing image of the preset area in the target sample remote sensing image and determines the remote sensing image as an original image or a filling image; and superposing the output results of the global recognizer and the local recognizer to be used as the recognition result of the remote sensing image of the target sample.
5. The method of claim 1, wherein the step of determining coordinate information of sensitive information contained in the remote sensing image to be processed comprises:
acquiring attribute information and coordinate information of each known sensitive information contained in a sensitive information base, and attribute information and coordinate information of each sensitive information contained in the remote sensing image to be processed;
detecting whether candidate sensitive information matched with the known sensitive information exists in the remote sensing image to be processed or not according to the attribute information and the coordinate information of the known sensitive information and the attribute information and the coordinate information of the sensitive information contained in the remote sensing image to be processed;
if so, determining the coordinate information of the candidate sensitive information as the coordinate information of the sensitive information contained in the remote sensing image to be processed;
if not, inputting the remote sensing image to be processed into a detection network to obtain coordinate information of sensitive information contained in the remote sensing image to be processed; the detection network is obtained by training according to the remote sensing image containing the sensitive information and the remote sensing image not containing the sensitive information.
6. The method according to claim 5, wherein after inputting the remote sensing image to be processed into a detection network and obtaining coordinate information of sensitive information contained in the remote sensing image to be processed, the method further comprises:
and storing the attribute information and the coordinate information of the sensitive information to the sensitive information library.
7. The method according to any one of claims 1-6, wherein the predetermined pixel value is 0 or 255.
8. The device for automatically shielding the sensitive information of the remote sensing image based on the deep learning is characterized by comprising the following components:
the to-be-processed image acquisition module is used for acquiring a to-be-processed remote sensing image;
the coordinate information determining module is used for determining coordinate information of sensitive information contained in the remote sensing image to be processed, and converting pixel values of a region corresponding to the coordinate information in the remote sensing image to be processed into preset pixel values to obtain a candidate remote sensing image;
the sensitive information shielding module is used for inputting the candidate remote sensing image into a model obtained by pre-training to obtain a target remote sensing image after pixel filling is carried out on a preset pixel value area in the candidate remote sensing image; the model is obtained by training candidate sample remote sensing images obtained by converting pixel values of a preset area of each sample remote sensing image into the preset pixel values according to each sample remote sensing image.
9. The apparatus of claim 8, further comprising:
the initial model building module is used for building an initial model; the initial model comprises a full convolution fill sub-network and a recognizer;
the sample image acquisition module is used for acquiring sample remote sensing images, and converting pixel values of preset areas of the sample remote sensing images into the preset pixel values to obtain candidate sample remote sensing images;
the model training module is used for inputting each sample remote sensing image and each corresponding candidate sample remote sensing image into the initial model, the full convolution filling sub-network fills each candidate sample remote sensing image to obtain a target sample remote sensing image, the recognizer recognizes each sample remote sensing image and each target sample remote sensing image, and when the ratio of the number of the remote sensing images recognized as original images by the recognizer to the number of the remote sensing images of the filled images is larger than a preset proportion threshold value, the current initial model is used as the model.
10. The apparatus of claim 9, wherein the full convolution fill sub-network comprises: a convolutional layer, a pooling layer, a void convolutional layer, and a reverse convolutional layer; the neural network training module is specifically configured to:
and after the candidate sample remote sensing images pass through a convolution layer, a pooling layer, a cavity convolution layer and a deconvolution layer, dimension reduction is carried out through a zero-padding deconvolution layer, and the target sample remote sensing image is obtained through up-sampling of the convolution layer.
CN201911329520.6A 2019-12-20 2019-12-20 Remote sensing image sensitive information automatic shielding method and device based on deep learning Active CN110992257B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911329520.6A CN110992257B (en) 2019-12-20 2019-12-20 Remote sensing image sensitive information automatic shielding method and device based on deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911329520.6A CN110992257B (en) 2019-12-20 2019-12-20 Remote sensing image sensitive information automatic shielding method and device based on deep learning

Publications (2)

Publication Number Publication Date
CN110992257A true CN110992257A (en) 2020-04-10
CN110992257B CN110992257B (en) 2024-06-14

Family

ID=70073787

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911329520.6A Active CN110992257B (en) 2019-12-20 2019-12-20 Remote sensing image sensitive information automatic shielding method and device based on deep learning

Country Status (1)

Country Link
CN (1) CN110992257B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112732912A (en) * 2020-12-30 2021-04-30 平安科技(深圳)有限公司 Sensitive tendency expression detection method, device, equipment and storage medium

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170076438A1 (en) * 2015-08-31 2017-03-16 Cape Analytics, Inc. Systems and methods for analyzing remote sensing imagery
WO2017071160A1 (en) * 2015-10-28 2017-05-04 深圳大学 Sea-land segmentation method and system for large-size remote-sensing image
CN108109126A (en) * 2018-01-12 2018-06-01 适普远景遥感信息技术(北京)有限公司 A kind of target area filling and method for amalgamation processing based on satellite remote-sensing image
WO2018214195A1 (en) * 2017-05-25 2018-11-29 中国矿业大学 Remote sensing imaging bridge detection method based on convolutional neural network
CN109255334A (en) * 2018-09-27 2019-01-22 中国电子科技集团公司第五十四研究所 Remote sensing image terrain classification method based on deep learning semantic segmentation network
CN109271972A (en) * 2018-11-05 2019-01-25 常熟理工学院 Intelligent image identifying system and method based on natural language understanding and image graphics
CN109446992A (en) * 2018-10-30 2019-03-08 苏州中科天启遥感科技有限公司 Remote sensing image building extracting method and system, storage medium, electronic equipment based on deep learning
CN109934200A (en) * 2019-03-22 2019-06-25 南京信息工程大学 A kind of RGB color remote sensing images cloud detection method of optic and system based on improvement M-Net
CN109961068A (en) * 2017-12-26 2019-07-02 阿里巴巴集团控股有限公司 Image recognition, training, searching method and device and equipment, medium
CN110175538A (en) * 2019-05-10 2019-08-27 国网福建省电力有限公司龙岩供电公司 A kind of substation's Bird's Nest recognition methods and system based on machine learning
CN110287869A (en) * 2019-06-25 2019-09-27 吉林大学 High-resolution remote sensing image Crop classification method based on deep learning
CN110399819A (en) * 2019-07-15 2019-11-01 北京洛斯达数字遥感技术有限公司 A kind of remote sensing image residential block extraction method based on deep learning

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170076438A1 (en) * 2015-08-31 2017-03-16 Cape Analytics, Inc. Systems and methods for analyzing remote sensing imagery
WO2017071160A1 (en) * 2015-10-28 2017-05-04 深圳大学 Sea-land segmentation method and system for large-size remote-sensing image
WO2018214195A1 (en) * 2017-05-25 2018-11-29 中国矿业大学 Remote sensing imaging bridge detection method based on convolutional neural network
CN109961068A (en) * 2017-12-26 2019-07-02 阿里巴巴集团控股有限公司 Image recognition, training, searching method and device and equipment, medium
CN108109126A (en) * 2018-01-12 2018-06-01 适普远景遥感信息技术(北京)有限公司 A kind of target area filling and method for amalgamation processing based on satellite remote-sensing image
CN109255334A (en) * 2018-09-27 2019-01-22 中国电子科技集团公司第五十四研究所 Remote sensing image terrain classification method based on deep learning semantic segmentation network
CN109446992A (en) * 2018-10-30 2019-03-08 苏州中科天启遥感科技有限公司 Remote sensing image building extracting method and system, storage medium, electronic equipment based on deep learning
CN109271972A (en) * 2018-11-05 2019-01-25 常熟理工学院 Intelligent image identifying system and method based on natural language understanding and image graphics
CN109934200A (en) * 2019-03-22 2019-06-25 南京信息工程大学 A kind of RGB color remote sensing images cloud detection method of optic and system based on improvement M-Net
CN110175538A (en) * 2019-05-10 2019-08-27 国网福建省电力有限公司龙岩供电公司 A kind of substation's Bird's Nest recognition methods and system based on machine learning
CN110287869A (en) * 2019-06-25 2019-09-27 吉林大学 High-resolution remote sensing image Crop classification method based on deep learning
CN110399819A (en) * 2019-07-15 2019-11-01 北京洛斯达数字遥感技术有限公司 A kind of remote sensing image residential block extraction method based on deep learning

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
刘文涛;李世华;覃驭楚;: "基于全卷积神经网络的建筑物屋顶自动提取", no. 11 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112732912A (en) * 2020-12-30 2021-04-30 平安科技(深圳)有限公司 Sensitive tendency expression detection method, device, equipment and storage medium
CN112732912B (en) * 2020-12-30 2024-04-09 平安科技(深圳)有限公司 Sensitive trend expression detection method, device, equipment and storage medium

Also Published As

Publication number Publication date
CN110992257B (en) 2024-06-14

Similar Documents

Publication Publication Date Title
CN111738110A (en) Remote sensing image vehicle target detection method based on multi-scale attention mechanism
CN109086668B (en) Unmanned aerial vehicle remote sensing image road information extraction method based on multi-scale generation countermeasure network
CN111222395A (en) Target detection method and device and electronic equipment
CN111914843B (en) Character detection method, system, equipment and storage medium
CN111310770B (en) Target detection method and device
US10373316B2 (en) Images background subtraction for dynamic lighting scenarios
CN112487848A (en) Character recognition method and terminal equipment
CN110020658B (en) Salient object detection method based on multitask deep learning
CN114926766A (en) Identification method and device, equipment and computer readable storage medium
CN111523558A (en) Ship shielding detection method and device based on electronic purse net and electronic equipment
CN110992257B (en) Remote sensing image sensitive information automatic shielding method and device based on deep learning
CN117765485A (en) Vehicle type recognition method, device and equipment based on improved depth residual error network
CN111179245B (en) Image quality detection method, device, electronic equipment and storage medium
CN111339808A (en) Vehicle collision probability prediction method and device, electronic equipment and storage medium
CN112001258B (en) Method, device, equipment and storage medium for identifying on-time arrival of logistics truck
CN111881996A (en) Object detection method, computer device and storage medium
CN116453086A (en) Method and device for identifying traffic sign and electronic equipment
CN112861979B (en) Trademark identification method, trademark identification device, computing equipment and computer storage medium
CN113298102B (en) Training method and device for target classification model
CN115393247A (en) Image processing method, base station feeder line detection method and related device
CN113326891A (en) Method, system and device for detecting small target object
CN108805190B (en) Image processing method and device
CN117934486B (en) Transformer element detection method, device, electronic equipment and storage medium
EP4398205A1 (en) Image anti-spoofing detection method and apparatus
CN116596867B (en) Target detection method for welding image

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant