CN110992257B - Remote sensing image sensitive information automatic shielding method and device based on deep learning - Google Patents
Remote sensing image sensitive information automatic shielding method and device based on deep learning Download PDFInfo
- Publication number
- CN110992257B CN110992257B CN201911329520.6A CN201911329520A CN110992257B CN 110992257 B CN110992257 B CN 110992257B CN 201911329520 A CN201911329520 A CN 201911329520A CN 110992257 B CN110992257 B CN 110992257B
- Authority
- CN
- China
- Prior art keywords
- remote sensing
- sensing image
- image
- sensitive information
- sample
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000013135 deep learning Methods 0.000 title claims abstract description 19
- 238000001514 detection method Methods 0.000 claims description 23
- 238000011176 pooling Methods 0.000 claims description 13
- 238000005070 sampling Methods 0.000 claims description 7
- 238000013528 artificial neural network Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 2
- 239000003086 colorant Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/40—Filling a planar surface by adding surface attributes, e.g. colour or texture
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the invention discloses a remote sensing image sensitive information automatic shielding method and device based on deep learning. The method comprises the following steps: acquiring a remote sensing image to be processed; determining coordinate information of sensitive information contained in the remote sensing image to be processed, and converting pixel values of a region corresponding to the coordinate information in the remote sensing image to be processed into preset pixel values to obtain candidate remote sensing images; inputting the candidate remote sensing images into a model obtained by pre-training to obtain a target remote sensing image after pixel filling of a preset pixel value area in the candidate remote sensing images; the model is obtained by training candidate sample remote sensing images obtained by converting pixel values of preset areas of each sample remote sensing image into preset pixel values according to each sample remote sensing image. By applying the scheme provided by the embodiment of the invention, the sensitive information can be ensured not to be disclosed, and the image quality of the released remote sensing image is not influenced.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to an automatic shielding method and device for remote sensing image sensitive information based on deep learning.
Background
At present, the resolution of the remote sensing satellite is higher and higher, and sensitive information which does not meet the disclosure requirements possibly exists in the image shot by the remote sensing satellite, such as special buildings, fixed facilities and the like. For the remote sensing image containing the sensitive information, the remote sensing image can be issued after being processed.
In known methods, the sensitive information may be masked by reducing the resolution of the image. However, this method may affect other image areas that do not need to be masked, for example, may cause the sharpness of other image areas to be reduced, thereby affecting the quality of the entire remote sensing image being distributed. Therefore, in order to improve the quality of the released remote sensing image, a method for shielding the sensitive information of the remote sensing image is needed.
Disclosure of Invention
The invention provides a remote sensing image sensitive information automatic shielding method and device based on deep learning, which are used for improving the quality of a released remote sensing image. The specific technical scheme is as follows.
In a first aspect, an embodiment of the present invention provides an automatic shielding method for sensitive information of a remote sensing image based on deep learning, where the method includes:
Acquiring a remote sensing image to be processed;
Determining coordinate information of sensitive information contained in the remote sensing image to be processed, and converting pixel values of a region corresponding to the coordinate information in the remote sensing image to be processed into preset pixel values to obtain candidate remote sensing images;
Inputting the candidate remote sensing images into a model obtained by pre-training to obtain a target remote sensing image after pixel filling of a preset pixel value area in the candidate remote sensing images; the model is obtained by training candidate sample remote sensing images obtained by converting pixel values of preset areas of each sample remote sensing image into the preset pixel values according to each sample remote sensing image.
Optionally, the training process of the model includes:
Constructing an initial model; the initial model comprises a full convolution filling sub-network and an identifier;
Acquiring sample remote sensing images, and converting pixel values of preset areas of each sample remote sensing image into preset pixel values to obtain candidate sample remote sensing images;
And inputting each sample remote sensing image and each corresponding candidate sample remote sensing image into the initial model, wherein the full convolution filling sub-network fills each candidate sample remote sensing image to obtain a target sample remote sensing image, the identifier identifies each sample remote sensing image and each target sample remote sensing image, and when the identifier identifies that the ratio of the number of the remote sensing images which are original images to the number of the remote sensing images which are filled images is greater than a preset ratio threshold, the current initial model is used as the model.
Optionally, the full convolution filling sub-network includes: the device comprises a convolution layer, a pooling layer, a cavity convolution layer and a deconvolution layer; the step of filling each candidate sample remote sensing image by the full convolution filling sub-network to obtain a target sample remote sensing image comprises the following steps:
and after each candidate sample remote sensing image passes through the convolution layer, the pooling layer, the cavity convolution layer and the deconvolution layer, performing dimensional reduction through the deconvolution layer with zero filling, and performing up-sampling through the convolution layer to obtain the target sample remote sensing image.
Optionally, the identifier includes a global identifier and a local identifier; the step of identifying each sample remote sensing image and each target sample remote sensing image by the identifier comprises the following steps:
For each sample remote sensing image, the global identifier identifies the sample remote sensing image and determines the sample remote sensing image as an original image or a filling image; the local identifier identifies the remote sensing image of the preset area in the sample remote sensing image, and determines the remote sensing image as an original image or a filling image; superposing the output results of the global identifier and the local identifier to serve as the identification result of the sample remote sensing image;
For each target sample remote sensing image, the global identifier identifies the target sample remote sensing image and determines the target sample remote sensing image as an original image or a filling image; the local identifier identifies the remote sensing image of the preset area in the target sample remote sensing image, and determines the target sample remote sensing image as an original image or a filling image; and superposing the output results of the global identifier and the local identifier to serve as the identification result of the target sample remote sensing image.
Optionally, the step of determining the coordinate information of the sensitive information included in the remote sensing image to be processed includes:
acquiring attribute information and coordinate information of known sensitive information contained in a sensitive information base, and attribute information and coordinate information of sensitive information contained in the remote sensing image to be processed;
Detecting whether candidate sensitive information matched with the known sensitive information exists in the remote sensing image to be processed according to the attribute information and the coordinate information of the known sensitive information and the attribute information and the coordinate information of the sensitive information contained in the remote sensing image to be processed;
if yes, determining the coordinate information of the candidate sensitive information as the coordinate information of the sensitive information contained in the remote sensing image to be processed;
If not, inputting the remote sensing image to be processed into a detection network to obtain coordinate information of sensitive information contained in the remote sensing image to be processed; the detection network is obtained through training according to the remote sensing image containing the sensitive information and the remote sensing image not containing the sensitive information.
Optionally, after the remote sensing image to be processed is input into the detection network to obtain the coordinate information of the sensitive information contained in the remote sensing image to be processed, the method further includes:
And storing attribute information and the coordinate information of the sensitive information into the sensitive information base.
Optionally, the preset pixel value is 0 or 255.
In a second aspect, an embodiment of the present invention provides an automatic remote sensing image sensitive information shielding device based on deep learning, where the device includes:
The image acquisition module to be processed is used for acquiring remote sensing images to be processed;
The coordinate information determining module is used for determining coordinate information of sensitive information contained in the remote sensing image to be processed, and converting pixel values of a region corresponding to the coordinate information in the remote sensing image to be processed into preset pixel values to obtain candidate remote sensing images;
the sensitive information shielding module is used for inputting the candidate remote sensing images into a model obtained by training in advance to obtain a target remote sensing image after pixel filling of a preset pixel value area in the candidate remote sensing images; the model is obtained by training candidate sample remote sensing images obtained by converting pixel values of preset areas of each sample remote sensing image into the preset pixel values according to each sample remote sensing image.
Optionally, the apparatus further includes:
the initial model building module is used for building an initial model; the initial model comprises a full convolution filling sub-network and an identifier;
the sample image acquisition module is used for acquiring sample remote sensing images, converting pixel values of preset areas of each sample remote sensing image into preset pixel values and then obtaining candidate sample remote sensing images;
The model training module is used for inputting each sample remote sensing image and each corresponding candidate sample remote sensing image into the initial model, the full convolution filling sub-network fills each candidate sample remote sensing image to obtain a target sample remote sensing image, the identifier identifies each sample remote sensing image and each target sample remote sensing image, and when the identifier identifies that the ratio of the number of remote sensing images of the original image to the number of remote sensing images of the filled image is greater than a preset ratio threshold, the current initial model is used as the model.
Optionally, the full convolution filling sub-network includes: the device comprises a convolution layer, a pooling layer, a cavity convolution layer and a deconvolution layer; the neural network training module is specifically configured to:
and after each candidate sample remote sensing image passes through the convolution layer, the pooling layer, the cavity convolution layer and the deconvolution layer, performing dimensional reduction through the deconvolution layer with zero filling, and performing up-sampling through the convolution layer to obtain the target sample remote sensing image.
Optionally, the identifier includes a global identifier and a local identifier; the neural network training module is specifically configured to:
For each sample remote sensing image, the global identifier identifies the sample remote sensing image and determines the sample remote sensing image as an original image or a filling image; the local identifier identifies the remote sensing image of the preset area in the sample remote sensing image, and determines the remote sensing image as an original image or a filling image; superposing the output results of the global identifier and the local identifier to serve as the identification result of the sample remote sensing image;
For each target sample remote sensing image, the global identifier identifies the target sample remote sensing image and determines the target sample remote sensing image as an original image or a filling image; the local identifier identifies the remote sensing image of the preset area in the target sample remote sensing image, and determines the target sample remote sensing image as an original image or a filling image; and superposing the output results of the global identifier and the local identifier to serve as the identification result of the target sample remote sensing image.
Optionally, the coordinate information determining module includes:
the information acquisition sub-module is used for acquiring attribute information and coordinate information of each known sensitive information contained in the sensitive information base and attribute information and coordinate information of each sensitive information contained in the remote sensing image to be processed;
the sensitive information matching sub-module is used for detecting whether candidate sensitive information matched with the known sensitive information exists in the remote sensing image to be processed according to the attribute information and the coordinate information of the known sensitive information and the attribute information and the coordinate information of the sensitive information contained in the remote sensing image to be processed;
The coordinate information determining submodule is used for determining the coordinate information of the candidate sensitive information as the coordinate information of the sensitive information contained in the remote sensing image to be processed when the sensitive information matching submodule detects the candidate sensitive information matched with the known sensitive information in the remote sensing image to be processed;
The sensitive information detection sub-module is used for inputting the remote sensing image to be processed into a detection network to obtain coordinate information of sensitive information contained in the remote sensing image to be processed when the sensitive information matching sub-module does not detect candidate sensitive information matched with the known sensitive information in the remote sensing image to be processed; the detection network is obtained through training according to the remote sensing image containing the sensitive information and the remote sensing image not containing the sensitive information.
Optionally, the apparatus further includes:
and the information storage module is used for storing the attribute information and the coordinate information of the sensitive information into the sensitive information base.
Optionally, the preset pixel value is 0 or 255.
As can be seen from the above, the method and the device for automatically shielding the sensitive information of the remote sensing image based on deep learning provided by the embodiments of the present invention can be used for matting out the sensitive information after determining the coordinate information of the sensitive information in the remote sensing image to be processed, and filling the remote sensing image after matting out the sensitive information based on the model obtained by training in advance, that is, only filling a proper picture in the region where the sensitive information is located in the remote sensing image, and not performing any processing on other regions, thereby not only ensuring that the sensitive information is not disclosed, but also not influencing the image quality of the released remote sensing image. Of course, it is not necessary for any one product or method of practicing the invention to achieve all of the advantages set forth above at the same time.
The innovation points of the embodiment of the invention include:
1. After the coordinate information of the sensitive information in the remote sensing image to be processed is determined, the sensitive information is scratched, and the remote sensing image with the sensitive information scratched is filled based on a model obtained by training in advance, namely, a proper picture is filled only in the area where the sensitive information is located in the remote sensing image, and any processing is not carried out on other areas, so that the sensitive information is not disclosed, and the image quality of the released remote sensing image is not influenced.
2. The candidate sample remote sensing image is an image obtained by matting out a certain area in the sample remote sensing image, and the correlation between the certain area in the remote sensing image and the images of the surrounding areas can be obtained through the sample remote sensing image and a model obtained by training the candidate sample remote sensing image, so that when the sensitive information of the remote sensing image is screened, the candidate remote sensing image with the sensitive information scratched out is input into the model, and the model can fill the sensitive information area according to the image characteristics of the surrounding areas of the sensitive information area, so that the complete remote sensing image is obtained, and the image quality of the released remote sensing image is ensured.
3. And storing the attribute information and the coordinate information of the sensitive information identified by the detection network into a sensitive information base, so that the coordinate information of the identified sensitive information can be directly determined according to the sensitive information base in the subsequent sensitive information shielding process, and the efficiency of sensitive information shielding is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It is apparent that the drawings in the following description are only some embodiments of the invention. Other figures may be derived from these figures without inventive effort for a person of ordinary skill in the art.
FIG. 1 is a flowchart of a remote sensing image sensitive information automatic shielding method based on deep learning according to an embodiment of the invention;
FIG. 2 is a flowchart of another method for automatically shielding sensitive information of a remote sensing image based on deep learning according to an embodiment of the present invention;
FIG. 3 is a flowchart of another method for automatically shielding sensitive information of a remote sensing image based on deep learning according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a training process of a model according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an automatic shielding device for sensing information of remote sensing images based on deep learning according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. It will be apparent that the described embodiments are only some, but not all, embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without any inventive effort, are intended to be within the scope of the invention.
It should be noted that the terms "comprising" and "having" and any variations thereof in the embodiments of the present invention and the accompanying drawings are intended to cover non-exclusive inclusions. A process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed but may alternatively include other steps or elements not listed or inherent to such process, method, article, or apparatus.
The embodiment of the invention discloses an automatic shielding method and device for sensitive information of remote sensing images based on deep learning, which can improve the quality of the released remote sensing images. The following describes embodiments of the present invention in detail.
Fig. 1 is a schematic flow chart of an automatic shielding method for remote sensing image sensitive information based on deep learning according to an embodiment of the present invention. The method is applied to the electronic equipment. The method specifically comprises the following steps.
S110: and obtaining the remote sensing image to be processed.
The remote sensing image to be processed can be a remote sensing image acquired by a remote sensing satellite. After the remote sensing satellite collects the remote sensing image, the remote sensing image can be sent to the electronic equipment, and the electronic equipment can take the received remote sensing image as the remote sensing image to be processed.
S120: and determining the coordinate information of the sensitive information contained in the remote sensing image to be processed, and converting the pixel value of the region corresponding to the coordinate information in the remote sensing image to be processed into a preset pixel value to obtain a candidate remote sensing image.
The sensitive information is the sensitive information which does not meet the disclosure requirement, and in the embodiment of the invention, the sensitive information can be also called as a target, and the remote sensing image can be issued after the sensitive information is shielded. The coordinate information of the sensitive information may be its longitude and latitude coordinates.
It can be understood that in the remote sensing image shot by the remote sensing satellite, the coordinate information of each sensitive information is usually marked. In addition, in some cases, it is known which sensitive information cannot be disclosed in the remote sensing image, and in some cases, it is unknown which sensitive information cannot be disclosed in the remote sensing image.
In the embodiment of the invention, in order to accurately determine the sensitive information in the remote sensing image to be processed, a sensitive information base can be constructed in advance, and the sensitive information base can comprise attribute information and coordinate information of known sensitive information. For example, the sensitive information base may include all known attribute information such as names, colors, uses, etc. of the sensitive information, and longitude and latitude coordinates of each sensitive information.
In one implementation, the process of determining the coordinate information of the sensitive information contained in the remote sensing image to be processed may include the following steps.
S210: and acquiring attribute information and coordinate information of each known sensitive information contained in the sensitive information base, and attribute information and coordinate information of each sensitive information contained in the remote sensing image to be processed.
Specifically, attribute information and coordinate information of each known sensitive information may be acquired from the corresponding storage location according to the storage location of the sensitive information library. The attribute information of each sensitive information in the remote sensing image to be processed can be obtained by carrying out sensitive information identification on the remote sensing image to be processed; or the attribute information of each sensitive information can be searched according to the existing data and the coordinate information of each sensitive information in the remote sensing image to be processed.
S220: detecting whether candidate sensitive information matched with the known sensitive information exists in the remote sensing image to be processed according to the attribute information and the coordinate information of each known sensitive information and the attribute information and the coordinate information of each sensitive information contained in the remote sensing image to be processed; if yes, step S230 is performed, and if no, step S240 is performed.
Candidate sensitive information matched with the known sensitive information is the sensitive information which is the same as the attribute information and the coordinate information of the known sensitive information.
S230: and determining the coordinate information of the candidate sensitive information as the coordinate information of the sensitive information contained in the remote sensing image to be processed.
When the candidate sensitive information matched with the known sensitive information exists in the remote sensing image to be processed, the candidate sensitive information in the remote sensing image to be processed is indicated to be the known sensitive information. That is, whether the sensitive information contained in the remote sensing image to be processed can be publicly known or not can be determined, and all the sensitive information in the remote sensing image to be processed can be determined only by the sensitive information base.
In this case, the coordinate information of the candidate sensitive information may be directly determined as the coordinate information of the sensitive information included in the remote sensing image to be processed. That is, the candidate sensitive information is the sensitive information which does not meet the disclosure requirement in the remote sensing image to be processed, and the coordinate information of the candidate sensitive information is the coordinate information of the known sensitive information matched with the candidate sensitive information.
S240: inputting the remote sensing image to be processed into a detection network to obtain coordinate information of sensitive information contained in the remote sensing image to be processed; the detection network is obtained through training according to the remote sensing image containing the sensitive information and the remote sensing image not containing the sensitive information.
When the candidate sensitive information matched with the known sensitive information does not exist in the remote sensing image to be processed, whether the sensitive information contained in the remote sensing image to be processed can be disclosed as unknown or not is indicated, and the sensitive information in the remote sensing image to be processed can not be determined only through the sensitive information base. In this case, the sensitive information in the remote sensing image to be processed can be detected, and the coordinate information of the sensitive information included in the sensitive information can be determined.
In the embodiment of the invention, the detection network capable of carrying out two classifications on the sensitive information in the remote sensing image can be obtained through pre-training. That is, the network can identify the sensitive information in the input remote sensing image, and determine that each sensitive information is not in accordance with the disclosure standard or is a publicly available sensitive information.
Specifically, the detection network can be obtained in advance according to the remote sensing image containing the sensitive information and the remote sensing image not containing the sensitive information. For example, a method of resnet network structure scene classification can be used for classification training to obtain a detection network. The basis for the detection network training is that the recognition results of the detection network training on the remote sensing images containing the sensitive information and the remote sensing images not containing the sensitive information are the same as the result of whether the marked remote sensing images contain the sensitive information or not.
After the remote sensing image to be processed is input into a trained detection network, the detection network can conduct sliding prediction on the remote sensing image to be processed in a block sliding window mode, and the position corresponding to the sensitive information is detected. Specifically, the detection network can convolutionally pool the input image, ensure the transmission of effective information in the image by connecting the shallow layer network characteristics and the deep layer network characteristics, train through the transmission of 50 layers, and finally output a binary classification result.
In one implementation, after identifying the coordinate information of the sensitive information in the remote sensing image to be processed, the attribute information and the coordinate information of the sensitive information may be stored in the sensitive information base. Therefore, in the subsequent sensitive information shielding process, the coordinate information of the identified sensitive information can be directly determined according to the sensitive information base, and the sensitive information shielding efficiency is improved.
After the coordinate information of the sensitive information is determined, the pixel value of the region corresponding to the coordinate information in the remote sensing image to be processed can be converted into a preset pixel value, and a candidate remote sensing image is obtained. The preset pixel value may be 0 or 255. That is, the sensitive information may be scratched out, and the area where the sensitive information is located may be changed to white or black.
S130: inputting the candidate remote sensing images into a model obtained by pre-training to obtain a target remote sensing image after pixel filling of a preset pixel value area in the candidate remote sensing images; the model is obtained by training candidate sample remote sensing images obtained by converting pixel values of preset areas of each sample remote sensing image into preset pixel values according to each sample remote sensing image.
In the embodiment of the invention, the model can be obtained by training the candidate sample remote sensing image after converting the pixel value of the preset area of each sample remote sensing image into the preset pixel value according to each sample remote sensing image in advance. The candidate sample remote sensing image is an image obtained by matting out a certain area in the sample remote sensing image, and the correlation between the certain area in the remote sensing image and the surrounding area image can be obtained through the sample remote sensing image and a model obtained by training the candidate sample remote sensing image.
When the sensitive information of the remote sensing image is shielded, the candidate remote sensing image with the sensitive information removed is input into the model, the model can fill the sensitive information area according to the image characteristics of the surrounding area of the sensitive information area, and the target remote sensing image after the preset pixel value area in the candidate remote sensing image is subjected to pixel filling, namely the complete remote sensing image is obtained, so that the image quality of the released remote sensing image is ensured.
As can be seen from the above, the method for automatically shielding the sensitive information of the remote sensing image based on deep learning provided by the embodiment of the invention can be used for matting out the sensitive information after determining the coordinate information of the sensitive information in the remote sensing image to be processed, and filling the remote sensing image after matting out the sensitive information based on the model obtained by training in advance, namely filling a proper picture only in the region where the sensitive information is located in the remote sensing image, and not performing any processing on other regions, thereby not only ensuring that the sensitive information is not disclosed, but also not influencing the image quality of the released remote sensing image.
As an implementation manner of the embodiment of the present invention, as shown in fig. 3, the training process of the model may include:
s310: constructing an initial model; the initial model includes a full convolution filling sub-network and an identifier.
S320: and obtaining sample remote sensing images, and converting pixel values of preset areas of the sample remote sensing images into preset pixel values to obtain candidate sample remote sensing images.
Each sample remote sensing image can be an image acquired by a remote sensing satellite. The remote sensing images of each sample may contain sensitive information which does not meet the disclosure requirement, or may contain sensitive information which can be disclosed, which is not limited in the embodiment of the invention.
The pixel value of the preset area of each sample remote sensing image is converted into a preset pixel value, namely the content of the preset area of each sample remote sensing image is scratched out, and the image with the preset area as a single pixel value is obtained and can be called as a candidate sample remote sensing image.
S330: and inputting each sample remote sensing image and each corresponding candidate sample remote sensing image into an initial model, filling each candidate sample remote sensing image by a full convolution filling sub-network to obtain a target sample remote sensing image, identifying each sample remote sensing image and each target sample remote sensing image by an identifier, and taking the current initial model as the model when the ratio of the number of the remote sensing images identified as the original images by the identifier to the number of the remote sensing images filled with the images is larger than a preset ratio threshold value.
Specifically, the process of filling each candidate sample remote sensing image by the full convolution filling sub-network may be that after each candidate sample remote sensing image passes through a convolution layer, a pooling layer, a hollow convolution layer and a deconvolution layer, dimension reduction is performed through a deconvolution layer with zero filling, and a target sample remote sensing image is obtained through up-sampling of the convolution layer.
The process of the identifier for identifying the sample remote sensing image and the target sample remote sensing image can be that the global identifier identifies the sample remote sensing image for each sample remote sensing image and determines the sample remote sensing image as an original image or a filling image; the local identifier identifies the remote sensing image of a preset area in the sample remote sensing image, namely identifies the area with the content removed, and determines the area as an original image or a filling image; and superposing the output results of the global identifier and the local identifier to serve as the identification result of the sample remote sensing image.
For each target sample remote sensing image, the global identifier identifies the target sample remote sensing image and determines the target sample remote sensing image as an original image or a filling image; the local identifier identifies the remote sensing image of a preset area in the target sample remote sensing image, namely identifies the area with the content removed, and determines the area as an original image or a filling image; and superposing the output results of the global identifier and the local identifier to serve as the identification result of the target sample remote sensing image.
In a specific implementation, as shown in fig. 4, a schematic diagram of a model training process in an embodiment of the present invention is shown.
After the sample remote sensing image 410 is obtained, the sample remote sensing image may be copied, and the copied image may be randomly filled with 0 values, so as to obtain a candidate sample remote sensing image 420. The paired images can then be put into the built initial model for training.
Aiming at the characteristics of large map range, unfixed ground features and the like of the high-definition remote sensing image, in the embodiment of the invention, the sample remote sensing image and the candidate sample remote sensing image can be read into a network for training in a mode of overlapping area and unfixed window size. Wherein, the network structure adopts convolution kernels with different scales.
The model in the embodiment of the invention comprises two parts, wherein the first part is a full convolution filling sub-network 430, the input candidate sample remote sensing image is subjected to convolution, pooling and cavity convolution, then the dimension reduction is performed by using the deconvolution of zero filling, and finally a new image is formed by convolution up-sampling, which can be called as a target sample remote sensing image.
The second part is a recognizer 440, which includes a global recognizer and a local recognizer, the global recognizer inputs the full-image size, the local recognizer inputs the 0-complement area, after multi-layer convolution pooling, the outputs of the two are finally connected in series, the output result is predicted, and whether the image is an actual image (true) or a filled image (false) is judged.
The candidate sample remote sensing image is an image obtained by matting out a certain area in the sample remote sensing image, and the correlation between the certain area in the remote sensing image and the images of the surrounding areas can be obtained through the sample remote sensing image and a model obtained by training the candidate sample remote sensing image, so that when the sensitive information of the remote sensing image is screened, the candidate remote sensing image with the sensitive information scratched out is input into the model, and the model can fill the sensitive information area according to the image characteristics of the surrounding areas of the sensitive information area, so that the complete remote sensing image is obtained, and the image quality of the released remote sensing image is ensured.
As shown in fig. 5, the device for automatically shielding sensitive information of remote sensing images based on deep learning according to the embodiment of the present invention may include:
the to-be-processed image obtaining module 510 is configured to obtain a to-be-processed remote sensing image;
the coordinate information determining module 520 is configured to determine coordinate information of sensitive information included in the remote sensing image to be processed, and convert a pixel value of a region corresponding to the coordinate information in the remote sensing image to be processed into a preset pixel value, so as to obtain a candidate remote sensing image;
the sensitive information shielding module 530 is configured to input the candidate remote sensing image into a model obtained by training in advance, so as to obtain a target remote sensing image after pixel filling is performed on a preset pixel value area in the candidate remote sensing image; the model is obtained by training candidate sample remote sensing images obtained by converting pixel values of preset areas of each sample remote sensing image into the preset pixel values according to each sample remote sensing image.
Optionally, the apparatus further includes:
the initial network construction module is used for constructing an initial model; the initial model comprises a full convolution filling sub-network and an identifier;
the sample image acquisition module is used for acquiring sample remote sensing images, converting pixel values of preset areas of each sample remote sensing image into preset pixel values and then obtaining candidate sample remote sensing images;
The model training module is used for inputting each sample remote sensing image and each corresponding candidate sample remote sensing image into the initial model, the full convolution filling sub-network fills each candidate sample remote sensing image to obtain a target sample remote sensing image, the identifier identifies each sample remote sensing image and each target sample remote sensing image, and when the identifier identifies that the ratio of the number of remote sensing images of the original image to the number of remote sensing images of the filled image is greater than a preset ratio threshold, the current initial model is used as the model.
Optionally, the full convolution filling sub-network includes: the device comprises a convolution layer, a pooling layer, a cavity convolution layer and a deconvolution layer; the neural network training module is specifically configured to:
and after each candidate sample remote sensing image passes through the convolution layer, the pooling layer, the cavity convolution layer and the deconvolution layer, performing dimensional reduction through the deconvolution layer with zero filling, and performing up-sampling through the convolution layer to obtain the target sample remote sensing image.
Optionally, the identifier includes a global identifier and a local identifier; the neural network training module is specifically configured to:
For each sample remote sensing image, the global identifier identifies the sample remote sensing image and determines the sample remote sensing image as an original image or a filling image; the local identifier identifies the remote sensing image of the preset area in the sample remote sensing image, and determines the remote sensing image as an original image or a filling image; superposing the output results of the global identifier and the local identifier to serve as the identification result of the sample remote sensing image;
For each target sample remote sensing image, the global identifier identifies the target sample remote sensing image and determines the target sample remote sensing image as an original image or a filling image; the local identifier identifies the remote sensing image of the preset area in the target sample remote sensing image, and determines the target sample remote sensing image as an original image or a filling image; and superposing the output results of the global identifier and the local identifier to serve as the identification result of the target sample remote sensing image.
Optionally, the coordinate information determining module 520 includes:
the information acquisition sub-module is used for acquiring attribute information and coordinate information of each known sensitive information contained in the sensitive information base and attribute information and coordinate information of each sensitive information contained in the remote sensing image to be processed;
the sensitive information matching sub-module is used for detecting whether candidate sensitive information matched with the known sensitive information exists in the remote sensing image to be processed according to the attribute information and the coordinate information of the known sensitive information and the attribute information and the coordinate information of the sensitive information contained in the remote sensing image to be processed;
The coordinate information determining submodule is used for determining the coordinate information of the candidate sensitive information as the coordinate information of the sensitive information contained in the remote sensing image to be processed when the sensitive information matching submodule detects the candidate sensitive information matched with the known sensitive information in the remote sensing image to be processed;
The sensitive information detection sub-module is used for inputting the remote sensing image to be processed into a detection network to obtain coordinate information of sensitive information contained in the remote sensing image to be processed when the sensitive information matching sub-module does not detect candidate sensitive information matched with the known sensitive information in the remote sensing image to be processed; the detection network is obtained through training according to the remote sensing image containing the sensitive information and the remote sensing image not containing the sensitive information.
Optionally, the apparatus further includes:
and the information storage module is used for storing the attribute information and the coordinate information of the sensitive information into the sensitive information base.
Optionally, the preset pixel value is 0 or 255.
As can be seen from the above, the remote sensing image sensitive information automatic shielding device based on deep learning provided by the embodiment of the invention can be used for matting out the sensitive information after determining the coordinate information of the sensitive information in the remote sensing image to be processed, and filling the remote sensing image after matting out the sensitive information based on the model obtained by training in advance, namely filling a proper picture only in the region where the sensitive information is located in the remote sensing image, and not performing any processing on other regions, thereby not only ensuring that the sensitive information is not disclosed, but also not influencing the image quality of the released remote sensing image.
The device embodiment corresponds to the method embodiment, and has the same technical effects as the method embodiment, and the specific description refers to the method embodiment. The apparatus embodiments are based on the method embodiments, and specific descriptions may be referred to in the method embodiment section, which is not repeated herein.
Those of ordinary skill in the art will appreciate that: the drawing is a schematic diagram of one embodiment and the modules or flows in the drawing are not necessarily required to practice the invention.
Those of ordinary skill in the art will appreciate that: the modules in the apparatus of the embodiments may be distributed in the apparatus of the embodiments according to the description of the embodiments, or may be located in one or more apparatuses different from the present embodiments with corresponding changes. The modules of the above embodiments may be combined into one module, or may be further split into a plurality of sub-modules.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (7)
1. The method for automatically shielding the sensitive information of the remote sensing image based on the deep learning is characterized by comprising the following steps of:
Acquiring a remote sensing image to be processed;
Determining coordinate information of sensitive information contained in the remote sensing image to be processed, and converting pixel values of a region corresponding to the coordinate information in the remote sensing image to be processed into preset pixel values to obtain candidate remote sensing images;
Inputting the candidate remote sensing images into a model obtained by pre-training to obtain a target remote sensing image after pixel filling of a preset pixel value area in the candidate remote sensing images; the model is obtained by training candidate sample remote sensing images obtained by converting pixel values of preset areas of each sample remote sensing image into preset pixel values according to each sample remote sensing image; the model comprises a full convolution filling sub-network and a recognizer, wherein the full convolution filling sub-network is used for filling each candidate sample remote sensing image to obtain a target sample remote sensing image, and the recognizer is used for recognizing each sample remote sensing image and each target sample remote sensing image;
Wherein the full convolution stuffing sub-network comprises: the device comprises a convolution layer, a pooling layer, a cavity convolution layer and a deconvolution layer; the full convolution filling sub-network is specifically used for: after each candidate sample remote sensing image passes through a convolution layer, a pooling layer, a cavity convolution layer and a deconvolution layer, performing dimension reduction through the deconvolution layer with zero filling, and performing up-sampling through the convolution layer to obtain a target sample remote sensing image;
The identifier comprises a global identifier and a local identifier; the identifier is particularly for: for each sample remote sensing image, the global identifier identifies the sample remote sensing image and determines the sample remote sensing image as an original image or a filling image; the local identifier identifies the remote sensing image of a preset area in the sample remote sensing image, and determines the remote sensing image as an original image or a filling image; superposing the output results of the global identifier and the local identifier to serve as the identification result of the sample remote sensing image; for each target sample remote sensing image, the global identifier identifies the target sample remote sensing image and determines the target sample remote sensing image as an original image or a filling image; the local identifier identifies the remote sensing image of the preset area in the target sample remote sensing image, and determines the target sample remote sensing image as an original image or a filling image; and superposing the output results of the global identifier and the local identifier to serve as the identification result of the target sample remote sensing image.
2. The method of claim 1, wherein the training process of the model comprises:
Constructing an initial model;
Acquiring sample remote sensing images, and converting pixel values of preset areas of each sample remote sensing image into preset pixel values to obtain candidate sample remote sensing images;
And inputting each sample remote sensing image and each corresponding candidate sample remote sensing image into the initial model, wherein the full convolution filling sub-network fills each candidate sample remote sensing image to obtain a target sample remote sensing image, the identifier identifies each sample remote sensing image and each target sample remote sensing image, and when the identifier identifies that the ratio of the number of the remote sensing images which are original images to the number of the remote sensing images which are filled images is greater than a preset ratio threshold, the current initial model is used as the model.
3. The method of claim 1, wherein the step of determining the coordinate information of the sensitive information contained in the remote sensing image to be processed comprises:
acquiring attribute information and coordinate information of known sensitive information contained in a sensitive information base, and attribute information and coordinate information of sensitive information contained in the remote sensing image to be processed;
Detecting whether candidate sensitive information matched with the known sensitive information exists in the remote sensing image to be processed according to the attribute information and the coordinate information of the known sensitive information and the attribute information and the coordinate information of the sensitive information contained in the remote sensing image to be processed;
if yes, determining the coordinate information of the candidate sensitive information as the coordinate information of the sensitive information contained in the remote sensing image to be processed;
If not, inputting the remote sensing image to be processed into a detection network to obtain coordinate information of sensitive information contained in the remote sensing image to be processed; the detection network is obtained through training according to the remote sensing image containing the sensitive information and the remote sensing image not containing the sensitive information.
4. The method according to claim 3, wherein after inputting the remote sensing image to be processed into a detection network to obtain the coordinate information of the sensitive information included in the remote sensing image to be processed, the method further comprises:
And storing attribute information and the coordinate information of the sensitive information into the sensitive information base.
5. The method according to any one of claims 1-4, wherein the preset pixel value is 0 or 255.
6. An automatic shielding device for sensitive information of remote sensing images based on deep learning, which is characterized by comprising:
The image acquisition module to be processed is used for acquiring remote sensing images to be processed;
The coordinate information determining module is used for determining coordinate information of sensitive information contained in the remote sensing image to be processed, and converting pixel values of a region corresponding to the coordinate information in the remote sensing image to be processed into preset pixel values to obtain candidate remote sensing images;
The sensitive information shielding module is used for inputting the candidate remote sensing images into a model obtained by training in advance to obtain a target remote sensing image after pixel filling of a preset pixel value area in the candidate remote sensing images; the model is obtained by training candidate sample remote sensing images obtained by converting pixel values of preset areas of each sample remote sensing image into preset pixel values according to each sample remote sensing image; the model comprises a full convolution filling sub-network and a recognizer, wherein the full convolution filling sub-network is used for filling each candidate sample remote sensing image to obtain a target sample remote sensing image, and the recognizer is used for recognizing each sample remote sensing image and each target sample remote sensing image;
Wherein the full convolution stuffing sub-network comprises: the device comprises a convolution layer, a pooling layer, a cavity convolution layer and a deconvolution layer; the full convolution filling sub-network is specifically used for: after each candidate sample remote sensing image passes through a convolution layer, a pooling layer, a cavity convolution layer and a deconvolution layer, performing dimension reduction through the deconvolution layer with zero filling, and performing up-sampling through the convolution layer to obtain a target sample remote sensing image;
The identifier comprises a global identifier and a local identifier; the identifier is particularly for: for each sample remote sensing image, the global identifier identifies the sample remote sensing image and determines the sample remote sensing image as an original image or a filling image; the local identifier identifies the remote sensing image of a preset area in the sample remote sensing image, and determines the remote sensing image as an original image or a filling image; superposing the output results of the global identifier and the local identifier to serve as the identification result of the sample remote sensing image; for each target sample remote sensing image, the global identifier identifies the target sample remote sensing image and determines the target sample remote sensing image as an original image or a filling image; the local identifier identifies the remote sensing image of the preset area in the target sample remote sensing image, and determines the target sample remote sensing image as an original image or a filling image; and superposing the output results of the global identifier and the local identifier to serve as the identification result of the target sample remote sensing image.
7. The apparatus of claim 6, wherein the apparatus further comprises:
The initial model building module is used for building an initial model;
the sample image acquisition module is used for acquiring sample remote sensing images, converting pixel values of preset areas of each sample remote sensing image into preset pixel values and then obtaining candidate sample remote sensing images;
The model training module is used for inputting each sample remote sensing image and each corresponding candidate sample remote sensing image into the initial model, the full convolution filling sub-network fills each candidate sample remote sensing image to obtain a target sample remote sensing image, the identifier identifies each sample remote sensing image and each target sample remote sensing image, and when the identifier identifies that the ratio of the number of remote sensing images of the original image to the number of remote sensing images of the filled image is greater than a preset ratio threshold, the current initial model is used as the model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911329520.6A CN110992257B (en) | 2019-12-20 | 2019-12-20 | Remote sensing image sensitive information automatic shielding method and device based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911329520.6A CN110992257B (en) | 2019-12-20 | 2019-12-20 | Remote sensing image sensitive information automatic shielding method and device based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110992257A CN110992257A (en) | 2020-04-10 |
| CN110992257B true CN110992257B (en) | 2024-06-14 |
Family
ID=70073787
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911329520.6A Active CN110992257B (en) | 2019-12-20 | 2019-12-20 | Remote sensing image sensitive information automatic shielding method and device based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110992257B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112732912B (en) * | 2020-12-30 | 2024-04-09 | 平安科技（深圳）有限公司 | Sensitive trend expression detection method, device, equipment and storage medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108109126A (en) * | 2018-01-12 | 2018-06-01 | 适普远景遥感信息技术（北京）有限公司 | A kind of target area filling and method for amalgamation processing based on satellite remote-sensing image |
Family Cites Families (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017040691A1 (en) * | 2015-08-31 | 2017-03-09 | Cape Analytics, Inc. | Systems and methods for analyzing remote sensing imagery |
| CN105513041B (en) * | 2015-10-28 | 2018-12-21 | 深圳大学 | A kind of method and system of large format remote sensing images sea land segmentation |
| CN107066995A (en) * | 2017-05-25 | 2017-08-18 | 中国矿业大学 | A kind of remote sensing images Bridges Detection based on convolutional neural networks |
| CN109961068A (en) * | 2017-12-26 | 2019-07-02 | 阿里巴巴集团控股有限公司 | Image recognition, training, searching method and device and equipment, medium |
| CN109255334B (en) * | 2018-09-27 | 2021-12-07 | 中国电子科技集团公司第五十四研究所 | Remote sensing image ground feature classification method based on deep learning semantic segmentation network |
| CN109446992B (en) * | 2018-10-30 | 2022-06-17 | 苏州中科天启遥感科技有限公司 | Remote sensing image building extraction method and system based on deep learning, storage medium and electronic equipment |
| CN109271972A (en) * | 2018-11-05 | 2019-01-25 | 常熟理工学院 | Intelligent image identifying system and method based on natural language understanding and image graphics |
| CN109934200B (en) * | 2019-03-22 | 2023-06-23 | 南京信息工程大学 | RGB color remote sensing image cloud detection method and system based on improved M-Net |
| CN110175538A (en) * | 2019-05-10 | 2019-08-27 | 国网福建省电力有限公司龙岩供电公司 | A kind of substation's Bird's Nest recognition methods and system based on machine learning |
| CN110287869B (en) * | 2019-06-25 | 2022-03-18 | 吉林大学 | High-resolution remote sensing image crop classification method based on deep learning |
| CN110399819A (en) * | 2019-07-15 | 2019-11-01 | 北京洛斯达数字遥感技术有限公司 | A kind of remote sensing image residential block extraction method based on deep learning |
-
2019
- 2019-12-20 CN CN201911329520.6A patent/CN110992257B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108109126A (en) * | 2018-01-12 | 2018-06-01 | 适普远景遥感信息技术（北京）有限公司 | A kind of target area filling and method for amalgamation processing based on satellite remote-sensing image |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110992257A (en) | 2020-04-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111738110A (en) | Remote sensing image vehicle target detection method based on multi-scale attention mechanism | |
| CN107944450B (en) | License plate recognition method and device | |
| CN111160379A (en) | Training method and device of image detection model and target detection method and device | |
| CN111210399B (en) | Imaging quality evaluation method, device and equipment | |
| CN112541448A (en) | Pedestrian re-identification method and device, electronic equipment and storage medium | |
| CN114926766A (en) | Identification method and device, equipment and computer readable storage medium | |
| CN111539456B (en) | Target identification method and device | |
| CN111523558A (en) | Ship shielding detection method and device based on electronic purse net and electronic equipment | |
| CN116071294A (en) | Optical fiber surface defect detection method and device | |
| CN112052702A (en) | Method and device for identifying two-dimensional code | |
| CN110992257B (en) | Remote sensing image sensitive information automatic shielding method and device based on deep learning | |
| CN112926667B (en) | Method and device for detecting saliency target of depth fusion edge and high-level feature | |
| CN111179245B (en) | Image quality detection method, device, electronic equipment and storage medium | |
| CN111753729B (en) | False face detection method and device, electronic equipment and storage medium | |
| CN113158963A (en) | High-altitude parabolic detection method and device | |
| CN115908988B (en) | Defect detection model generation method, device, equipment and storage medium | |
| CN111881996A (en) | Object detection method, computer device and storage medium | |
| CN116580026A (en) | Automatic optical detection method, equipment and storage medium for appearance defects of precision parts | |
| CN112149698A (en) | Method and device for screening difficult sample data | |
| CN114973064A (en) | Method and device for generating pseudo label frame and electronic equipment | |
| CN113298102B (en) | Training method and device for target classification model | |
| CN115311680A (en) | Human body image quality detection method and device, electronic equipment and storage medium | |
| CN114998194A (en) | Product defect detection method, system and storage medium | |
| CN108805190B (en) | Image processing method and device | |
| CN113326891A (en) | Method, system and device for detecting small target object |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |