CN109284684A - A kind of information processing method, device and computer storage medium - Google Patents

Abstract

The embodiment of the present invention discloses an information processing method, a device and a computer storage medium. By acquiring an image to be processed; based on a first training model, sensitive information is detected from the to-be-processed image and the type of the sensitive information is obtained; Generate replacement information corresponding to the sensitive information based on the second training model corresponding to the type of the sensitive information; perform replacement processing on the sensitive information in the to-be-processed image according to the replacement information, and obtain the processed image; thus realizing the hiding of the user's real sensitive information, and protecting the user's real sensitive information without reducing the beauty of the original image.

Description

An information processing method, device and computer storage medium

technical field

The present invention relates to the technical field of image processing, and in particular, to an information processing method, an apparatus and a computer storage medium.

Background technique

With the development of science and technology, people pay more and more attention to the security of personal information. How to avoid the leakage of personal information is a current research hotspot. and broader prospects. For example, in some selfies and life photos, people often block areas involving personal information by mosaicking or using some small patterns. These personal information can include ID numbers, license plate numbers, flight numbers, and even face images. However, mosaicking and masking with small patterns can indeed hide personal information, but they often make photos lose part of their beauty.

SUMMARY OF THE INVENTION

The main purpose of the present invention is to provide an information processing method, a device and a computer storage medium, which can hide the user's real sensitive information by generating virtual replacement information of "fake the real", so that the user's real sensitive information can be hidden without reducing the beauty of the original image. Real sensitive information is protected.

In order to achieve the above object, the technical scheme of the present invention is achieved in this way:

In a first aspect, an embodiment of the present invention provides an information processing method, the method includes:

Get the image to be processed;

Based on the first training model, sensitive information is detected from the to-be-processed image and the type of the sensitive information is obtained;

generating replacement information corresponding to the sensitive information based on the second training model corresponding to the type of the sensitive information;

Perform replacement processing on the sensitive information in the to-be-processed image according to the replacement information to obtain a processed image.

In the above solution, the obtaining of the image to be processed specifically includes:

Receive a camera turn-on command to turn on the camera;

A photographing instruction is received, and the to-be-processed image is acquired according to the photographing instruction.

In the above solution, before the sensitive information is detected from the to-be-processed image based on the first training model and the type of the sensitive information is obtained, the method further includes:

Obtain an image sample; wherein, each image in the image sample contains sensitive information;

Perform block division on each image in the image sample to obtain a divided image block set;

Marking each image block in the divided image block set to obtain a marked image block set;

Taking the marked set of image blocks as the first training set, by training the first training set and the marking corresponding to each image block in the first training set, a first training model is obtained; wherein, the The first training model is used to detect whether the image to be processed contains sensitive information and the type of the sensitive information.

In the above solution, the first training model obtained by training the first training set and the label corresponding to each image block in the first training set specifically includes:

Inputting the first training set into a convolutional neural network model, and performing image recognition based on the convolutional neural network model to determine the category of each image block in the first training set;

Determine the value of the loss function based on the determined category of each image block and the label corresponding to each image block;

Based on the value of the loss function, the parameters of the convolutional neural network model are adjusted to re-determine the category of each image block in the first training set according to the parameter-adjusted convolutional neural network model, and re-determine the category of each image block in the first training set. The value of the loss function, until the value of the loss function is less than a preset threshold, it is determined that the training of the convolutional neural network model is completed;

Based on the trained convolutional neural network model, a first training model is obtained.

In the above solution, before generating the replacement information corresponding to the sensitive information based on the second training model corresponding to the type of the sensitive information, the method further includes:

Obtain an image sample; wherein, each image in the image sample contains sensitive information;

Perform sensitive information detection on each image in the image sample based on the first training model;

Based on the detection result, obtain the type of the sensitive information in each image;

grouping the image samples based on the obtained multiple types to obtain multiple sets of images;

Using the multiple sets of image sets as the second training set, by performing separate training on each set of image sets in the second training set, multiple sets of second training models are obtained; wherein, the multiple sets of second training models are There is a corresponding relationship with the plurality of types.

In the above solution, generating the replacement information corresponding to the sensitive information based on the second training model corresponding to the type of the sensitive information specifically includes:

obtaining a second training model corresponding to the type of the sensitive information according to the correspondence between the multiple groups of second training models and the multiple types;

The sensitive information is generated and trained based on the corresponding second training model to obtain replacement information corresponding to the sensitive information.

In the above solution, performing replacement processing on the sensitive information in the to-be-processed image according to the replacement information to obtain a processed image specifically includes:

determining the sensitive information area corresponding to the sensitive information in the to-be-processed image;

The replacement information is overlaid on the sensitive information area to obtain a processed image.

In the above scheme, the method further includes:

Based on the first training model, if no sensitive information is detected from the to-be-processed image, the information processing flow of the to-be-processed image is ended.

In the above solution, after the sensitive information is detected from the to-be-processed image based on the first training model and the type of the sensitive information is obtained, the method further includes:

Send a consultation instruction, where the consultation instruction is used to confirm whether the sensitive information needs to be processed;

If a confirmation instruction is received, continue the information processing flow of the to-be-processed image according to the confirmation instruction;

If a cancellation instruction is received, the information processing flow of the image to be processed is ended according to the cancellation instruction.

In a second aspect, an embodiment of the present invention provides an information processing apparatus, where the information processing apparatus includes: a network interface, a memory, and a processor; wherein,

The network interface is used for receiving and sending signals in the process of sending and receiving information with other external network elements;

the memory for storing a computer program executable on the processor;

The processor is configured to execute the steps of the information processing method in any one of the first aspect when running the computer program.

In a third aspect, an embodiment of the present invention provides a computer storage medium, where an information processing program is stored in the computer storage medium, and when the information processing program is executed by at least one processor, implements the information in any one of the first aspects The steps of the method of processing.

An information processing method, a device, and a computer storage medium provided by the embodiments of the present invention acquire an image to be processed; based on a first training model, sensitive information is detected from the image to be processed and the type of the sensitive information is obtained ; Based on the second training model corresponding to the type of the sensitive information, generate replacement information corresponding to the sensitive information; Carry out replacement processing on the sensitive information in the to-be-processed image according to the replacement information, and obtain the processed Therefore, the real sensitive information of the user is hidden, and the real sensitive information of the user is protected without reducing the beauty of the original image.

Description of drawings

1 is a schematic flowchart of an information processing method according to an embodiment of the present invention;

2 is a schematic diagram of a hardware structure of a mobile terminal according to an embodiment of the present invention;

3 is a detailed schematic flowchart of an information processing method provided by an embodiment of the present invention;

4 is a schematic structural diagram of an original image of an ID card provided by an embodiment of the present invention;

5 is a schematic structural diagram of an identity card original image division block provided by an embodiment of the present invention;

6 is a schematic structural diagram of a processed ID card image according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of the composition and structure of an information processing apparatus according to an embodiment of the present invention;

FIG. 8 is a schematic diagram of the composition and structure of another information processing apparatus provided by an embodiment of the present invention;

FIG. 9 is a schematic diagram of the composition and structure of another information processing apparatus provided by an embodiment of the present invention;

FIG. 10 is a schematic diagram of the composition and structure of still another information processing apparatus provided by an embodiment of the present invention;

FIG. 11 is a schematic diagram of the composition and structure of still another information processing apparatus provided by an embodiment of the present invention;

FIG. 12 is a schematic diagram of a specific hardware structure of an information processing apparatus according to an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

Convolutional Neural Network (CNN) is a feedforward neural network that has been successfully applied to image recognition. The basic structure of CNN includes two layers, one is the feature extraction layer, the input of each neuron is connected to the local receptive field of the previous layer, and the local features are extracted; the other is the feature mapping layer, each calculation of the network. A layer consists of multiple feature maps, each feature map is a plane, and the weights of all neurons on the plane are equal. CNN is mainly used to identify displacement, scaling, image information and other forms of distortion invariant two-dimensional graphics, and the feature extraction layer of CNN is learned through training data, avoiding explicit feature extraction, and implicitly extracting features from training data. At the same time, based on the same neuron weights on the same feature mapping surface, the CNN network can learn in parallel, which is also a major advantage of the convolutional neural network compared to the network where neurons are connected to each other. Convolutional neural network has unique advantages in speech recognition and image processing with its special structure of local weight sharing, its layout is closer to the actual biological neural network, and weight sharing reduces the complexity of the network, especially multi-dimensional The fact that the image of the input vector can be directly fed into the network avoids the complexity of data reconstruction in the process of feature extraction and classification.

Generative Adversarial Networks (GAN) is a deep learning model proposed in 2014. It is one of the most promising methods for unsupervised learning on complex distributions in recent years. This method is based on the idea of "game theory". There are (at least) two models in the construction framework: a Generative Model (G) that captures the distribution of the data, and a Discriminative Model (D) that estimates the probability that samples come from the training data. The dynamic "game process" constituted, in the most ideal state, G can generate enough data to be "real", while D cannot correctly distinguish the generated data from the real data.

Deep learning is a powerful technology derived from neural networks. Through the construction of multi-level neurons and repeated training of a large number of data samples, deep learning technology is increasingly used in image recognition and information hiding applications. . In the embodiment of the present invention, sensitive personal information, such as ID card number, license plate number, flight number, and face image, is detected from the acquired image through a first training model (eg, a convolutional neural network model). A second training model (such as a generative adversarial network model) is used to generate a set of virtual replacement information that is "real", and then the virtual replacement information is used to cover the real sensitive information in the original image, so as to realize the real sensitive information to the user. The hiding of sensitive information also protects the user's real sensitive information without reducing the beauty of the original image; the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

Example 1

Referring to FIG. 1, it shows an information processing method provided by an embodiment of the present invention. The method may include:

S101: acquire the image to be processed;

S102: Based on the first training model, detect sensitive information from the to-be-processed image and obtain the type of the sensitive information;

S103: Generate replacement information corresponding to the sensitive information based on the second training model corresponding to the type of the sensitive information;

S104: Perform replacement processing on the sensitive information in the to-be-processed image according to the replacement information to obtain a processed image.

Based on the technical solution shown in FIG. 1, the image to be processed is acquired; based on the first training model, sensitive information is detected from the image to be processed and the type of the sensitive information is obtained; based on the type corresponding to the sensitive information The second training model generates replacement information corresponding to the sensitive information; performs replacement processing on the sensitive information in the to-be-processed image according to the replacement information, and obtains a processed image; thereby realizing the real user experience The hiding of sensitive information also protects the user's real sensitive information without reducing the beauty of the original image.

For the technical solution shown in FIG. 1, in a possible implementation manner, the acquiring the image to be processed specifically includes:

Receive a camera turn-on command to turn on the camera;

A photographing instruction is received, and the to-be-processed image is acquired according to the photographing instruction.

It should be noted that the to-be-processed image is acquired, and the to-be-processed image may contain sensitive information. The to-be-processed image may be obtained from an existing image library, downloaded from the Internet, or directly captured by a camera; for example, after the camera is turned on, a photo-taking instruction can be used to obtain the image to be processed. Process images.

For the technical solution shown in FIG. 1, in a possible implementation manner, before the sensitive information is detected from the to-be-processed image based on the first training model and the type of the sensitive information is obtained, the Methods also include:

Obtain an image sample; wherein, each image in the image sample contains sensitive information;

Perform block division on each image in the image sample to obtain a divided image block set;

Marking each image block in the divided image block set to obtain a marked image block set;

Taking the marked set of image blocks as the first training set, by training the first training set and the marking corresponding to each image block in the first training set, a first training model is obtained; wherein, the The first training model is used to detect whether the image to be processed contains sensitive information and the type of the sensitive information.

In the above implementation manner, specifically, the mark includes the category of the image block and the boundary of the sensitive information; wherein the category includes the type of the sensitive information and the non-sensitive information.

It should be noted that the obtained image sample contains multiple images, and each image contains sensitive information; among them, sensitive information is private information used to represent personal identity, such as ID number, mobile phone number, flight numbers, license plate numbers, face images, etc. Specifically, the image samples can be obtained by downloading images from an online or local image library, or can be obtained from a pre-established image sample library; in order to facilitate the unification of image samples, all images in the image samples can be classified into It is normalized to a uniform resolution, for example, the resolution is 448×448, or the resolution is 224×224; however, in this embodiment of the present invention, the resolution of the image sample is not specifically limited.

It should also be noted that after obtaining the image samples, in order to facilitate subsequent labeling and model training, each image in the image samples can also be divided into blocks, for example, each image is divided into 10×10 images. Then mark each image block, that is, add a mark for each image block, the mark contains two parts: the category of the image block and the boundary of sensitive information; the category of the image block includes sensitive information type (such as ID card, contact number, flight, license plate, face, etc.) and non-sensitive information, and for non-sensitive information, the bounding box of sensitive information is invalid; when the image block is marked, all The marked content is used as a label of the image block, which can be used to determine whether the image block contains sensitive information and the type and boundary of sensitive information; the marked image block set is used as the first training set, using the existing volume The accumulated neural network model trains the first training set and the label, so as to obtain a first training model, such as a VGG model (Visual Geometry Group Network, VGGNet).

It can be understood that, for the specific training process of the first training model, the embodiment of the present invention takes the convolutional neural network model as an example. The label corresponding to each image block in a training set is trained to obtain a first training model, including:

Inputting the first training set into a convolutional neural network model, and performing image recognition based on the convolutional neural network model to determine the category of each image block in the first training set;

Determine the value of the loss function based on the determined category of each image block and the label corresponding to each image block;

Based on the value of the loss function, the parameters of the convolutional neural network model are adjusted to re-determine the category of each image block in the first training set according to the parameter-adjusted convolutional neural network model, and re-determine the category of each image block in the first training set. The value of the loss function, until the value of the loss function is less than a preset threshold, it is determined that the training of the convolutional neural network model is completed;

Based on the trained convolutional neural network model, a first training model is obtained.

It should be noted that the preset threshold is a judgment value used to measure whether the training of the convolutional neural network model is completed. In this embodiment of the present invention, it is assumed that the determined category of each image block is defined as The label corresponding to each image block is defined as Y _i , i=1,2,...,N, where N represents the number of image blocks in the first training set; L2 level norm is used to calculate The difference between Yi and _Yi , and then take the square of the difference value as the value of the loss function, and use Loss to represent the loss function, then the value of the loss function is The calculated value of the loss function is compared with a preset threshold, and if the value of the loss function is greater than the preset threshold, the parameters of the convolutional neural network model are continuously adjusted; and according to the adjusted convolutional neural network The network model re-determines the value of the loss function; when the value of the loss function is less than the preset threshold, it can be determined that the training of the convolutional neural network model is completed, and the first training model is obtained.

For the technical solution shown in FIG. 1, in a possible implementation manner, before the second training model corresponding to the type of the sensitive information generates replacement information corresponding to the sensitive information, the method Also includes:

Obtain an image sample; wherein, each image in the image sample contains sensitive information;

Perform sensitive information detection on each image in the image sample based on the first training model;

Based on the detection result, obtain the type of the sensitive information in each image;

grouping the image samples based on the obtained multiple types to obtain multiple sets of images;

Using the multiple sets of image sets as the second training set, by performing separate training on each set of image sets in the second training set, multiple sets of second training models are obtained; wherein, the multiple sets of second training models are There is a corresponding relationship with the plurality of types.

It should be noted that the acquired image sample contains multiple images, and each image contains sensitive information; at this time, sensitive information detection can be performed on each image in the image sample according to the first training model, Thereby, the sensitive information of each image and the type of the sensitive information can be obtained; the image samples are grouped according to the obtained type, so that multiple sets of images can be obtained; for example, the image set corresponding to the ID card type and the license plate type The set of images and the set of images corresponding to the contact phone type, etc.; these sets of images are used as the second training set, and each set of images in the second training set is trained separately. After training, more A group of second training models, and multiple groups of second training models correspond to multiple types of sensitive information; for example, after the image set corresponding to the ID card type is trained, the second training model corresponding to the ID card type is obtained; license plate After the image set corresponding to the type is trained, a second training model corresponding to the license plate type is obtained; after the image set corresponding to the contact phone type is trained, the second training model corresponding to the contact phone type is obtained.

It should also be noted that, for the specific training process of the second training model, a generative adversarial network model is used as an example in the embodiment of the present invention, wherein the generative adversarial network model includes a generative model G and a discriminant model D, according to the generative adversarial network model. The model trains a certain set of images in the second training set. During the training process, the goal of generating model G is to generate images that are as close to real as possible to allow the discriminant network D to discriminate, and the goal of the discriminant network D is to try to make G The generated images are distinguished from the real images in a certain set of images; thus G and D constitute a dynamic "game process"; the final result of training enables G to generate images that are "real" enough, while D is difficult to It is determined whether the image generated by G is a real image, and the probability of determination is 0.5; thus, a second training model is obtained, which can generate virtual information that is "real".

In the above implementation manner, specifically, generating the replacement information corresponding to the sensitive information based on the second training model corresponding to the type of the sensitive information includes:

obtaining a second training model corresponding to the type of the sensitive information according to the correspondence between the multiple groups of second training models and the multiple types;

The sensitive information is generated and trained based on the corresponding second training model to obtain replacement information corresponding to the sensitive information.

It should be noted that, after determining the type of sensitive information in the image to be processed, the second training model corresponding to the type of sensitive information can be obtained according to the correspondence between multiple sets of second training models and multiple types. model; and then input the sensitive information into the corresponding second training model for training, thereby generating virtual replacement information that is infinitely close to the sensitive information, that is, obtaining the replacement information corresponding to the sensitive information .

For the technical solution shown in FIG. 1, in a possible implementation manner, performing replacement processing on the sensitive information in the to-be-processed image according to the replacement information to obtain a processed image specifically includes:

determining the sensitive information area corresponding to the sensitive information in the to-be-processed image;

The replacement information is overlaid on the sensitive information area to obtain a processed image.

It should be noted that, after obtaining the replacement information corresponding to the sensitive information, in order to facilitate hiding the sensitive information in the image to be processed; the sensitive information area corresponding to the sensitive information in the image to be processed can be determined. The information area is obtained based on the bounding box of the sensitive information; then the sensitive information area is overlaid with the replacement information, so that the processed image can be obtained. Here, in the processed image, the real sensitive information has been hidden by the replacement information, so that the real sensitive information of the user can be protected without reducing the beauty of the original image.

Understandably, the image to be processed may or may not contain sensitive information; if it is detected that the image to be processed does not contain sensitive information, the information processing flow of the image to be processed does not need to be executed; The technical solution shown, in a possible implementation, the method further includes:

Based on the first training model, if no sensitive information is detected from the to-be-processed image, the information processing flow of the to-be-processed image is ended.

It should be noted that, based on the first training model, if sensitive information is detected from the image to be processed, the information processing flow of the image to be processed is continued, such as obtaining the type of the sensitive information, and then corresponding to the type of the sensitive information based on the type of the sensitive information. The second training model of the second training model is used to generate replacement information corresponding to the sensitive information; according to the replacement information, the sensitive information in the to-be-processed image is subjected to replacement processing to obtain a processed image; based on the first training model, If no sensitive information is detected from the to-be-processed image, the information processing flow of the to-be-processed image is ended, and it is not necessary to perform image information processing on the to-be-processed image.

Understandably, when it is detected that the image to be processed contains sensitive information, the sensitive information may not be so sensitive, or the image to be processed may not be released to the public, in which case the sensitive information does not need to be processed. , there is no need to execute the information processing flow of the image to be processed; therefore, for the technical solution shown in FIG. 1 , in a possible implementation manner, the method further includes:

After the sensitive information is detected from the to-be-processed image based on the first training model and the type of the sensitive information is obtained, the method further includes:

Send a consultation instruction, where the consultation instruction is used to confirm whether the sensitive information needs to be processed;

If a confirmation instruction is received, continue the information processing flow of the to-be-processed image according to the confirmation instruction;

If a cancellation instruction is received, the information processing flow of the image to be processed is ended according to the cancellation instruction.

It should be noted that when it is determined that the image to be processed contains sensitive information, a consultation instruction can be sent at this time; then the user confirms whether the sensitive information needs to be processed; when a confirmation instruction is received, the sensitive information is confirmed. The information needs to be processed, that is to say, the information processing flow of the image to be processed needs to be continued at this time; when a cancellation instruction is received, it is confirmed that the sensitive information does not need to be processed, that is to say, the description can be ended at this time. The information processing flow of the image to be processed; thus, it can not only protect the user's real sensitive information without reducing the beauty of the original image, but also avoid some unnecessary processing operations to hide sensitive information, saving system resources .

This embodiment provides an information processing method, by acquiring an image to be processed; based on a first training model, sensitive information is detected from the image to be processed and the type of the sensitive information is obtained; based on the type of the sensitive information The corresponding second training model generates replacement information corresponding to the sensitive information; performs replacement processing on the sensitive information in the to-be-processed image according to the replacement information, and obtains a processed image; The hiding of real sensitive information also protects the real sensitive information of users without reducing the beauty of the original image.

Embodiment 2

Based on the same inventive concept of the foregoing embodiments, referring to FIG. 2 , it shows a structural example of a mobile terminal that can be applied to the technical solutions of the foregoing embodiments. The mobile terminal 200 has a camera function and a display function, and may be, but not limited to, a mobile phone , tablet computers, personal digital assistants, e-book readers, multimedia playback devices, smart camera devices and portable electronic devices such as wearable devices. As shown in FIG. 2 , the structure of the mobile terminal 200 may include: a radio frequency (RadioFrequency, RF) unit 210, a memory 220, an input unit 230, a display unit 240, a camera 250, a sensor 260, a processor 270, a power supply 280 and other components; Among them, the main functions of each component of the mobile terminal shown in FIG. 2 are introduced as follows:

The radio frequency unit 210 is used to send and receive information or to receive and transmit signals during a call; the radio frequency unit 210 includes but is not limited to an antenna, at least one amplifier, a transceiver, a duplexer, etc. Specifically, after receiving the downlink information of the base station, send to the processor 270 for processing; in addition, send the uplink data to the base station;

The memory 220 is used to store software programs and various data. The memory 220 mainly includes a program storage area and a data storage area, wherein the storage program area can store an operating system, an application program required for at least one function (such as an application program for taking pictures) , the first training model application for detecting sensitive information and sensitive information types, the second training model application for generating replacement information, etc.), etc.; the storage data area can store data created according to the use of the mobile terminal;

The input unit 230 is used for receiving input digital or character information, and generating key signal input related to user settings and function control of the mobile terminal; specifically, the input unit 230 may include a touch panel 231 (also called a touch screen) and Other input devices 232 (including but not limited to physical keyboards, function keys (such as volume control keys, switch keys, etc.), mouse, etc.).

The display unit 240 is used to display information input by the user or information provided to the user, the display unit 240 includes a display panel 241, and the touch panel 231 can cover the display panel 241; although in FIG. 2, the touch panel 231 and the display panel 241 It is used as two independent components to realize the input and input functions of the mobile terminal, but in some embodiments, the touch panel 231 and the display panel 241 can be integrated to realize the input and output functions of the mobile terminal;

The camera 250 is used for still pictures, continuous shooting function or short video shooting. The camera 250 is divided into built-in and external. The built-in camera means that the camera is inside the mobile terminal, which is more convenient to use; the external camera is connected through the data cable or the interface of the mobile terminal. Connecting with an external camera to complete the shooting function, the weight of the mobile terminal can be reduced; the camera 250 generally has functions such as video recording/distribution and still image capture, and is used to transmit the captured information to the memory 107 through a serial-parallel port or other interface;

The mobile terminal also includes at least one sensor 260, such as a light sensor, a gravity sensor, a gyroscope and other sensors; when the gravity sensor or gyroscope inside the mobile terminal detects that the mobile terminal is in a shaking state, the processor 270 can execute corresponding Photo processing methods, such as turning on the anti-shake function.

The processor 270 is the control center of the mobile terminal, using various interfaces and lines to connect various parts of the mobile terminal, by running or executing the software programs and/or modules stored in the memory 220, and calling the data stored in the memory 220, Execute various functions of the mobile terminal and process data, so as to realize the overall monitoring of the mobile terminal;

The mobile terminal also includes a power supply 280 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 270 through a power management system, so as to manage charging, discharging, and power consumption management functions through the power management system.

Those skilled in the art can understand that the structure of the mobile terminal shown in FIG. 2 does not constitute a limitation on the mobile terminal, and the mobile terminal may include more or less components than the one shown, or combine some components, or different components layout.

Based on the above-mentioned structural example of the mobile terminal 20, see FIG. 3, which shows a detailed process of an information processing method provided by an embodiment of the present invention, and the detailed process may include:

S301: Receive a camera turn-on instruction to turn on the camera;

S302: Receive a photographing instruction, and obtain the to-be-processed image according to the photographing instruction;

For example, taking the mobile terminal 200 shown in FIG. 2 as an example, when the mobile terminal 200 needs to take a photo, the camera 250 needs to be turned on first, and then the camera 250 needs to be turned on through the input unit 230 (such as a physical button or a photo touch button on the touch panel 231 ). ) to input a photographing instruction, and an original image can be obtained according to the photographing instruction, which is used as an image to be processed for subsequent sensitive information detection and processing. The following takes an original image of an ID card as an example, referring to FIG. 4 , which shows a schematic diagram of an original image of an ID card captured by a mobile terminal according to an embodiment of the present invention.

S303: Perform sensitive information detection on the to-be-processed image based on the first training model;

S304: If sensitive information is detected from the to-be-processed image, obtain the type of the sensitive information;

S305: If no sensitive information is detected from the to-be-processed image, end the information processing flow of the to-be-processed image;

For example, taking the mobile terminal 200 shown in FIG. 2 as an example, the application program of the first training model and the application program of the second training model have been pre-stored in the memory 220; Taking the original image of the ID card as an example, the processor 270 performs sensitive information detection on the original image of the ID card through the pre-stored first training model; in the process of detecting the sensitive information, the processor 270 can also perform image block division for the original image 5 , the original image of the ID card is divided into 5×5 image blocks, and then the first training model performs sensitive information detection on each image block, which is not specifically limited in the embodiment of the present invention; wherein, After the sensitive information detection of the first training model, if no sensitive information is detected from the original image, for example, no ID number, mobile phone number, flight number, license plate number, face image, etc. are detected from the original image. If sensitive information is detected from the original image, for example, the information in the area shown in 501 is detected as an ID card number, and the information in the area shown in 502 is detected as a face image; Since both the ID card number and the face image are sensitive information, the information processing flow of the original image needs to be continued.

S306: Send a consultation instruction, where the consultation instruction is used to confirm whether the sensitive information needs to be processed;

S307: if a confirmation instruction is received, continue the information processing flow of the to-be-processed image according to the confirmation instruction;

S308: If a cancellation instruction is received, end the information processing flow of the to-be-processed image according to the cancellation instruction;

It should be noted that, after step S304, step S306 is performed; when it is determined that the sensitive information needs to be processed, step S307 is performed; when it is determined that the sensitive information does not need to be processed, step S308 is performed;

For example, taking the mobile terminal 200 shown in FIG. 2 as an example, the application program of the first training model and the application program of the second training model have been pre-stored in the memory 220; Taking the original image of the ID card as an example, when it is detected that there is sensitive information in the original image, considering that some sensitive information is not so sensitive, or the original image will not be released to the public, the processor 270 will also notify the user at this time. Send a consultation instruction, for example, in the form of a pop-up dialog box to consult the user; when the user confirms that the sensitive information needs to be processed, the user can click the "Confirm" button to send a confirmation instruction to the mobile terminal 200, and the processor 270 receives the confirmation according to the instruction. The instruction will continue the information processing flow of the original image; when the user confirms that the sensitive information does not need to be processed, the user can click the "Cancel" button to send the cancellation instruction to the mobile terminal 200, and the processor 270 will directly send the cancellation instruction according to the received cancellation instruction. The information processing flow of the original image is ended; thus, some unnecessary processing operations of hiding sensitive information can be avoided, thereby saving system resources.

S309: Based on the second training model corresponding to the type of the sensitive information, generate replacement information corresponding to the sensitive information;

S310: Determine the sensitive information area corresponding to the sensitive information in the to-be-processed image;

S311: Overlay the replacement information on the sensitive information area to obtain a processed image.

For example, taking the mobile terminal 200 shown in FIG. 2 as an example, the application program of the first training model and the application program of the second training model have been pre-stored in the memory 220; Take the original image of the ID card as an example. When it is confirmed that the sensitive information in the original image needs to be processed, according to the type of sensitive information, such as the type of ID card, the second training model corresponding to the type of ID card is selected to generate a corresponding model corresponding to the ID card number. replacement information; for the face image type, select the second training model corresponding to the face image type to generate the replacement information corresponding to the face image; wherein, the generated replacement information is infinitely close to the real sensitive information; then According to the determined sensitive information area corresponding to the sensitive information in the image to be processed, after covering the generated replacement information on the sensitive information area, a processed image is obtained, such as the processed ID card shown in FIG. 6 . Image example; thus realizing the protection of the user's real sensitive information without reducing the beauty of the original image.

Through the above embodiments, the specific implementation of the above embodiments is described in detail, from which it can be seen that through the technical solutions of the above embodiments, the user's real sensitive information can be hidden, without reducing the beauty of the original image. It also protects the user's real sensitive information.

Embodiment 3

Based on the same inventive concept of the foregoing embodiments, see FIG. 7 , which shows the composition of an information processing apparatus 70 provided by an embodiment of the present invention. The information processing apparatus 70 may include: a first acquisition part 701 , a first detection part 701 part 702, generation part 703 and replacement part 704; where,

The first acquisition part 701 is configured to acquire images to be processed;

The first detection part 702 is configured to detect sensitive information from the to-be-processed image and obtain the type of the sensitive information based on the first training model;

The generating part 703 is configured to generate replacement information corresponding to the sensitive information based on the second training model corresponding to the type of the sensitive information;

The replacement part 704 is configured to perform replacement processing on the sensitive information in the to-be-processed image according to the replacement information to obtain a processed image.

In the above solution, the first obtaining part 701 is specifically configured as:

Receive a camera turn-on command to turn on the camera;

A photographing instruction is received, and the to-be-processed image is acquired according to the photographing instruction.

In the above solution, referring to FIG. 8 , the information processing apparatus 70 further includes a first training part 705 configured as:

Obtain an image sample; wherein, each image in the image sample contains sensitive information;

Perform block division on each image in the image sample to obtain a divided image block set;

Marking each image block in the divided image block set to obtain a marked image block set;

Taking the marked set of image blocks as the first training set, by training the first training set and the marking corresponding to each image block in the first training set, a first training model is obtained; wherein, the The first training model is used to detect whether the image to be processed contains sensitive information and the type of the sensitive information.

In the above solution, the first training part 705 is specifically configured as:

Inputting the first training set into a convolutional neural network model, and performing image recognition based on the convolutional neural network model to determine the category of each image block in the first training set;

Determine the value of the loss function based on the determined category of each image block and the label corresponding to each image block;

Based on the value of the loss function, the parameters of the convolutional neural network model are adjusted to re-determine the category of each image block in the first training set according to the parameter-adjusted convolutional neural network model, and re-determine the category of each image block in the first training set. The value of the loss function, until the value of the loss function is less than a preset threshold, it is determined that the training of the convolutional neural network model is completed;

Based on the trained convolutional neural network model, a first training model is obtained.

In the above solution, referring to FIG. 9 , the information processing apparatus 70 further includes a second training part 706 configured as:

Obtain an image sample; wherein, each image in the image sample contains sensitive information;

Perform sensitive information detection on each image in the image sample based on the first training model;

Based on the detection result, obtain the type of the sensitive information in each image;

grouping the image samples based on the obtained multiple types to obtain multiple sets of images;

Using the multiple sets of image sets as the second training set, by performing separate training on each set of image sets in the second training set, multiple sets of second training models are obtained; wherein, the multiple sets of second training models are There is a corresponding relationship with the plurality of types.

In the above solution, the generating part 703 is specifically configured as:

obtaining a second training model corresponding to the type of the sensitive information according to the correspondence between the multiple groups of second training models and the multiple types;

The sensitive information is generated and trained based on the corresponding second training model to obtain replacement information corresponding to the sensitive information.

In the above solution, the replacement part 704 is specifically configured as:

determining the sensitive information area corresponding to the sensitive information in the to-be-processed image;

The replacement information is overlaid on the sensitive information area to obtain a processed image.

In the above solution, referring to FIG. 10 , the information processing apparatus 70 further includes a second detection part 707 configured as:

Based on the first training model, if no sensitive information is detected from the to-be-processed image, the information processing flow of the to-be-processed image is ended.

In the above solution, referring to FIG. 11 , the information processing device 70 further includes a consultation part 708, which is configured as:

Send a consultation instruction, where the consultation instruction is used to confirm whether the sensitive information needs to be processed;

If a confirmation instruction is received, continue the information processing flow of the to-be-processed image according to the confirmation instruction;

If a cancellation instruction is received, the information processing flow of the image to be processed is ended according to the cancellation instruction.

It can be understood that, in this embodiment, a "part" may be a part of a circuit, a part of a processor, a part of a program or software, etc., of course, it may also be a unit, or a module or non-modularity.

In addition, each component in this embodiment may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware, or can be implemented in the form of software function modules.

If the integrated unit is implemented in the form of a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or The part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product, the computer software product is stored in a storage medium, and includes several instructions for making a computer device (which can be It is a personal computer, a server, or a network device, etc.) or a processor (processor) that executes all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes: U disk, removable hard disk, Read Only Memory (ROM, Read Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes.

Therefore, this embodiment provides a computer storage medium, where the computer storage medium stores an information processing program, and when the information processing program is executed by at least one processor, implements the steps of the information processing method described in the first embodiment above.

Based on the composition of the above-mentioned information processing apparatus 70 and the computer storage medium, see FIG. 12 , which shows the specific hardware structure of the information processing apparatus 70 provided by the embodiment of the present invention, which may include: a network interface 1201, a memory 1202, and a processor 1203; The various components are coupled together by a bus system 1204 . It will be appreciated that the bus system 1204 is used to implement connection communication between these components. In addition to the data bus, the bus system 1204 also includes a power bus, a control bus, and a status signal bus. However, for clarity of illustration, the various buses are labeled as bus system 1204 in FIG. 12 . Among them, the network interface 1201 is used for receiving and sending signals in the process of sending and receiving information with other external network elements;

memory 1202 for storing computer programs that can run on processor 1203;

The processor 1203 is configured to, when running the computer program, execute:

Get the image to be processed;

Based on the first training model, sensitive information is detected from the to-be-processed image and the type of the sensitive information is obtained;

generating replacement information corresponding to the sensitive information based on the second training model corresponding to the type of the sensitive information;

Perform replacement processing on the sensitive information in the to-be-processed image according to the replacement information to obtain a processed image.

It can be understood that the memory 1202 in the embodiment of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Wherein, the non-volatile memory may be Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (Erasable PROM, EPROM), Erase programmable read-only memory (Electrically EPROM, EEPROM) or flash memory. The volatile memory may be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double DataRate SDRAM, DDRSDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (Synchlink DRAM, SLDRAM) and Direct memory bus random access memory (Direct Rambus RAM, DRRAM). The memory 1202 of the systems and methods described herein is intended to include, but not be limited to, these and any other suitable types of memory.

The processor 1203 may be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above-mentioned method can be completed by an integrated logic circuit of hardware in the processor 1203 or an instruction in the form of software. The above-mentioned processor 1203 may be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), an off-the-shelf programmable gate array (Field Programmable Gate Array, FPGA) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Various methods, steps, and logical block diagrams disclosed in the embodiments of the present invention can be implemented or executed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in conjunction with the embodiments of the present invention may be directly embodied as executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other storage media mature in the art. The storage medium is located in the memory 1202, and the processor 1203 reads the information in the memory 1202, and completes the steps of the above method in combination with its hardware.

It will be appreciated that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more Application Specific Integrated Circuits (ASIC), Digital Signal Processing (DSP), Digital Signal Processing Device (DSP Device, DSPD), programmable logic Devices (Programmable Logic Device, PLD), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present invention or a combination thereof.

For a software implementation, the techniques described herein may be implemented through modules (eg, procedures, functions, etc.) that perform the functions described herein. Software codes may be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

Optionally, as another embodiment, the processor 1203 is further configured to execute the steps of the information processing method described in Embodiment 1 above when the computer program is executed.

Optionally, an embodiment of the present invention further provides a mobile terminal, where the mobile terminal includes the information processing apparatus 70 described in any one of the foregoing embodiments.

It should be noted that the technical solutions described in the embodiments of the present invention may be combined arbitrarily unless there is a conflict.

The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited to this. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention. should be included within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims (11)

1. an information processing method, it is characterised in that the method comprises: Get the image to be processed; Based on the first training model, sensitive information is detected from the to-be-processed image and the type of the sensitive information is obtained; generating replacement information corresponding to the sensitive information based on the second training model corresponding to the type of the sensitive information; Perform replacement processing on the sensitive information in the to-be-processed image according to the replacement information to obtain a processed image. 2. The method according to claim 1, wherein the acquiring the image to be processed specifically comprises: Receive a camera turn-on command to turn on the camera; A photographing instruction is received, and the to-be-processed image is acquired according to the photographing instruction. 3. The method according to claim 1, characterized in that, before the sensitive information is detected from the to-be-processed image based on the first training model and the type of the sensitive information is obtained, the method further comprises: : Obtain an image sample; wherein, each image in the image sample contains sensitive information; Perform block division on each image in the image sample to obtain a divided image block set; Marking each image block in the divided image block set to obtain a marked image block set; Taking the marked set of image blocks as the first training set, by training the first training set and the marking corresponding to each image block in the first training set, a first training model is obtained; wherein, the The first training model is used to detect whether the image to be processed contains sensitive information and the type of the sensitive information. 4 . The method according to claim 3 , wherein the first training model is obtained by training the first training set and the label corresponding to each image block in the first training set, specifically comprising: 5 . : Inputting the first training set into a convolutional neural network model, and performing image recognition based on the convolutional neural network model to determine the category of each image block in the first training set; Determine the value of the loss function based on the determined category of each image block and the label corresponding to each image block; Based on the value of the loss function, the parameters of the convolutional neural network model are adjusted to re-determine the category of each image block in the first training set according to the parameter-adjusted convolutional neural network model, and re-determine the category of each image block in the first training set. The value of the loss function, until the value of the loss function is less than a preset threshold, it is determined that the training of the convolutional neural network model is completed; Based on the trained convolutional neural network model, a first training model is obtained. 5. The method according to claim 1, wherein, before generating the replacement information corresponding to the sensitive information based on the second training model corresponding to the type of the sensitive information, the method further comprises: Obtain an image sample; wherein, each image in the image sample contains sensitive information; Perform sensitive information detection on each image in the image sample based on the first training model; Based on the detection result, obtain the type of the sensitive information in each image; grouping the image samples based on the obtained multiple types to obtain multiple sets of images; Using the multiple sets of image sets as the second training set, by performing separate training on each set of image sets in the second training set, multiple sets of second training models are obtained; wherein, the multiple sets of second training models are There is a corresponding relationship with the plurality of types. 6. The method according to claim 5, wherein generating replacement information corresponding to the sensitive information based on the second training model corresponding to the type of the sensitive information, specifically comprising: obtaining a second training model corresponding to the type of the sensitive information according to the correspondence between the multiple groups of second training models and the multiple types; The sensitive information is generated and trained based on the corresponding second training model to obtain replacement information corresponding to the sensitive information. 7 . The method according to claim 1 , wherein, performing replacement processing on the sensitive information in the to-be-processed image according to the replacement information to obtain a processed image, specifically comprising: 8 . determining the sensitive information area corresponding to the sensitive information in the to-be-processed image; The replacement information is overlaid on the sensitive information area to obtain a processed image. 8. The method of claim 1, wherein the method further comprises: Based on the first training model, if no sensitive information is detected from the to-be-processed image, the information processing flow of the to-be-processed image is ended. 9 . The method according to claim 1 , wherein after the sensitive information is detected from the to-be-processed image based on the first training model and the type of the sensitive information is obtained, the method further comprises: 10 . : Send a consultation instruction, where the consultation instruction is used to confirm whether the sensitive information needs to be processed; If a confirmation instruction is received, continue the information processing flow of the to-be-processed image according to the confirmation instruction; If a cancellation instruction is received, the information processing flow of the image to be processed is ended according to the cancellation instruction. 10. An information processing device, characterized in that the information processing device comprises: a network interface, a memory and a processor; wherein, The network interface is used for receiving and sending signals in the process of sending and receiving information with other external network elements; the memory for storing a computer program executable on the processor; The processor is configured to execute the steps of the information processing method according to any one of claims 1 to 9 when running the computer program. 11. A computer storage medium, characterized in that the computer storage medium stores an information processing program, and when the information processing program is executed by at least one processor, the information processing method according to any one of claims 1 to 9 is implemented A step of.