CN111460247A - Automatic detection method for network picture sensitive characters - Google Patents

Abstract

The invention discloses an automatic detection method for sensitive text of network pictures, which crawls and downloads websites containing pictures to be detected, collects pictures and adds them to a database by means of online crawling and offline loading; and obtains the pictures from the picture database. Image and image target detection (text area positioning, image text recognition) and sensitive text information detection and other processing. Using the Faster R-CNN deep network architecture based on the Region Proposal Network (RPN), a two-level sensitive text information classifier is adopted in the sensitive text information detection link. Among them, the first-level classifier performs rough screening of sensitive words in the input sentence by using a method based on multi-dimensional expansion of sensitive word library. The second-layer filter performs deep-level semantic refinement of sensitive text information based on the combination of sentiment polarity thesaurus and SVM classifier to confirm whether the text information is sensitive information. The automatic detection of image-sensitive text is effectively realized, the detection efficiency is high, and the system response time delay is fast.

Description

Automatic detection method of sensitive text in network pictures

technical field

The invention relates to digital image processing and related algorithms of deep learning, belongs to the field of machine vision and natural language processing, and particularly relates to an automatic detection method for sensitive text in network pictures.

Background technique

With the advancement of science and technology, my country's Internet industry has entered a stage of rapid development. Among them, live broadcast platforms including Douyu and Huya were born, and online social platforms such as WeChat, Weibo and QQ have been continuously updated and improved. These live broadcast platforms and social platforms not only have huge user groups, but also are very active, especially Loved by young and teenage users. With the transmission of massive data and information, such a huge amount of information interaction makes it easy for people to obtain diverse data information on the network, but these data information are often filled with a large amount of sensitive information. The sensitive text filtering technology based on traditional text information is relatively mature, and the monitoring of sensitive information contained in images is relatively difficult, so the transmission of sensitive images is also more concealed. In order to evade the supervision of Internet information by surveillance departments such as the government, many organizations and individuals use images inserted into text to disseminate sensitive information, including pornographic information, anti-social information and violent information. one. According to relevant surveys, more than 10% of websites contain content related to sensitive information. Not only that, many criminals disseminate sensitive information through the user avatars of Tencent QQ, WeChat, and live broadcast platforms. The images of pornographic sensitive information that are full of them not only adversely affect the physical and mental health of young people, but also contain reactionary and violence-related images. Information also has the potential to interfere with social stability. The characteristics of the network itself, such as data sharing, interconnection, and resource openness, are the fundamental reasons why criminals and organizations dare to disseminate sensitive information. The main features of image and text sensitive information are:

(1) The manifestations of sensitive information vary greatly

The scope of sensitive information is very wide, covering many aspects such as ideological and political issues, social issues, cultural issues, etc., and the expressions of sensitive information on different topics are very different. Sensitivity in the context and other aspects shows different degrees. Words such as "bloodbath" and "lore" are mostly used in sports-themed information to mean victory in the game, while in other themes, they are likely to be signs of sensitive information.

(2) Character recognition out of the original text is likely to cause significant ambiguity

Considering the possibility of illegal content of sensitive texts, criminals will deliberately use synonyms, homophones, pinyin, and left and right structural glyphs to separate input and replace to create sensitive text pictures. This increases the difficulty of character recognition.

Due to the different forms of pictures on the Internet, different pictures are very different in terms of text size, text color, text size, text relative position, text font, etc. Before identifying the text in it, it is necessary to locate the text in the image For the included text area, accurate image text area positioning is the basis for subsequent recognition work. The traditional methods of text area localization include the method based on the image connected domain feature, the method based on the image texture feature, and the method based on the image edge feature. With the development of machine learning technology, the effect of image feature target detection algorithms based on machine learning has been greatly improved in recent years. Among them, the scheme detection effect based on deep learning technology is very significant. In 2014, Girshick took the lead in proposing the RCNN (Region with CNN Features) scheme based on Region Proposal (candidate region). The core idea of this method is to replace the public dataset PASCAL VOC with some representative candidate regions in the image to increase the highest detection rate from 35% to 53%. In 2015, the author proposed a new detection method fast RCNN based on RCNN. This method greatly reduces the time complexity of algorithm training and testing while ensuring that the detection accuracy is comparable to that of RCNN. The total training time of the detection system was reduced from 84 hours to 9.5 hours, and the test time was reduced from 47 seconds to about 0.3 seconds. In the same year, the author team proposed faster RCNN. The core of this method is to integrate several main modules of the previous RCNN into the same deep network framework for end-to-end processing. In terms of text area positioning, relevant feature learning can be performed by extracting image content features (including image color features, texture features, and edge features, etc.), and the image regions can be classified according to the learned relevant features to realize the determination of text regions.

To sum up, although a lot of research results have been achieved in the identification of sensitive information in pictures, there are still some limitations and deficiencies. Sensitive information recognition based on image text content mainly has adverse effects such as difficulty in locating text regions similar to natural scene image text information, low text recognition accuracy, and difficulty in discriminating sensitive information in short texts. At present, the current identification of sensitive information in pictures is an area of focus and research, and it is also a focus problem that network supervision departments need to improve technical means to solve.

SUMMARY OF THE INVENTION

The purpose of the present invention is to overcome the shortcomings of the existing image text sensitive information detection technology. This paper mainly studies the improved Faster R-CNN, which improves the detection effect of small target areas. The improved algorithm is used in network image text detection. With better recognition effect, its recognition accuracy will be higher. Aiming at the method research of multi-level classifier based on short text sensitive information, the sensitive character library is expanded on the original basis, and the sensitive character classifier is improved. The overall block diagram of the present invention is shown in FIG. 1 .

The traditional detection of sensitive text information in pictures has long relied on manual supervision and bans, and the detection time of traditional manual reports is generally at the hour level, and the sensitive information may have been widely disseminated during the time interval between the publication of the picture and the report. Sensitive information in the form of pictures and text is wandering on the fringes of supervision, profoundly affecting the healthy environment of the Internet and the physical and mental health of netizens. The present invention realizes the automatic detection of the sensitive text of network pictures based on the deep learning algorithm and the machine learning algorithm.

In view of this, the technical solution adopted in the present invention is: a method for automatic detection of sensitive text in network pictures, comprising the following steps:

Step S1, use a web crawler to crawl a website containing pictures; save the basic information of the pictures in the data source database, and collect the pictures in the picture database for subsequent use;

Step S2, obtain the picture from the picture database and use the FasterR-CNN deep network based on the region proposal network to detect the text object of the picture, and after completion, extract the text information of the picture recognition and convert it into the picture text information;

Step S3, using the classifier to detect the sensitive text information of the extracted image text information, including the first-level classifier performing rough screening of sensitive words on the input sentence based on the multi-dimensional expansion of the sensitive word library, and using the rough screened text information. Chinese word segmentation processing, and then through the second-level classifier based on sentiment polarity thesaurus and SVM classifier, the deep-level sensitive information is finely screened, and the automatic detection of sensitive text information in network pictures is completed.

Further, the basic information of the picture includes the link of the picture, the size of the picture, and the name of the picture.

The process of performing text target detection on the image described in step S2 includes performing maximum pooling sampling reduction and deconvolution operation amplification on the shared convolution layer of the region proposal network, and then generating the feature map output by the feature mapping layer of the network for the candidate region. Perform average pooling to generate fixed-size target candidate regions. The region pooling layer of the candidate region optimization network generates the target candidate region output by the network according to the candidate region, and performs region pooling on the feature map output by the feature mapping layer of the candidate region generation network. , generate a fixed-size region feature;

According to the softmax layer, the classification probability of whether each target candidate area contains a target or background is output, and only target candidate areas with a probability greater than a preset threshold are output, and most of the invalid candidate areas can be excluded to obtain the optimized target candidate area, and then the target classification According to the optimized target candidate region, the regression network extracts regional features from the generated shared feature map, and performs final target text category discrimination and target bounding box regression correction.

In step S3, the sensitive information is finely screened, the emotional polarity words are added to the data set of the existing sensitive information short texts, and the text information is marked in combination with the emotional tendency judgment, and the SVM model is used to analyze the sensitive information short texts containing emotional polar words. dataset for training.

The SVM classifier performs Chinese word segmentation on the training set, then encodes the text in the training set in the form of word vectors, uses multi-dimensional vectors to represent the vocabulary of the text, and performs feature extraction and model training on it, and finally uses The trained classification model judges the short text after rough screening to confirm whether the short text is sensitive text information text. The cross-validation function of libsvm is used to realize the optimization of vector parameters, and the optimal parameter values are obtained by searching the parameter value space. After text preprocessing, feature extraction, feature representation, and normalization, the original text information has been abstracted into a vectorized sample set, and then the similarity between the sample set and the trained template file is calculated to further confirm Whether the short text is sensitive text information text.

It also includes tracking and alarming pictures that are determined to contain sensitive text information, and displaying the picture's address link, picture name information, and picture size information.

The present invention finally realizes the function of automatic detection of image-sensitive text, greatly reduces the system response time delay compared with the traditional method, and improves the accuracy of system detection of sensitive information of images. In particular, the recognition and detection of texts in small target areas on pictures, the inclination of texts, ambiguous texts and more complex and sensitive semantics have been significantly improved.

Description of drawings

Fig. 1 is the flow chart of the present invention's picture text target detection;

Fig. 2 is the flow chart of sensitive text information detection of the present invention;

Fig. 3 is the network structure diagram of the second-level classifier.

Detailed ways

The invention includes two parts: target detection of pictures (location of text area and text recognition) and detection of sensitive text information. In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, but are not intended to limit the present invention. .

The web image crawling module uses web crawlers to crawl images from specific websites containing images, saves the basic information of the images in the data source database, and collects images into the image database for subsequent use. Appropriate manual classification of pictures in the picture library for later inspection and supervision. First, set up a picture acquisition rule for crawling the content of the website on the website, use the web crawler in the prior art to find the webpage through the link address of the webpage, and keep looping until the pictures of all the webpages of the website have been crawled. . In the specific application implementation process, in order to obtain the pictures of the website faster, some non-picture content that does not need to be obtained can be omitted through the preset information acquisition rules, so as to reduce the workload of crawling the content. The image acquisition rule used in this method is set to be acquired every 5 minutes, and the acquired website depth involves the home page of the website to be detected, the first and second layers of links on the home page, and subsequent pages. To save the detection report in text format, it is conceivable that the periodic acquisition can be set to a longer or shorter time as needed, and the crawled images can be saved to the website according to the actual needs of the detection. In the picture database, other data information is saved in the data source database.

As shown in the embodiment shown in FIG. 1 , the image text target detection module further includes a region candidate network extraction part (used for the spatial features of text images) and a Fast-R-CNN detection part. The main specific steps of the image text target detection module are:

(1) Reasonably divide the data set, use the standard data set for standardization, unify the input dimensions, and speed up the training speed;

(2) Integrate the convolution modules of different layers together, which can extract multi-layer features, high-level abstract features and low-level detailed features, make the map of the first layer a max pooling for downsampling to reduce, and reduce the last layer. The map does a deconv (deconvolution) for upsampling to enlarge, and then connects the convolution outputs of the first 5 layers, and the effect of connecting layers 1, 3, and 5 is better (because there is an interval layer in the middle, and the There is little correlation between the features). Finally, we use local response normalization (LRN) to normalize multiple feature maps. Without normalization, large features will overwhelm small features. Multiple feature maps are then combined into a single output cube, which we call cube Featuremaps. The last layer is added to the deconvolution layer;

(3) Operate parallel convolution. In the second convolution module, 5*5 and 7*7 convolutions are performed in parallel, and the features extracted by convolution kernels of different sizes are different, and are extracted and fused differently;

(4) Introduce a cross convolution kernel, convert the square convolution kernel into an asymmetric convolution structure, convert the 5*5 convolution kernel into 5*1 and 1*5 convolution kernels, and perform maximum pooling sampling reduction and deconvolution operation to enlarge the operation;

(5) The fixed candidate region pooling layer, whose input is composed of feature maps obtained by multiple different depth convolution layers, and the three depth feature maps are jointly used as the input of the candidate region pooling layer. The purpose of this layer is to make the size different The candidate frame of one is converted into the output feature map with a fixed size for use in the next step. Then average pooling is performed on the feature map output by the feature mapping layer of the candidate region generation network to generate a fixed size region feature;

(6) The candidate region optimization network adopts a Gaussian distribution with zero mean and standard deviation a randomly initialized, uses the trained faster-R-CNN network to generate training data, trains the candidate region optimization network separately, and inputs the training images in the training set into In the network, the target candidate region output by the candidate region generation network is used as the training data of the candidate region optimization network. The data whose intersection ratio IOU (intersection and union ratio) of any annotation frame is greater than the target candidate area is regarded as a positive sample, and the data whose intersection ratio IOU with any annotation frame is smaller than the target candidate area is regarded as a negative sample;

(7) Use non-maximum suppression to obtain 100 high-scoring suggestion windows. These suggestion windows can basically cover all the text areas that appear. If there are too many selections, the suggestion windows will overlap, which will increase the useless calculation amount. Perform edge refinement correction, which can predict the precise vertical position through the position offset. The formula is as follows:

是x轴的实际边缘坐标，它是从实际边界框和锚点位置预先计算的。

是x轴的锚点的中心，w^a是固定的锚点宽度w^a＝16。o、o^*分别表示预测和实际偏移量。使用边缘提议的偏移量来优化最终的文本行边界框。where x _side is the predicted x-coordinate of the closest horizontal side to the current anchor,

is the actual edge coordinate of the x-axis, which is precomputed from the actual bounding box and anchor position.

is the center of the anchor for the x-axis, and w ^a is the fixed anchor width w ^a =16. o, o ^* represent the predicted and actual offsets, respectively. The final text line bounding box is refined using the offsets of the edge proposals.

分别表示的是文本/分文本的二分类损失，坐标损失和边缘细化损失。根据最小损失规则，最小化图像的总体目标函数(L)最小化：We employ multi-task learning to jointly optimize model parameters. According to the source of the output data, three loss functions are introduced:

They represent the binary classification loss of text/sub-text, coordinate loss and edge refinement loss, respectively. The overall objective function (L) of the image is minimized according to the minimum loss rule:

是与第k个锚点关联的x轴的预测和实际偏移量。

是我们使用Softmax损失区分文本和非文本的分类损失。

和

是回归损失。N_s,N_v,N_o是标准化参数，表示

分别使用的锚点总数。where each anchor is a training sample and i is the index of an anchor in a mini-batch. S _i is the predicted probability of anchor i as the actual text. k is the index of the edge anchor, which is defined as a set of anchors within a horizontal distance (eg, 8 pixels) to the left or right of the actual text line bounding box. o _k and

are the predicted and actual offsets of the x-axis associated with the kth anchor.

is our classification loss to distinguish text from non-text using Softmax loss.

and

is the regression loss. N _s , N _v , and N _o are standardized parameters, indicating

The total number of anchors used respectively.

Finally, the suggestion window is merged through the text line construction algorithm, that is, every two similar 8*h small suggestion windows are formed into a pair, and then different pairs are merged until they can no longer be merged, and finally a complete suggestion box is generated. The construction of a text line is very simple. Text lines are constructed as follows. First, we define a paired neighbor (B _j ,B _i ) of B _i for the proposal as B _j ->B _i , when B _j is the closest horizontal distance to B _i , the distance is less than 50 pixels, and their vertical overlap greater than 0.6. Second, if B _j ->B _i and B _i ->B _j , then group the two proposals as a pair. Text lines are then constructed by sequentially concatenating proposal pairs with the same proposal, and the pair performs the completed object class discrimination and object bounding box regression correction. Through the subsequent CNN+CTC method, the deep feature extraction of the input image, the sequence label probability prediction and the label transcription function are completed. On this basis, Tesseract secondary recognition is added, and the Fast-R-CNN detection part inputs the recognized text lines in the form of strings into the sensitive text information detection module for text-sensitive semantic detection.

In the embodiment shown in FIG. 2 , the sensitive text information detection module includes a first-level classifier, a word segmentation module and a second-level classifier. The first-level classifier performs rough screening of sensitive words in the input sentence by means of a word rule filtering engine based on multi-dimensional expansion of sensitive word database. The original sensitive words are multi-dimensionally expanded, including synonyms, homophones, pinyin, side-by-side characters and other ambiguous methods to establish a new sensitive information thesaurus, which includes reaction, pornography, and violence. The function of the word segmentation module is to perform word segmentation processing on the text. Since there is no space segmentation method in the Chinese text as in the Western text, the Chinese word segmentation process needs to be performed first. The second-level classifier will perform Chinese word segmentation on the completed training set, then encode the text in the training set in the form of word vectors, use multi-dimensional vectors to represent the vocabulary of the text, and perform feature extraction and model training on it. Finally, Use the trained classification model to judge the short text after rough screening to confirm whether the short text is sensitive text information text.

In the embodiment shown in Figure 3, the secondary classifier is an SVM classifier. SVM is a discriminative classifier defined by a classification hyperplane, that is to say, given a set of labeled training samples, the algorithm will output an optimal hyperplane to classify new samples (test samples), find a hyperplane, And it has the largest distance to the training sample closest to him. That is, the optimal segmentation hyperplane maximizes the training sample boundary.

Support vector machine classification, first of all, it is a classification problem, which corresponds to two important steps in the classification process, one is to use the training data set to train the classifier, and the other is to use the test data set to evaluate the classification accuracy of the classifier. As sensitive information text classification, the implementation process of text classification implementation based on libsvm is as follows:

(1) Select the text training data set and test data set: both the training set and the test set are known with known class labels;

(2) Training set text preprocessing: This mainly includes Chinese word segmentation, removal of stop words, and establishment of word vector model;

(3) Select the feature vector (word vector) used for text classification: The ultimate goal is to make the final selected feature vector have a certain degree of category distinction between multiple categories, so as to realize the classification and screening of the feature vector, due to the Chinese word segmentation After obtaining a large number of words, by selecting dimensionality reduction technology, the amount of calculation can be well reduced, and the classification accuracy can be maintained;

(4) Output the training sample set file of quantified emotional polarity words supported by libsvm: the mapping conversion of category name, each word element in the feature vector to the number number, and the quantized text training set based on the category and feature vector, which can satisfy The data format required for training with libsvm;

(5) Test data set preprocessing: It also includes Chinese word segmentation (which needs to be consistent with the word segmentation device used in the training process), removing stop words, and establishing a word vector model (inverted list), but at this time, it is necessary to load the generated words during the training process. The feature vector of , and use the feature vector to exclude redundant words that are not in the feature vector (also known as dimensionality reduction);

(6) Output the quantized test sample set file supported by libsvm: the output format is the same as the output of the preprocessing stage of the training data set. Use the quantified sentiment polarity word dataset file output in the training set preprocessing stage, and finally output the classification model file. Use the libsvm toolkit to train a text classifier. At the beginning of using libsvm, you need to do a scale transformation operation, which is conducive to libsvm to train a better model. The training data formats used by libsvm are all digital, so it is necessary to quantify the documents in the training set. We use the TF-IDF metric to represent the correlation between words and documents. In the previous output data, the value of TF-IDF is used for each dimension vector, but the value of TF-IDF may be in an irregular range (depending on the values of TF and IDF), such as 0.19872 ~ 8.3233, so You can use libsvm to transform all values into the same range, such as 0~1.0, or -1.0~1.0, which can be selected according to actual needs;

(7) Use libsvm to verify the accuracy of the classification model: use the quantified data set file and classification model file output in the test set preprocessing stage to verify the accuracy of the classification, select the appropriate kernel function, and set the cost coefficient c, the default is 1, indicating When computing a linear classification surface, it is possible to tolerate a point being misclassified. At this time, use cross-validation (CrossValidation) to gradually optimize the calculation and select the most suitable parameters;

(8) Optimization of classification model parameters: If the accuracy of the classification model trained by libsvm is poor, you can continue to optimize the parameters through the cross-validation function that comes with libsvm, and obtain the best parameters by searching the parameter value space. value. After text preprocessing, feature extraction, feature representation, and normalization, the original text information has been abstracted into a vectorized sample set, and then the similarity between the sample set and the trained template file is calculated, that is, Determine the similarity probability of specific categories (pornography, violence, reaction) of sensitive text information after comparing the text to be tested with the template file. If it does not belong to this category, it will be calculated with the template files of other categories until it is classified into the corresponding specific category;

(9) The final detection result output Tracks and alarms the detection report of the sensitive text of the picture and its corresponding website address, etc., prompts the picture that contains sensitive text information, and displays the address link and picture name of the picture in the relevant area. information, image size, etc.

Claims (7)

1. an automatic detection method for network image sensitive text, characterized in that, comprising the following steps: Step S1, use a web crawler to crawl a website containing pictures; save the basic information of the pictures in the data source database, and collect the pictures in the picture database for subsequent use; Step S2, obtain the picture from the picture database and use the Faster R-CNN deep network based on the region proposal network to perform text target detection on the picture, and after completion, extract the text information recognized by the picture and convert it into picture text information; Step S3, using the classifier to detect the sensitive text information of the extracted image text information, including the first-level classifier performing rough screening of sensitive words on the input sentence based on the multi-dimensional expansion of the sensitive word library, and using the rough screened text information. Chinese word segmentation processing, and then through the second-level classifier based on sentiment polarity thesaurus and SVM classifier, the deep-level sensitive information is finely screened, and the automatic detection of sensitive text information in network pictures is completed. 2 . The automatic detection method for sensitive text in network pictures according to claim 1 , wherein the basic information of the picture includes the link of the picture, the size of the picture, and the name of the picture. 3 . 3. The method for automatic detection of sensitive text in network pictures according to claim 1, characterized in that: the process of carrying out text target detection to the picture described in step S2 comprises that the shared convolution layer of the regional suggestion network is subjected to maximum pooling sampling reduction and The deconvolution operation is enlarged, and then the feature map output by the feature mapping layer of the candidate region generation network is averagely pooled to generate a fixed-size target candidate region. The region pooling layer of the candidate region optimization network generates the target output of the network according to the candidate region. Candidate region, perform regional pooling on the feature map output by the feature mapping layer of the candidate region generation network, and generate regional features of a fixed size; According to the softmax layer, the classification probability of whether each target candidate region contains the target or the background is output, and only the target candidate region whose probability is greater than the preset threshold is output. The target classification and regression network extracts the generated shared feature map according to the optimized target candidate region. Region features, the final target text category discrimination and target bounding box regression correction. 4. according to the described automatic detection method of the network picture sensitive text of claim 1, it is characterized in that: described in the step S3, the multi-dimensional expansion of the sensitive word library specifically includes synonyms, homophones, pinyin, and side half-words by the multi-dimensional expansion of original sensitive words. A new sensitive information thesaurus, the thesaurus includes reaction, pornography and violence. 5. according to the described automatic detection method of the network picture sensitive text of claim 4, it is characterized in that: the sensitive information fine screening described in step S3, the emotion polar word is added in the data set of the existing sensitive information short text, combine emotion Tendency judgment, labeling text information, and using the SVM model to train a dataset containing short texts with sensitive information of sentiment polarity words. 6. according to the described automatic detection method of network picture sensitive text of claim 5, it is characterized in that: SVM classifier described in step S3, the training set is carried out Chinese word segmentation processing, then by the form of word vector, the text in the training set is encoded , using multi-dimensional vectors to represent the vocabulary of the text, and perform feature extraction and model training on it, and finally use the trained classification model to judge the short text after rough screening to confirm whether the short text is sensitive text information text. 7. according to the described network picture sensitive text automatic detection method of claim 1-6, it is characterized in that: also comprise for determining that the picture that contains sensitive text information carries out tracking alarm, displays the address link, picture name information, picture size information of this picture .

Images