CN111798360B

CN111798360B - Watermark detection method and device, electronic equipment and storage medium

Info

Publication number: CN111798360B
Application number: CN202010618268.7A
Authority: CN
Inventors: 孙斌; 焦大原; 刘亚萍
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2020-06-30
Filing date: 2020-06-30
Publication date: 2023-08-15
Anticipated expiration: 2040-06-30
Also published as: CN111798360A

Abstract

The application discloses a watermark detection method, a watermark detection device, electronic equipment and a storage medium, and relates to the field of artificial intelligence and deep learning. The specific scheme is as follows: inputting an image to be identified into a pre-trained single-target watermark detector; watermark detection is carried out on the current sample image through a single-target watermark detector, so that a watermark detection frame in the image to be identified and the confidence corresponding to the watermark detection frame are obtained; responding to the confidence coefficient corresponding to the watermark detection frame being greater than or equal to a first confidence coefficient threshold value, and adjusting the confidence coefficient corresponding to the watermark detection frame; and responding to the fact that the adjusted confidence coefficient corresponding to the watermark detection frame is larger than or equal to a preset second confidence coefficient threshold value, carrying out brand authorization verification on the image to be identified, and detecting that the watermark in the watermark detection frame is an obtained or unauthorized watermark. The embodiment of the application can effectively identify the watermark and the version authorization relation thereof, greatly reduce the manual auditing cost and effectively improve the watermark detection accuracy.

Description

Watermark detection method and device, electronic equipment and storage medium

Technical Field

The application relates to the technical field of computers, in particular to a watermark detection method, a watermark detection device, electronic equipment and a storage medium, which relate to the field of artificial intelligence and deep learning.

Background

The watermark is an effective means of copyright protection, and can display brand information more, so that the watermark is a common element in the current Internet picture materials, but the appearance of the watermark can influence the picture attractiveness and possibly has infringement risks, and the watermark is high in various risks, so that the risks of the watermark picture are required to be checked and controlled. With the increasing picture quantity, huge labor cost is consumed by a manual auditing mode, and auditing timeliness is poor. The watermark identification machine strategy with high accuracy and high recall rate is required to be constructed, and the watermark is detected and identified, so that the on-line picture has a plurality of kinds, the background of the picture is complex, the type of the watermark is fast to increase, and the identification difficulty is high. The identification accuracy and recall rate are improved as much as possible, the brand names corresponding to the watermarks are identified, the corresponding brand authorization relations are checked, and the brand and trademark infringement risk is reduced.

At present, watermark detection mainly adopts the following two schemes: (1) traditional detection scheme: and detecting and identifying the watermark by adopting image feature calculation matching methods such as edge feature calculation, a histogram, a feature vector method and the like. The digital watermark is independently provided with an airspace, a transform domain, discrete cosine transform, a robust difference value and other algorithms; (2) detection scheme based on deep learning: watermark region identification is performed through manually marked samples, such as common target detection algorithms of Faster RCNN, SSD, yolo and the like.

The two schemes have different use scenes and different problems and disadvantages. Aiming at the scheme (1): the robustness of the mode based on regional pixel difference and feature matching is poor, the identification can be carried out only for specific categories, and the identification precision and speed are difficult to meet the requirement of content auditing; aiming at scheme (2): target detection relies on a large amount of data labeling, so that the ever-increasing watermarks of various patterns on the Internet are difficult to be covered on the whole, meanwhile, picture texts are difficult to distinguish from the watermarks, and misrecognition is easy to cause. In addition, the copyright identification problem is not considered in all the current watermark identification schemes, the low-quality watermark, the pirate infringement watermark and the watermark conforming to the authorization cannot be distinguished, and the trademark copyright problem is a problem that the content auditing needs to be considered seriously.

Disclosure of Invention

The application provides a watermark detection method, a watermark detection device, electronic equipment and a storage medium, which can effectively identify watermarks and version authorization relations thereof, and can also greatly reduce manual auditing cost and effectively improve watermark detection accuracy.

In a first aspect, the present application provides a watermark detection method, the method comprising:

Inputting an image to be identified into a pre-trained single-target watermark detector; watermark detection is carried out on the current sample image through the single-target watermark detector, so that a watermark detection frame in the image to be identified and confidence degrees corresponding to the watermark detection frame are obtained;

responding to the fact that the confidence coefficient corresponding to the watermark detection frame is larger than or equal to the first confidence coefficient threshold value, adjusting the confidence coefficient corresponding to the watermark detection frame according to a preset method to obtain an adjusted confidence coefficient corresponding to the watermark detection frame;

and responding to the fact that the adjusted confidence coefficient corresponding to the watermark detection frame is larger than or equal to a preset second confidence coefficient threshold value, carrying out brand authorization verification on the image to be identified, and detecting whether the watermark in the watermark detection frame is an authorized watermark or an unauthorized watermark.

In a second aspect, the present application provides a watermark detection apparatus, the apparatus comprising: the device comprises a detection module, an adjustment module and a judgment module; wherein, the liquid crystal display device comprises a liquid crystal display device,

the detection module is used for inputting the image to be identified into a pre-trained single-target watermark detector; watermark detection is carried out on the current sample image through the single-target watermark detector, so that a watermark detection frame in the image to be identified and confidence degrees corresponding to the watermark detection frame are obtained;

The adjusting module is used for adjusting the confidence coefficient corresponding to the watermark detection frame according to a preset method to obtain the adjusted confidence coefficient corresponding to the watermark detection frame in response to the fact that the confidence coefficient corresponding to the watermark detection frame is larger than or equal to the first confidence coefficient threshold;

and the judging module is used for carrying out brand authorization verification on the image to be identified in response to the fact that the adjusted confidence coefficient corresponding to the watermark detection frame is larger than or equal to a preset second confidence coefficient threshold value, and detecting whether the watermark in the watermark detection frame is an authorized watermark or an unauthorized watermark.

In a third aspect, an embodiment of the present application provides an electronic device, including:

one or more processors;

a memory for storing one or more programs,

the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the watermark detection method described in any embodiment of the application.

In a fourth aspect, an embodiment of the present application provides a storage medium having stored thereon a computer program which, when executed by a processor, implements the watermark detection method described in any embodiment of the present application.

The technical scheme provided by the application can effectively identify the watermark and the version authorization relation thereof, and can also greatly reduce the manual auditing cost and effectively improve the watermark detection accuracy.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.

Drawings

The drawings are included to provide a better understanding of the present application and are not to be construed as limiting the application. Wherein:

fig. 1 is a flowchart of a watermark detection method according to a first embodiment of the present application;

fig. 2 is a flow chart of a watermark detection method according to a second embodiment of the present application;

fig. 3 is a flow chart of a watermark detection method according to a third embodiment of the present application;

fig. 4 is a schematic structural diagram of a watermark detection device according to a fourth embodiment of the present application;

fig. 5 is a block diagram of an electronic device for implementing a watermark detection method of an embodiment of the application.

Detailed Description

Exemplary embodiments of the present application will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present application are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Example 1

Fig. 1 is a schematic flow chart of a watermark detection method according to an embodiment of the present application, where the method may be performed by a watermark detection device or an electronic device, and the device or the electronic device may be implemented by software and/or hardware, and the device or the electronic device may be integrated into any intelligent device having a network communication function. As shown in fig. 1, the watermark detection method may include the steps of:

s101, inputting an image to be identified into a pre-trained single-target watermark detector; watermark detection is carried out on the current sample image through a single-target watermark detector, and a watermark detection frame in the image to be identified and the confidence corresponding to the watermark detection frame are obtained.

In a specific embodiment of the application, the electronic device may input the image to be identified to a pre-trained single-target watermark detector; watermark detection is carried out on the current sample image through a single-target watermark detector, and a watermark detection frame in the image to be identified and the confidence corresponding to the watermark detection frame are obtained. Specifically, the electronic device may obtain four vertex coordinates of the watermark detection frame through the single-target watermark detector, and may obtain the watermark detection frame based on the four vertex coordinates. In one embodiment, the electronic device may first use the first layer convolutional neural network of the single target watermark detector as the current layer convolutional neural network; taking the image to be identified as a detection object of the convolutional neural network of the current layer; then, performing image downsampling on a detection object of the current layer convolutional neural network through the current layer convolutional neural network to obtain an image feature extraction result corresponding to the current layer convolutional neural network; then taking an image characteristic extraction result corresponding to the current layer convolutional neural network as a detection object of a next layer convolutional neural network of the current layer convolutional neural network; the next layer of convolutional neural network is used as the current layer of convolutional neural network, and the operation is repeatedly executed until the image feature extraction result corresponding to the nth layer of convolutional neural network is extracted from the detection object of the nth layer of convolutional neural network of the single-target watermark detector; wherein N is a natural number greater than 1; and finally, obtaining a watermark detection frame of the image to be identified and confidence corresponding to the watermark detection frame based on image feature extraction results corresponding to each layer of convolutional neural network in the first layer of convolutional neural network to the N layer of convolutional neural network.

And S102, adjusting the confidence coefficient corresponding to the watermark detection frame according to a preset method in response to the confidence coefficient corresponding to the watermark detection frame being greater than or equal to a first confidence coefficient threshold value, so as to obtain the adjusted confidence coefficient corresponding to the watermark detection frame.

In a specific embodiment of the present application, the electronic device may adjust the confidence coefficient corresponding to the watermark detection frame according to a preset method in response to the confidence coefficient corresponding to the watermark detection frame being greater than or equal to the first confidence coefficient threshold value, to obtain an adjusted confidence coefficient corresponding to the watermark detection frame; and judging that no watermark exists in the watermark detection frame in response to the confidence coefficient corresponding to the watermark detection frame being smaller than a preset first confidence coefficient threshold value.

And S103, responding to the fact that the adjusted confidence coefficient corresponding to the watermark detection frame is larger than or equal to a preset second confidence coefficient threshold value, carrying out brand authorization verification on the image to be identified, and detecting whether the watermark in the watermark detection frame is an authorized watermark or an unauthorized watermark.

In a specific embodiment of the present application, the electronic device may perform brand authorization verification on the image to be identified in response to the adjusted confidence coefficient corresponding to the watermark detection frame being greater than or equal to a preset second confidence coefficient threshold, and detect that the watermark in the watermark detection frame is an authorized watermark or an unauthorized watermark; and judging that the watermark detection frame is free of the watermark in response to the adjusted confidence coefficient corresponding to the watermark detection frame being smaller than a second confidence coefficient threshold value.

The watermark detection method provided by the embodiment of the application comprises the steps of inputting an image to be identified into a pre-trained single-target watermark detector; watermark detection is carried out on the current sample image through a single-target watermark detector, so that a watermark detection frame in the image to be identified and the confidence corresponding to the watermark detection frame are obtained; responding to the fact that the confidence coefficient corresponding to the watermark detection frame is larger than or equal to a preset first confidence coefficient threshold value, adjusting the confidence coefficient corresponding to the watermark detection frame according to a preset method to obtain an adjusted confidence coefficient corresponding to the watermark detection frame; and responding to the fact that the adjusted confidence coefficient corresponding to the watermark detection frame is larger than or equal to a preset second confidence coefficient threshold value, carrying out brand authorization verification on the image to be identified, and detecting whether the watermark in the watermark detection frame is an authorized watermark or an unauthorized watermark. That is, the present application can verify the watermark and its version authorization relationship by a single target watermark detector. In the existing watermark detection method, the conventional detection scheme and the detection scheme based on deep learning do not consider the problem of copyright identification, and the low-quality watermark, the pirate infringement watermark and the watermark conforming to authorization cannot be distinguished. Because the application adopts the technical means of watermark detection and preset confidence threshold corresponding to version authorization through the single-target watermark detector, the technical problem that the low-quality watermark, pirate infringement watermark and watermark conforming to authorization cannot be distinguished in the prior art is solved; in addition, the technical scheme of the embodiment of the application is simple and convenient to realize, convenient to popularize and wider in application range.

Example two

Fig. 2 is a flowchart of a watermark detection method according to a second embodiment of the present application. As shown in fig. 2, the watermark detection method may include the steps of:

s201, inputting an image to be identified into a pre-trained single-target watermark detector; watermark detection is carried out on the current sample image through a single-target watermark detector, and a watermark detection frame in the image to be identified and the confidence corresponding to the watermark detection frame are obtained.

And S202, adjusting the confidence coefficient corresponding to the watermark detection frame according to a preset method in response to the confidence coefficient corresponding to the watermark detection frame being greater than or equal to a first confidence coefficient threshold value, so as to obtain the adjusted confidence coefficient corresponding to the watermark detection frame.

In a specific embodiment of the present application, the electronic device may adjust the confidence level corresponding to the watermark detection frame according to a preset method in response to the confidence level corresponding to the watermark detection frame being greater than or equal to the first confidence level threshold, to obtain an adjusted confidence level corresponding to the watermark detection frame. In one embodiment, the electronic device may perform word number recognition on the watermark detection frame through an optical character recognition method (Optical Character Recognition, abbreviated as OCR) to obtain the number of words in the image to be recognized; and in response to the number of characters in the watermark detection frame being greater than a preset character number threshold, reducing the confidence corresponding to the watermark detection frame by a preset value. In another embodiment, the electronic device may further perform text region number recognition on the watermark detection frame by using an optical character recognition method, to obtain the number of text regions in the watermark detection frame; and in response to the number of the text areas in the watermark detection frame being greater than a preset text area number threshold, reducing the confidence corresponding to the watermark detection frame by a preset value. In another embodiment, the electronic device may further perform text region proportion recognition on the watermark detection frame by using an optical character recognition method, to obtain a proportion of the watermark detection frame to the image to be recognized; and in response to the proportion of the watermark detection frame to the image to be identified being greater than a preset proportion threshold, reducing the confidence corresponding to the watermark detection frame by a preset value. In another embodiment, the electronic device may further extract four edge regions of the watermark detection frame by an edge pixel detection algorithm; then calculating pixel variance values of all edge areas in the four edge areas; based on the pixel variance values of the edge areas, obtaining the maximum pixel variance value of the four edge areas; and reducing the confidence corresponding to the watermark detection frame by a preset value in response to the maximum pixel variance value being smaller than a preset variance threshold.

And S203, judging that no watermark exists in the watermark detection frame in response to the adjusted confidence coefficient corresponding to the watermark detection frame being smaller than a preset second confidence coefficient threshold value.

In a specific embodiment of the present application, the electronic device may determine that there is no watermark in the watermark detection frame in response to the adjusted confidence level corresponding to the watermark detection frame being less than a preset second confidence level threshold; the method can also include the steps of firstly extracting a feature map vector corresponding to the watermark detection frame based on the watermark detection frame in response to the adjusted confidence coefficient corresponding to the watermark detection frame being greater than or equal to a second confidence coefficient threshold value; then carrying out vector matching on the feature map vector corresponding to the watermark detection frame and each feature map vector in the pre-constructed watermark brand feature library; in response to the fact that the feature map vector corresponding to the watermark detection frame is not successfully matched with each feature map vector in the watermark brand feature library, determining that the watermark in the watermark detection frame is an unauthorized watermark; in response to successful matching of the feature map vector corresponding to the watermark detection frame and any one of the feature map vectors in the watermark brand feature library, performing authorized matching on the watermark name of any one of the vectors and each brand authorization word in the watermark brand authorization word list; responding to the successful matching of the watermark name of any vector and any brand authorization word in the watermark brand authorization word list, and judging that the watermark in the watermark detection frame is authorized; and in response to the fact that the watermark name of any vector is not successfully matched with all brand authorization words in the watermark brand authorization word list, judging that the watermark in the watermark detection box is an unauthorized watermark.

Preferably, in the specific embodiment of the application, in order to solve the technical problem of difficulty in identifying the watermark with high transparency and small target, the application also provides a method for replacing a Drop Block module with a Drop out module commonly used in a deep network feature extraction layer, which is favorable for network learning more information, can better focus the watermark with high transparency, and can accelerate the network convergence speed. In addition, multi-scale information can be added into the single-target watermark detector to be fused, the fitting capacity of the small-target watermark is improved by utilizing the high-resolution detail features of the shallow network, a plurality of scale features can be obtained in the feature map pyramid network (Feature Pyramid Networks, FPN for short) according to scene recognition requirements, for example, 3 scale in yolov3 can be increased to 4 scale or 5 scale, and the detection effect of the small watermark is further improved. In addition, in order to solve the technical problem of accuracy of the detection frame, the application also provides a mechanism for introducing weights into the loss function, and compared with the original frame coordinate cross entropy regression loss, the method can take the distance, the overlapping rate and the scale between the prediction frame and the real frame into consideration, so that the frame regression is more stable and accurate. Specifically, the loss function introducing the weights to be weighted can be expressed as:

Wherein, the liquid crystal display device comprises a liquid crystal display device,the value of the weighted loss function is taken; b and b ^gt Respectively representing the coordinates of the central points of the prediction frame and the real frame, ρ ² Representing the Euclidean distance between two center points; c ² A rectangular diagonal length representing a minimum closure region of the predicted and real frames; is a weight coefficient. The application can increase the weight occupied by regression loss and lead the network to pay more attention to the learning of frame information. The application expects that the detection result can contain the whole watermark information as far as possible, in order to prevent the frame from returning inaccurately, leading to the occurrence of truncation and affecting the recognition accuracy, a certain scale (specific size or proportion) expansion can be added after the detection result, for example, a plurality of pixels are expanded to the periphery on the result coordinates of the predicted frame, and the adjusted frame information contains more watermark information, thereby being beneficial to improving the accuracy of the subsequent watermark brand recognition.

Example III

Fig. 3 is a flowchart of a watermark detection method according to a third embodiment of the present application. As shown in fig. 3, the watermark detection method may include the steps of:

s301, inputting an image to be identified into a pre-trained single-target watermark detector; watermark detection is carried out on the current sample image through a single-target watermark detector, and a watermark detection frame in the image to be identified and the confidence corresponding to the watermark detection frame are obtained.

And S302, adjusting the confidence coefficient corresponding to the watermark detection frame according to a preset method in response to the confidence coefficient corresponding to the watermark detection frame being greater than or equal to a first confidence coefficient threshold value, so as to obtain the adjusted confidence coefficient corresponding to the watermark detection frame.

And S303, responding to the fact that the adjusted confidence coefficient corresponding to the watermark detection frame is larger than or equal to a second confidence coefficient threshold value, and extracting a feature map vector corresponding to the watermark detection frame based on the watermark detection frame.

In a specific embodiment of the present application, the electronic device may extract the feature map vector corresponding to the watermark detection box based on the watermark detection box in response to the adjusted confidence level corresponding to the watermark detection box being greater than or equal to the second confidence level threshold. In one embodiment, the electronic device may use a multi-layer convolutional network (5 layers before VGG or 18 layers before res net may be selected) to extract the semantic features of the context in combination with a Channel-Wise Attention mechanism (Channel-Wise Attention), and may further add an RMAC to perform L2 regularization and PCR whitening on the feature map, so as to better extract the watermark texture features, and compress the features into a vector, where the vector is 128 or 256 dimensions; and then searching the compressed vector in a watermark brand library by using a search engine, and taking the cosine distance of ascending order as a final search result.

S304, carrying out vector matching on the feature map vector corresponding to the watermark detection frame and each feature map vector in the pre-built watermark brand feature library.

In a specific embodiment of the present application, the electronic device may perform vector matching on the feature map vector corresponding to the watermark detection frame and each feature map vector in the pre-built watermark brand feature library. In one embodiment, the electronic device may calculate a cosine distance between the feature map vector corresponding to the watermark detection frame and each feature map vector in the watermark brand feature library, where the cosine distance indicates a matching degree between the feature map vector corresponding to the watermark detection frame and each feature map vector in the watermark brand feature library. Specifically, the cosine distance can be expressed as:

wherein dist (A, B) is the calculated cosine distance, A _i The feature map vector corresponding to the watermark detection frame; b (B) _i Is any feature map vector in the watermark brand feature library.

And S305, in response to successful matching of the feature map vector corresponding to the watermark detection frame and any one of the feature map vectors in the watermark brand feature library, performing authorized matching on the watermark name of any one of the vectors and each brand authorization word in the watermark brand authorization word list.

In a specific embodiment of the present application, the electronic device may perform authorization matching on the watermark name of any one of the feature map vectors and each brand authorization word in the watermark brand authorization word list in response to successful matching of the feature map vector corresponding to the watermark detection frame and any one of the feature map vectors in the watermark brand feature library. Specifically, the electronic device may calculate a matching value between the watermark name of the arbitrary vector and each brand authorization word in the watermark brand authorization word list, and determine whether the watermark name of the arbitrary vector matches each brand authorization word in the watermark brand authorization word list according to the matching value.

S306, in response to the fact that the watermark name of any vector is successfully matched with any brand authorization word in the watermark brand authorization word list, determining that the watermark in the watermark detection frame is authorized.

In a specific embodiment of the present application, the electronic device may determine that the watermark in the watermark detection box has been authorized in response to the watermark name of the arbitrary vector successfully matching with any brand authorization word in the watermark brand authorization vocabulary; and determining that the watermark in the watermark detection frame is an unauthorized watermark in response to the fact that the watermark name of any one vector is not successfully matched with each brand authorization word in the watermark brand authorization word list.

Preferably, in a specific embodiment of the present application, the electronic device may also train the single-target watermark detector before inputting the image to be identified into the pre-trained single-target watermark detector. Specifically, in response to the single-target watermark detector not meeting a preset convergence condition, firstly extracting a sample image from a pre-constructed sample image library to serve as a current sample image; wherein the sample image comprises: positive sample image or negative sample image; then inputting the current sample image to a single-target watermark detector, and training the single-target watermark detector by using the current sample image; the above operations are repeatedly performed until the single-target watermark detector satisfies the convergence condition.

Preferably, in a specific embodiment of the present application, the electronic device may also construct the sample image library before extracting a sample image from the pre-constructed sample image library as the current sample image. Specifically, in response to the number of sample images in the sample image library being smaller than a preset image number threshold, extracting a watermark template from a pre-constructed watermark template library as a current watermark template; then constructing at least one sample image corresponding to the current watermark template based on the current watermark template; and adding at least one sample image into the sample image library, and repeatedly executing the operation until the number of the sample images in the sample image library is equal to a preset image number threshold.

The existing target detection algorithm based on deep learning is based on a large amount of labeled data, and the detection effect of the algorithm is affected by the data quantity, the data diversity, the data distribution and the like. The watermark pictures are difficult to collect and cover real data of product lines comprehensively, the watermark pictures have the characteristic of long-tail data, the frequency of occurrence of individual mass watermarks is low, and the models with less data in training sets are difficult to fit and learn. The application adopts various data construction enhancement modes, collects a batch of online real watermark data templates, expands template data sets according to enhancement modes of different fonts, sizes, colors, inclination angles, transparency and the like, randomly covers the templates to the cleaned watermark-free data sets, thereby constructing a generated sample image library, and stores the vertex coordinates of the upper left corner and the lower right corner of the template watermark in the mapping process. The construction mode can cover a large amount of real watermark data, only the watermark template needs to be collected, the quick expansion can be realized, and a large amount of labor cost does not need to be saved by labeling. In addition, the electronic equipment can splice watermark pictures generated by a plurality of structures to obtain a picture containing a plurality of watermark targets, the data set can be enriched by the structure mode, and the problem of unbalanced positive and negative samples of the data set is solved to a certain extent. For example, 4 watermarked pictures are randomly selected, rearranged to the same size, and spliced in a 2×2 manner to obtain a picture containing 4 watermarks of different types, and the positions of the watermarks in the picture are recorded as tag information. The method can iteratively collect the negative example sample obtained by misidentification of the initial edition detection model as a basic sample set for template mapping again, and the watermark picture synthesis is carried out in the same way as the above, so that more negative example information can be added into model training, and the generalization capability of the model is enhanced. It can be seen that the data set construction mode provided by the application can continuously and iteratively generate a large number of water-logging data sets, is flexible and expandable, and has no artificial data marking requirement in the whole flow.

Example IV

Fig. 4 is a schematic structural diagram of a watermark detection device according to a fourth embodiment of the present application. As shown in fig. 4, the apparatus 400 includes: a detection module 401, an adjustment module 402 and a judgment module 403; wherein, the liquid crystal display device comprises a liquid crystal display device,

the detection module 401 is configured to input an image to be identified to a pre-trained single-target watermark detector; watermark detection is carried out on the current sample image through the single-target watermark detector, so that a watermark detection frame in the image to be identified and confidence degrees corresponding to the watermark detection frame are obtained;

the adjusting module 402 is configured to adjust, according to a preset method, the confidence coefficient corresponding to the watermark detection frame in response to the confidence coefficient corresponding to the watermark detection frame being greater than or equal to the first confidence coefficient threshold value, to obtain an adjusted confidence coefficient corresponding to the watermark detection frame;

the determining module 403 is configured to perform brand authorization verification on the image to be identified, where the watermark in the watermark detection frame is detected to be an authorized watermark or an unauthorized watermark in response to the adjusted confidence coefficient corresponding to the watermark detection frame being greater than or equal to a second preset confidence coefficient threshold.

Further, the determining module 403 is further configured to determine that there is no watermark in the watermark detection frame in response to the confidence coefficient corresponding to the watermark detection frame being less than the first confidence coefficient threshold; or, in response to the adjusted confidence coefficient corresponding to the watermark detection frame being smaller than the second confidence coefficient threshold value, determining that no watermark exists in the watermark detection frame.

Further, the adjusting module 402 is specifically configured to perform word number recognition on the watermark detection frame by using an optical character recognition method, so as to obtain the number of words in the image to be recognized; and in response to the number of characters in the watermark detection frame being greater than a preset character number threshold, reducing the confidence corresponding to the watermark detection frame by a preset value.

Further, the adjusting module 402 is specifically configured to perform text region number recognition on the watermark detection frame by using an optical character recognition method, so as to obtain the number of text regions in the watermark detection frame; and in response to the number of the text areas in the watermark detection frame being greater than a preset text area number threshold, reducing the confidence corresponding to the watermark detection frame by a preset value.

Further, the adjusting module 402 is specifically configured to perform text region proportion recognition on the watermark detection frame by using an optical character recognition method, so as to obtain a proportion of the watermark detection frame in the image to be recognized; and in response to the proportion of the watermark detection frame to the image to be identified being greater than a preset proportion threshold, reducing the confidence corresponding to the watermark detection frame by a preset value.

Further, the adjusting module 402 is specifically configured to extract four edge areas of the watermark detection frame through an edge pixel detection algorithm; calculating pixel variance values of all edge areas in the four edge areas; obtaining the maximum pixel variance value in the four edge areas based on the pixel variance values of the edge areas; and responding to the maximum pixel variance value being smaller than a preset variance threshold, and reducing the confidence corresponding to the watermark detection frame by a preset value.

Further, the judging module 403 is specifically configured to extract a feature map vector corresponding to the watermark detection box based on the watermark detection box; vector matching is carried out on the feature map vector corresponding to the watermark detection frame and the vector in the pre-constructed watermark brand feature library; judging that the watermark in the watermark detection frame is an unauthorized watermark in response to the fact that the feature map vector corresponding to the watermark detection frame is not successfully matched with each vector in the watermark brand feature library; in response to successful matching of the feature map vector corresponding to the watermark detection frame and any one vector in the watermark brand feature library, performing authorized matching on the watermark name of any one vector and a watermark brand authorization vocabulary; responding to the successful matching of the watermark name of any vector and any watermark brand in the watermark brand authorization vocabulary, and judging that the watermark in the watermark detection frame is authorized; and judging that the watermark in the watermark detection frame is an unauthorized watermark in response to the fact that the watermark name of any vector is not successfully matched with each watermark brand in the watermark brand authorization vocabulary.

Further, the device further comprises: a training module 404 (not shown in the figure) for extracting a sample image from a pre-constructed sample image library as a current sample image in response to the single target watermark detector not meeting a pre-set convergence condition; wherein the sample image comprises: positive sample image or negative sample image; inputting the current sample image to the single-target watermark detector, and training the single-target watermark detector by using the current sample image; the above operation is repeatedly performed until the single target watermark detector satisfies the convergence condition.

Further, the training module 404 is further configured to extract a watermark template from a pre-constructed watermark template library as a current watermark template in response to the number of sample images in the sample image library being less than a preset image number threshold; constructing at least one sample image corresponding to the current watermark template based on the current watermark template; and adding the at least one sample image into the sample image library, and repeatedly executing the operation until the number of the sample images in the sample image library is equal to a preset image number threshold.

The watermark detection device can execute the method provided by any embodiment of the application, and has the corresponding functional modules and beneficial effects of the execution method. Technical details not described in detail in this embodiment may be referred to the watermark detection method provided in any embodiment of the present application.

Example five

According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

As shown in fig. 5, a block diagram of an electronic device of a watermark detection method according to an embodiment of the application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the applications described and/or claimed herein.

As shown in fig. 5, the electronic device includes: one or more processors 501, memory 502, and interfaces for connecting components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the electronic device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In other embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple electronic devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 501 is illustrated in fig. 5.

Memory 502 is a non-transitory computer readable storage medium provided by the present application. The memory stores instructions executable by the at least one processor to cause the at least one processor to perform the watermark detection method provided by the present application. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to execute the watermark detection method provided by the present application.

The memory 502 is used as a non-transitory computer readable storage medium, and may be used to store a non-transitory software program, a non-transitory computer executable program, and modules, such as program instructions/modules (e.g., the detection module 401, the adjustment module 402, and the determination module 403 shown in fig. 4) corresponding to the watermark detection method in the embodiment of the application. The processor 501 executes various functional applications of the server and data processing, i.e., implements the watermark detection method in the above-described method embodiments, by running non-transitory software programs, instructions, and modules stored in the memory 502.

Memory 502 may include a storage program area that may store an operating system, at least one application program required for functionality, and a storage data area; the storage data area may store data created according to the use of the electronic device of the watermark detection method, and the like. In addition, memory 502 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 502 may optionally include memory remotely located with respect to the processor 501, which may be connected to the electronic device of the watermark detection method via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The electronic device of the watermark detection method may further include: an input device 503 and an output device 504. The processor 501, memory 502, input devices 503 and output devices 504 may be connected by a bus or otherwise, for example in fig. 5.

The input device 503 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic device of the watermark detection method, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointer stick, one or more mouse buttons, a track ball, a joystick, etc. The output devices 504 may include a display device, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device may be a touch screen.

Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASIC (application specific integrated circuit), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

These computing programs (also referred to as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), the internet, and blockchain networks.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

According to the technical scheme of the embodiment of the application, an image to be identified is firstly input to a pre-trained single-target watermark detector; watermark detection is carried out on the current sample image through a single-target watermark detector, so that a watermark detection frame in the image to be identified and the confidence corresponding to the watermark detection frame are obtained; responding to the fact that the confidence coefficient corresponding to the watermark detection frame is larger than or equal to a preset first confidence coefficient threshold value, adjusting the confidence coefficient corresponding to the watermark detection frame according to a preset method to obtain an adjusted confidence coefficient corresponding to the watermark detection frame; and responding to the fact that the adjusted confidence coefficient corresponding to the watermark detection frame is larger than or equal to a preset second confidence coefficient threshold value, carrying out brand authorization verification on the image to be identified, and detecting whether the watermark in the watermark detection frame is an authorized watermark or an unauthorized watermark. That is, the present application can verify the watermark and its version authorization relationship by a single target watermark detector. In the existing watermark detection method, the conventional detection scheme and the detection scheme based on deep learning do not consider the problem of copyright identification, and the low-quality watermark, the pirate infringement watermark and the watermark conforming to authorization cannot be distinguished. Because the application adopts the technical means of watermark detection and preset confidence threshold corresponding to version authorization through the single-target watermark detector, the technical problem that the low-quality watermark, pirate infringement watermark and watermark conforming to authorization cannot be distinguished in the prior art is solved; in addition, the technical scheme of the embodiment of the application is simple and convenient to realize, convenient to popularize and wider in application range.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present application may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution disclosed in the present application can be achieved, and are not limited herein.

The above embodiments do not limit the scope of the present application. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of the present application.

Claims

1. A method of watermark detection, the method comprising:

inputting an image to be identified into a pre-trained single-target watermark detector; watermark detection is carried out on the current sample image through the single-target watermark detector, so that a watermark detection frame in the image to be identified and the confidence corresponding to the watermark detection frame are obtained;

responding to the fact that the confidence coefficient corresponding to the watermark detection frame is larger than or equal to a preset first confidence coefficient threshold value, adjusting the confidence coefficient corresponding to the watermark detection frame according to a preset method to obtain an adjusted confidence coefficient corresponding to the watermark detection frame; the preset method comprises the following steps: a character number recognition method, a character area proportion recognition method or an edge area detection method;

And responding to the fact that the adjusted confidence coefficient corresponding to the watermark detection frame is larger than or equal to a preset second confidence coefficient threshold value, carrying out brand authorization verification on the image to be identified, and detecting that the watermark in the watermark detection frame is an authorized watermark or an unauthorized watermark through the brand authorization verification.

2. The method according to claim 1, wherein the method further comprises:

determining that no watermark exists in the watermark detection frame in response to the confidence coefficient corresponding to the watermark detection frame being smaller than the first confidence coefficient threshold; or, in response to the adjusted confidence coefficient corresponding to the watermark detection frame being smaller than the second confidence coefficient threshold value, determining that no watermark exists in the watermark detection frame.

3. The method according to claim 1, wherein the adjusting the confidence level corresponding to the watermark detection frame according to the preset method includes:

performing character number recognition on the watermark detection frame by an optical character recognition method to obtain the number of characters in the image to be recognized; and in response to the number of characters in the watermark detection frame being greater than a preset character number threshold, reducing the confidence corresponding to the watermark detection frame by a preset value.

4. The method according to claim 1, wherein the adjusting the confidence level corresponding to the watermark detection frame according to the preset method includes:

carrying out character area number identification on the watermark detection frame by an optical character identification method to obtain the number of character areas in the watermark detection frame; and in response to the number of the text areas in the watermark detection frame being greater than a preset text area number threshold, reducing the confidence corresponding to the watermark detection frame by a preset value.

5. The method according to claim 1, wherein the adjusting the confidence level corresponding to the watermark detection frame according to the preset method includes:

performing text region proportion recognition on the watermark detection frame by an optical character recognition method to obtain the proportion of the watermark detection frame to the image to be recognized; and in response to the proportion of the watermark detection frame to the image to be identified being greater than a preset proportion threshold, reducing the confidence corresponding to the watermark detection frame by a preset value.

6. The method according to claim 1, wherein the adjusting the confidence level corresponding to the watermark detection frame according to the preset method includes:

Extracting four edge areas of the watermark detection frame through an edge pixel detection algorithm;

calculating pixel variance values of all edge areas in the four edge areas; obtaining the maximum pixel variance value in the four edge areas based on the pixel variance values of the edge areas;

and responding to the maximum pixel variance value being smaller than a preset variance threshold, and reducing the confidence corresponding to the watermark detection frame by a preset value.

7. The method of claim 1, wherein the performing brand authorization verification on the image to be identified, detecting whether the watermark in the watermark detection box is an authorized watermark or an unauthorized watermark, comprises:

extracting a feature map vector corresponding to the watermark detection frame based on the watermark detection frame;

carrying out vector matching on the feature map vector corresponding to the watermark detection frame and each feature map vector in a pre-constructed watermark brand feature library;

judging that the watermark in the watermark detection frame is an unauthorized watermark in response to the fact that the feature map vector corresponding to the watermark detection frame is not successfully matched with each feature map vector in the watermark brand feature library; in response to successful matching of the feature map vector corresponding to the watermark detection frame and any one of the feature map vectors in the watermark brand feature library, performing authorized matching on the watermark name of any one of the vectors and each brand authorization word in the watermark brand authorization word list; responding to the successful matching of the watermark name of any vector and any brand authorization word in the watermark brand authorization word list, and judging that the watermark in the watermark detection frame is authorized; and judging that the watermark in the watermark detection frame is an unauthorized watermark in response to the fact that the watermark name of any one vector is not successfully matched with each brand authorization word in the watermark brand authorization word list.

8. The method of claim 1, wherein prior to said inputting the image to be identified to a pre-trained single-target watermark detector, the method further comprises:

in response to the single-target watermark detector not meeting a preset convergence condition, extracting a sample image from a pre-constructed sample image library as a current sample image; wherein the sample image comprises: positive sample image or negative sample image;

inputting the current sample image to the single-target watermark detector, and training the single-target watermark detector by using the current sample image; the above operation is repeatedly performed until the single target watermark detector satisfies the convergence condition.

9. The method of claim 8, wherein prior to extracting a sample image from the library of pre-constructed sample images as the current sample image, the method further comprises:

extracting a watermark template from a pre-constructed watermark template library to serve as a current watermark template in response to the number of sample images in the sample image library being smaller than a preset image number threshold;

constructing at least one sample image corresponding to the current watermark template based on the current watermark template; and adding the at least one sample image into the sample image library, and repeatedly executing the operation until the number of the sample images in the sample image library is equal to a preset image number threshold.

10. A watermark detection device, the device comprising: the device comprises a detection module, an adjustment module and a judgment module; wherein, the liquid crystal display device comprises a liquid crystal display device,

the detection module is used for inputting the image to be identified into a pre-trained single-target watermark detector; watermark detection is carried out on the current sample image through the single-target watermark detector, so that a watermark detection frame in the image to be identified and the confidence corresponding to the watermark detection frame are obtained;

the adjusting module is used for adjusting the confidence coefficient corresponding to the watermark detection frame according to a preset method to obtain the adjusted confidence coefficient corresponding to the watermark detection frame in response to the fact that the confidence coefficient corresponding to the watermark detection frame is larger than or equal to a preset first confidence coefficient threshold value; the preset method comprises the following steps: a character number recognition method, a character area proportion recognition method or an edge area detection method;

and the judging module is used for carrying out brand authorization verification on the image to be identified in response to the fact that the adjusted confidence coefficient corresponding to the watermark detection frame is larger than or equal to a preset second confidence coefficient threshold value, and detecting that the watermark in the watermark detection frame is an authorized watermark or an unauthorized watermark through the brand authorization verification.

11. The apparatus of claim 10, wherein the determining module is further configured to determine that there is no watermark in the watermark detection box in response to a confidence level corresponding to the watermark detection box being less than the first confidence threshold; or, in response to the adjusted confidence coefficient corresponding to the watermark detection frame being smaller than the second confidence coefficient threshold value, determining that no watermark exists in the watermark detection frame.

12. The apparatus according to claim 10, wherein:

the adjusting module is specifically configured to perform word number recognition on the watermark detection frame through an optical character recognition method, so as to obtain the number of words in the image to be recognized; and in response to the number of characters in the watermark detection frame being greater than a preset character number threshold, reducing the confidence corresponding to the watermark detection frame by a preset value.

13. The apparatus according to claim 10, wherein:

the adjusting module is specifically configured to perform text region number identification on the watermark detection frame through an optical character identification method, so as to obtain the number of text regions in the watermark detection frame; and in response to the number of the text areas in the watermark detection frame being greater than a preset text area number threshold, reducing the confidence corresponding to the watermark detection frame by a preset value.

14. The apparatus according to claim 10, wherein:

the adjusting module is specifically configured to perform text region proportion identification on the watermark detection frame by using an optical character identification method, so as to obtain the proportion of the watermark detection frame in the image to be identified; and in response to the proportion of the watermark detection frame to the image to be identified being greater than a preset proportion threshold, reducing the confidence corresponding to the watermark detection frame by a preset value.

15. The apparatus according to claim 10, wherein:

the adjusting module is specifically configured to extract four edge areas of the watermark detection frame through an edge pixel detection algorithm; calculating pixel variance values of all edge areas in the four edge areas; obtaining the maximum pixel variance value in the four edge areas based on the pixel variance values of the edge areas; and responding to the maximum pixel variance value being smaller than a preset variance threshold, and reducing the confidence corresponding to the watermark detection frame by a preset value.

16. The apparatus according to claim 10, wherein:

the judging module is specifically used for extracting a feature map vector corresponding to the watermark detection frame based on the watermark detection frame; vector matching is carried out on the feature map vector corresponding to the watermark detection frame and the vector in the pre-constructed watermark brand feature library; judging that the watermark in the watermark detection frame is an unauthorized watermark in response to the fact that the feature map vector corresponding to the watermark detection frame is not successfully matched with each vector in the watermark brand feature library; in response to successful matching of the feature map vector corresponding to the watermark detection frame and any one vector in the watermark brand feature library, performing authorized matching on the watermark name of any one vector and a watermark brand authorization vocabulary; responding to the successful matching of the watermark name of any vector and any watermark brand in the watermark brand authorization vocabulary, and judging that the watermark in the watermark detection frame is authorized; and judging that the watermark in the watermark detection frame is an unauthorized watermark in response to the fact that the watermark name of any vector is not successfully matched with each watermark brand in the watermark brand authorization vocabulary.

17. The apparatus of claim 10, wherein the apparatus further comprises: the training module is used for responding that the single-target watermark detector does not meet a preset convergence condition, and extracting a sample image from a pre-constructed sample image library to serve as a current sample image; wherein the sample image comprises: positive sample image or negative sample image; inputting the current sample image to the single-target watermark detector, and training the single-target watermark detector by using the current sample image; the above operation is repeatedly performed until the single target watermark detector satisfies the convergence condition.

18. The apparatus according to claim 17, wherein:

the training module is further configured to extract a watermark template from a watermark template library constructed in advance as a current watermark template in response to the number of sample images in the sample image library being less than a preset image number threshold; constructing at least one sample image corresponding to the current watermark template based on the current watermark template; and adding the at least one sample image into the sample image library, and repeatedly executing the operation until the number of the sample images in the sample image library is equal to a preset image number threshold.

19. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein, the liquid crystal display device comprises a liquid crystal display device,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9.

20. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-9.