CN111062903A - Automatic processing method and system for image watermark, electronic equipment and storage medium - Google Patents

Abstract

The invention discloses an automatic processing method, a system, electronic equipment and a storage medium of image watermarks, wherein the automatic processing method comprises a first model training method which comprises the steps of obtaining a synthetic image; acquiring an original background image which does not contain the watermark and corresponds to the synthetic image; dividing the synthetic image and the original background image into a training set and a prediction set; training the images in the training set in a first machine learning model; predicting the probability of the watermark and the watermark prediction position in the trained machine learning model by using the images in the prediction set; and judging whether the accuracy and the recall rate of the prediction result of the prediction set reach a detection accuracy threshold and a detection recall threshold, and if so, finishing the training of the first model. The automatic processing method can automatically detect whether the image contains the watermark or not and the specific position of the watermark, thereby not only avoiding the low efficiency of manually searching and smearing the watermark, but also avoiding the defects of image tampering and low removal efficiency caused by directly inputting the image.

Description

Automatic processing method and system for image watermark, electronic equipment and storage medium

Technical Field

The present invention relates to the field of image recognition and image processing technologies, and in particular, to an automatic processing method and system for image watermarking, an electronic device, and a storage medium.

Background

The image can ensure the visual representation and transmission of information, so that the large-scale transmission and application in the Internet environment can be realized, and the user experience can be greatly improved by properly displaying the image. Because the source of the image is usually complex, a large number of images pushed from different sources often contain partial images with watermarks, and the direct display affects the browsing of the images by users, even can cause infringement legal problems.

The removal of the watermark is a common requirement, and the current removal of the watermark comprises two common methods, wherein the first method is to manually remove the watermark by smearing pixel points filling a watermark region by using an image processing tool in an interactive mode, and the method needs to process the watermark pictures one by one, so that the processing speed is low, and the efficiency is low. The second method for removing water is applied to the situation that a large number of pictures containing watermarks exist, and the method can be uniformly removed through an algorithm, however, the current method for removing the watermarks through the algorithm usually directly inputs a watermark image into an algorithm model to output a watermark-free picture, whereas the watermark usually occupies only a small local area of the image, and the direct input of the model can cause the image in a non-watermark area to be tampered. In addition, the existing watermark removing algorithm estimates the position and transparency of the watermark by utilizing a large number of pictures containing the same type of watermark, only can process a single type of known watermark, but cannot process watermarks of various types and unknown patterns or positions.

Disclosure of Invention

The invention provides an automatic processing method of an image watermark, an electronic device and a readable storage medium, aiming at overcoming the defects of low watermark removal efficiency, inaccurate removed watermark position and incapability of removing complex watermarks in the prior art.

The invention solves the technical problems through the following technical scheme:

the invention provides an automatic processing method of image watermarks, which comprises a first model training method:

the first model training method comprises the following steps:

acquiring a composite image, wherein the composite image comprises a watermark;

acquiring an original background image which does not contain the watermark and corresponds to the synthetic image;

dividing the synthetic image and the original background image into a training set and a prediction set;

taking images in a training set as input, and taking whether each image contains a watermark and the specific position of the watermark in the image as output to a first machine learning model for training until a first loss function is converged;

taking the images in the prediction set as input into a trained machine learning model to obtain the probability that each image in the prediction set contains the watermark and the watermark prediction position;

and judging whether the accuracy and the recall rate of the prediction result of the prediction set reach a detection accuracy threshold and a detection recall threshold, if so, determining that the trained first machine learning model is a trained first model.

Wherein the first machine learning model is a deep convolutional neural network model.

Wherein, the deep convolutional neural network model comprises 48 convolutional layers and 3 pyramid modules.

In the invention, whether the image to be processed contains the watermark or not and the specific position of the watermark in the image can be automatically detected through the trained first model, so that the low efficiency of manually searching and smearing the watermark is avoided, the defect that the original image is distorted due to the fact that the image to be processed is directly input into a watermark removing algorithm or the watermark removing efficiency is low due to the fact that the removed part is a non-watermark part and the like because of inaccurate watermark positioning is avoided, a foundation is laid for subsequent targeted removal of the related watermark through automatically and accurately finding the specific position of the watermark, and the efficiency of automatically removing the watermark is greatly improved.

Preferably, the first loss function is:

Class loss＝-α_t(1-p_t)^γlog(p_t),γ>0,α^t∈[0,1]

wherein Class Loss represents a probability Loss function, location Loss represents a position Loss function, and the first Loss function is the sum of the probability Loss function and the position Loss function;

wherein p is_tIs the probability of the image containing the watermark, x is the predicted location of the watermark, α_tAnd γ are both weight coefficients.

In the invention, the first model can be optimized through the combined first loss function, so that more accurate prediction results of whether the watermark exists and the specific position of the watermark can be obtained.

Preferably, the first and second liquid crystal films are made of a polymer,

obtaining the composite image by:

acquiring an original watermark image;

acquiring an original background image;

automatically embedding the original watermark image into the original background image to generate the composite image.

In the invention, a large-scale synthetic image, namely an image watermark data set is constructed, and the defect that the application of the image watermark detection and removal technology in an OTA (on-line travel agency) scene is restricted due to the lack of an image multi-class watermark data set in the prior art is overcome.

Preferably, the automatic processing method includes:

obtaining an original watermark image by the following steps:

acquiring various types of watermark pictures;

removing the background color of the watermark picture;

automatically converting the watermark picture with the background color removed into an RGBA (red, green and blue) (image color representation method containing transparency) format to obtain the original watermark image;

and/or the presence of a gas in the gas,

the step of automatically embedding the original watermark image into the original background image to generate the composite image specifically includes:

randomly selecting the original watermark image and the original background image;

randomly setting the size, transparency and position of the selected original watermark image in the original background image;

embedding the selected original watermark image into the selected original background image to generate a composite image;

the original watermark images of the same type are embedded in different original background images with different sizes, transparencies and positions respectively.

According to the invention, a large-scale non-repetitive synthetic image is constructed, and the model training efficiency is further improved.

Preferably, the automatic processing method includes:

inputting an image to be processed into the first model to detect whether the image to be processed contains a watermark and the position of the watermark in the image to be processed.

In the invention, the watermark area can be effectively positioned before removing the watermark, and the specific position of the watermark in the image is detected to remove the watermark pertinently and accurately, thereby avoiding the defect that the image of the non-watermark area is easy to be falsified because the watermark image is directly input into an algorithm to output a watermark-free image in the prior art.

Preferably, the first and second liquid crystal films are made of a polymer,

in the step of inputting an image to be processed into the first model to detect whether the image to be processed contains a watermark, if the image to be processed contains the watermark, cutting an image area containing the watermark based on the position of the watermark in the image to be processed to obtain a first image;

the automatic processing method further comprises a second model training method;

the second model training method includes:

dividing the first image into a training set and a prediction set;

taking the training set in the first image as input and the non-watermark image corresponding to the first image as output to a generating network for training until a generating loss function is converged;

inputting the prediction set in the first image into a trained generation network to obtain a second image without a watermark;

and judging whether the accuracy and the recall rate of the second image without the watermark respectively reach a generation accuracy threshold and a recall accuracy threshold, if so, determining the trained generation network as the trained generation network.

The generation network is a U-net (an extreme learning model) -based full convolution network, the U-net-based full convolution network comprises l convolution modules, and the ith convolution module is in hopping connection with the l-i convolution modules.

Preferably, the generating loss function is defined as follows:

wherein α weight coefficient, L_L1Is a distance loss function of the pixel, L_L1Is defined as follows:

L_L1(x,y)＝‖f(x)-y‖₁

wherein x and y represent the x, y coordinates of each pixel, respectively, f (x) represents the second image through the generation network, y represents the true, watermark-free image corresponding to the second image, | f (x) -y |₁A norm representing f (x) -y;

wherein L is_plA loss function obtained by computing semantic feature distances of an image for perceptual loss, which is defined as follows:

where Φ represents VGG16 (a type of visual object recognition software study) trained on ImageNet (a large visual database for visual object recognition software research)Neural network model), j represents the j-th layer of the network,

representing said second image through the generation network, y representing a true, watermark-free image corresponding to the second image,

to represent

Image semantic features on the jth layer of the trained network, [ phi ] j (y) denotes the image semantic features of y on the jth layer of the trained network, C_j、H_jAnd W_jRespectively representing the number, height and width of feature maps of the relu2.2 level features,

to represent

The second paradigm of (1).

Preferably, the second model training method further comprises:

dividing each of the second images into a training set and a prediction set;

taking the training set in the second image as input and a plurality of watermark-free feature maps corresponding to the second image as output to a discriminant network for training until the cross entropy loss function is converged;

inputting the prediction set in the second image into the trained discrimination network to obtain a result of whether each feature map in each second image in the prediction set contains a watermark;

and judging whether the accuracy and the recall rate of the result reach a judgment accuracy threshold and a judgment recall threshold, if so, the trained judgment network is a trained judgment network.

The discriminant network is a PatchGAN (machine learning model) -based full convolution network, the PatchGAN-based full convolution network comprises four convolution layers, the step length of each convolution layer is 2, and the size of each convolution kernel is 3 x 3.

Preferably, the second model training method further comprises: and judging whether the generated network and the judging network are trained completely, if so, taking the second model comprising the trained generated network and the trained judging network as a trained second model.

The invention overcomes the defects that the image of the current watermark is various and unknown, and the efficiency of a manual image processing tool processing mode is low. According to the image data characteristics printed with the watermarks, an algorithm model is established, a network and a judgment network are further generated through training according to the positions of the watermarks of the detected images output by the first model, so that the purpose of removing the watermarks can be effectively achieved based on the deep convolution neural network, the maintenance cost of operation can be greatly saved, the reasonable and proper display requirements of the images can be met, and the user experience is improved.

Preferably, the image to be processed comprises a first image and the rest images of the first image are eliminated;

after training the second model, the automatic processing method further comprises:

inputting the first image into the trained second model to obtain a third image without a watermark corresponding to the first image;

replacing the first image in the to-be-processed image with the third image to obtain a fourth image;

and/or the presence of a gas in the gas,

based on the position of the watermark in the image to be processed, the step of cutting out the image area containing the watermark to obtain a first image comprises the following steps:

and expanding the range of the detected image area containing the watermark by a preset times value, and then cutting the expanded range to obtain a first image.

In the invention, the whole image without the watermark can be obtained by replacing the watermark part with the watermark part, and the defect that part of the watermark is not completely removed due to errors can be avoided by cutting the enlarged image area, thereby improving the accuracy of removing the watermark.

Preferably, the step of replacing the first image in the to-be-processed image with the third image to obtain a fourth image further includes:

adjusting pixel values of regions of the third image in the fourth image according to the following formula:

where y represents the adjusted pixel value, w represents the pixel replacement weighting factor, and x₁For pixels containing watermark image blocks, x₂And if the automatic processing method further comprises the step of expanding the range of the detected image area containing the watermark by a preset times value and then cutting the expanded range to obtain a first image, the pixel replacement weighting factor is the preset times value.

In the invention, the watermark-free picture without loss of image quality can be obtained by adjusting the pixel value of the replacement area.

The invention also provides an automatic processing system of the image watermark, which comprises a first model training module;

the first model training module comprises a synthetic image acquisition unit, an original background image acquisition unit, a first division unit, a first training unit, a first prediction unit and a first judgment unit:

the composite image acquisition unit is used for acquiring a composite image, and the composite image comprises a watermark;

the original background image acquisition unit is used for acquiring an original background image which does not contain a watermark and corresponds to the composite image;

the first dividing unit is used for dividing the synthetic image and the original background image into a training set and a prediction set;

the first training unit is used for taking images in a training set as input, judging whether each image contains a watermark and outputting the specific position of the watermark in the image to a first machine learning model for training until a first loss function is converged;

the first prediction unit is used for inputting the images in the prediction set into the trained machine learning model to obtain the probability that each image in the prediction set contains the watermark and the watermark prediction position;

the first judging unit is used for judging whether the accuracy and the recall rate of the prediction result of the prediction set reach a detection accuracy threshold and a detection recall threshold, and if so, the trained first machine learning model is a trained first model.

Wherein the first machine learning model is a deep convolutional neural network model.

Wherein, the deep convolutional neural network model comprises 48 convolutional layers and 3 pyramid modules.

In the invention, whether the image to be processed contains the watermark or not and the specific position of the watermark in the image can be automatically detected through the trained first model, so that the low efficiency of manually searching and smearing the watermark is avoided, the defect that the original image is distorted due to the fact that the image to be processed is directly input into a watermark removing algorithm or the watermark removing efficiency is low due to the fact that the removed part is a non-watermark part and the like because of inaccurate watermark positioning is avoided, a foundation is laid for subsequent targeted removal of the related watermark through automatically and accurately finding the specific position of the watermark, and the efficiency of automatically removing the watermark is greatly improved.

Preferably, the first loss function is:

Class loss＝-α_t(1-p_t)^γlog(p_t),γ>0,α^t∈[0,1]

wherein Class Loss represents a probability Loss function, location Loss represents a position Loss function, and the first Loss function is the sum of the probability Loss function and the position Loss function;

wherein p is_tIs the probability of the image containing the watermark, x is the predicted location of the watermark, α_tAnd γ are both weight coefficients.

In the invention, the first model can be optimized through the combined first loss function, so that more accurate prediction results of whether the watermark exists and the specific position of the watermark can be obtained.

Preferably, the composite image obtaining unit includes a watermark image obtaining subunit, a background image obtaining subunit and an embedding subunit;

the watermark image acquisition subunit is used for acquiring an original watermark image;

the background image acquisition subunit is used for acquiring an original background image;

the embedding subunit is configured to automatically embed the original watermark image into the original background image to generate the composite image.

In the invention, a large-scale synthetic image, namely an image watermark data set is constructed, and the defect that the application of the image watermark detection and removal technology in an OTA scene is restricted due to the lack of an image multi-class watermark data set in the prior art is overcome.

Preferably, the watermark image obtaining subunit is specifically configured to obtain multiple types of watermark images, remove a background color of the watermark images, and automatically convert the watermark images with the background color removed into an RGBA format to obtain the original watermark image;

and/or the presence of a gas in the gas,

the embedding subunit is specifically configured to randomly select the original watermark image and the original background image, randomly set the size, transparency, and position of the selected original watermark image in the original background image, and embed the selected original watermark image into the selected original background image, so as to generate a composite image;

the original watermark images of the same type are embedded in different original background images with different sizes, transparencies and positions respectively.

According to the invention, a large-scale non-repetitive synthetic image is constructed, and the model training efficiency is further improved.

Preferably, the automatic processing system further comprises: the watermark detection module is used for inputting the image to be processed into the first model so as to detect whether the image to be processed contains the watermark or not and the position of the watermark in the image to be processed.

In the invention, the watermark area can be effectively positioned before removing the watermark, and the specific position of the watermark in the image is detected to remove the watermark pertinently and accurately, thereby avoiding the defect that the image of the non-watermark area is easy to be falsified because the watermark image is directly input into an algorithm to output a watermark-free image in the prior art.

The generation network is a U-net-based full convolution network, the U-net-based full convolution network comprises l convolution modules, and the ith convolution module is in hopping connection with the l-i convolution modules.

Preferably, the watermark detection module is further configured to, when it is detected that the to-be-processed image includes a watermark, clip an image area including the watermark based on a position of the watermark in the to-be-processed image to obtain a first image;

the automated processing system further comprises a second model training module;

the second model training module comprises: the device comprises a second dividing unit, a second training unit, a second prediction unit and a second judgment unit;

the second dividing unit is used for dividing the first image into a training set and a prediction set;

the second training unit is used for taking a training set in the first image as input and taking a non-watermark image corresponding to the first image as output to a generation network for training until a generation loss function is converged;

the second prediction unit is used for inputting the prediction set in the first image into a trained generation network to obtain a second image without a watermark;

the second judging unit is used for judging whether the accuracy and the recall rate of the second image without the watermark respectively reach a generation accuracy threshold and a recall accuracy threshold, if so, the trained generation network is the trained generation network.

Preferably, the generating loss function is defined as follows:

wherein α weight coefficient, L_L1Is a distance loss function of the pixel, L_L1Is defined as follows:

L_L1(x,y)＝‖f(x)-y‖₁

wherein x and y represent the x, y coordinates of each pixel, respectively, f (x) represents the second image through the generation network, y represents the true, watermark-free image corresponding to the second image, | f (x) -y |₁A norm representing f (x) -y;

wherein L is_plA loss function obtained by computing semantic feature distances of an image for perceptual loss, which is defined as follows:

where Φ represents the VGG16 network trained on ImageNet, j represents the j-th layer of the network,

representing said second image through the generation network, y representing a true, watermark-free image corresponding to the second image,

to represent

Semantic features of the image at level j of the trained net,. phi.j (y) denotes that y is in the trained netImage semantic features on layer j of the network, C_j、H_jAnd W_jRespectively representing the number, height and width of feature maps of the relu2.2 level features,

to represent

The second paradigm of (1).

Preferably, the second model training module further comprises: the device comprises a third dividing unit, a third training unit, a third prediction unit and a third judgment unit;

the third dividing unit is used for dividing each second image into a training set and a prediction set;

the third training unit is used for taking a training set in the second image as input and a plurality of watermark-free feature maps corresponding to the second image as output to a discriminant network for training until a cross entropy loss function converges;

the third prediction unit is used for inputting the prediction set in the second image into the trained discrimination network to obtain a result of whether each feature map in each second image in the prediction set contains a watermark or not;

the third judging unit is used for judging whether the accuracy and the recall rate of the result reach a judgment accuracy threshold and a judgment recall threshold, if so, the trained judgment network is a trained judgment network.

The judgment network is a PatchGAN-based full convolution network, the PatchGAN-based full convolution network comprises four convolution layers, the step length of each convolution layer is 2, and the size of each convolution kernel is 3 x 3.

Preferably, the second model training module further includes a training completion determining unit, configured to determine whether both the generating network and the determining network are trained, and if so, the second model including the trained generating network and the trained determining network is a trained second model.

The invention overcomes the defects that the image of the current watermark is various and unknown, and the efficiency of a manual image processing tool processing mode is low. According to the image data characteristics printed with the watermarks, an algorithm model is established, a network and a judgment network are further generated through training according to the positions of the watermarks of the detected images output by the first model, so that the purpose of removing the watermarks can be effectively achieved based on the deep convolution neural network, the maintenance cost of operation can be greatly saved, the reasonable and proper display requirements of the images can be met, and the user experience is improved.

Preferably, the image to be processed comprises a first image and the rest images of the first image are eliminated;

the automated processing system further comprises: a watermark-free image generation module and a replacement module;

the watermark-free image generation module is used for inputting the first image into the trained second model to obtain a watermark-free third image corresponding to the first image;

the replacing module is used for replacing the first image in the to-be-processed image with the third image to obtain a fourth image;

and/or the presence of a gas in the gas,

the watermark detection module is further used for expanding the range of the detected image area containing the watermark by a preset times value and then cutting the expanded range to obtain the first image.

In the invention, the whole image without the watermark can be obtained by replacing the watermark part with the watermark part, and the defect that part of the watermark is not completely removed due to errors can be avoided by cutting and enlarging the image area by using the image, thereby improving the accuracy of removing the watermark.

Preferably, the automatic processing system further comprises a pixel adjusting module for adjusting pixel values of a region of the third image in the fourth image according to the following formula:

where y represents the adjusted pixel value, w represents the pixel replacement weighting factor, and x₁For pixels containing watermark image blocks, x₂And if the automatic processing method further comprises the step of expanding the range of the detected image area containing the watermark by a preset times value and then cutting the expanded range to obtain a first image, the pixel replacement weighting factor is the preset times value.

In the invention, the watermark-free picture without loss of image quality can be obtained by adjusting the pixel value of the replacement area.

The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, and is characterized in that the processor realizes the automatic processing method when executing the computer program.

The invention also provides a computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned automatic processing method.

The positive progress effects of the invention are as follows:

in the invention, whether the image to be processed contains the watermark or not and the specific position of the watermark in the image can be automatically detected through the trained watermark detection model, so that the low efficiency of manually searching and smearing the watermark is avoided, the defect that the original image is distorted due to the fact that the image to be processed is directly input into a watermark removal algorithm or the watermark removal efficiency is low due to the fact that the removed part is a non-watermark part and the like because of inaccurate watermark positioning is avoided, a foundation is laid for subsequent targeted removal of the related watermark through automatically and accurately finding the specific position of the watermark, and the efficiency of automatically removing the watermark is greatly improved.

Drawings

Fig. 1 is a flowchart of a first model training method in the automatic processing method of image watermarking according to embodiment 1.

Fig. 2 is a flowchart of a specific implementation method of step 101 in embodiment 2.

Fig. 3 is a partial flowchart of a second model training method in the automatic processing method for image watermarking according to embodiment 3.

Fig. 4 is a partial flowchart of a second model training method in the automatic processing method for image watermarking according to embodiment 3.

Fig. 5 is a partial flowchart of a method for automatically processing an image watermark according to embodiment 4.

Fig. 6 is a schematic diagram of an image to be processed of embodiment 4.

Fig. 7 is a schematic diagram of an image from which a watermark has been removed according to embodiment 4.

Fig. 8 is a schematic diagram of a first model training module in the automatic processing system for image watermarking according to embodiment 5.

Fig. 9 is a block diagram of a composite image acquisition unit of embodiment 6.

Fig. 10 is a partial schematic diagram of a second model training module in the automatic processing system for image watermarking according to embodiment 7.

Fig. 11 is a partial schematic diagram of a second model training module in the automatic processing system for image watermarking according to embodiment 7.

Fig. 12 is a block diagram showing an automatic processing system for image watermarking according to embodiment 8.

Fig. 13 is a schematic diagram of a hardware structure of an electronic device according to embodiment 9 of the present invention.

Detailed Description

The invention is further illustrated by the following examples, which are not intended to limit the scope of the invention.

Example 1

The embodiment provides an automatic processing method of an image watermark, the automatic processing method includes a first model training method, as shown in fig. 1, the first model training method specifically includes:

step 101, acquiring a composite image.

And 102, acquiring an original background image which corresponds to the composite image and does not contain the watermark.

Step 103, dividing the synthetic image and the original background image into a training set and a prediction set.

And step 104, taking the images in the training set as input, and outputting whether each image contains the watermark and the specific position of the watermark in the image to the first machine learning model for training.

And 105, inputting the images in the prediction set into the trained machine learning model to obtain the probability that each image in the prediction set contains the watermark and the watermark prediction position.

And step 106, judging whether the accuracy and the recall ratio of the prediction result of the prediction set reach a detection accuracy threshold value and a detection recall threshold value, if so, executing step 107, and if not, executing step 108.

And step 107, determining the trained first machine learning model as the trained first model.

Step 104 is performed after adjusting parameters of the first machine learning model and/or adding images in the training set, step 108.

Wherein the first machine learning model is a deep convolutional neural network model.

The deep convolutional neural network model comprises 48 convolutional layers and 3 pyramid modules.

The size of the input image may be fixed, for example, the size of the input image may be 480 × 320 according to the average value of the size aspect ratio based on the synthesized image and the original background image and the depth of the mesh.

In step 104, when the first loss function converges to the stable range, the training is stopped.

In the embodiment, whether the image to be processed contains the watermark or not and the specific position of the watermark in the image can be automatically detected through the trained first model, so that the low efficiency of manually searching and smearing the watermark is avoided, the defect that the original image is falsified due to the fact that the image to be processed is directly input into a watermark removing algorithm or the defect that the watermark removing efficiency is low due to the fact that the removed part is a non-watermark part and the like due to inaccurate watermark positioning is also avoided, a foundation is laid for subsequent targeted removal of the related watermark through automatically and accurately finding the specific position of the watermark, and the efficiency of automatically removing the watermark is greatly improved.

Example 2

The present embodiment provides an automatic processing method of image watermarks, and is an improvement on embodiment 1.

As shown in fig. 2, in step 101, a specific method for acquiring a composite image includes:

and step 1011, acquiring various types of watermark pictures.

And step 1012, removing the background color of the watermark picture.

And 1013, automatically converting the watermark picture without the background color into an RGBA format to obtain an original watermark image.

And 1014, acquiring an original background image.

Step 1015, randomly selecting the original watermark image and the original background image.

And step 1016, randomly setting the size, transparency and position of the selected original watermark image in the original background image.

Step 1017, embedding the selected original watermark image into the selected original background image to generate a composite image.

In step 1017, the original watermark images of the same type are embedded into different original background images with different sizes, transparencies and positions.

Because a large-scale image multi-class watermark data set with an open source does not exist at present, the application of image watermark detection and identification in an OTA scene is restricted. In this embodiment, a first large-scale image watermark data set is constructed, which includes a large number of composite images. The method comprises the steps of removing a picture background to convert a watermark picture from an RGB (red, green and blue) format to an RGBA format, wherein the watermark picture only keeps a watermark foreground. Then, a large number of images containing various scenes, such as not less than fifty thousand images, are downloaded in an OTA picture library on a system or a network to be used as an original background image, the type, the size, the position and the transparency of the watermark random watermark are synthesized with the background image, and the background image without the watermark before synthesis is reserved.

In step 104, the first loss function is:

Class loss＝-α_t(1-p_t)^γlog(p_t),γ>0,α^t∈[0,1]

wherein Class Loss represents a probability Loss function, location Loss represents a position Loss function, and the first Loss function is the sum of the probability Loss function and the position Loss function;

wherein p is_tIs the probability of the image containing the watermark, x is the predicted location of the watermark, α_tAnd γ are both weight coefficients.

The automatic processing method for image watermarks in the embodiment further includes:

step 201, inputting an image to be processed into the first model to detect whether the image to be processed contains a watermark and a position of the watermark in the image to be processed.

The position of the watermark in the figure can be represented by the position coordinates (x1, y1, x2 and y2) of a rectangular frame surrounding the outer part of the watermark, x1 and y1 respectively represent the abscissa and ordinate of the upper left corner of the rectangular frame including the watermark, and x2 and y2 respectively represent the abscissa and ordinate of the lower right corner of the rectangular frame including the watermark.

In the embodiment, the watermark area can be effectively positioned before the watermark is removed, and the specific position of the watermark in the image is detected to remove the watermark pertinently and accurately, so that the defect that the image of a non-watermark area is easy to be falsified because the watermark image is directly input into an algorithm to output a watermark-free image in the prior art is avoided.

In the application scenario, a large-scale image is typically composed of a small portion of an aqueous print and a large portion of an anhydrous print. Therefore, it is a great challenge to effectively distinguish the watermark pattern from the large-scale image. In this embodiment, a first model for watermark pre-detection is created that is designed for watermark detection so that the watermark can be removed with accurate identification of the watermark and its location.

Example 3

The present embodiment provides an automatic processing method of image watermarks, and is an improvement on embodiment 2.

If it is detected in step 201 that the image to be processed contains the watermark, the image area containing the watermark is clipped based on the position (x1, y1, x2, y2) of the watermark in the image to be processed to obtain the first image.

In order to efficiently and accurately remove the watermark in the first image, the automatic processing method in this embodiment further includes a second model training method. As shown in fig. 3, the second model training method includes:

step 301, dividing the first image into a training set and a prediction set.

Step 302, using the training set in the first image as input and the non-watermark image corresponding to the first image as output to the generation network for training.

And step 303, inputting the prediction set in the first image into the trained generation network to obtain a second image without the watermark.

And step 304, judging whether the accuracy and the recall ratio of the second image without the watermark reach a generation accuracy threshold and a recall accuracy threshold respectively, if so, executing step 305, and if not, executing step 306.

And 305, determining the trained first machine learning model as the trained first model.

Step 306, after adjusting the parameters of the generated network and/or adding images in the training set, step 302 is performed.

In step 302, the training is stopped when the generated loss function converges to a stable range.

The generated network is a U-net-based full convolution network, the U-net full convolution network in the embodiment comprises l convolution modules, and the ith convolution module is in hopping connection with the l-i convolution modules.

Wherein the generating loss function is defined as follows:

wherein α weight coefficient, L_L1Is a distance loss function of the pixel, L_L1Is defined as follows:

L_L1(x,y)＝‖f(x)-y‖₁

wherein x and y represent the x, y coordinates of each pixel, respectively, f (x) represents the second image through the generation network, y represents the true, watermark-free image corresponding to the second image, | f (x) -y |₁A norm representing f (x) -y;

wherein L is_plA loss function obtained by computing semantic feature distances of an image for perceptual loss, which is defined as follows:

where Φ represents the VGG16 network trained on ImageNet, j represents the j-th layer of the network,

representing said second image through the generation network, y representing a true, watermark-free image corresponding to the second image,

to represent

Image semantic features on the jth layer of the trained network, [ phi ] j (y) denotes the image semantic features of y on the jth layer of the trained network, C_j、H_jAnd W_jRespectively representing the number, height and width of feature maps of the relu2.2 level features,

to represent

The second paradigm of (1).

In order to determine whether the watermark in the image passing through the above generation network can be effectively removed, as shown in fig. 4, the second model training method in this embodiment further includes:

step 307, divide each second image into a training set and a prediction set.

And 308, taking the training set in the second image as input and a plurality of watermark-free feature maps corresponding to the second image as output to a discriminant network for training.

Step 309, inputting the prediction set in the second image into the trained discrimination network to obtain a result whether each feature map in each second image in the prediction set contains a watermark.

And 310, judging whether the accuracy and the recall rate of the result reach a judgment accuracy threshold value and a judgment recall threshold value, if so, executing a step 311, and if not, executing a step 312.

And 311, determining the trained discrimination network as the trained discrimination network.

Step 312, after adjusting the parameters of the discriminant network and/or adding images in the training set, step 308 is performed.

In step 308, the training is stopped when the cross entropy loss function converges to a stable range.

The judgment network is a PatchGAN-based full convolution network, the PatchGAN-based full convolution network comprises four convolution layers, the step length of each convolution layer is 2, and the size of each convolution kernel is 3 x 3.

In the embodiment, the size of the second image input to the full convolution network based on the PatchGAN is W × H, the second image with the size of W × H includes a plurality of image blocks, each image block corresponds to a feature map F with a size of M × N, the feature map F is input to the determination network, the size of the original image corresponding to each feature map F is X × Y, that is, a small block on the original image with the size of W × H, and the determination of whether the watermark is correctly removed is performed within the size range of the corresponding original image. For example, the input watermark region size is 256 × 256, and a feature map with a size of 32 × 32 is output through a full convolution network, in this embodiment, the network includes four convolution layers, the step size of each convolution layer is 2, and the convolution kernel size is 3 × 3, so that for one point on the feature map, the receptive field size corresponding to the original map is 31 × 31.

In this embodiment, the second model training method further includes:

and 313, judging whether the generated network and the judgment network are trained completely, if so, determining that the second model comprising the trained generated network and the trained judgment network is the trained second model, and if not, continuing to execute 302 or 308.

In the embodiment, the defects that the image types of the current watermarks are various and unknown, and the mode of manual processing by using an image processing tool is low in efficiency are overcome. According to the image data characteristics printed with the watermarks, an algorithm model is established, a network and a judgment network are further generated through training according to the positions of the watermarks of the detected images output by the first model, so that the purpose of removing the watermarks can be effectively achieved based on the deep convolution neural network, the maintenance cost of operation can be greatly saved, the reasonable and proper display requirements of the images can be met, and the user experience is improved.

Example 4

The present embodiment provides an automatic processing method of image watermarks, and is an improvement on embodiment 3.

In step 201, if it is detected that the image to be processed includes the watermark, the range of the image area including the watermark is enlarged by a preset multiple value, and then the enlarged range is cut to obtain the first image. The image region range including the watermark in this embodiment is (x1, y1, x2, y2), and the enlarged image region range is (x1 ', y 1', x2 ', y 2').

The image to be processed in the embodiment comprises a first image with an image range of (x1 ', y 1', x2 ', y 2') and the rest of the image with the first image removed.

As shown in fig. 5, after the training of the second model in step 313, the automatic processing method in this embodiment further includes:

step 401, inputting the first image into the trained second model to obtain a third image without watermark corresponding to the first image;

step 402, replacing the first image in the to-be-processed image with the third image to obtain a fourth image;

step 403, adjusting the pixel value of the area of the third image in the fourth image according to the pixel adjustment formula.

The pixel adjustment formula is as follows:

where y represents the adjusted pixel value, w represents the pixel replacement weighting factor, and x₁For pixels containing watermark image blocks, x₂And generating the pixel of the waterless image block, wherein the pixel replacement weighting factor is the preset times value.

In the embodiment, the whole image without the watermark can be obtained by replacing the watermark part with the watermark part, and the defect that part of the watermark is not completely removed due to errors can be avoided by cutting the enlarged image area, so that the accuracy of removing the watermark is improved.

In this embodiment, a watermark-free picture without loss of image quality can be obtained by adjusting the pixel values of the replacement regions.

For better understanding of the present embodiment, the following describes an overall flow of the present embodiment by using a specific example:

for example, after a batch of pictures needing removing the watermark are obtained, the pictures are input into a first model, after the first model, n pictures with the watermark are detected, and the specific position of the watermark in each picture with the watermark is detected through the first model, as shown in fig. 6, after the image 41 to be processed is input into the first model, an area 411 containing the watermark is detected, and then the area 411 is expanded into an area 412 and the area 412 is cut out. Then 412 is input to the second model, after the second model, a picture 412 '(see fig. 7) with the watermark removed can be output, then 412' is spliced with the original area 410 not containing the watermark (i.e. the part with the watermark removed 412), and the pixel of 412 'is adjusted by a pixel adjustment formula, as shown in fig. 7, 41' is the final image with the watermark removed.

The embodiment can not only identify the watermarks of various types and forms in a large batch, but also accurately identify the positions of the watermarks, so that the watermarks can be quickly and accurately detected and effectively removed, the operation and maintenance cost can be greatly saved, the accuracy and the attractiveness of picture display are ensured, and the user experience under the Internet environment is effectively improved.

Example 5

The present embodiment provides an automatic processing system for image watermarking, which includes a first model training module 50, as shown in fig. 8.

The first model training module 50 includes a synthesized image obtaining unit 501, an original background image obtaining unit 502, a first dividing unit 503, a first training unit 504, a first prediction unit 505, and a first determining unit 506.

The composite image obtaining unit 501 is configured to obtain a composite image, where the composite image includes a watermark.

The original background image obtaining unit 502 is configured to obtain an original background image corresponding to the composite image and not containing a watermark.

The first division unit 503 is used to divide the synthesized image and the original background image into a training set and a prediction set.

The first training unit 504 is configured to take images in a training set as input, and take whether each image includes a watermark and a specific position of the watermark in the image as output to the first machine learning model for training until the first loss function converges.

The first prediction unit 505 is configured to input the images in the prediction set into the trained machine learning model to obtain a probability that each image in the prediction set contains a watermark and a watermark prediction position.

The first determining unit 506 is configured to determine whether the accuracy and the recall of the prediction result of the prediction set reach a detection accuracy threshold and a detection recall threshold, and if so, the trained first machine learning model is the trained first model.

Wherein the first machine learning model is a deep convolutional neural network model.

The deep convolutional neural network model comprises 48 convolutional layers and 3 pyramid modules.

The size of the input image may be fixed, for example, the size of the input image may be 480 × 320 according to the average value of the size aspect ratio based on the synthesized image and the original background image and the depth of the mesh.

The first training unit 504 is further configured to stop training when the first loss function converges to a stable range.

In the embodiment, whether the image to be processed contains the watermark or not and the specific position of the watermark in the image can be automatically detected through the trained first model, so that the low efficiency of manually searching and smearing the watermark is avoided, the defect that the original image is falsified due to the fact that the image to be processed is directly input into a watermark removing algorithm or the defect that the watermark removing efficiency is low due to the fact that the removed part is a non-watermark part and the like due to inaccurate watermark positioning is also avoided, a foundation is laid for subsequent targeted removal of the related watermark through automatically and accurately finding the specific position of the watermark, and the efficiency of automatically removing the watermark is greatly improved.

Example 6

The present embodiment provides an automatic processing system for image watermarking, and is an improvement of embodiment 5.

As shown in fig. 9, the synthesized image acquiring unit 501 specifically includes: a watermark image acquisition sub-unit 5011, a background image acquisition sub-unit 5012 and an embedding sub-unit 5013.

The watermark image obtaining subunit 5011 is configured to obtain an original watermark image;

the background image acquisition subunit 5012 is used to acquire an original background image.

The embedding subunit 5013 is configured to automatically embed the original watermark image into the original background image to generate the composite image.

The watermark image obtaining subunit 5011 is configured to obtain multiple types of watermark images, remove a background color of the watermark images, and automatically convert the watermark images with the background color removed into an RGBA format to obtain the original watermark image.

The embedding subunit 5013 is specifically configured to randomly select the original watermark image and the original background image, randomly set the size, transparency, and position of the selected original watermark image in the original background image, and embed the selected original watermark image into the selected original background image, so as to generate a composite image.

Wherein, the original watermark images of the same type are respectively embedded into different original background images with different sizes, transparencies and positions.

Because a large-scale image multi-class watermark data set with an open source does not exist at present, the application of image watermark detection and identification in an OTA scene is restricted. In this embodiment, a first large-scale image watermark data set is constructed, which includes a large number of composite images. The method comprises the steps of removing a picture background to convert a watermark picture from an RGB (red, green and blue) format to an RGBA format, wherein the watermark picture only keeps a watermark foreground. Then, a large number of images containing various scenes, such as not less than fifty thousand images, are downloaded in an OTA picture library on a system or a network to be used as an original background image, the type, the size, the position and the transparency of the watermark random watermark are synthesized with the background image, and the background image without the watermark before synthesis is reserved.

Wherein the first loss function is:

Class loss＝-α_t(1-p_t)^γlog(p_t),γ>0,α^t∈[0,1]

wherein Class Loss represents a probability Loss function, location Loss represents a position Loss function, and the first Loss function is the sum of the probability Loss function and the position Loss function;

wherein p is_tIs the probability of the image containing the watermark, x is the predicted location of the watermark, α_tAnd γ are both weight coefficients.

Wherein, the automatic processing system in this embodiment further includes: a watermark detection module 60, configured to input an image to be processed into the first model to detect whether the image to be processed includes a watermark and a position of the watermark in the image to be processed.

The position of the watermark in the figure can be represented by the position coordinates (x1, y1, x2 and y2) of a rectangular frame surrounding the outer part of the watermark, x1 and y1 respectively represent the abscissa and ordinate of the upper left corner of the rectangular frame including the watermark, and x2 and y2 respectively represent the abscissa and ordinate of the lower right corner of the rectangular frame including the watermark.

In the embodiment, the watermark area can be effectively positioned before the watermark is removed, and the specific position of the watermark in the image is detected to remove the watermark pertinently and accurately, so that the defect that the image of a non-watermark area is easy to be falsified because the watermark image is directly input into an algorithm to output a watermark-free image in the prior art is avoided.

In the application scenario, a large-scale image is typically composed of a small portion of an aqueous print and a large portion of an anhydrous print. Therefore, it is a great challenge to effectively distinguish the watermark pattern from the large-scale image. In this embodiment, a first model for watermark pre-detection is created that is designed for watermark detection so that the watermark can be removed with accurate identification of the watermark and its location.

Example 7

The present embodiment provides an automatic processing system for image watermarking, and is an improvement of embodiment 6.

In this embodiment, the watermark detection module 60 is further configured to, when it is detected that the to-be-processed image includes the watermark, crop the image area including the watermark based on the position (x1, y1, x2, y2) of the watermark in the to-be-processed image to obtain the first image.

As shown in FIG. 10, the automated processing system in this embodiment further includes a second model training module 70.

The second model training module 70 includes: a second partitioning unit 701, a second training unit 702, a second prediction unit 703, and a second determining unit 704.

The second dividing unit 701 is configured to divide the first image into a training set and a prediction set.

The second training unit 702 is configured to take the training set in the first image as input and the non-watermark image corresponding to the first image as output to a generation network for training until the generation loss function converges.

The second prediction unit 703 is configured to input the prediction set in the first image to the trained generation network to obtain a second image without a watermark.

The second determining unit 704 is configured to determine whether the accuracy and the recall rate of the obtained second image without the watermark respectively reach a generation accuracy threshold and a recall accuracy threshold, and if so, the trained generation network is the trained generation network.

The generated network is a U-net-based full convolution network, the U-net full convolution network in the embodiment comprises l convolution modules, and the ith convolution module is in hopping connection with the l-i convolution modules.

Wherein the generating loss function is defined as follows:

wherein α weight coefficient, L_L1Is a distance loss function of the pixel, L_L1Is defined as follows:

L_L1(x,y)＝‖f(x)-y‖₁

wherein x and y represent the x, y coordinates of each pixel, respectively, f (x) represents the second image through the generation network, y represents the true, watermark-free image corresponding to the second image, | f (x) -y | 1 represents a norm of f (x) -y;

wherein L is_plA loss function obtained by computing semantic feature distances of an image for perceptual loss, which is defined as follows:

where Φ represents the VGG16 network trained on ImageNet, j represents the j-th layer of the network,

representing said second image through the generation network, y representing a true, watermark-free image corresponding to the second image,

to represent

Image semantic features on the jth layer of the trained network, [ phi ] j (y) denotes the image semantic features of y on the jth layer of the trained network, C_j、H_jAnd W_jRespectively representing the number, height and width of feature maps of the relu2.2 level features,

to represent

The second paradigm of (1).

In order to determine whether the watermark in the image passing through the above-mentioned generating network can be effectively removed, as shown in fig. 11, the second model training module 70 in this embodiment further includes: a third dividing unit 705, a third training unit 706, a third prediction unit 707, and a third determination unit 708.

The third dividing unit 705 is configured to divide each of the second images into a training set and a prediction set.

The third training unit 706 is configured to output the training set in the second image as input and a plurality of watermark-free feature maps corresponding to the second image as output to a discriminant network for training until the cross entropy loss function converges.

The third prediction unit 707 is configured to input the prediction set in the second image to the trained discrimination network to obtain a result of whether each feature map in each second image in the prediction set includes a watermark.

The third determining unit 708 is configured to determine whether the accuracy and the recall of the result reach a determination accuracy threshold and a determination recall threshold, and if so, the trained discrimination network is a trained discrimination network.

The third training unit 706 is further configured to stop training when the cross entropy loss function converges to a stable range.

The judgment network is a PatchGAN-based full convolution network, the PatchGAN-based full convolution network comprises four convolution layers, the step length of each convolution layer is 2, and the size of each convolution kernel is 3 x 3.

In the embodiment, the size of the second image input to the full convolution network based on the PatchGAN is W × H, the second image with the size of W × H includes a plurality of image blocks, each image block corresponds to a feature map F with a size of M × N, the feature map F is input to the determination network, the size of the original image corresponding to each feature map F is X × Y, that is, a small block on the original image with the size of W × H, and the determination of whether the watermark is correctly removed is performed within the size range of the corresponding original image. For example, the input watermark region size is 256 × 256, and a feature map with a size of 32 × 32 is output through a full convolution network, in this embodiment, the network includes four convolution layers, the step size of each convolution layer is 2, and the convolution kernel size is 3 × 3, so that for one point on the feature map, the receptive field size corresponding to the original map is 31 × 31.

In this embodiment, the second model training module 70 further includes a training completion determining unit 709, configured to determine whether both the generating network and the determining network are trained, and if so, the second model including the trained generating network and the trained determining network is the trained second model.

In the embodiment, the defects that the image types of the current watermarks are various and unknown, and the mode of manual processing by using an image processing tool is low in efficiency are overcome. According to the image data characteristics printed with the watermarks, an algorithm model is established, a network and a judgment network are further generated through training according to the positions of the watermarks of the detected images output by the first model, so that the purpose of removing the watermarks can be effectively achieved based on the deep convolution neural network, the maintenance cost of operation can be greatly saved, the reasonable and proper display requirements of the images can be met, and the user experience is improved.

Example 8

The present embodiment provides an automatic processing system for image watermarking, and is an improvement of embodiment 7. If the watermark detection module 60 detects that the image to be processed contains the watermark, the range of the image area containing the watermark is enlarged by a preset times value, and then the enlarged range is cut off to obtain the first image. The image region range including the watermark in this embodiment is (x1, y1, x2, y2), and the enlarged image region range is (x1 ', y 1', x2 ', y 2').

The image to be processed in the embodiment comprises a first image with an image range of (x1 ', y 1', x2 ', y 2') and the rest of the image with the first image removed.

As shown in fig. 12, the automatic processing system in this embodiment further includes: a watermark-free image generation module 80, a replacement module 81 and a pixel adjustment module 82.

After the training completion determining unit 709 determines that the training is completed, the watermark-free image generating module 80 is executed to input the first image to the trained second model to obtain a watermark-free third image corresponding to the first image.

The replacing module 81 is configured to replace the first image in the to-be-processed image with the third image to obtain a fourth image.

The pixel adjusting module 82 is configured to adjust pixel values of a region of the third image in the fourth image according to the following formula:

where y represents the adjusted pixel value, w represents the pixel replacement weighting factor, and x₁For pixels containing watermark image blocks, x₂And if the automatic processing method further comprises the step of expanding the range of the detected image area containing the watermark by a preset times value and then cutting the expanded range to obtain a first image, the pixel replacement weighting factor is the preset times value.

In the embodiment, the whole image without the watermark can be obtained by replacing the watermark part with the watermark part, and the defect that part of the watermark is not completely removed due to errors can be avoided by cutting the enlarged image area, so that the accuracy of removing the watermark is improved.

In this embodiment, a watermark-free picture without loss of image quality can be obtained by adjusting the pixel values of the replacement regions.

For better understanding of the present embodiment, the following describes an overall flow of the present embodiment by using a specific example:

for example, after a batch of pictures needing removing the watermark are obtained, the pictures are input into a first model, after the first model, n pictures with the watermark are detected, and the specific position of the watermark in each picture with the watermark is detected through the first model, as shown in fig. 6, after the image 41 to be processed is input into the first model, an area 411 containing the watermark is detected, and then the area 411 is expanded into an area 412 and the area 412 is cut out. Then 412 is input to the second model, after the second model, a picture 412 '(see fig. 7) with the watermark removed can be output, then 412' is spliced with the original area 410 not containing the watermark (i.e. the part with the watermark removed 412), and the pixel of 412 'is adjusted by a pixel adjustment formula, as shown in fig. 7, 41' is the final image with the watermark removed.

The embodiment can not only identify the watermarks of various types and forms in a large batch, but also accurately identify the positions of the watermarks, so that the watermarks can be quickly and accurately detected and effectively removed, the operation and maintenance cost can be greatly saved, the accuracy and the attractiveness of picture display are ensured, and the user experience under the Internet environment is effectively improved.

Example 9

The present embodiment provides an electronic device, which may be represented in the form of a computing device (for example, may be a server device), and includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program to implement any automatic processing method in embodiments 1 to 4.

Fig. 13 shows a schematic diagram of a hardware structure of the present embodiment, and as shown in fig. 9, the electronic device 9 specifically includes:

at least one processor 91, at least one memory 92, and a bus 93 for connecting the various system components (including the processor 91 and the memory 92), wherein:

the bus 93 includes a data bus, an address bus, and a control bus.

Memory 92 includes volatile memory, such as Random Access Memory (RAM)921 and/or cache memory 922, and can further include Read Only Memory (ROM) 923.

Memory 92 also includes a program/utility 925 having a set (at least one) of program modules 924, such program modules 924 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.

The processor 91 executes various functional applications and data processing, such as any of the automatic processing methods according to embodiments 1 to 4 of the present invention, by executing the computer program stored in the memory 92.

The electronic device 9 may further communicate with one or more external devices 94 (e.g., a keyboard, a pointing device, etc.). Such communication may be through an input/output (I/O) interface 95. Also, the electronic device 9 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet) via the network adapter 96. The network adapter 96 communicates with the other modules of the electronic device 9 via the bus 93. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device 9, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.

It should be noted that although in the above detailed description several units/modules or sub-units/modules of the electronic device are mentioned, such a division is merely exemplary and not mandatory. Indeed, the features and functionality of two or more of the units/modules described above may be embodied in one unit/module, according to embodiments of the application. Conversely, the features and functions of one unit/module described above may be further divided into embodiments by a plurality of units/modules.

Example 10

The present embodiment provides a computer-readable storage medium on which a computer program is stored, the program implementing the steps of the automatic processing method of any one of embodiments 1 to 4 when executed by a processor.

More specific examples, among others, that the readable storage medium may employ may include, but are not limited to: a portable disk, a hard disk, random access memory, read only memory, erasable programmable read only memory, optical storage device, magnetic storage device, or any suitable combination of the foregoing.

In a possible implementation manner, the present invention can also be implemented in the form of a program product including program code for causing a terminal device to execute steps implementing the automatic processing method according to any one of embodiments 1 to 4 when the program product is run on the terminal device.

Where program code for carrying out the invention is written in any combination of one or more programming languages, the program code may be executed entirely on the user device, partly on the user device, as a stand-alone software package, partly on the user device and partly on a remote device or entirely on the remote device.

While specific embodiments of the invention have been described above, it will be appreciated by those skilled in the art that this is by way of example only, and that the scope of the invention is defined by the appended claims. Various changes and modifications to these embodiments may be made by those skilled in the art without departing from the spirit and scope of the invention, and these changes and modifications are within the scope of the invention.

While specific embodiments of the invention have been described above, it will be appreciated by those skilled in the art that this is by way of example only, and that the scope of the invention is defined by the appended claims. Various changes and modifications to these embodiments may be made by those skilled in the art without departing from the spirit and scope of the invention, and these changes and modifications are within the scope of the invention.

Claims (24)

1. An automatic processing method for image watermarking, which is characterized in that the automatic processing method comprises a first model training method:

the first model training method comprises the following steps:

acquiring a composite image, wherein the composite image comprises a watermark;

acquiring an original background image corresponding to the composite image that does not contain a watermark,

dividing the synthetic image and the original background image into a training set and a prediction set;

taking images in a training set as input, and taking whether each image contains a watermark and the specific position of the watermark in the image as output to a first machine learning model for training until a first loss function is converged;

taking the images in the prediction set as input into a trained machine learning model to obtain the probability that each image in the prediction set contains the watermark and the watermark prediction position;

and judging whether the accuracy and the recall rate of the prediction result of the prediction set reach a detection accuracy threshold and a detection recall threshold, if so, determining that the trained first machine learning model is a trained first model.

2. The automated processing method of claim 1,

the first loss function is:

Class loss＝-α_t(1-p_t)^γlog(p_t),γ>0,α^t∈[0,1]

wherein Class Loss represents a probability Loss function, location Loss represents a position Loss function, and the first Loss function is the sum of the probability Loss function and the position Loss function;

wherein p is_tIs the probability of the image containing the watermark, x is the predicted location of the watermark, α_tAnd γ are both weight coefficients.

3. The automated processing method of claim 1, wherein the automated processing method comprises:

obtaining the composite image by:

acquiring an original watermark image;

acquiring an original background image;

automatically embedding the original watermark image into the original background image to generate the composite image.

4. The automated processing method of claim 3, wherein the automated processing method comprises:

obtaining an original watermark image by the following steps:

acquiring various types of watermark pictures;

removing the background color of the watermark picture;

automatically converting the watermark picture without the background color into an RGBA format to obtain the original watermark image;

and/or the presence of a gas in the gas,

the step of automatically embedding the original watermark image into the original background image to generate the composite image specifically includes:

randomly selecting the original watermark image and the original background image;

randomly setting the size, transparency and position of the selected original watermark image in the original background image;

embedding the selected original watermark image into the selected original background image to generate a composite image;

the original watermark images of the same type are embedded in different original background images with different sizes, transparencies and positions respectively.

5. The automated processing method according to any one of claims 1 to 4, characterized in that the automated processing method comprises:

inputting an image to be processed into the first model to detect whether the image to be processed contains a watermark and the position of the watermark in the image to be processed.

6. The automated processing method of claim 5,

in the step of inputting an image to be processed into the first model to detect whether the image to be processed contains a watermark, if the image to be processed contains the watermark, cutting an image area containing the watermark based on the position of the watermark in the image to be processed to obtain a first image;

the automatic processing method further comprises a second model training method;

the second model training method includes:

dividing the first image into a training set and a prediction set;

taking the training set in the first image as input and the non-watermark image corresponding to the first image as output to a generating network for training until a generating loss function is converged;

inputting the prediction set in the first image into a trained generation network to obtain a second image without a watermark;

and judging whether the accuracy and the recall rate of the second image without the watermark respectively reach a generation accuracy threshold and a recall accuracy threshold, if so, determining the trained generation network as the trained generation network.

7. The process of claim 6 wherein the generative loss function is defined as follows:

wherein α weight coefficient, L_L1Is a distance loss function of the pixel, L_L1Is defined as follows:

L_L1(x,y)＝‖f(x)-y‖₁

wherein x and y represent the x, y coordinates of each pixel, respectively, f (x) represents the second image through the generation network, y represents the true, watermark-free image corresponding to the second image, | f (x) -y |₁A norm representing f (x) -y;

wherein L is_plA loss function obtained by computing semantic feature distances of an image for perceptual loss, which is defined as follows:

where Φ represents the VGG16 network trained on ImageNet, j represents the j-th layer of the network,

representing said second image through the generation network, y representing a true, watermark-free image corresponding to the second image,

to represent

Image semantic features on the jth layer of the trained network, [ phi ] j (y) denotes the image semantic features of y on the jth layer of the trained network, C_j、H_jAnd W_jRespectively representing the number, height and width of feature maps of the relu2.2 level features,

to represent

The second paradigm of (1).

8. The automated processing method of claim 6, wherein the second model training method further comprises:

dividing each of the second images into a training set and a prediction set;

taking the training set in the second image as input and a plurality of watermark-free feature maps corresponding to the second image as output to a discriminant network for training until the cross entropy loss function is converged;

inputting the prediction set in the second image into the trained discrimination network to obtain a result of whether each feature map in each second image in the prediction set contains a watermark;

and judging whether the accuracy and the recall rate of the result reach a judgment accuracy threshold and a judgment recall threshold, if so, the trained judgment network is a trained judgment network.

9. The automated processing method of claim 8, wherein the second model training method further comprises: and judging whether the generated network and the judging network are trained completely, if so, taking the second model comprising the trained generated network and the trained judging network as a trained second model.

10. The automated processing method of claim 9,

the image to be processed comprises a first image and residual images of the first image are removed;

after training the second model, the automatic processing method further comprises:

inputting the first image into the trained second model to obtain a third image without a watermark corresponding to the first image;

replacing the first image in the to-be-processed image with the third image to obtain a fourth image;

and/or the presence of a gas in the gas,

based on the position of the watermark in the image to be processed, the step of cutting out the image area containing the watermark to obtain a first image comprises the following steps:

and expanding the range of the detected image area containing the watermark by a preset times value, and then cutting the expanded range to obtain a first image.

11. The automated processing method of claim 10, wherein the step of replacing the first image of the images to be processed with the third image to obtain a fourth image further comprises:

adjusting pixel values of regions of the third image in the fourth image according to the following formula:

wherein y represents the adjusted pixel value,w represents a pixel replacement weighting factor, x₁For pixels containing watermark image blocks, x₂And if the automatic processing method further comprises the step of expanding the range of the detected image area containing the watermark by a preset times value and then cutting the expanded range to obtain a first image, the pixel replacement weighting factor is the preset times value.

12. An automatic processing system for image watermarking, comprising a first model training module;

the first model training module comprises a synthetic image acquisition unit, an original background image acquisition unit, a first division unit, a first training unit, a first prediction unit and a first judgment unit:

the composite image acquisition unit is used for acquiring a composite image, and the composite image comprises a watermark;

the original background image acquisition unit is used for acquiring an original background image which does not contain a watermark and corresponds to the composite image;

the first dividing unit is used for dividing the synthetic image and the original background image into a training set and a prediction set;

the first training unit is used for taking images in a training set as input, judging whether each image contains a watermark and outputting the specific position of the watermark in the image to a first machine learning model for training until a first loss function is converged;

the first prediction unit is used for inputting the images in the prediction set into the trained machine learning model to obtain the probability that each image in the prediction set contains the watermark and the watermark prediction position;

the first judging unit is used for judging whether the accuracy and the recall rate of the prediction result of the prediction set reach a detection accuracy threshold and a detection recall threshold, and if so, the trained first machine learning model is a trained first model.

13. The automated processing system of claim 12,

the first loss function is:

Class loss＝-α_t(1-p_t)^γlog(p_t),γ>0,α^t∈[0,1]

wherein Class Loss represents a probability Loss function, location Loss represents a position Loss function, and the first Loss function is the sum of the probability Loss function and the position Loss function;

wherein p is_tIs the probability of the image containing the watermark, x is the predicted location of the watermark, α_tAnd γ are both weight coefficients.

14. The automated processing system of claim 12, wherein the composite image acquisition unit comprises a watermark image acquisition subunit, a background image acquisition subunit, and an embedding subunit:

the watermark image acquisition subunit is used for acquiring an original watermark image;

the background image acquisition subunit is used for acquiring an original background image;

the embedding subunit is configured to automatically embed the original watermark image into the original background image to generate the composite image.

15. The automatic processing system according to claim 14, wherein the watermark image obtaining sub-unit is specifically configured to obtain multiple types of watermark images, remove a background color of the watermark images, and automatically convert the watermark images with the background color removed into an RGBA format to obtain the original watermark image;

and/or the presence of a gas in the gas,

the embedding subunit is specifically configured to randomly select the original watermark image and the original background image, randomly set the size, transparency, and position of the selected original watermark image in the original background image, and embed the selected original watermark image into the selected original background image, so as to generate a composite image;

the original watermark images of the same type are embedded in different original background images with different sizes, transparencies and positions respectively.

16. The automated processing system according to any one of claims 12-15, wherein the automated processing system further comprises: the watermark detection module is used for inputting the image to be processed into the first model so as to detect whether the image to be processed contains the watermark or not and the position of the watermark in the image to be processed.

17. The automated processing system of claim 16,

the watermark detection module is further used for cutting an image area containing the watermark to obtain a first image based on the position of the watermark in the image to be processed when the image to be processed is detected to contain the watermark;

the automated processing system further comprises a second model training module;

the second model training module comprises: the device comprises a second dividing unit, a second training unit, a second prediction unit and a second judgment unit;

the second dividing unit is used for dividing the first image into a training set and a prediction set;

the second training unit is used for taking a training set in the first image as input and taking a non-watermark image corresponding to the first image as output to a generation network for training until a generation loss function is converged;

the second prediction unit is used for inputting the prediction set in the first image into a trained generation network to obtain a second image without a watermark;

the second judging unit is used for judging whether the accuracy and the recall rate of the second image without the watermark respectively reach a generation accuracy threshold and a recall accuracy threshold, if so, the trained generation network is the trained generation network.

18. The processing system of claim 17, wherein the generative loss function is defined as follows:

wherein α weight coefficient, L_L1Is a distance loss function of the pixel, L_L1Is defined as follows:

L_L1(x,y)＝‖f(x)-y‖₁

wherein x and y represent the x, y coordinates of each pixel, respectively, f (x) represents the second image through the generation network, y represents the true, watermark-free image corresponding to the second image, | f (x) -y |₁A norm representing f (x) -y;

wherein L is_plA loss function obtained by computing semantic feature distances of an image for perceptual loss, which is defined as follows:

where Φ represents the VGG16 network trained on ImageNet, j represents the j-th layer of the network,

representing said second image through the generation network, y representing a true, watermark-free image corresponding to the second image,

to represent

Image semantic features on the jth layer of the trained network, [ phi ] j (y) denotes image semantic features of y on the jth layer of the trained network,C_j、H_jand W_jRespectively representing the number, height and width of feature maps of the relu2.2 level features,

to represent

The second paradigm of (1).

19. The automated processing system of claim 17, wherein the second model training module further comprises: the device comprises a third dividing unit, a third training unit, a third prediction unit and a third judgment unit;

the third dividing unit is used for dividing each second image into a training set and a prediction set;

the third training unit is used for taking a training set in the second image as input and a plurality of watermark-free feature maps corresponding to the second image as output to a discriminant network for training until a cross entropy loss function converges;

the third prediction unit is used for inputting the prediction set in the second image into the trained discrimination network to obtain a result of whether each feature map in each second image in the prediction set contains a watermark or not;

the third judging unit is used for judging whether the accuracy and the recall rate of the result reach a judgment accuracy threshold and a judgment recall threshold, if so, the trained judgment network is a trained judgment network.

20. The automated processing system of claim 19, wherein the second model training module further comprises a training completion determination unit configured to determine whether the generating network and the discriminating network are both trained, and if so, the second model comprising the trained generating network and the trained discriminating network is the trained second model.

21. The automated processing system of claim 20,

the image to be processed comprises a first image and residual images of the first image are removed;

the automated processing system further comprises: a watermark-free image generation module and a replacement module;

the watermark-free image generation module is used for inputting the first image into the trained second model to obtain a watermark-free third image corresponding to the first image;

the replacing module is used for replacing the first image in the to-be-processed image with the third image to obtain a fourth image;

and/or the presence of a gas in the gas,

the watermark detection module is further used for expanding the range of the detected image area containing the watermark by a preset times value and then cutting the expanded range to obtain the first image.

22. The automated processing system of claim 21, further comprising a pixel adjustment module to adjust pixel values of a region of the third image in the fourth image according to the formula:

where y represents the adjusted pixel value, w represents the pixel replacement weighting factor, and x₁For pixels containing watermark image blocks, x₂And if the automatic processing method further comprises the step of expanding the range of the detected image area containing the watermark by a preset times value and then cutting the expanded range to obtain a first image, the pixel replacement weighting factor is the preset times value.

23. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the automated processing method of any one of claims 1 to 11 when executing the computer program.

24. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the automatic processing method of any one of claims 1 to 11.

Images