CN115426537A

CN115426537A - Method and system for adding invisible watermark in video picture and tracing information source

Info

Publication number: CN115426537A
Application number: CN202211075592.4A
Authority: CN
Inventors: 黄安; 白刚
Original assignee: Shanghai Sailian Information Technology Co ltd
Current assignee: Shanghai Sailian Information Technology Co ltd
Priority date: 2022-09-05
Filing date: 2022-09-05
Publication date: 2022-12-02
Anticipated expiration: 2042-09-05

Abstract

The invention provides a method and a system for adding an invisible watermark in a video picture of a video conference. The method comprises the following steps: converting the continuous frames of the video conference pictures in the time domain interval into the continuous frames of the video conference pictures in the frequency domain interval in real time based on a frequency domain conversion algorithm; dividing the watermark map according to a certain rule to obtain a plurality of local watermark maps; respectively dispersedly and superpositioning the local watermark images in real time in different continuous frames of the video conference picture of the frequency domain interval by taking the frame as a unit; converting the continuous frames of the video conference pictures in the frequency domain interval into the continuous frames of the video conference pictures in the time domain interval by a reverse frequency domain conversion algorithm to obtain the continuous frames of the video conference pictures on which the invisible local watermark is superimposed; presenting the user with successive frames of the video conference picture superimposed with the invisible partial watermark. The method for adding the invisible watermark effectively improves the difficulty of a cracker in obtaining complete watermark information and avoids the obvious influence on the picture quality caused by embedding the watermark.

Description

Method and system for adding invisible watermark in video picture and tracing information source

Technical Field

The invention relates to the technical field of communication, in particular to a method and a system for adding an invisible watermark in a video picture and tracing information.

Background

The video communication technology based on cloud computing has wider and wider application range. In these applications, video data is transmitted to each terminal, and if people transmit video conference contents in a form of screen recording or the like, privacy and interests of others may be damaged. The security and traceability of information are particularly important.

Because many people often enter the meeting at the same time in the video conference, the tracing difficulty of the video conference content is high, and the picture needs to be encrypted by technical means such as adding watermarks and the like, so that the video conference content is convenient to trace and trace.

In the common prior art, a method of overlaying a clear watermark is adopted when the watermark is overlaid, and the watermark is directly overlaid on a video picture. However, the content of the picture is shielded by the plain watermark, so that the viewing experience of the user is influenced; and secondly, because the plain watermark can be directly observed, the plain watermark can be easily removed by tools such as Photoshop and the like when the plain watermark is attacked, the watermark information borne in the video picture can not be effectively protected, and the use experience is influenced.

To address these problems, invisible watermarks have been proposed and developed, such as:

patent CN201811563428 discloses a frequency domain-based image blind watermarking method, which comprises: 1. generating watermark information into a QR code watermark image according to the version number of the QR code and the error correction level of the QR code; 2. dividing the input into equal parts according to the size of a QR code watermark image, and calculating the gradient of each part by utilizing the image differential; 3. performing space-domain to frequency-domain conversion on the plurality of regional images with the highest average gradient in the step 2; 4. encoding the watermark image and superposing the watermark image with the frequency domain obtained in the step 3; 5. and performing inverse spatial-to-frequency domain transformation to obtain output after adding the watermark.

Patent CN202010906543 discloses a method, apparatus, device and storage medium for adding and extracting watermarks from video, wherein the method comprises: acquiring a target image frame in video data; performing time-frequency conversion on the target image frame to obtain target frequency domain data, wherein the target frequency domain data comprises a matrix consisting of frequency domain coefficients; modifying the frequency domain coefficient in the target frequency domain data according to the watermark data to obtain watermark frequency domain data; performing time-frequency inverse conversion on the watermark frequency domain data to obtain a watermark image frame; and synthesizing watermark video data according to the watermark image frame.

Patent CN202111518217 discloses a video watermark generation method and apparatus, wherein the method includes: the graphics processor responds to an encryption instruction triggered in the terminal equipment and analyzes a two-dimensional map corresponding to each video frame from a video to be processed stored in a video memory space; converting a watermark mapping to be added in a video memory space from a time domain space to a frequency domain space to obtain a first frequency domain map, and converting a two-dimensional mapping corresponding to each video frame from the time domain space to the frequency domain space to obtain a second frequency domain map corresponding to each video frame; and generating an anti-counterfeiting image corresponding to each video frame in a video memory space according to the first frequency domain image and the second frequency domain image, wherein the anti-counterfeiting image carries a video watermark corresponding to the watermark map.

Patent CN201710306245 discloses a video tracing method based on watermark technology, which includes: extracting an original video, and acquiring equipment information and user information; transforming the original video by adopting a discrete cosine transform algorithm, transforming the original video from a time domain to a frequency domain, and generating a watermark to be embedded by adopting a watermark generation algorithm; embedding a watermark to be embedded in an original video transform domain by adopting a watermark embedding algorithm to generate a video containing the watermark; and transforming the video containing the watermark by adopting an inverse transformation algorithm, transforming the video from a frequency domain to a time domain, and generating the original video containing the watermark.

The invisible watermarks disclosed in the above four patents convert an original picture or video from a time domain to a frequency domain based on invisibility of frequency domain information, superimpose watermark information in a frequency domain interval, and reversely convert an image from the frequency domain to the time domain, thereby obtaining the image or video superimposed with the invisible watermark. However, the method only simply superimposes the invisible watermarks on the images or videos, and only utilizes the common technical point of common technicians, namely invisibility of frequency domain information, so that the method is easy to crack, the watermark information is erased or modified, and the real tracing to the video conference content leakage source cannot be realized. Since the prior art adds watermarks to high-frequency regions (corners) in the frequency domain interval of the whole image or video frame, the identifiability of people or characters in the image is obviously reduced. In addition, the watermark information is inserted into the frequency domain section in the form of a picture, and if the content included in the watermark information is large, the picture is large. The picture quality is also significantly degraded if a larger watermark is inserted in the frequency domain interval.

Disclosure of Invention

The invention provides a method and a system for adding an invisible watermark in a video picture of a video conference and tracing the video conference information, aiming at the video picture, the difficulty of a cracker in obtaining complete watermark information is effectively improved, and the cracker is prevented from simply obtaining complete watermark information from one frame of picture; the obvious influence on the picture quality caused by embedding the watermark is avoided; but also watermark maps containing a large amount of conference tracing information can be embedded.

In a first aspect, the present invention provides a method for adding an invisible watermark to a video frame of a video conference, where the method includes:

converting the continuous frames of the video conference pictures in the time domain interval into the continuous frames of the video conference pictures in the frequency domain interval in real time based on a frequency domain conversion algorithm;

dividing the watermark map according to a certain rule to obtain a plurality of local watermark maps;

respectively dispersedly and superpositioning the local watermark images in real time in different continuous frames of the video conference picture of the frequency domain interval by taking the frames as units;

converting the continuous frames of the video conference pictures in the frequency domain interval into the continuous frames of the video conference pictures in the time domain interval by a reverse frequency domain conversion algorithm to obtain the continuous frames of the video conference pictures on which the invisible local watermark is superimposed;

and presenting the continuous frames of the video conference picture on which the invisible local watermark is superimposed to a user.

Preferably, the method further comprises:

removing people and characters from the video conference picture of the time domain interval to obtain the residual video conference picture of the time domain interval;

dividing the rest video conference pictures in the time domain interval according to the specified block size to obtain a plurality of blocks, and selecting the position of a certain complete block as a low attention area according to a set mode;

converting the low attention area of the continuous frame of the time domain interval into the low attention area of the continuous frame of the frequency domain interval in real time based on a frequency domain conversion algorithm;

respectively dispersedly superposing the local watermark images in the corners of low attention areas of different continuous frames of the frequency domain interval in real time by taking the frames as units;

converting the low attention region of the continuous frame of the frequency domain interval overlapped with the local watermark map into a low attention region of the continuous frame of the time domain interval overlapped with the local watermark map by an inverse frequency domain conversion algorithm;

and correspondingly replacing the low attention area of the video conference picture of the continuous frame of the time domain interval by the low attention area of the continuous frame of the time domain interval on which the local watermark is superposed, so as to obtain the continuous frame of the video conference picture on which the invisible local watermark is superposed.

In a second aspect, the present invention further provides a method for tracing video conference information on a video conference picture added with an invisible watermark based on the method, where the method includes:

converting continuous frames of the video conference pictures to be traced and superimposed with the invisible local watermark images from time domain intervals to frequency domain intervals through the frequency domain conversion algorithm, and obtaining the local watermark images from the corners of the frequency domain intervals;

rearranging the local watermark map according to a certain rule to obtain a complete watermark map;

and identifying watermark information including a conference number, conference time, conference account information, a conference terminal number, a conference terminal location and the like according to the complete watermark map.

In a third aspect, the present invention further provides a system for adding an invisible watermark to a video frame of a video conference, where the system includes:

converting means for converting, in real time, the successive frames of the videoconference frames for the time domain intervals into successive frames of the videoconference frames for the frequency domain intervals based on a frequency domain conversion algorithm;

the dividing device is used for dividing the watermark pattern according to a certain rule to obtain a plurality of local watermark patterns;

the superposition device is used for dispersedly superposing the local watermark images in different continuous frames of the video conference picture of the frequency domain interval in real time by taking the frames as units;

the conversion device is also used for converting the continuous frames of the video conference pictures in the frequency domain interval into the continuous frames of the video conference pictures in the time domain interval through a reverse frequency domain conversion algorithm to obtain the continuous frames of the video conference pictures on which the invisible local watermark is superimposed;

and the presenting device is used for presenting the continuous frames of the video conference pictures on which the invisible local watermark pictures are superposed to the user.

In a fourth aspect, the present invention further provides a system for tracing video conference information on a video conference picture added with an invisible watermark based on the system, wherein the system comprises:

the second conversion device is used for converting the continuous frames of the video conference pictures which are to be traced and are superposed with the invisible local watermark images from the time domain interval to the frequency domain interval by the frequency domain conversion algorithm, and obtaining the local watermark images from the corners of the frequency domain interval;

the rearrangement device is used for rearranging the local watermark map according to a certain rule to obtain a complete watermark map;

and the identification device is used for identifying watermark information including a conference number, conference time, conference account information, a conference terminal number, a conference terminal belonging place and the like according to the complete watermark map.

The invention provides a method and a system for adding an invisible watermark in a video picture of a video conference and tracing the video conference information, wherein the method comprises the following steps: firstly, aiming at a video picture, dividing a complete watermark image according to a certain rule, and then disordering and recombining the complete watermark image into a plurality of local watermark images; secondly, the local watermark images are dispersedly superposed in different continuous frames of the video conference picture, the first point and the second point greatly improve the difficulty of a cracker in acquiring complete watermark information by disorganizing and recombining the complete watermark images and dispersedly superposed in different frames, effectively prevent the cracker from simply obtaining the complete watermark information from a single frame picture, and simultaneously realize the purpose of embedding the larger watermark image by dividing the larger watermark image into a plurality of smaller local watermark images which are respectively embedded into different video frames; thirdly, a low attention area of the video conference picture is introduced, the perception of the user on the deformed area superposed with the invisible watermark is reduced, and meanwhile, the frequency domain and time domain conversion can be only carried out on the low attention area of the continuous frames of the video conference picture, so that the efficiency of converting the frequency domain and the time domain is improved, a local watermark picture cannot be obtained under the condition that a cracker does not know the distribution of the low attention area, and the cracking difficulty is increased again; and fourthly, the terminal watermark information generated by the conference terminal in real time can be synthesized into a larger watermark image due to the fact that the local watermark images which are dispersedly embedded into the video frames, so that a large amount of information related to the video conference and the terminal recording the video conference can be obtained, and the accurate positioning of the conference terminal can be conveniently carried out during tracing.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.

Fig. 1 is a flowchart of a method for adding an invisible watermark in a video frame of a video conference according to an embodiment of the present invention;

FIG. 2a is a complete watermark diagram provided by an embodiment of the present invention;

FIGS. 2b-2e are four partial watermark diagrams provided by embodiments of the present invention;

FIG. 3 is a schematic view of a high attention area provided by an embodiment of the present invention;

fig. 4 is a flowchart of a method for determining a low attention area and adding an invisible watermark to the low attention area of a video frame of a video conference according to an embodiment of the present invention;

FIG. 5 is a diagram of frequency domain intervals provided by an embodiment of the present invention;

fig. 6 is a flowchart of a method for determining a low attention area by using an attention mechanism and deep learning and adding an invisible watermark in the low attention area of a video frame of a video conference according to an embodiment of the present invention;

fig. 7 is a flowchart of a method for tracing video conference information of a video conference picture added with an invisible watermark according to the method provided by the embodiment of the present invention;

fig. 8 is a schematic diagram of a system 100 for adding an invisible watermark to a video frame of a video conference according to an embodiment of the present invention;

fig. 9 is a schematic diagram of a system 200 for tracing video conference information of a video conference picture added with an invisible watermark according to the system in an embodiment of the present invention.

Detailed Description

The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.

Summary of The Invention

As described above, the present invention provides a method and system for adding an invisible watermark in a video frame of a video conference and tracing the video conference information, which realizes the tracing of the video conference information, improves the security of the video conference, further effectively improves the difficulty of a cracker in obtaining complete watermark information and the information amount of a watermark image containing the conference tracing information, reduces the perception of a user on a malformed region after the invisible watermark is superimposed, and simultaneously can only perform frequency domain and time domain conversion on a low attention region of a continuous frame of the video conference frame, thereby improving the efficiency of frequency domain to time domain conversion.

Exemplary method

Fig. 1 is a flowchart of a method for adding an invisible watermark in a video frame of a video conference, according to an embodiment of the present invention, where the embodiment includes the following steps:

s101: the method includes converting, in real-time, the successive frames of the video conference picture in the time domain interval to successive frames of the video conference picture in the frequency domain interval based on a frequency domain conversion algorithm.

The time and frequency domains are fundamental properties of the signal, so that the signal can be analyzed in a number of ways, each way providing a different angle.

The Time domain is a function describing a mathematical function or a physical signal versus Time. For example, a time domain waveform of a signal may express the change of the signal over time. Is the real world and is the only actual domain present. The time domain argument is time, i.e., the horizontal axis is time and the vertical axis is the change of the signal. The dynamic signal x (t) is a function describing the value of the signal at different moments.

The frequency domain is a coordinate system used to describe the frequency characteristics of a signal. In electronics, control system engineering and statistics, frequency domain plots show the amount of signal in each given frequency band over a range of frequencies. The most important properties of the frequency domain are: it is not true, but a mathematical construct. The time domain is the only objective domain, while the frequency domain is a mathematical category that follows certain rules, and is also known by some scholars as the god view. Therefore, the frequency domain information has invisibility, the watermark information is superposed in the frequency domain interval, and then the image is converted into the time domain through the inverse frequency domain conversion algorithm, so that the image superposed with the invisible watermark can be obtained. The frequency domain argument is the frequency, i.e. the horizontal axis is the frequency and the vertical axis is the amplitude of the frequency signal, i.e. the spectrogram in general.

Wherein the frequency domain transform algorithm includes, but is not limited to, fourier transform, short-time fourier transform, wavelet packet decomposition, and the like.

The continuous frames of the video conference pictures are continuous pictures converted from the video conference pictures. It is known that the human visual system has a short-term memory of the picture, and when different motions of the same character occur continuously, the viewer can overlap the previous impression when watching the next picture as long as the motion switching speed of the character is fast enough, thereby creating the illusion that the character is moving. How many continuous pictures are shot by the camera within 1 second is the frame rate, which is commonly referred to as the number of pictures of animation or video. The FPS measures the amount of information used to store and display the motion video. The greater the number of frames per second, the more fluid the displayed motion will be. Typically, a minimum of 30 to avoid motion blindness is used. Some computer video formats can only provide 15 frames per second. The invention requires the insertion of watermarks in consecutive frames, where consecutive frames do not necessarily mean every frame, but may also be inserted with a watermark every other multiple of frames, for example, in frames 5, 10, 15. Therefore, the acquisition of the continuous frames of the video conference pictures in which the watermarks need to be inserted needs to be determined according to the position of the frame in which the watermarks are embedded after a section of video is selected from the actual video pictures and is subjected to frequency domain conversion.

S102: and dividing the watermark map according to a certain rule to obtain a plurality of local watermark maps.

As shown in fig. 2a, the watermark map is generated in real time at the conference terminal. The watermark map includes but is not limited to the watermark information such as the conference number, the conference time, the account number information of the participating conference, the number of the conference terminal, the affiliated place of the conference terminal, etc.

Terminal watermark information generated by the conference terminal in real time facilitates accurate positioning of the conference terminal during tracing.

Preferably, the rule is to cut the watermark pattern according to a predetermined mask and recombine the cut watermark pattern into the partial watermark pattern, for example, the complete watermark pattern shown in fig. 2a is cut according to a predetermined mask and recombined into four partial watermark patterns shown in fig. 2b-2 e.

The mask is a rule of watermark cutting and recombination, for example, a watermark is cut into 32 units, and then four local watermark images are combined according to a predetermined rule, each local watermark image comprises 8 units, and the four local watermark images can be divided into quarters and respectively embedded into four corners of a frequency domain interval image. Since the masking rule is predetermined, the complete watermark map can be restored by performing segmentation and rearrangement according to the predetermined rule during tracing.

The steps not only can make the watermark be disordered and not easy to crack, but also can realize the purpose of embedding the larger watermark by dividing the larger watermark into a plurality of smaller local watermark images which are respectively embedded into different video frames.

S103: and respectively dispersedly and superpositioning the local watermark images in real time in different continuous frames of the video conference picture of the frequency domain interval by taking the frame as a unit.

Preferably, the partial watermark maps are superimposed in successive frames, the next partial watermark map is changed by a fixed number of frames, and the superimposition is repeated cyclically. For example, the partial watermark map shown in FIG. 2b is superimposed on frames 1-3, the partial watermark map shown in FIG. 2c is superimposed on frames 4-6, the partial watermark map shown in FIG. 2d is superimposed on frames 7-9, and the partial watermark map shown in FIG. 2e is superimposed on frames 10-12.

In addition to the above method, the local watermark images can be superimposed according to other rules. For example, the partial watermark map shown in FIG. 2b is superimposed on frames 1-2, the partial watermark map shown in FIG. 2c is superimposed on frames 3-5, the partial watermark map shown in FIG. 2d is superimposed on frames 6-9, and the partial watermark map shown in FIG. 2e is superimposed on frames 10-14.

In a word, the

steps

102 and 103 greatly improve the difficulty of a cracker in obtaining complete watermark information by disorganizing and recombining the complete watermark image and dispersedly overlapping the complete watermark image in different frames, and effectively prevent the cracker from simply obtaining the complete watermark information from a single frame image.

S104: and converting the continuous frames of the video conference pictures in the frequency domain interval into the continuous frames of the video conference pictures in the time domain interval by using a reverse frequency domain conversion algorithm to obtain the continuous frames of the video conference pictures on which the invisible local watermark is superimposed.

The reverse frequency domain conversion algorithm and the frequency domain conversion algorithm are mutually reverse in path, and by utilizing the invisibility of frequency domain information in a time domain interval, the superposed local watermark image of the continuous frame of the video conference picture in the time domain interval after reverse conversion becomes invisible and becomes the invisible watermark.

S105: and presenting the continuous frames of the video conference picture on which the invisible partial watermark is superposed to the user.

Preferably, the invention also introduces a low attention area, which is generally an area of little attention of the user, and aims to reduce the perception of the user on the deformed area after the invisible watermark is superimposed; only the low attention area of the continuous frame of the video conference picture is converted into the frequency domain and the time domain, so that the efficiency of converting the frequency domain into the time domain is improved; and when tracing, because the low attention area must be determined first, and the low attention area is converted from the time domain area to the frequency domain area, the local watermark image can be observed, and then the local watermark image is intercepted, therefore, under the condition that a cracker does not know the distribution of the low attention area, the local watermark image can not be obtained, and the cracking difficulty is increased.

The attention of the human being is itself determined by the perception of the human brain and human eye, and areas of high attention gather information that is of most interest to the human being and is most useful to the human being.

As shown in fig. 3, the right image is a high attention area image of the left image, and the higher the probability, the higher the probability of representing an important object in the image, i.e., the focus of attention of human eyes, the highest probability is at the head position, and the higher probability is at the legs and the tail, which is really useful information in the image.

Here, a method of determining a low attention area and adding an invisible watermark in the low attention area of a video picture of a video conference is enumerated, as shown in fig. 4, the method comprising:

s401: and removing people and characters from the video conference picture in the time domain interval to obtain the residual video conference picture in the time domain interval.

Step S401 is mainly to remove a high attention area in the video conference screen. The method for removing people and characters can adopt an edge matting method or a rectangular removing method, for example, if the length of a person in a picture is 16cm and the widest position is 4cm, all parts of the person in the picture are directly removed by 16cm multiplied by 4 cm.

Usually people and text belong to high attention areas. The high attention area includes bright colors, moving parts in the video (e.g., running train, running dog, flapping feathers), etc., in addition to people and text.

S402: and dividing the rest video conference pictures in the time domain interval according to the specified block size to obtain a plurality of blocks, and selecting the position of a certain complete block as a low attention area according to a set mode.

The invention can also adjust the parameters of the frequency domain conversion algorithm and the reverse frequency domain conversion algorithm based on the resolution of the video conference picture and the configuration of the equipment so as to avoid increasing the time delay when the watermark is superposed.

For example, the predetermined block size is specifically set according to the resolution of the video conference screen, and the block size set at the high resolution is small, and the block size set at the low resolution is large.

The predetermined manner may be to select the first complete block from left to right from top to bottom, or to select the first complete block closest to the edge of the screen preferentially according to the top, bottom, left and right direction order, which is not limited herein.

The low attention area is usually selected to be background without special patterns or located in the corners of the picture. It is therefore also possible to fixedly define the low attention area at a corner of the picture, e.g. a local area of fixed size arranged in the upper left corner, e.g. an area of 200 x 300 pixels in the upper left corner.

S403: and converting the low attention region of the continuous frame of the time domain interval into the low attention region of the continuous frame of the frequency domain interval in real time based on a frequency domain conversion algorithm.

The continuous frames are continuous frames of a video conference picture. The low attention area of the continuous frames is convenient for information tracing, and a common area of the rest parts of the first frames of the video conference picture after the high attention area is removed can be selected as the low attention area of all the frames; the low attention area of each frame may also be selected as the low attention area of each frame.

It is also possible to divide the video picture of each frame into the same blocks according to a predetermined rule, and then select a common divided block of the remaining portions of the first frames of the video conference picture after removing the high attention area as the low attention area of all frames. During tracing, a frame of picture can be selected according to a preset blocking rule for segmentation, then frequency domain conversion is carried out on each segmentation block, and the segmentation block on which the local watermark is superimposed is searched. The segmented blocks corresponding to other frames are then frequency domain transformed to recover the complete watermark map.

Only the low attention area of the continuous frame of the video conference picture is subjected to time domain and frequency domain conversion, compared with the time domain and frequency domain conversion of the complete picture of the continuous frame, the picture to be converted is smaller, so that the conversion is more efficient and faster, and the cracking difficulty is increased.

S404: and dispersedly superposing the local watermark images in the corners of the low attention areas of different continuous frames of the frequency domain interval in real time by taking the frames as units respectively.

As shown in fig. 5, the picture in the second row is a corresponding frequency domain interval map of the image (time domain interval) in the first row, the center of the frequency domain interval map is at a low frequency, and the center corresponds to a continuous color block region displaying the time domain interval map, and the edge of the frequency domain interval map is at a high frequency, and the detail or edge contour portion of the time domain interval map is correspondingly displayed. After high frequency parts of edges are removed in the frequency domain interval graph in the 4 th row in the graph, the graph is converted into a corresponding time domain interval graph which is not greatly different from the original graph shown in the 3 rd row; after the center low frequency is removed from the frequency domain interval diagram in column 6, most of the color blocks of the whole time domain interval diagram disappear, and only some contour portions corresponding to the high frequency components remain.

Therefore, specifically, the local watermark map is superimposed in the four corners of the low attention area in the frame of the frequency domain section, i.e., the high frequency part of the low attention area, for example, assuming that the 1 st column and the 2 nd column of fig. 5 are the low attention areas of the video conference screen, the four equal division units of the local watermark map shown in fig. 2d and fig. 2b are superimposed in the four corners of the frequency domain section map of the 1 st column and the 2 nd column, respectively, as shown in fig. 5. In a word, the low-frequency part is prevented from being covered by the watermark, so that the low attention area in the time domain interval is greatly changed, and the appearance of a user is influenced.

S405: and converting the low attention area of the continuous frame of the frequency domain interval overlapped with the local watermark map into the low attention area of the continuous frame of the time domain interval overlapped with the local watermark map by an inverse frequency domain conversion algorithm.

S406: and correspondingly replacing the low attention area of the video conference picture of the continuous frame of the time domain interval with the low attention area of the continuous frame of the local watermark map, so as to obtain the continuous frame of the video conference picture with the invisible local watermark map.

In another embodiment, determining a low attention region using an attention mechanism and deep learning is shown in fig. 6, the method comprising:

s601: and training a neural network model for detecting the high-attention area by using an attention mechanism and deep learning.

Attention Mechanism (Attention Mechanism) stems from the study of human vision. In cognitive science, humans selectively focus on a portion of all information while ignoring other visible information due to bottlenecks in information processing. The above mechanism is commonly referred to as an attention mechanism. Different parts of the human retina have different degrees of information processing capabilities, i.e., acuity, with only the foveal part having the greatest acuity. In order to make reasonable use of limited visual information processing resources, a human needs to select a specific portion in a visual region and then focus on it. For example, when a person is reading, only a few words to be read are usually attended to and processed. In summary, the attention mechanism has two main aspects: deciding which part of the input needs to be focused on; limited information processing resources are allocated to the important parts.

S602: the neural network model for detecting the high attention area extracts the characteristics of the image through a convolution network, classifies the high and low attention areas by using an attention mechanism, and finally outputs the high attention area.

S603: excluding the high attention area, the remaining area obtained is the low attention area.

Steps S604 to S607 correspond to steps S403 to S406, and are a method of adding an invisible watermark to a low attention area of a video screen.

Fig. 7 is a flowchart of a method for tracing video conference information of a video conference picture added with an invisible watermark according to an embodiment of the present invention, where the embodiment includes the following steps:

s701: converting continuous frames of the video conference pictures to be traced and superimposed with the invisible local watermark images from time domain intervals to frequency domain intervals through the frequency domain conversion algorithm, and obtaining the local watermark images from the corners of the frequency domain intervals;

s702: rearranging the local watermark map according to a certain rule to obtain a complete watermark map;

s703: and identifying watermark information including a conference number, conference time, conference account information, a conference terminal number, a conference terminal location and the like according to the complete watermark map.

Step S701 further specifically includes:

removing people and characters from the video conference picture on which the invisible local watermark is superimposed to obtain a residual video conference picture in a time domain interval, wherein the video conference picture on which the invisible local watermark is superimposed is a picture in the time domain interval;

dividing the rest video conference pictures in the time domain interval according to the specified block size to obtain a plurality of blocks, and selecting the position of a certain complete block as a low attention area according to a set mode, wherein the step is the same as the step S402;

converting, by the frequency domain conversion algorithm, the low attention region of the continuous frame on which the local watermark is superimposed into a low attention region of the continuous frame on which the local watermark is superimposed of a frequency domain section;

the local watermark map is truncated from the corners of the low attention areas of the consecutive frames of the frequency domain interval on which the local watermark map is superimposed.

Certain rules in step S702 are the same as the correspondence of step S102. The partial watermark image is firstly divided according to a certain rule and then recombined into a complete watermark image.

Step S703 specifically includes performing noise removal on the complete watermark image to eliminate the influence of the high-frequency component in the frequency domain interval on the character recognition in the watermark information.

Exemplary System

Correspondingly, the embodiment of the invention also provides a system for adding the invisible watermark in the video picture of the video conference. Fig. 8 is a schematic diagram of a system 100 for adding an invisible watermark in a video frame of a video conference according to an embodiment of the present invention, and as shown in fig. 8, the system 100 according to the embodiment includes:

converting means 101 for converting, in real time, the successive frames of the videoconference picture for the time domain interval into the successive frames of the videoconference picture for the frequency domain interval based on a frequency domain conversion algorithm;

the dividing device 102 is configured to divide the watermark map according to a certain rule to obtain a plurality of local watermark maps, where the rule is to cut the watermark map according to a predetermined mask, and recombine the cut watermark maps into the local watermark maps, where the watermark maps are generated at a conference terminal and include watermark information such as a conference number, a conference time, conference account information, a conference terminal number, and a location where the conference terminal belongs;

superimposing means 103 for dispersedly superimposing the local watermark images in real time in different consecutive frames of the video conference picture of the frequency domain interval in units of frames, respectively;

the conversion device 101 is further configured to convert the continuous frames of the video conference picture in the frequency domain interval into the continuous frames of the video conference picture in the time domain interval by using a reverse frequency domain conversion algorithm, so as to obtain the continuous frames of the video conference picture on which the invisible local watermark is superimposed;

a presentation device 104 for presenting the consecutive frames of the video conference picture superimposed with the invisible partial watermark to a user.

The system 100 further comprises a removal module 105 and a selection module 106,

the removing module 105 is configured to remove people and characters from the video conference picture in the time domain interval, and obtain a remaining video conference picture in the time domain interval;

the selection module 106 is configured to divide the remaining video conference pictures in the time domain interval according to a predetermined block size to obtain a plurality of blocks, and select a position of a complete block as a low attention area according to a predetermined manner;

the converting means 101 is further configured to convert, in real time, the low attention region of the continuous frame of the time-domain interval into the low attention region of the continuous frame of the frequency-domain interval based on a frequency-domain conversion algorithm;

the superimposing device 103 is further configured to superimpose the local watermark map in a frame unit in a real-time dispersed manner in the corners of the low attention areas of different consecutive frames of the frequency domain interval;

the converting means 101 is further configured to convert the low attention region of the continuous frame of the frequency domain interval superimposed with the local watermark into the low attention region of the continuous frame of the time domain interval superimposed with the local watermark by an inverse frequency domain conversion algorithm;

the system 100 further comprises a replacing device 107 for correspondingly replacing the low attention area of the video conference picture of the continuous frame of the time domain interval on which the local watermark is superimposed with the low attention area of the continuous frame of the time domain interval, so as to obtain the continuous frame of the video conference picture on which the invisible local watermark is superimposed.

The system 100 further comprises training means 108 and excluding means 109:

the training device 108 is configured to train a neural network model for detecting a high attention region by using an attention mechanism and deep learning, perform feature extraction on an image by using the neural network model for detecting the high attention region through a convolution network, classify high and low attention regions by using the attention mechanism, and finally output the high attention region;

the excluding device 109 is configured to exclude the high attention area, and the obtained remaining area is the low attention area;

the converting means 101 is further configured to convert the low attention region of the continuous frame of the frequency domain interval superimposed with the local watermark into the low attention region of the continuous frame of the time domain interval superimposed with the local watermark by using an inverse frequency domain conversion algorithm;

the replacing device 107 is configured to correspondingly replace the low attention area of the video conference picture of the continuous frame of the time domain interval on which the local watermark is superimposed, so as to obtain the continuous frame of the video conference picture on which the invisible local watermark is superimposed.

The superimposing means 103 comprises a replacement cycle module 1031, which is configured to superimpose the local watermark map in the form of continuous frames, and replace the next local watermark map by a fixed number of frames, and perform cyclic superimposition.

The system further comprises an adjusting means 110 for adjusting parameters of the frequency domain conversion algorithm and the inverse frequency domain conversion algorithm based on the resolution of the video conference picture and the configuration of the device to avoid adding delay in overlaying the watermark.

Fig. 9 is a schematic diagram of a system 200 for performing video conference information tracing on a video conference picture added with an invisible watermark according to an embodiment of the present invention, and as shown in fig. 9, the system 200 according to the embodiment includes:

a second conversion device 201, configured to convert, by the frequency domain conversion algorithm, the continuous frames of the video conference picture to be traced and superimposed with the invisible local watermark map from the time domain interval to the frequency domain interval, and obtain the local watermark map from the corner of the frequency domain interval;

a rearranging device 202, configured to rearrange the local watermark pattern according to a certain rule, so as to obtain a complete watermark pattern;

and the identifying device 203 is used for identifying watermark information such as a conference number, conference time, conference account information, a conference terminal number, a conference terminal belonging place and the like according to the complete watermark map.

The second conversion means 201 comprise a second removal module 204 and a second selection module 205,

the second removing module 204 is configured to remove people and characters from the video conference picture on which the invisible local watermark is superimposed, and obtain a remaining video conference picture in the time domain interval;

the second selecting module 205 is configured to divide the remaining video conference pictures in the time domain interval according to a specified block size to obtain a plurality of blocks, and select a position of a complete block as a low attention area according to a predetermined manner;

the second converting means 201 is further configured to convert, by the frequency domain converting algorithm, the low attention region of the continuous frame on which the local watermark is superimposed into the low attention region of the continuous frame on which the local watermark is superimposed, and the local watermark is extracted from the corner of the low attention region of the continuous frame on which the local watermark is superimposed.

It should be noted that although the operations of the present invention for adding invisible watermarks in video frames of a video conference and for tracing video conference information are depicted in the drawings in a particular order, this does not require or imply that these operations must be performed in that particular order, or that all of the illustrated operations must be performed to achieve the desired results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.

Furthermore, although in the above detailed description reference has been made to adding invisible watermarks in video pictures of a video conference and to several devices, units, or modules of a video conference information sourcing system, such a division is merely exemplary and not mandatory. Indeed, the features and functionality of two or more of the modules described above may be embodied in one module according to embodiments of the invention. Conversely, the features and functions of one module described above may be further divided into embodiments by a plurality of modules.

While the spirit and principles of the invention have been described with reference to several particular embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, nor is the division of aspects, which is for convenience only as the features in such aspects may not be combined to benefit. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

The present invention provides:

1. a method for adding invisible watermarks in video pictures of a video conference, the method comprising:

2. The method of adding an invisible watermark according to item 1, wherein the method further comprises:

and correspondingly replacing the low attention area of the video conference picture of the continuous frame of the time domain interval with the low attention area of the continuous frame of the local watermark map, so as to obtain the continuous frame of the video conference picture with the invisible local watermark map.

3. The method of adding an invisible watermark according to item 1, wherein the method further comprises:

training a neural network model for detecting a high-attention area by using an attention mechanism and deep learning;

the neural network model for detecting the high attention area extracts the characteristics of the image through a convolution network, classifies the high and low attention areas by using an attention mechanism, and finally outputs the high attention area;

excluding the high attention area, and obtaining the residual area as the low attention area;

separately and dispersedly superposing the local watermark images in the corners of the low attention areas of different continuous frames of the frequency domain interval in real time by taking the frames as units;

converting the low attention area of the continuous frame of the frequency domain interval on which the local watermark is superposed into the low attention area of the continuous frame of the time domain interval on which the local watermark is superposed by an inverse frequency domain conversion algorithm;

4. The method for adding the invisible watermark according to any one of items 1 to 3, wherein the rule is to cut the watermark image according to a predetermined mask and recombine the cut watermark image into the partial watermark image.

5. The method for adding the invisible watermark according to any one of items 1 to 4, wherein the step of respectively dispersing and superposing the local watermark images in real time by taking the frames as units specifically comprises the steps of:

and overlapping the local watermark images in a continuous frame form, replacing the next local watermark image at intervals of a fixed frame number, and circularly overlapping.

6. The method for adding the invisible watermark according to any one of items 1 to 5, wherein the watermark is generated at a conference terminal and comprises watermark information such as a conference number, conference time, conference account information, a conference terminal number, a conference terminal location and the like.

7. The method for adding the invisible watermark, according to any one of the items 1-6, is characterized in that the method further comprises: and adjusting parameters of the frequency domain conversion algorithm and the reverse frequency domain conversion algorithm based on the resolution of the video conference picture and the configuration of the equipment so as to avoid increasing delay when the watermark is superposed.

8. A method for tracing video conference information on a video conference picture added with an invisible watermark based on any one of items 1 to 7, the method comprising:

rearranging the local watermark image according to a certain rule to obtain a complete watermark image;

9. The method for tracing video conference information according to item 8, wherein the step of converting the continuous frames of the video conference picture superimposed with the invisible local watermark image to be traced by the frequency domain conversion algorithm from the time domain interval to the frequency domain interval and obtaining the local watermark image from the corner of the frequency domain interval specifically comprises:

removing people and characters from the video conference picture on which the invisible local watermark is superimposed to obtain a residual video conference picture of the time domain interval;

10. A system for adding an invisible watermark to a video frame of a video conference, the system comprising:

11. The system for adding an invisible watermark according to item 10,

the system further comprises a removal module and a selection module,

the removing module is used for removing people and characters from the video conference picture in the time domain interval to obtain the residual video conference picture in the time domain interval;

the selection module is used for dividing the rest video conference pictures in the time domain interval according to the specified square block size to obtain a plurality of square blocks, and selecting the position of a certain complete square block as a low attention area according to a set mode;

the converting means is further configured to convert, in real time, the low attention region of the continuous frame of time-domain intervals into a low attention region of the continuous frame of frequency-domain intervals based on a frequency-domain conversion algorithm;

the superposition device is also used for dispersedly superposing the local watermark images in the corners of the low attention areas of different continuous frames of the frequency domain interval in real time by taking the frames as units;

the converting device is further used for converting the low attention area of the continuous frame of the frequency domain interval superposed with the local watermark into the low attention area of the continuous frame of the time domain interval superposed with the local watermark through an inverse frequency domain conversion algorithm;

the system further comprises a replacing device which is used for correspondingly replacing the low attention area of the video conference picture of the continuous frame of the time domain interval on which the local watermark is superimposed, so as to obtain the continuous frame of the video conference picture on which the invisible local watermark is superimposed.

12. The system for adding invisible watermarks according to item 10, further comprising training means and excluding means:

the training device is used for training a neural network model for detecting a high attention area by using an attention mechanism and deep learning, extracting the characteristics of an image by the neural network model for detecting the high attention area through a convolution network, classifying the high and low attention areas by using the attention mechanism, and finally outputting the high attention area;

the excluding device is used for excluding a high attention area, and the obtained remaining area is the low attention area;

the conversion device is also used for converting the low attention area of the continuous frame of the time domain interval into the low attention area of the continuous frame of the frequency domain interval in real time based on a frequency domain conversion algorithm;

13. The system for adding an invisible watermark according to any one of the items 10-12, wherein the rule is to cut the watermark image according to a predetermined mask and recombine the cut watermark image into the partial watermark image.

14. The system for adding the invisible watermark according to any one of the items 10 to 13, wherein the superimposing apparatus includes a replacement loop module for superimposing the local watermark patterns in the form of continuous frames, and replacing the next local watermark pattern by a fixed number of frames, and performing the loop superimposition.

15. The system for adding the invisible watermark according to any one of items 10 to 14, wherein the watermark is generated at a conference terminal and includes watermark information such as a conference number, conference time, conference account information, a conference terminal number, a conference terminal location, and the like.

16. The system for adding the invisible watermark according to any one of the items 10 to 15, characterized in that the system further comprises an adjusting device for adjusting parameters of the frequency domain conversion algorithm and the inverse frequency domain conversion algorithm based on the resolution of the video conference picture and the configuration of the device so as to avoid adding delay when the watermark is superimposed.

17. A system for tracing video conference information on a video conference picture added with an invisible watermark based on any one of the systems of items 10 to 16, the system comprising:

and the identification device is used for identifying watermark information such as a conference number, conference time, conference account information, a conference terminal number, a conference terminal location and the like according to the complete watermark map.

18. The system for tracing video conference information as recited in claim 17, wherein the second converting means comprises a second removing module and a second selecting module,

the second removing module is used for removing people and characters from the video conference picture on which the invisible local watermark is superimposed to obtain the residual video conference picture in the time domain interval;

the second selection module is used for dividing the rest video conference pictures in the time domain interval according to the specified square block size to obtain a plurality of square blocks, and selecting the position of a certain complete square block as a low attention area according to a set mode;

the second converting means is further configured to convert, by the frequency domain conversion algorithm, the low attention region of the continuous frame on which the local watermark is superimposed into a low attention region of a frequency domain section on which the continuous frame of the local watermark is superimposed, the local watermark being cut from a corner of the low attention region of the frequency domain section on which the continuous frame of the local watermark is superimposed.

Claims

2. The method of adding an invisible watermark according to claim 1, further comprising:

3. The method for adding the invisible watermark according to claim 1 or 2, wherein the watermark pattern is generated at a conference terminal and comprises watermark information such as a conference number, a conference time, conference account number information, a conference terminal number, a conference terminal belonging place and the like.

4. A method for tracing video conference information of a video conference picture added with an invisible watermark according to any one of claims 1-3, wherein the method comprises:

5. The method for tracing video conference information according to claim 4, wherein the step of converting the continuous frames of the video conference picture superimposed with the invisible local watermark to be traced by the frequency domain conversion algorithm from the time domain section to the frequency domain section and obtaining the local watermark from the corner of the frequency domain section specifically comprises:

removing people and characters from the video conference picture on which the invisible local watermark is superimposed to obtain the residual video conference picture in the time domain interval;

converting, by the frequency domain conversion algorithm, a low attention region of the continuous frame on which the local watermark is superimposed into a low attention region of the continuous frame on which the local watermark is superimposed, of a frequency domain section;

6. A system for adding an invisible watermark to a video frame of a video conference, the system comprising:

converting means for converting, in real time, the successive frames of the videoconference frames for the time domain intervals into the successive frames of the videoconference frames for the frequency domain intervals based on a frequency domain conversion algorithm;

the superposition device is used for dispersedly superposing the local watermark images in different continuous frames of the video conference picture of the frequency domain interval in real time by taking the frame as a unit;

7. The system for adding invisible watermarks according to claim 6,

the system further comprises a removal module and a selection module,

the selection module is used for dividing the rest video conference pictures of the time domain interval according to the specified square block size to obtain a plurality of square blocks, and selecting the position of a certain complete square block as a low attention area according to a set mode;

8. The system for adding the invisible watermark is characterized in that the watermark pattern is generated at a conference terminal and comprises watermark information such as a conference number, conference time, conference account number information, a conference terminal number, a conference terminal belonging place and the like.

9. A system for tracing video conference information on video conference pictures added with invisible watermarks according to any one of claims 6-8, the system comprising:

10. The video conference information traceability system of claim 9, wherein the second transformation means comprises a second removal module and a second selection module,

the second selection module is used for dividing the rest video conference pictures of the time domain interval according to the specified square block size to obtain a plurality of square blocks, and selecting the position of a certain complete square block as a low attention area according to a set mode;