KR102555534B1 - Method and apparatus for detecting wartermark for spherical panoramic image - Google Patents

Abstract

In the watermark detection method according to an embodiment of the present invention, a target image is input to a viewpoint information derivation model generated by learning viewpoint information of a learning image and a learning panoramic image, and a 360° watermark-embedded image is obtained. obtaining viewpoint information of the target image for the panoramic image; generating a target original image corresponding to the target image by inversely rendering the target image based on the viewpoint information; and comparing a watermark pattern included in the target original image with a watermark pattern included in the 360° panoramic image to determine whether the target image corresponds to a part of the 360° panoramic image.

Description

Watermark detection method and apparatus for spherical panoramic image

The present invention relates to a watermark detection method and apparatus, and more particularly, to a watermark detection method and apparatus for detecting a watermark from a target image in order to determine whether a target image originates from a panoramic image and extract copyright information.

A panoramic image is an image capable of securing a 360° foreground view, and allows the user to freely select a viewpoint within the image, providing a feeling as if the user is located in a virtual space and is directly viewing the image.

As interest in panoramic images has recently increased, copyright protection for panoramic images has become important. However, compared to the rapidly developing panoramic video industry, the technology to protect copyrights is not yet properly established.

There is digital watermarking as a technology to protect copyrights from illegal copying and illegal distribution of existing media contents. Currently, digital watermarking technology for contents such as 2D, stereoscopic 3D, and 3D mesh is continuously researched. However, there is currently no watermarking technology for protecting panoramic images.

In particular, due to the nature of the panoramic image, the perspective of the panoramic image is leaked rather than the entire panoramic image being leaked, and copyright infringement may occur. For example, when a concert venue is made into a panoramic video, the viewpoint video in the direction of the stage has content value even if it is not a full panoramic video, and it is possible to collect and regenerate the video from various angles and recreate it as panoramic content. There is a need for a technology capable of detecting a watermark in an image.

Korean Patent Publication, No. 10-2003-0010220

An object to be solved by an embodiment of the present invention is to provide a technology capable of detecting a watermark from a viewpoint image of a panoramic image.

However, the technical problems to be achieved by the embodiments of the present invention are not limited to the above-mentioned problems, and various technical problems may be derived from the contents to be described below within a range apparent to those skilled in the art.

In the watermark detection method according to an embodiment of the present invention, a target image is input to a viewpoint information derivation model generated by learning viewpoint information of a learning image and a learning panoramic image, and a 360° watermark-embedded image is obtained. obtaining viewpoint information of the target image for the panoramic image; generating a target original image corresponding to the target image by inversely rendering the target image based on the viewpoint information; and comparing a watermark pattern included in the target original image with a watermark pattern included in the 360° panoramic image to determine whether the target image corresponds to a part of the 360° panoramic image.

The determining whether the target image corresponds to a part of the 360° panoramic image may include dividing the target original image into a plurality of blocks to correspond to the positions of the plurality of blocks; selecting a block from among the plurality of blocks of the target original image in which pixels included in the plurality of blocks are reconstructed by a first threshold value or higher; For each of the selected blocks, converting a spatial domain of an image in each selected block into a frequency coefficient; calculating a correlation between the frequency coefficient in each selected block and a watermark pattern included in the 360° panoramic image; and determining that the target image is a part of the 360° panoramic image when an average of the correlation coefficients calculated from each of the selected blocks is greater than or equal to a second threshold.

Also, the transforming into the frequency coefficient may include transforming the spatial domain into the frequency coefficient through a discrete Fourier transform.

In addition, calculating the correlation coefficient of the watermark pattern may include a correlation coefficient between a magnitude coefficient among the frequency coefficients in each selected block and a watermark pattern included in the 360° panoramic image can be calculated.

In addition, the viewpoint information derivation model may be implemented as an SRNet model that performs pooling only in the last layer type.

In addition, the target original image may include at least one of an equirectangular image, a conical image, a cylindrical image, a pseudo-cylindrical image, a cylindrical image, a fisheye image, a Mercator image, and a sinusoidal image. can include

A watermark detection apparatus according to an embodiment of the present invention inputs a target image to a viewpoint information derivation model generated by learning viewpoint information of a learning image and a learning panoramic image to obtain a 360° watermark-embedded image. a viewpoint information derivation unit for obtaining viewpoint information of the target image for the panoramic image; a reverse rendering unit generating a target original image corresponding to the target image by reverse-rendering the target image based on the viewpoint information; and a watermark detection unit comparing a watermark pattern included in the target original image and a watermark pattern included in the 360° panoramic image to determine whether the target image corresponds to a part of the 360° panoramic image.

The watermark detector divides the target original image into a plurality of blocks to correspond to the positions of the plurality of blocks, and a pixel included in the plurality of blocks among the plurality of blocks of the target original image has a first threshold value. A block that has been restored above is selected, and for each of the selected blocks, a spatial domain of an image in each selected block is converted into a frequency coefficient, and the frequency coefficient in each selected block and the 360° A correlation coefficient of a watermark pattern included in a panoramic image is calculated, and when an average of the correlation coefficients calculated from each of the selected blocks is greater than or equal to a second threshold value, the target image is a part of the 360° panoramic image. can be determined as

Also, the watermark detector may transform the spatial domain into the frequency coefficient through a discrete Fourier transform.

Also, the watermark detection unit may calculate a correlation between a magnitude coefficient among the frequency coefficients in each selected block and a watermark pattern included in the 360° panoramic image.

In addition, the viewpoint information derivation model may be implemented as an SRNet model that performs pooling only in the last layer type.

In addition, the target original image may include at least one of an equirectangular image, a conical image, a cylindrical image, a pseudo-cylindrical image, a cylindrical image, a fisheye image, a Mercator image, and a sinusoidal image. can include

According to an embodiment of the present invention, it is possible to prevent copyright infringement of a panoramic image by detecting a watermark not only in the entire panoramic image but also in a viewpoint image corresponding to a specific viewpoint of the panoramic image.

1 is a functional block diagram of a watermark embedding device according to an embodiment of the present invention.
2 is a flow chart showing the process of a watermark embedding method according to an embodiment of the present invention.
3 is a functional block diagram of a watermark detection apparatus according to an embodiment of the present invention.
4 is a flow chart showing the process of a watermark detection method according to an embodiment of the present invention.
5 is an exemplary view for explaining viewpoint information at which a target image is located in a panoramic image.
6 is a diagram illustrating the structure of a viewpoint information derivation model according to an embodiment of the present invention.
7 is an exemplary view for explaining restoring a pixel of a hole through image interpolation according to an embodiment of the present invention.

Objects and technical configurations of the present invention and details of the operational effects thereof will be more clearly understood by the following detailed description based on the accompanying drawings in the specification of the present invention. An embodiment according to the present invention will be described in detail with reference to the accompanying drawings.

The embodiments disclosed herein should not be construed or used as limiting the scope of the present invention. It goes without saying that the description, including the embodiments herein, has a variety of applications for those skilled in the art. Therefore, any embodiments described in the detailed description of the present invention are illustrative for better explaining the present invention, and the scope of the present invention is not intended to be limited to the examples.

The functional blocks shown in the drawings and described below are only examples of possible implementations. Other functional blocks may be used in other implementations without departing from the spirit and scope of the detailed description. Also, while one or more functional blocks of the present invention are represented as separate blocks, one or more of the functional blocks of the present invention may be a combination of various hardware and software configurations that perform the same function.

In addition, the expression that certain components are included simply indicates that the corresponding components exist as an open expression, and should not be understood as excluding additional components.

Furthermore, it should be understood that when a component is referred to as being connected or connected to another component, it may be directly connected or connected to the other component, but other components may exist in the middle.

In addition, expressions such as 'first and second' are expressions used only to classify a plurality of components, and do not limit the order or other characteristics between the components.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

1 is a functional block diagram of a watermark embedding device 100 according to an embodiment of the present invention.

As shown in FIG. 1, the watermark embedding apparatus 100 according to an embodiment of the present invention includes a block allocation unit 110, a conversion unit 120, an embedding unit 130, a restoration unit 140, and a panorama unit. It includes an image generator 150.

The block allocator 110 divides the original image to be rendered as a panoramic image into a plurality of blocks. At this time, 'block' is used as the same meaning as 'patch' in the term patch image meaning a predetermined part of a specific image. On the other hand, the original image of the format that can be rendered as a panoramic image is an equirectangular image, a conical image, a cylindrical image, a pseudo-cylindrical image, a cylindrical image, a fisheye image, a Mercator image, a sinusoidal curve ( sinusoidal) images, etc.

The transform unit 120 transforms the spatial domain of an image included in each block into a frequency coefficient. For example, each block is transformed into a frequency coefficient in the spatial domain through a discrete Fourier transform.

The inserter 130 may insert a watermark pattern into frequency coefficients of each block. If discrete Fourier transform is used, the inserter 130 may insert a watermark pattern into a magnitude coefficient among transformed frequency coefficients.

The restoration unit 140 restores the frequency coefficients to which the watermark pattern is inserted for each block in the spatial domain, and the panoramic image generator 150 restores the original image in which the watermark pattern is inserted to the frequency coefficients in each block. Render to create a panoramic image.

At this time, a detailed process in which each component of the watermark embedding apparatus 100 operates will be described together with FIG. 2 .

Meanwhile, the block allocation unit 110, the conversion unit 120, the insertion unit 130, the restoration unit 140, and the panoramic image generation unit 150 included in the above-described embodiment use commands programmed to perform their functions. It may be implemented by a computing device including a memory including and a microprocessor that executes these instructions.

2 is a flow chart showing the process of a watermark embedding method according to an embodiment of the present invention. Each step of the watermark embedding method according to FIG. 2 may be performed by the watermark embedding apparatus 100 described with reference to FIG. 1 .

Prior to explaining each step, a panoramic image is produced in a form that can be seen from a viewpoint of 360 ° x 180 ° (horizontal angle x vertical angle) by taking a picture of an actual space using a camera and then rendering it. At this time, the original image of the format capable of generating a panoramic image is an equirectangular image obtained by projecting a sphere onto a two-dimensional plane, a conical image, a cylindrical image, a pseudo-cylindrical image, a cylindrical image, a fisheye image, and a Mercator image. This includes Mercator images, sinusoidal images, and the like.

In particular, the most widely used format when dealing with panoramic images is an equirectangular image. An equirectangular image is one of the cylindrical projection methods. The width of an equirectangular image is twice the height, which means that the width of the image is 360°, one turn in the longitudinal direction from the sphere, and the height of the image is 180 degrees, the angle between the ceiling and the floor of the sphere. Because it means °. Although an equirectangular image is shown as an original image in the drawings of this specification, this is only an example, and an embodiment of the present invention can be applied using an original image of a format other than the equirectangular image. Here, each step will be explained.

First, the block allocation unit 110 divides an original image to be rendered as a panoramic image into a plurality of blocks (S210). A plurality of blocks is a position where a watermark is inserted, and when a watermark is inserted in units of a plurality of blocks, the watermark can be detected in an arbitrary area of the panoramic image, and the watermark is efficiently protected from cropping attacks. can do.

At this time, the block allocator 110 may allocate a plurality of blocks in a grid format that does not overlap each other. If a plurality of blocks are divided into a lattice form, watermarks can be inserted in all areas without missing parts from the original image, so watermarks can be extracted from images from all perspectives.

In addition, when a viewpoint image of a panoramic image is reverse-rendered, the area and quality of the restored image vary according to the vertical angle of the viewpoint image. When the range of vertical angles of a panoramic image is -90 degrees to +90 degrees, viewpoint images with vertical angles close to -90 degrees and +90 degrees are restored to a large area during inverse rendering, but the quality is low, and the vertical angle is 0 degrees. The near-view image has a small area restored during inverse rendering, but high quality. That is, the quality and area of an image restored during inverse rendering are inversely proportional. At this time, if a plurality of blocks are allocated in a grid form and a watermark is inserted into each block, the number of watermarks that can be extracted increases because a large area is restored even if the restored quality is low. Therefore, when a watermark is inserted into each block divided in a grid form, it is possible to detect a watermark from more blocks at a vertical angle restored with low quality, thereby increasing the robustness of watermark detection.

Next, the transform unit 120 transforms the spatial domain of the image included in each block into a frequency coefficient (S220). For example, a spatial domain may be transformed into a frequency coefficient through a discrete Fourier transform.

The inserter 130 may generate a watermark pattern from the secret key through the reference pattern generator (S225), and insert the watermark pattern into the frequency coefficients converted within each block (S230). Specifically, the inserter 130 may insert a watermark pattern into a magnitude coefficient among frequency coefficients.

At this time, if a watermark is inserted in a low-frequency band, the invisibility of the watermark may be reduced, and if a watermark is inserted in a high-frequency band, the watermark is vulnerable to attacks such as compression, so the insertion unit 130 detects the watermark. In order to increase the robustness of , a watermark pattern may be inserted into frequency coefficients of an intermediate frequency band through a zigzag scan method. (Barni, Mauro, et al. "A DCT-domain system for robust image watermarking." Signal processing 66.3 (1998): 357-372.)

The restoration unit 140 restores the frequency coefficients into the spatial domain in each block to which the watermark pattern is inserted (S240). For example, a frequency coefficient into which a watermark pattern is inserted may be restored to a spatial domain through an inverse discrete Fourier transform.

Thereafter, the panoramic image generation unit 150 combines the images of each block restored through the restoration unit 140 (S245), and renders an original image in which a watermark pattern is inserted into frequency coefficients in each block to create a panorama. An image may be generated (S250).

The watermark detection apparatus 300 according to an embodiment of the present invention checks whether the target image corresponds to a part of the panoramic image generated according to the embodiment of FIGS. 1 and 2 through watermark detection. 3 is a functional block diagram of a watermark detection apparatus 300 according to an embodiment of the present invention.

As shown in FIG. 3, the watermark detection apparatus 300 according to an embodiment of the present invention includes a storage unit 310, a view information derivation model generator 320, a view information derivation unit 330, and a reverse rendering It includes a unit 340 and a watermark detection unit 350.

The storage unit 310 stores a view information derivation model for deriving view information on a location of a target image in a panoramic image. At this time, the view information derivation model stored by the storage unit 310 may be generated by the view information derivation model generator 320 learning a learning image for the learning panoramic image, or may be received from an external device and stored. there is.

The viewpoint information deriving unit 330 derives viewpoint information where the target image is located in the panoramic image, and the inverse rendering unit 340 reverse-renders the target image based on the viewpoint information derived by the viewpoint information deriving unit 330. A target original image corresponding to the target image is generated.

The watermark detection unit 350 compares the watermark pattern included in the target original image generated by the inverse rendering unit 340 with the watermark pattern included in the panoramic image, and determines whether the target image corresponds to a part of the panoramic image.

At this time, a detailed process in which each component of the watermark detection apparatus 300 operates will be described together with FIG. 4 .

Meanwhile, the view information deriving unit 330, the inverse rendering unit 340, and the watermark detection unit 350 included in the above-described embodiment include a memory including instructions programmed to perform their functions and a microprocessor executing these instructions. It may be implemented by an arithmetic device including a.

4 is a flow chart showing the process of a watermark detection method according to an embodiment of the present invention. Each step of the watermark detection method according to FIG. 4 may be performed by the watermark detection apparatus 300 described with reference to FIG. 3 .

First of all, since a watermark inserted into a panoramic image according to the embodiments of FIGS. 1 and 2 can be detected from an original image obtained by reverse-rendering the panoramic image, in order to detect the watermark from a target image assumed to have been leaked, the target image The watermark must be detected from the target original image by inverse rendering.

At this time, in order to restore the same as the target original image, for example, an equirectangular image, inverse rendering setting values are required to determine the number of values to be inversely rendered, so viewpoint information indicating which part of the panoramic image the target image corresponds to is essential. do. 5 is an exemplary view for explaining viewpoint information at which a target image is located in a panoramic image. As shown in FIG. 5, when it is assumed that the panoramic image has a spherical shape, the viewpoint information is the vertical angle v and the horizontal angle of the line segment connecting the center of the sphere to the center point of the area determined to be the target image from the reference axis. means h.

Subsequently, it is possible to determine whether the target image corresponds to a part of the panoramic image by detecting a watermark from the target original image obtained by reverse-rendering the target image based on viewpoint information of the target image. Here, each step will be explained.

The storage unit 310 stores a view information derivation model for outputting view information on a target image location in the panoramic image (S420). To this end, the storage unit 310 may receive and store a viewpoint information derivation model from an external device or may store one generated by the viewpoint information derivation model generator 320 . Hereinafter, the latter embodiment will be described as a premise.

The viewpoint information derivation model generation unit 320 learns the learning image derived from the learning panoramic image and generates a viewpoint information derivation model (S430). The view information derivation model generation unit 320 according to an embodiment may learn view information on a learning panoramic image from a learning image through machine learning such as deep learning. For example, the viewpoint information derivation model generator 320 may generate a viewpoint information derivation model through supervised learning. Supervised learning refers to a machine learning method for learning learning data composed of an input and a corresponding label, and a label may mean a correct answer to an input among learning data, that is, information at a point in time. Here, the viewpoint information may include at least one of horizontal viewpoint information and vertical viewpoint information.

The viewpoint information derivation model generation unit 320 following the supervised learning method receives as input a learning image including at least one of a direct image divided from a learning panoramic image and an indirect image in which viewpoint information is changed therefrom, and learning of the corresponding learning image It is possible to learn learning data in which viewpoints of panoramic images are mapped to labels. Specifically, the viewpoint information derivation model generator 320 may learn a relationship between an input learning image and correct viewpoint information, and generate a viewpoint information derivation model as a result of the learning.

In contrast, the view information derivation model generator 320 performs semisupervised learning in which learning is performed by using some training data labeled with view information in training images and remaining training data consisting of unlabeled training images. Thus, a viewpoint information derivation model can be created. Alternatively, the view information derivation model generator 320 generates a view information derivation model through reinforcement learning using feedback about whether the result of evaluating the view information of the view information derivation model generated through learning is correct. can do.

The view information derivation model generated by the view information derivation model generation unit 320 may be constructed in consideration of the field of application of the video security model, the purpose of learning, or the computer performance of the device. A viewpoint information derivation model according to an embodiment may be a model based on a neural network. For example, the viewpoint information derivation model may be implemented as a model such as a convolution neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), or a bidirectional recurrent deep neural network (BRDNN), but is not limited thereto. does not

6 is a diagram illustrating the structure of a viewpoint information derivation model according to an embodiment of the present invention. Here, LT means a layer type, SC means a skip connection, and numbers in parentheses mean a kernel number.

A viewpoint information derivation model according to an embodiment may be implemented as an SRNet model among CNNs. Compared to the ResNet model, which shows good performance in detecting high-level features, the SRNet model is advantageous in detecting noise. Therefore, the view information derivation model implemented as the SRNet model is suitable for outputting view information by detecting distortion caused by rendering.

Referring to FIG. 6, the view information derivation model implemented as the SRNet model is composed of three types of layers. The first layer type and the second layer type do not have a pooling layer, and only the third layer type has a pooling layer. Since it is difficult to detect fine noise when pooling is performed at the beginning of the network, the view information derivation model performs pooling only in the third layer type to output view information by detecting distortion caused by rendering.

The inverse rendering unit 340 reverse-renders the target image based on the viewpoint information derived by the viewpoint information deriving unit 330 to generate a target original image corresponding to the target image (S440). At this time, the target original image includes an equirectangular image, a conical image, a cylindrical image, a pseudo-cylindrical image, a cylindrical image, a fisheye image, a Mercator image, and a sinusoidal image. Since the inverse rendering unit 340 can accurately know in which part of the panoramic image the target image is located through viewpoint information, it can generate a target original image through inverse rendering by reflecting the viewpoint information.

Meanwhile, since only pixel information included in the target image can be inversely rendered, as shown in FIG. 7 , only a partial region of the original image, which is the basis of the panoramic image, is reconstructed. In addition, since distortion occurs during inverse rendering, a hole is also generated in a region corresponding to the restored part in the original target image. Since the hole does not have pixel information when watermark is detected, robustness of watermark detection may be lowered.

To prevent this, the inverse rendering unit 340 may restore the pixel of the hole through image interpolation (S445). 7 is an exemplary view for explaining restoring a pixel of a hole through image interpolation according to an embodiment of the present invention. Referring to FIG. 7 , the inverse rendering unit 340 may interpolate a hole pixel with an average value of information on non-hole pixels among pixels around a pixel to be interpolated through image interpolation.

Thereafter, the watermark detection unit 350 compares the watermark pattern included in the target original image generated by the inverse rendering unit 340 with the watermark pattern included in the panoramic image to determine whether the target image corresponds to a part of the panoramic image. Do (S450).

In more detail, in step S450, the target original image is divided into a plurality of blocks to correspond to the positions of the plurality of blocks allocated to the original image, and pixels included in blocks among the plurality of blocks of the target original image are reconstructed above a predetermined threshold. Blocks can be selected (S451).

Thereafter, the spatial domain of the image in each selected block may be converted into a frequency coefficient (S452). For example, the watermark detector 350 transforms a spatial domain into a frequency coefficient through discrete Fourier transform.

In addition, in order to compare the converted frequency coefficient with the watermark pattern included in the panoramic image, a watermark pattern is generated from the secret key used in the panoramic image through the reference pattern generator (S453), and within each block converted in step S452. A correlation coefficient may be calculated by comparing the magnitude coefficient among the frequency coefficients with the watermark pattern generated in step S453 (S454). If discrete Fourier transform is used, the watermark detection unit 350 may compare the magnitude coefficient among the transformed frequency coefficients with the watermark pattern of the panoramic image.

At this time, in order to remove outliers, after excluding some of the blocks with the highest correlation coefficient and some of the blocks with the lowest correlation coefficient, the average of the remaining correlation coefficients is calculated. can be judged to be

According to the above-described embodiment, it is possible to prevent copyright infringement of a panoramic image by detecting a watermark not only in the entire panoramic image but also in a viewpoint image corresponding to a specific viewpoint of the panoramic image.

In particular, when a panoramic image is generated by rendering an original image, only vertical distortion exists and there is also an error. The watermark detection method and apparatus according to the present invention have shift-invariant characteristics This problem can be solved by using the magnitude coefficient of the discrete Fourier transform with .

In addition, when machine learning such as deep learning is used, since the vertical viewpoint information has an accuracy of 99% or more within about 5 °, it is possible to detect a watermark within 5 ° of the vertical viewpoint information from the magnitude coefficient of the discrete Fourier transform.

The above-described embodiments of the present invention may be implemented through various means. For example, embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof.

According to an embodiment of the present invention, the above-described watermark detection method and apparatus have industrial applicability because they can be used in various fields such as at home or at industrial sites.

100: watermark insertion device
110: block allocation unit
120: conversion unit
130: insertion part
140: restoration unit
150: panoramic image generator
300: watermark detection device
310: storage unit
320: viewpoint information derivation model generation unit
330: viewpoint information extraction unit
340: reverse rendering unit
350: watermark detection unit

Claims (14)

Acquiring viewpoint information of the target image for a 360° panoramic image with a watermark inserted therein by inputting a target image to a viewpoint information derivation model generated by learning viewpoint information of the learning image and the learning panoramic image. ;
generating a target original image corresponding to the target image by inversely rendering the target image based on the viewpoint information; and
Comparing a watermark pattern included in the target original image and a watermark pattern included in the 360° panoramic image to determine whether the target image corresponds to a part of the 360° panoramic image; Including,
Determining whether the target image corresponds to a part of the 360 ° panoramic image,
Dividing the target original image into a plurality of blocks;
selecting a block from among the plurality of blocks of the target original image in which pixels included in the plurality of blocks are reconstructed by a first threshold value or higher;
For each of the selected blocks, converting a spatial domain of an image in each selected block into a frequency coefficient; and
Calculating a correlation between the frequency coefficient in each selected block and a watermark pattern included in the 360° panoramic image;
Calculating the correlation coefficient of the watermark pattern,
Calculating a magnitude coefficient among the frequency coefficients in each selected block and the correlation coefficient of a watermark pattern included in the 360° panoramic image;
Watermark detection method. According to claim 1,
Determining whether the target image corresponds to a part of the 360 ° panoramic image,
Determining that the target image is a part of the 360 ° panoramic image when the average of the correlation coefficients calculated from each of the selected blocks is greater than or equal to a second threshold
Watermark detection method. According to claim 2,
The step of converting to the frequency coefficient,
Transforming the spatial domain into the frequency coefficient through a discrete Fourier transform.
Watermark detection method. delete According to claim 1,
The viewpoint information derivation model,
Implemented as an SRNet model that performs pooling only on the last layer type.
Watermark detection method. According to claim 1,
The target original image,
At least one of an equirectangular image, a conical image, a cylindrical image, a pseudo-cylindrical image, a cylindrical image, a fisheye image, a Mercator image, and a sinusoidal image
Watermark detection method. A point in time to acquire the viewpoint information of the target image for the 360° panoramic image in which the watermark is inserted by inputting the target image to the viewpoint information derivation model generated by learning the viewpoint information of the learning image and the learning panoramic image. information extraction unit;
a reverse rendering unit generating a target original image corresponding to the target image by reverse-rendering the target image based on the viewpoint information; and
A watermark detection unit that compares a watermark pattern included in the target original image and a watermark pattern included in the 360° panoramic image to determine whether the target image corresponds to a part of the 360° panoramic image,
The watermark detection unit,
The target original image is divided into a plurality of blocks, blocks in which pixels included in the plurality of blocks are reconstructed above a first threshold value are selected among the plurality of blocks of the target original image, and for each of the selected blocks , Converting the spatial domain of the image in each selected block into a frequency coefficient, and calculating a correlation coefficient between the frequency coefficient in each selected block and a watermark pattern included in the 360° panoramic image, Calculating the magnitude coefficient among the frequency coefficients in the selected block of and the correlation coefficient of the watermark pattern included in the 360 ° panoramic image
Watermark detection device. According to claim 7,
The watermark detection unit,
Determining that the target image is a part of the 360 ° panoramic image when the average of the correlation coefficients calculated from each of the selected blocks is greater than or equal to a second threshold
Watermark detection device. According to claim 8,
The watermark detection unit,
Converting the spatial domain into the frequency coefficient through a discrete Fourier transform
Watermark detection device. delete According to claim 7,
The viewpoint information derivation model,
Implemented as an SRNet model that performs pooling only on the last layer type.
Watermark detection device. According to claim 7,
The target original image,
At least one of an equirectangular image, a conical image, a cylindrical image, a pseudo-cylindrical image, a cylindrical image, a fisheye image, a Mercator image, and a sinusoidal image
Watermark detection device. As a computer program stored on a computer-readable recording medium,
Acquiring viewpoint information of the target image for a 360° panoramic image with a watermark inserted therein by inputting a target image to a viewpoint information derivation model generated by learning viewpoint information of the learning image and the learning panoramic image. ;
generating a target original image corresponding to the target image by inversely rendering the target image based on the viewpoint information; and
Comparing a watermark pattern included in the target original image and a watermark pattern included in the 360° panoramic image to determine whether the target image corresponds to a part of the 360° panoramic image; Including,
Determining whether the target image corresponds to a part of the 360 ° panoramic image,
Dividing the target original image into a plurality of blocks;
selecting a block from among the plurality of blocks of the target original image in which pixels included in the plurality of blocks are reconstructed by a first threshold value or higher;
For each of the selected blocks, converting a spatial domain of an image in each selected block into a frequency coefficient; and
Calculating a correlation coefficient between the frequency coefficient in each selected block and a watermark pattern included in the 360° panoramic image;
Calculating the correlation coefficient of the watermark pattern,
Calculating the correlation coefficient of the magnitude coefficient among the frequency coefficients in each selected block and the watermark pattern included in the 360° panoramic image; instructions for causing a processor to perform a watermark detection method including including
computer program. A computer-readable recording medium storing a computer program,
Acquiring viewpoint information of the target image for a 360° panoramic image with a watermark inserted therein by inputting a target image to a viewpoint information derivation model generated by learning viewpoint information of the learning image and the learning panoramic image. ;
generating a target original image corresponding to the target image by inversely rendering the target image based on the viewpoint information; and
Comparing a watermark pattern included in the target original image and a watermark pattern included in the 360° panoramic image to determine whether the target image corresponds to a part of the 360° panoramic image; Including,
Determining whether the target image corresponds to a part of the 360 ° panoramic image,
Dividing the target original image into a plurality of blocks;
selecting a block from among the plurality of blocks of the target original image in which pixels included in the plurality of blocks are reconstructed by a first threshold value or higher;
For each of the selected blocks, converting a spatial domain of an image in each selected block into a frequency coefficient; and
Calculating a correlation coefficient between the frequency coefficient in each selected block and a watermark pattern included in the 360° panoramic image;
Calculating the correlation coefficient of the watermark pattern,
Calculating the correlation coefficient of the magnitude coefficient among the frequency coefficients in each selected block and the watermark pattern included in the 360° panoramic image; instructions for causing a processor to perform a watermark detection method comprising: including
A computer-readable recording medium.

Images