KR102166392B1 - Method for detecting illlegal pornographic video - Google Patents

Abstract

The present invention relates to a method for detecting illegal footage dissemination. The method is to detect whether comparison target footage is footage illegally disseminated from reference footage. The method includes: (a) a step of extracting key frames from the reference footage; (b) a step of extracting a feature point from the key frames extracted from the reference footage by using a speed-up robust feature (SURF) algorithm and clustering the extracted feature point; (c) a step of storing the feature point clustered from the reference footage in a JSON format; (d) a step of extracting key frames from the comparison target footage; (e) a step of extracting a feature point from the key frames extracted from the comparison target footage by using a SURF algorithm and clustering the extracted feature point; (f) a step of retrieving the JSON format feature point clustered from the reference footage; (g) a step of obtaining feature point pairs by matching between the feature point clustered from the reference footage and the feature point clustered from the comparison target footage; (h) a step of obtaining homography from the obtained feature point pairs; and (i) a step of determining from the obtained homography whether the comparison target footage is footage illegally disseminated from the reference footage.

Description

How to detect the spread of illegally photographed video{METHOD FOR DETECTING ILLLEGAL PORNOGRAPHIC VIDEO}

The present invention relates to a method for detecting the distribution of illegally photographed images, and more particularly, to a method for determining whether a comparison target image is distributed from a reference image by using clustered feature points of a reference image and a comparison target image.

Recently, digital sex crime has emerged as a serious social problem. Digital sexual offenses include photographing, disseminating, intimidating, storing, and displaying the body of the other person without consent, and infringing on the sexual autonomy and personal rights of others in cyberspace.

In particular, among digital sex crimes, the dissemination of illegally filmed videos without relative consent damages the victim's social personality and causes serious problems, such as causing the victim to eventually commit suicide. Accordingly, in order to prevent the spread of such illegally photographed images, the Act on Special Cases concerning the Punishment of Sexual Violence Crimes, etc. stipulate the dissemination of illegally photographed images as a subject of criminal penalties.

However, apart from criminal penalties, in the information age caused by the recent Internet spread, if photos or videos are spread and spread on the Internet, deletion becomes difficult and remains semi-permanently on the Internet.

Victims of these illegally filmed images directly find and report the circulated illegally filmed images, or they are entrusted with the deletion by paying a high cost to a private company called a digital funeral director. However, as described above, there is a limit to deleting and blocking all illegally photographed images distributed on the Internet with limited personnel.

Republic of Korea Patent Publication No. 10-1753360

The present invention was devised to solve the above-described problem, by extracting a key frame from a reference image and an image to be compared, and determining whether the comparison target image is distributed from the reference image using the clustered feature points, An object of the present invention is to provide a method for detecting the distribution of illegally photographed images that can detect and delete illegally photographed images that have been widely distributed.

The technical problem to be achieved by the present invention is not limited to the technical problem mentioned above, and various technical problems may be included within the range that is apparent to a person skilled in the art from the contents to be described below.

In the method for detecting the distribution of illegally photographed images according to an embodiment of the present invention for solving the above problem, in the method of detecting whether an image to be compared is an image illegally distributed from a reference image, (a) in the reference image Extracting a plurality of key frames; (b) extracting feature points from a plurality of key frames extracted from the reference image using a speed-up robust feature (SURF) algorithm, and clustering the extracted feature points; (c) storing the feature points clustered in the reference image in JSON format; (d) extracting a plurality of key frames from the comparison target image; (e) extracting feature points from a plurality of key frames extracted from the comparison target image using a SURF algorithm, and clustering the extracted feature points; (f) calling feature points clustered from the reference image stored in the JSON format; (g) obtaining a plurality of pairs of feature points by matching feature points clustered in the reference image and feature points clustered in the comparison target image; (h) obtaining a homography from the obtained pairs of feature points; And (i) determining whether the comparison target image is an image illegally distributed from the reference image from the acquired homography. It may include.

In an embodiment, the key frame may be a frame having the largest motion vector among frames.

In one embodiment, the step (g) may be characterized in that a plurality of feature point pairs are acquired using Fast Library For Approximate Nearest Neighbors (FLANN).

In an embodiment, prior to the step (h), removing an outlier pair from among the plurality of obtained feature point pairs using a random sample consensus (RANSAC) algorithm; It may further include.

In an embodiment, the step (i) includes: when the obtained homography is non-linear, determining that the comparison target image is not illegally distributed from the reference image; It may include.

In an embodiment, the step (i) includes: if the obtained homography is linear, determining that the comparison target image is illegally distributed from the reference image; It may include.

An apparatus for detecting distribution of illegally photographed images according to an embodiment of the present invention for solving the above problem includes a processor; And a memory for storing instructions executable by the processor. Including, the processor, by executing the instructions, extracting a plurality of key frames from the reference image, and extracting feature points from the plurality of key frames extracted from the reference image using a speed-up robust feature (SURF) algorithm And, the extracted feature points are clustered, the feature points clustered from the reference image are stored in JSON format, a plurality of key frames are extracted from the comparison target image, and the SURF algorithm is used from the plurality of key frames extracted from the comparison target image. Then, the feature points are extracted, the extracted feature points are clustered, and the feature points clustered from the reference image stored in the JSON format are loaded, and the feature points clustered from the reference image and the feature points clustered from the comparison target image are matched to obtain a plurality of It is possible to obtain a feature point pair, obtain a homography from the obtained plurality of feature point pairs, and determine whether the comparison target image is an image illegally distributed from the reference image from the acquired homography. .

In an embodiment, when the acquired homography is nonlinear, the processor may determine that the comparison target image is not illegally distributed from the reference image.

In an embodiment, when the acquired homography is linear, the processor may determine that the comparison target image is illegally distributed from the reference image.

On the other hand, the present invention for solving the above problems can provide a computer program stored in a computer-readable storage medium in order to execute the above-described method for detecting the distribution of illegally photographed images using a computer.

The method for detecting the distribution of illegally photographed images proposed by the present invention extracts a key frame from the reference image and the comparison target image, and determines whether the comparison target image is distributed from the reference image using the clustered feature points. It can detect and delete disseminated illegal footage.

The effects of the present invention are not limited to the above-mentioned effects, and other effects that are not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a flowchart showing the overall flow of a method for detecting distribution of illegally photographed images according to an embodiment of the present invention.

The above-described and additional aspects of the invention are embodied through embodiments described with reference to the accompanying drawings. However, the embodiments described below are merely exemplary, and are not intended to limit the scope of the present invention to the described embodiments. In addition, components of each embodiment may be various combinations within the embodiment or between the embodiments as long as there is no other mention or contradiction between them.

In addition, when a part "includes" a certain component, this means that the component must be included regardless of other components, and the possibility of including other components is not excluded.

In addition, throughout the specification, when a part is said to be "connected" to another part, it is not only "directly connected", but also "electrically connected" with another element interposed therebetween. Include. Furthermore, throughout the specification, a signal means an amount of electricity such as voltage or current.

In addition, the flow charts shown in the drawings are merely an exemplary sequence in order to obtain the most desirable result in carrying out the present invention, and of course, other steps may be added or some steps may be deleted.

Hereinafter, a method of detecting the distribution of illegally photographed images according to an embodiment of the present invention will be described in detail with reference to FIG. 1.

1 is a flowchart showing the overall flow of a method for detecting distribution of illegally photographed images according to an embodiment of the present invention.

Referring to FIG. 1, the method for detecting distribution of an illegally photographed image according to an exemplary embodiment of the present invention may extract a plurality of key frames from a reference image (S100).

Here, the reference video means an illegally photographed video. An illegally photographed image refers to an image in which a part of the body or a specific activity is illegally captured. So-called hidden cameras and revenge porn can be the case.

In addition, the reference image may be an original image as an illegally photographed image, or an image that has already been determined to be illegally distributed from the reference image among the comparison target images.

In this case, the comparison target image may be an image for which it is necessary to confirm whether or not it is illegally distributed from the reference image among illegally photographed images distributed on the Internet.

Meanwhile, the reference image and the comparison target image are composed of a key frame and other frames. Here, the key frame means a frame having full information on an image between the start frame and the last frame. That is, the key frame may be a frame having the largest motion vector among frames of an image.

In the method for detecting distribution of an illegally photographed image according to an embodiment of the present invention, a plurality of key frames of the reference image may be extracted in step S100, and a plurality of key frames of the comparison target image may be extracted in step S400 to be described later.

Next, in the method of detecting the distribution of illegally photographed images according to an embodiment of the present invention, feature points are extracted from a plurality of key frames extracted from a reference image using a speed-up robust feature (SURF) algorithm, and the extracted feature points are clustered. Can be (S200).

The SURF algorithm is an algorithm that extracts feature points from an image, and can be divided into a feature extraction section that extracts feature points by generating an integral image and a feature description section that describes information about images around the feature points.

The reason why the feature extraction part in the SURF algorithm generates an integral image is to quickly perform a box-type filter and convolution operation. On the other hand, in the fast-Hessian matrix of the SURF algorithm, after obtaining the Heisian matrix through convolution operation between the second derivative LoG (Laplacian of Gaussian) box filter approximated on the x, y, and xy axes and the integral image, Compute the determinant value of the matrix. In addition, when the Determinant is greater than the threshold value and the determinant is the largest compared to the adjacent scale through Non-maxima Supperssion, the feature point is determined.

In the feature description part of the SURF algorithm, the dominant direction of the extracted feature point is determined, and a descriptor of the surrounding image is generated.

The SURF algorithm can extract feature points that do not change even with image conversion such as obscuration, noise, reduction, enlargement, and rotation, and can extract feature points at high speed, so whether the comparison target image in the present invention is an illegally distributed image from the reference image. It is a suitable algorithm to detect whether or not.

Meanwhile, since the reference image has a plurality of key frames, the method for detecting the distribution of illegally photographed images according to an embodiment of the present invention uses feature points of a plurality of key frames extracted from the reference image to compare the reference image and the comparison target image. Can be clustered.

In the method of detecting the distribution of illegally photographed images according to an embodiment of the present invention, a weighted KNN (K-Nearest Neighbor) may be used to cluster feature points.

The weighted KNN defines a feature point of an unknown key frame as a label of a training sample having up to three weights for the feature point of an unknown key frame. That is, when three label weights are determined for the feature points of the unknown key frame, the feature points of the unknown key frame are defined to have three label weights based on each determined training sample, and the cluster of training samples is rearranged to create a new one again. The feature points for all key frames can be clustered by redefining the feature points of unknown key frames.

Next, in the method for detecting distribution of illegally photographed images according to an embodiment of the present invention, feature points clustered from the reference image may be stored in the form of JavaScript Object Notation (JSON) (S300).

JSON is one of the formats used to send and receive data, and is a lightweight data exchange format. It is originally derived from the JavaScript language and follows the syntax form of JavaScript, but is a language independent data format. In other words, since it is independent of programming language or platform, it can be used in several programming languages such as C, C++, C#, JAVA, and PYTHON.

JSON is a lightweight data format, and most programming languages provide a library that can handle JSON format data, so the method for detecting the distribution of illegally photographed images according to an embodiment of the present invention captures the features clustered in the reference image. It is a data format suitable for saving and loading.

Next, the method for detecting the distribution of illegally photographed images according to an embodiment of the present invention extracts a plurality of key frames from the comparison target image (S400), and uses a SURF algorithm from a plurality of key frames extracted from the comparison target image. Is extracted, and the extracted feature points may be clustered (S500).

Steps S400 and S500 with respect to the image to be compared go through the same process as steps S100 and S200 described based on the reference image, so if you have ordinary knowledge in the art to which the present invention belongs, steps S100 and S200 are used. Accordingly, a key frame can be extracted from the comparison target image, feature points are extracted from the key frame extracted from the comparison target image using the SURF algorithm, and the extracted feature points can be clustered, so detailed descriptions of steps S400 and S500 are omitted.

Next, in the method for detecting the distribution of illegally photographed images according to an embodiment of the present invention, after calling the feature points clustered from the reference image stored in JSON format (S600), the feature points clustered in the reference image and the comparison target image A plurality of feature point pairs may be obtained by matching the feature points clustered in step S700.

Here, in the method of detecting the distribution of illegally photographed images according to an embodiment of the present invention, a pair of feature points may be obtained using Fast Library For Approximate Nearest Neighbors (FLANN).

FLANN is a library that defines the data structure for the ANN (Approximate Nearest Neighbors) algorithm. FLANN greatly reduces spatial (memory) and time (calculation) complexity when searching for Nearest Neighbors, and uses KD-Tree (KD-Tree is a spatial partitioning data structure for structuring K dimensions, and uses logN in a fast time search). Therefore, fast feature point matching is possible for feature points having multi-dimensional (64-dimensional) descriptors such as the SURF algorithm used in the method for detecting distribution of illegally photographed images according to an embodiment of the present invention.

Next, in the method for detecting distribution of an illegally photographed image according to an embodiment of the present invention, a homography may be obtained from a plurality of pairs of feature points (S800).

Homography is a 2D image conversion relationship model for a 3D planar object, and the present invention can determine whether the comparison target image is an image illegally distributed from the reference image by using homography in step S900 to be described later.

Meanwhile, in an embodiment of the present invention, before step S800, the step of removing an outlier pair from among a plurality of feature point pairs obtained in step S700 using a random sample consensus (RANSAC) algorithm may be performed first. .

The RANSAC algorithm is a method of predicting appropriate model parameters from original data with high measurement noise, and creates homography from this data. Based on this theory, the RANSAC algorithm randomly samples the minimum pair of feature points necessary to determine homography from among the candidates for a homography, and obtains an optimal pair of feature points through an iterative process.

The RANSAC algorithm acquires four sample feature point pairs from all feature point pair candidates, assumes the acquired sample feature point pair as a true value, predicts homography, determines whether the predicted homography is correct, and if not, the process above. Has an order of execution that repeats

That is, in the method of detecting the distribution of illegally photographed images according to an embodiment of the present invention, after approximating a pair of feature points using RANSAC, and removing all outliers from the matched feature points using only the feature point pairs located on the epipola line position, Acquire homography.

Finally, in the method for detecting the distribution of illegally photographed images according to an embodiment of the present invention, it may be determined whether the comparison target image is an image illegally distributed from the reference image from the homography obtained in step S800 (S900).

In this case, if the homography obtained in step S800 is non-linear, it is determined that the image to be compared is not illegally distributed from the reference image, and if the homography obtained in step S800 is linear, the image to be compared is illegal from the reference image. It can be determined that it has been circulated.

Here, when homography is non-linear, it means that abnormal conversion of overturning, warping, or concave occurs, and when homography is linear, it means that normal conversion occurs rather than an abnormal conversion of overturning, warping, or concave. do.

As described above, the present invention extracts a key frame from the reference image and the comparison target image, and determines whether the comparison target image has been distributed from the reference image using the clustered feature points, thereby accurately and quickly obtaining an illegal image. Can be detected and deleted.

Meanwhile, the method for detecting the distribution of illegally photographed images according to an embodiment of the present invention can be written as a program, and codes and code segments constituting the program can be easily inferred by a programmer in the field. In addition, a program relating to a method for detecting the distribution of illegally photographed images may be stored in an information storage medium that can be read by an electronic device such as a computer, and read and executed by an electronic device.

In addition, the method for detecting the distribution of illegally photographed images according to an embodiment of the present invention may be performed by an apparatus for detecting distribution of illegally photographed images including a processor and a memory. In this case, the memory may store instructions executable by the processor, and the processor may perform the above-described method of detecting the distribution of illegally photographed images by executing the instructions.

In addition, the technical features of the method for detecting the distribution of illegally photographed images according to an embodiment of the present invention and the implementation for executing the same are implemented as digital electronic circuits, or computer software including the structures and structural equivalents thereof described in this specification. , Firmware or hardware, or a combination of one or more of them. In addition, the implementation implementing the technical features described in this specification is a computer program product, that is, a module relating to computer program instructions encoded on a tangible program storage medium for execution by or for controlling the operation of a processing system. It can also be implemented.

Meanwhile, in the present specification, the term "system" includes all devices, devices, and machines for processing data, including, for example, a processor, a computer, or multiple processors or computers. The processing system includes, in addition to hardware, all elements forming an execution environment for a computer program, for example, code constituting the processor firmware, a protocol stack, a database management system, an operating system, or a combination of two or more of them. can do. Computer programs known as programs, software, software applications, scripts, or codes can be written in any form of a compiled or interpreted language or a programming language, including a priori or procedural language, and can be written as a standalone program, module, component, or sub-program. The routine may be implemented in various forms, as well as other units suitable for use in a computer environment.

Although the present invention has been described as described above, those of ordinary skill in the art to which the present invention pertains will recognize that the present invention can be implemented in other forms while maintaining the technical spirit and essential features of the present invention. .

The scope of the present invention will be defined by the claims, but not only the configuration directly derived from the description of the claims, but also all changes or modified forms derived from the configuration equivalent thereto are included in the scope of the present invention. Should be interpreted as.

Claims (10)

In the method of detecting whether the comparison target image is an illegally distributed image from the reference image,
(a) extracting a plurality of key frames from a reference image;
(b) extracting feature points from a plurality of key frames extracted from the reference image using a speed-up robust feature (SURF) algorithm, and clustering the extracted feature points;
(c) storing the feature points clustered in the reference image in JSON format;
(d) extracting a plurality of key frames from the comparison target image;
(e) extracting feature points from a plurality of key frames extracted from the comparison target image using a SURF algorithm, and clustering the extracted feature points;
(f) calling feature points clustered from the reference image stored in the JSON format;
(g) obtaining a plurality of pairs of feature points by matching feature points clustered in the reference image and feature points clustered in the comparison target image;
(h) obtaining a homography from the obtained pairs of feature points; And
(i) determining from the acquired homography whether the comparison target image is an image illegally distributed from the reference image; Including,
Step (i),
If the obtained homography is nonlinear, determining that the comparison target image is not illegally distributed from the reference image; And
If the obtained homography is linear, determining that the comparison target image is illegally distributed from the reference image; Containing,
How to detect the spread of illegally photographed video. The method of claim 1,
The key frame,
Characterized in that the motion vector is the largest frame among the frames,
How to detect the spread of illegally photographed video. The method of claim 1,
The step (g),
Characterized in that obtaining a plurality of feature point pairs using FLANN (Fast Library For Approximate Nearest Neighbors),
How to detect the spread of illegally photographed video. The method of claim 1,
Before step (h),
Removing an outlier pair from among the plurality of acquired feature point pairs using a random sample consensus (RANSAC) algorithm; Further comprising,
How to detect the spread of illegally photographed video. delete delete Processor; And
A memory storing instructions executable by the processor; Including,
The processor, by executing the instructions,
Extract a plurality of key frames from the reference image,
A feature point is extracted from a plurality of key frames extracted from the reference image using a speed-up robust feature (SURF) algorithm, and the extracted feature points are clustered,
The feature points clustered in the reference image are stored in JSON format,
Extracting a plurality of key frames from the image to be compared,
A feature point is extracted using a SURF algorithm from a plurality of key frames extracted from the comparison target image, and the extracted feature points are clustered,
Load the feature points clustered from the reference image stored in the JSON format,
A plurality of feature point pairs are obtained by matching the feature points clustered in the reference image and the feature points clustered in the comparison target image,
Acquiring homography from the obtained pairs of feature points,
It is determined from the obtained homography whether the comparison target image is an image illegally distributed from the reference image,
If the obtained homography is nonlinear, it is determined that the comparison target image is not illegally distributed from the reference image,
If the obtained homography is linear, determining that the comparison target image is illegally distributed from the reference image,
A device for detecting the spread of illegally photographed video. delete delete A computer program stored in a computer-readable storage medium for executing the method of any one of claims 1 to 4 using a computer.

Images