KR101437286B1 - Method and apparatus for identifying digital contents - Google Patents

Abstract

The present invention relates to a method and an apparatus for identifying digital content, capable of identifying the digital content using structural features and non-structural features of the digital content. The method for identifying the digital content according to one embodiment of the present invention includes the steps of: maintaining a database storing the structural features and non-structural features of reference digital content; extracting the structural features including a line segment of a frame in target digital content, which is formed with a frame structure; extracting the non-structural features of the target digital content; and comparing the extracted structural features and non-structural features with the structural features and non-structural features of the reference digital content stored in the database.

Description

[0001] METHOD AND APPARATUS FOR IDENTIFYING DIGITAL CONTENTS [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a digital content identification method and apparatus, and more particularly, to a digital content identification method and apparatus for identifying contents using structural and non-structural features of digital contents.

Digital content identification technology can be used as a technology to protect copyrights on publications, particularly content that is illegally scanned and distributed on the Web.

One technique for identifying digital content containing images is to use an optical character reader (OCR).

A method of identifying contents using OCR is a technique of extracting text existing in digital contents and then recognizing the similarity by determining the similarity degree of an index word or the like. As a method of identifying contents using OCR, it is impossible to practically use OCR processing time of about 4 ~ 5 seconds per page and recognition accuracy of about 40 ~ 60% on average.

For example, it takes 5 to 10 minutes to convert a comic book composed of 100 pages into text using OCR software.

Korean Patent Laid-Open No. 10-2011-0060450 discloses a digital contents identification system web service apparatus and method for correcting information that has been changed efficiently when a DB structure is changed or a business process is changed.

However, Korean Patent Laid-Open No. 10-2011-0060450 does not disclose a digital content identification technique for protecting the copyright of illegally distributed content.

Therefore, it is necessary to study digital contents identification technology for detecting illegal scan digital contents accurately and quickly.

It is an object of the present invention to provide a method and apparatus for accurately and quickly identifying digital contents using structural and non-structural features of digital contents.

According to an embodiment of the present invention, there is provided a digital content management method comprising: maintaining a database in which structural and non-structural features are stored in a reference digital content; Extracting a structural feature including a linear component of a frame from an object digital content having a frame structure; Extracting non-structural features of the digital content; And comparing the extracted structural and non-structural features with structural and non-structural features of the reference digital content stored in the database.

According to an aspect of the present invention, there is provided an information processing apparatus including an input unit for receiving an input of digital content; A database in which structural and non-structural features are stored in the reference digital content; A first feature extracting unit for extracting a structural feature including a linear component of a frame in an object digital content having a frame structure; A second feature extraction unit for extracting non-structural features of the target digital content; A comparison unit comparing the extracted structural and non-structural features with structural and non-structural features of the reference digital content stored in the database; And a control unit for controlling the input unit, the database, the first feature extraction unit, the second feature extraction unit, and the comparison unit.

The digital content identification method and apparatus according to an embodiment of the present invention can accurately and quickly identify digital contents using structural features and non-structural features of digital contents.

1 is a block diagram of a digital content identification apparatus according to an embodiment of the present invention.
2 is a flowchart illustrating a digital content identification method in accordance with an embodiment of the present invention.
3 is a diagram illustrating an example of a preprocessing process of the digital content identification method shown in FIG.
FIG. 4 is a diagram illustrating an example of a process of extracting structural features of digital contents in the digital content identification method shown in FIG. 2. Referring to FIG.
FIG. 5 shows a linear component extracted through the process of FIG.

Hereinafter, a digital content identification method and apparatus according to an embodiment of the present invention will be described with reference to the drawings.

As used herein, the singular forms "a", "an" and "the" include plural referents unless the context clearly dictates otherwise. In this specification, the terms "comprising ", or" comprising "and the like should not be construed as necessarily including the various elements or steps described in the specification, Or may be further comprised of additional components or steps.

In this specification, digital content refers to content that can be produced, distributed and consumed in the form of electronic files in contrast with offline contents. For example, the digital content may include a scanned copy of an off-line publication.

The digital contents according to an embodiment of the present invention may include digital contents having a frame structure.

In this specification, the reference digital content refers to content used to identify which content is digital content (hereinafter referred to as 'target digital content') to be identified. For example, the original content may be the reference digital content, and the content circulated on the web may be the target digital content.

Hereinafter, an example of digital contents will be described as a scan comic book which is a scan of a comic book as an offline publication, but the present invention is not limited thereto.

1 is a block diagram of a digital content identification apparatus according to an embodiment of the present invention.

2, the digital content identification apparatus 100 includes an input unit 110, a database 120, a preprocessor 130, a first feature extraction unit 140, a second feature extraction unit 150, 160, a candidate extracting unit 170, and a control unit 180.

The input unit 110 may receive input of digital contents. The digital content input and reception may be performed through a user interface.

The database 120 may store structural and non-structural features for the reference digital content. The structural and non-structural features of the reference digital content will be described later.

The preprocessing unit 130 performs a preprocessing operation to more accurately identify the digital contents to be processed. For example, the preprocessing unit 130 performs image processing on scans in which geometric / image degradation has occurred through processes such as rotation correction, noise removal, brightness adjustment, page division, and size generalization to identify scan comics can do.

The first feature extraction unit 140 may extract the structural features of the digital content. The structural characteristic of the digital content may be a coarse level fingerprint for content identification. The structural characteristic of the digital contents may include a structural characteristic of the frame constituting the digital contents.

The second feature extraction unit 150 can extract the non-structural features of the digital content. The non-structural feature of the digital content may be a fine level fingerprint for content identification. The unstructured characteristic of the digital content may include specific characteristics of the image existing in the frame of the digital content.

The comparing unit 160 compares the structural feature extracted by the first feature extracting unit 140 and the unstructured feature extracted from the second feature extracting unit 150 with the structural data of the reference digital content stored in the database 120 The similarity can be searched by comparing features and non-structural features.

The candidate extracting unit 170 may extract the candidate content among the reference digital contents stored in the database 120 using the structural features extracted from the first feature extracting unit 140. [ For example, the candidate extracting unit 170 may extract the reference digital content having the similarity within the specific threshold range with the structural feature extracted from the first feature extracting unit 140, and determine the candidate digital content as the candidate content. The candidate content extraction may be performed to reduce the range of the reference content to be compared after the fingerprint check of the coarse level in order to reduce the calculation amount for digital content identification.

The control unit 180 may include the input unit 110, the database 120, the preprocessor 130, the first feature extraction unit 140, the second feature extraction unit 150, the comparison unit 160, And the functions performed by the candidate extractor 170 can be organically controlled.

2 is a flowchart illustrating a digital content identification method in accordance with an embodiment of the present invention.

The control unit 180 may maintain the database 120 in which structural and non-structural features of the reference digital content are stored (S210). The structural characteristic of a digital content may include a structural characteristic of a frame constituting digital contents. For example, the structural feature may comprise a linear component of the frame. The straight line component may include a number of straight lines forming the frame, a length of a straight line forming the frame, and a direction of a straight line forming the frame.

In addition, the unstructured characteristic of the digital content may include specific characteristics of the image existing in the frame of the digital content. For example, the non-structural feature of the digital content may include a descriptor including feature points (e.g., coordinate information) of the image and feature information about the feature points.

In the following embodiments, a fingerprint using a feature based on a scan comic book structure is defined as a coarse level fingerprint, and a fingerprint using a more accurate and specific image characteristic is defined as a fine-level fingerprint separately from the comic book structure do.

The preprocessing unit 130 may perform a preprocessing operation to facilitate identification of the inputted digital content (S220). The preprocessing unit 130 performs a process of identifying the scan comics in the target digital contents (for example, illegal scanned images) in which the geometric / image degradation has occurred through processes such as rotation correction, noise removal, luminance adjustment, page division, size generalization, It is possible to perform image processing on the image.

3 is a diagram illustrating an example of a preprocessing process of the digital content identification method shown in FIG.

As shown, in the case of a comic book scan, geometric / image degradation may occur due to illegal scanning operations that are not formalized. In the preprocessing process according to an exemplary embodiment of the present invention, image processing may be performed to geometrically correct an image rotated clockwise or counterclockwise. In the preprocessing process, the noise and the degree of contrast deteriorate due to the nature of the scan pattern. In preparation for the feature extraction (i.e., extraction of the fingerprint), the noise is removed, And a process of improving the contrast based on the histogram of the area and the dark area.

In addition, in the case of the target digital contents (for example, illegal scanned images), there may be a case where a single image includes a comic book page on both sides rather than a comic book page on one side. Therefore, in the preprocessing process, in order to minimize errors in the fingerprint extraction, page separation can be performed when there are pages on both sides. The separated pages are subjected to a size generalizing process to adjust the size of the horizontal and vertical images.

In the digital content identification method according to an embodiment of the present invention, feature extraction may be performed step by step for effective content identification.

First, the first feature extraction unit 140 may extract a structural feature of a scan comic book having a frame structure (S230).

In order to extract the fingerprint of the coarse level, the frame structure of a comic book made up of a straight line can be utilized in an embodiment of the present invention. Comic books are divided into several rectangular frames within a page, and each frame is composed of a horizontal and vertical straight line. It may also include straight lines in various directions rather than only linear components of 0 and 90.

 However, in order to accurately extract such a frame by a square, there is a limitation of image processing in spite of effective preprocessing. Therefore, in this embodiment, a straight line component of a frame is extracted as a coarse-level fingerprint.

FIG. 4 is a diagram illustrating an example of a process of extracting structural features of digital contents in the digital content identification method shown in FIG. 2. Referring to FIG.

4 (a) shows the original image of the inputted target scan comic book. 4 (b) shows an image segmentation. Segmentation is performed on the original image to distinguish the frame from the background area. In this case, the segmentation technique can be used for region growing, k-mean clustering and thresholding.

Edge detection can be performed on a frame divided into segments (Fig. 4 (c)). In this way, the edge component of the frame can be extracted, and the edge component can be further improved by performing morphological filtering (FIG. 4 (d)) for more accurate linear component detection from the edge information. Linear component extraction from the edge can be done by Hough transform. From this, corner and straight line components in each frame can be obtained (Fig. 4 (e)).

FIG. 5 shows a linear component extracted through the process of FIG. The straight line component may include the number of straight lines forming the frame, the length of the straight line forming the frame, the direction of the straight line forming the frame, and the like.

As shown, the extracted linear components are divided according to each direction. Due to the nature of the comic book frame structure, rectangles other than rectangles or other types of frames exist, so that the direction can be considered as a forward direction.

 These straight lines are labeled according to the direction, and the fingerprints accordingly are as follows.

1. Number of labels in the corresponding direction (number of straight lines - 0 degrees in FIG. 5: 10, 90 degrees: 10)

2. The length of the straight line corresponding to each label

The linear component may be used as a coarse-level fingerprint for similarity search between the target scan comic book and the stored reference scan comic book.

If the extracted fingerprint information is the same as the fingerprint information of the reference scan comic book stored in the database 120, the target scan comic book may be recognized as a scanned copy (or a copy) of the stored reference scan comic book.

According to an embodiment of the present invention, an error may occur in an image processing process, so that an appropriate threshold is set for each fingerprint, thereby increasing the efficiency and accuracy of the similarity search.

After extracting the coarse-level fingerprint, the candidate extracting unit 170 may extract a candidate comic book stored in the database 120 using the result of the similarity search (S240). That is, the candidate extracting unit 170 extracts a reference comic book having similarity to the extracted linear component and within a predetermined threshold range, and can determine a candidate comic book (hereinafter referred to as a candidate comic book).

Coarse level fingerprints contain rough information that excludes details in comics. Therefore, candidate groups having similar frame structures can be detected according to the complexity of the frame structure.

This is because the fingerprints of each fingerprint are relatively insufficient to improve the speed at the coarse level. In this case, accurate matching candidates should be extracted again among the candidates.

For this, Fine level fingerprint detection and matching can be performed after the coarse level. That is, when the coarse-level fingerprint extraction and the candidate extraction process are performed, the second feature extraction unit 150 can perform fine-level fingerprint extraction for extracting non-structural features of the target scan comic book (S250). That is, in this embodiment, the unstructured characteristic of the target scan comic book can be a fine-level fingerprint.

Fine level fingerprints extract unstructured features, whereas coarse level fingerprints extract structural features of comic book pages. Fine level fingerprints contain more unique features of each comic book page. A fine level fingerprint can be obtained from the picture information in addition to the frame of the comic book. Therefore, a fine level fingerprint requires a larger number of bits and a lot of time is spent in search and similarity check. In this embodiment, a fine level test is performed on candidates narrowed after the inspection of the coarse level to reduce the overall calculation amount.

Fine level fingerprints can consist of feature points and descriptors. Feature points represent coordinate information in an image, and a vector is a descriptor that contains a unique feature of the feature point. When the similarity degree is practically examined, the similarity degree of the descriptor of each point is checked.

Feature Point detection is determined to be stronger in the corner part than the frame part. In order to shorten the execution time and ensure accuracy, the number of feature points can be determined according to the number of candidate groups obtained by the similarity search result by the coarse fingerprint. That is, as the number of candidate groups is smaller, a smaller number of characteristic points are extracted, and as the number of candidate groups is larger, a larger number of characteristic points can be extracted.

Next, the second feature extracting unit 150 can generate a descriptor that can explain the feature of the feature point. The descriptor may include the size of the frame including the feature point, and distribution information of neighboring pixels adjacent to the predetermined region based on the feature point. In this embodiment, the coordinates of the feature point are utilized as the main information of the descriptor. This is because there is no point in matching the feature points of completely different positions in the problem of judging the same comic responsibility.

The comparing unit 160 may perform the similarity search by comparing feature points and descriptors with respect to feature points and descriptors of the target scan comic book and candidates stored in the database 120 (S260).

Finally, the control unit 180 can identify whether the target scan comic book (i.e., test comic book) is the same as the comic book stored in the database 120 (S270). That is, the control unit 180 checks whether the fingerprint identical or similar to the fine level fingerprint extracted from the target scan comic book is stored in the database 120, so that the target scan comic is scanned with the reference comic book stored in the database 120 It is possible to judge whether it is a copy (or a copy).

Fine level fingerprints can be altered due to the contrast, sharpness, and brightness of scanned comic book images. Therefore, according to the embodiment of the present invention, after the structural similarity search through the fingerprint of the coarse level is preceded, the similarity search of the non-structural feature is performed through the fine level fingerprint, so that more accurate and robustness similarity search is performed .

The above-described digital content identification method may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable recording medium. At this time, the computer-readable recording medium may include program commands, data files, data structures, and the like, alone or in combination. On the other hand, the program instructions recorded on the recording medium may be those specially designed and configured for the present invention or may be available to those skilled in the art of computer software.

The computer-readable recording medium includes a magnetic recording medium such as a magnetic medium such as a hard disk, a floppy disk and a magnetic tape, an optical medium such as a CD-ROM and a DVD, a magnetic disk such as a floppy disk, A magneto-optical media, and a hardware device specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like.

The recording medium may be a transmission medium, such as a light or metal line, a wave guide, or the like, including a carrier wave for transmitting a signal designating a program command, a data structure, and the like.

The program instructions also include machine language code, such as those generated by the compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

The above-described digital contents identification method and apparatus are not limited to the configuration and method of the above-described embodiments, but the embodiments may be modified such that all or some of the embodiments are selectively And may be configured in combination.

100: digital content identification device
110: input unit
120: Database
130:
140: First feature extracting unit
150: second feature extracting unit
160:
170: candidate extracting unit
180:

Claims (14)

Maintaining a database in which structural and non-structural features are stored in the reference digital content;
Extracting a structural feature including a linear component of a frame in an object digital content composed of a frame structure;
Extracting candidate contents to be compared with unstructured features of the reference digital contents stored in the database using the extracted structural features;
Determining a number of unstructured features based on the number of extracted candidate contents and extracting the unstructured features of the target digital content according to the determined number of unstructured features, - the more the number of contents, the more; And
Comparing the extracted non-structural features of the target digital content with non-structural features of the candidate content. The method of claim 1,
A number of straight lines forming the frame, a length of a straight line forming the frame, and a direction of a straight line forming the frame. 2. The method of claim 1, wherein the non-
And a descriptor including feature points including specific coordinate information of an image existing in a frame of the target digital content and feature information about the feature point. 4. The method of claim 3, wherein the technician
Wherein the digital content identification information includes at least one of size information of a frame including the feature point and information about neighboring pixels within a predetermined region based on the feature point. delete delete The method of claim 1, wherein the reference digital content and the target digital content
And a scanned version of the offline publication scanned. An input unit for receiving input of digital contents;
A database in which structural and non-structural features are stored in the reference digital content;
A first feature extracting unit for extracting a structural feature including a linear component of a frame in an object digital content composed of a frame structure;
A candidate extractor for extracting candidate contents to be compared with unstructured features of the reference digital contents stored in the database using the extracted structural features;
A second feature extraction unit for determining the number of non-structural features based on the number of extracted candidate contents and extracting non-structural features of the target digital content according to the determined number of non-structural features; The number of candidate contents increases as the number of candidate contents increases;
A comparing unit comparing non-structural features of the extracted target digital contents with non-structural features of the candidate contents; And
And a control unit for controlling the input unit, the database, the first feature extraction unit, the second feature extraction unit, and the comparison unit. 9. The method of claim 8,
A number of straight lines forming the frame, a length of a straight line forming the frame, and a direction of a straight line forming the frame. 9. The method of claim 8, wherein the non-
And a descriptor including feature points including specific coordinate information of an image existing in a frame of the target digital content and feature information about the feature point. 11. The method of claim 10, wherein the technician
A size information of a frame including the feature point, and information about neighboring pixels within a predetermined region based on the feature point. delete delete The method of claim 8, wherein the reference digital content and the target digital content
And a scanned version of the offline publication scanned.

Images