KR101998593B1 - System and method for identifying online comics based on region of interest - Google Patents

Abstract

The present invention relates to a system and method for identifying an on-line cartoon based on a region of interest, and more particularly, to a system and method for identifying a region of interest on an on- And an online cartoon identification system based on the extracted region of interest and an on-line cartoon identification system based on the region of interest and a method thereof.

Description

[0001] The present invention relates to an online cartoon identification system and method,

The present invention relates to a system and method for identifying an on-line cartoon based on a region of interest, and more particularly, to a system and method for identifying a region of interest on an on- And an online cartoon identification system based on the extracted region of interest and an on-line cartoon identification system based on the region of interest and a method thereof.

Currently, content distribution and sharing functions via online are enabled, and copyright holders who produce and distribute contents upload and distribute their contents through various portal sites, Accordingly, the user can easily acquire a variety of content contents on-line.

However, since the contents can be easily acquired online, it is easier to reproduce the content online. As a result, the content displayed through the portal site is downloaded or captured, thereby generating duplicate content and distributing it illegally through online The frequency of illegal cases is increasing.

As a result, the rights of copyright holders are not only infringed, but also the users who provide legitimate fees for the work and subscribe to the content are also damaged.

Of these contents, online cartoon-related contents such as web toon are one of the most popular contents, and the most illegal copy contents are present. Therefore, in order to prevent pirated contents related to such online cartoon Techniques for identifying and preventing pirated content by applying various technologies are emerging.

For example, a method has been proposed in which all the minutiae of the individual images constituting the illegal copy content duplicated on-line cartoon content are extracted and compared in order. However, in order to apply such a scheme, It is necessary to identify the area corresponding to the space area where the minutiae extracted from the illegal copy content is located.

However, since the pirate replaces the on-line cartoon content of a certain number of times with an arbitrary variety of sizes or clips a part thereof to generate pirated contents, when the above-mentioned scheme is applied, the space area corresponding to the minutiae extracted from the on- It is difficult to match all the minutiae points because it is difficult to specify the minutiae in the contents. In addition to this, it is necessary to extract a large number of minutiae points data for identification of one image, which greatly increases the amount of processing, Is required. Such a scheme is not only inefficient, but also has a problem of guaranteeing performance.

In addition, when the server for publishing the online comic contents on the web site divides the original image constituting the on-line comic contents into a plurality of images and manages them, a specific scene is divided into a plurality of images In this case, even if the minutiae are extracted for a specific scene divided into a plurality of images constituting the original image, the minutiae are extracted from the single original image and a comparison with the illegally copied image is performed A difficult problem arises.

At this time, the specific scene may be defined as a single cut, a single frame, a single panel, or a single frame.

In recent years, some scenes (or cuts) of the original image are cut out in the same manner as capturing and used for various purposes such as the emoticon use of the dialogue window and the prevention of the squeezing of the cutout image. A user who shows an interest in a scene image uses a search tool based on various image search to search for contents including the scene image. However, such a scene image may also be produced in various sizes, so that the feature points are compared with the original image Detection is not easy.

Accordingly, it is possible to easily identify the content of the online comics reproduced on-line, thereby preventing the distribution of the pirated content in advance, and at the same time, providing information about the content related to the specific scene image of interest There is a need for an alternative that can enhance the user's search convenience.

Korean Patent Publication No. 10-2005-0026769

The present invention identifies a region of interest composed of one meaningful scene (or cut) in an image, extracts and normalizes the region of interest, and then extracts feature points through HOG-based image analysis, By supporting the feature points to be compared according to the interest areas of the original images constituting the content, it is possible to greatly reduce the amount of computation compared to the existing feature point extraction method, thereby improving the processing speed for the content identification corresponding to the identification target image, And to increase the accuracy of identification of the content associated with the image to be identified, which is produced in various sizes

The interest area-based on-line cartoon identification system according to an embodiment of the present invention includes an interest area extraction unit that extracts a region of interest through boundary detection in at least one original image constituting online cartoon content, An image analyzer for dividing the normalized image into a plurality of blocks having a predetermined size and calculating one or more governing slopes for each block based on Histogram of Oriented Gradient (HOG) A feature point extracting unit for generating feature point information of N-bit values obtained by binarizing the at least one governing slope for each block constituting the ROI according to a preset reference, corresponding to the ROI; And an image corresponding to the original image And the feature point information corresponding to the identification object image is generated by controlling the ROI extraction unit, the normalization unit, the image analysis unit, and the feature point extraction unit when receiving the identification target image, And a controller that compares the reference minutia information with the reference minutia information and provides content information matched with mutually matching reference minutia information in association with the identification target image.

As an example related to the present invention, the content information may include a rotation information for a specific rotation of the on-line cartoon content.

In one embodiment of the present invention, the ROI extracting unit detects the boundary line by scanning the original image in the horizontal and vertical directions, and when the source images constituting the online comic contents are plural, the source images are sequentially connected And generating a single image and then detecting the boundary line.

In one embodiment of the present invention, the ROI extracting unit may extract the ROI from the original ROI image if the ROI exists in the ROI, And the extracted region of interest is connected to the specific region of interest and is set as one region of interest and extracted.

According to an embodiment of the present invention, the feature point extracting unit may divide a unit block into a plurality of cells of a predetermined number, calculate a magnitude of a directional gradient by collecting a slope calculated for each unit cell based on HOG, At least one dominant gradient having a magnitude equal to or larger than the magnitude of the unit block is calculated corresponding to the unit block.

In one embodiment of the present invention, the feature point extracting unit may determine whether or not a ruling slope exists for each of a plurality of predetermined angular ranges according to the directions of the at least one ruling slopes, and determine whether or not the ruling slope exists for each of the plurality of angle ranges And then the binarization process is performed.

According to an embodiment of the present invention, the control unit may further include a storage unit for storing the reference minutia information and the content information in a matching manner under the control of the control unit.

In one embodiment of the present invention, the control unit identifies content information according to reference minutia information matching the minutia information of each of the plurality of ROIs when the target image includes a plurality of ROIs, When the plurality of different contents information are identified in correspondence with the region of interest, the contents information most corresponding to the identified object image is provided as the contents information corresponding to the identification object image.

The region of interest of the server including the DB in which the reference minutia information about the minutiae of each region of interest of the original image corresponding to each of the plurality of different online comic contents according to the embodiment of the present invention is matched with the different minutia information, An on-line cartoon identification method includes: receiving an image to be identified; extracting a region of interest through boundary detection of the image to be identified; generating a normalized image obtained by normalizing the region of interest to a predetermined normalization size; Dividing the normalized image into a plurality of blocks having a preset size, calculating at least one dominant slope for each block on the basis of a Histogram of Oriented Gradient (HOG), calculating the at least one dominant slope for each block constituting the ROI The N-bit binarization process is performed by binarizing the gradient according to a preset reference Comparing the minutia information with the reference minutia information to generate minutia information corresponding to the minutia information when the minutia information corresponding to the minutia information is present, And determining the identification target image as a content corresponding to the content information.

The present invention identifies a region of interest composed of one meaningful scene (or cut) in an image, extracts and normalizes the region of interest, and then extracts feature points for the region of interest through HOG-based image analysis, It is possible to compare minutiae points of interest between the image and the identification target image, thereby making it possible to greatly improve the processing speed in the process of identifying the on-line cartoon content related to the identification target image by comparing the original image with the scene unit excluding the unnecessary background It is possible to accurately compare the minutiae points of the original image and the identification target image because the minutiae point related characteristics based on the HOG are not changed and compared with each other in the normalized region of interest even when the identification target image is configured in various sizes, Associated Online Only It has the effect of greatly improving the identification accuracy of the content.

In addition, in the process of identifying the on-line cartoon contents associated with the image to be identified, the feature point can be calculated by binarizing the governing slope extracted through the HOG-based image analysis algorithm, Not only the number of feature points extracted corresponding to one image can be reduced as compared with the conventional feature point calculation method but also the feature point comparison is performed in units of scenes to improve the identification accuracy while minimizing the influence of changes in image quality and resolution.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram of an on-line cartoon identification system based on a region of interest according to an embodiment of the present invention; FIG.
FIG. 2 through FIG. 4 illustrate a process of extracting a ROI of an ROI based on a ROI according to an embodiment of the present invention; FIG.
FIG. 5 to FIG. 7 are diagrams illustrating a feature point extraction process for a ROI-based ROI of an ROI-based on-line cartoon recognition system according to an embodiment of the present invention;
FIG. 8 to FIG. 9 illustrate an operation example of content identification associated with an identification target image of an online cartoon identification system based on a ROI according to an embodiment of the present invention; FIG.
FIG. 10 is a table for comparison of performance between an on-line cartoon identification system based on a ROI and an existing feature point extraction method according to an embodiment of the present invention.
11 is a flowchart of a method for identifying an online cartoon based on a region of interest according to an embodiment of the present invention.

It extracts feature points of each successive image from a plurality of images constituting an illegal scan cartoon in the form of an image file generated by illegal scanning of a comic book composed of conventional paper, and sequentially compares the feature points extracted from each image constituting the original cartoon In the case of technology that identifies whether illegal copying is performed according to whether or not it is coincidental, it is easy to compare scan comics and feature points extracted from original cartoons, This makes it easy to identify illegal scanned comics.

However, in the case of online cartoon contents such as a web toon which is produced in the form of an image without passing through a paper and is directly displayed online, it is constituted of a single image of a unit of rotation without page breaks, And may be configured differently for each rotation.

In addition, the unauthorized copying of the on-line cartoon contents may capture a single image (original image) of each of the images, divide the image into a plurality of sizes to generate a plurality of image files, To create illegal copy content and distribute it without permission.

Therefore, unlike scanned comics, which are composed of original images constituting online comic contents and replica images constituting counterfeit clips, as well as page units, which are easy to identify, It is impossible to distinguish between duplicate images, because it is composed of a form divided into a plurality of images or a part of images belonging to different sequences. Thus, one-to-one matching through extracting feature points between the original image and duplicate images is impossible It is difficult to apply the detection method used for identification of illegal scan comics to detect illegal copy of online cartoon contents.

Recently, some scenes (or cuts) of an original image are cut out in the same manner as capturing and used for various purposes such as use of emoticons in a dialogue window or prevention of squeezing of a postage, and a scene image However, such a scene image is also produced in various sizes and it is not easy to detect it by comparing feature points with the original image.

According to the patent application No. 10-2016-0152738 previously filed by the same applicant, minutiae points are extracted based on the frequency characteristics of the image for the whole area of the original image constituting the online comic contents, and based on this, And the on-line cartoon contents are compared with each other, the position and the distribution of the minutiae do not change due to the same frequency characteristics of the illegal copy content and the online comic content due to the pixel change, The present invention proposes a configuration that accurately identifies feature points of pirated contents to increase the accuracy of identification of illegal copy contents. However, the present invention can further shorten the identification processing time of illegal contents of the existing registered patent, Retract content Some cut scenes image scenes from the original image to be too easily identified by presenting a plan to ensure the comfort of the user's search to search for content based on the image.

Prior to the description, the term 'scene' as described in the present invention may be defined as a single cut, a single cell, a single panel or a single frame, May mean a comic image or an image area comprising a single cut, a single frame, a single panel or a single frame. Also, the original image may be composed of one or more cuts, squares, panels, or frames.

At this time, a plurality of cuts, a plurality of cells, a plurality of panels, or a plurality of frames may be constituted by one region of interest according to the arrangement of the cuts or cells constituting the original image.

Hereinafter, detailed embodiments of the present invention will be described with reference to the drawings.

1 is a block diagram of a system for identifying an on-line cartoon based on a ROI according to an embodiment of the present invention. As shown in FIG. 1, the ROI extraction unit 110, the normalization unit 120, the image analysis unit 130, An extraction unit 140 and a control unit 150. [

It should be noted that at least one of the components constituting the on-line comic book identification system may be included in the other components, and in addition to the components described above, the on-line comic book identification system may further comprise additional components .

In addition, each constituent unit constituting the on-line comic book identification system may communicate with each other by configuring at least a part of the components of the on-line comic book identification system in each of a plurality of different apparatuses that are configured in the server or mutually communicable.

For example, the interest region extracting unit 110, the normalizing unit 120, the image analyzing unit 130, and the feature point extracting unit 140 are configured in the server, and the control unit 150 controls the terminal , A user terminal).

In addition, the on-line cartoon identification system may be composed of application-related data executed in a server or a user terminal.

Hereinafter, the detailed operation configuration of each constituent unit will be described in detail with reference to the configuration of FIG. 1 through the following drawings taking the online comic book identification system configured in the server as an example.

First, as shown in FIGS. 2 and 3, the ROI extracting unit 110 may extract one or more original images constituting the online comic contents for each rotation.

At this time, the server may include a content DB 101 matching one or more original images and content information stored in the content matching the online cartoon (or online cartoon content) related content information, And one or more original images may be stored in the content DB 101. FIG.

The control unit 150 may extract an original image matched with specific content information from the content DB 101 and provide the extracted original image to the ROI extraction unit 110. The control unit 150 may extract Matching with the specific content information extracted together with the original image when receiving the minutia information, and storing the matched information in the minutia DB 102 included in the server.

Also, the content information may include content identification information corresponding to on-line cartoon content, and rotation information on any one of a plurality of rotation angles constituting the on-line cartoon content, and the rotation information may include a plurality of rotation numbers Image identification information corresponding to (or matched with) any one of the original images.

For example, content information corresponding to each of a plurality of different time series corresponding to a specific online cartoon content can be stored in advance in the content DB 101, and a plurality of differently differentiated content The information may include mutually different pieces of discrepancy information including mutually identical pieces of content identification information.

In addition, the date of creation or the date of publication may be different from each other between the original images of different times, and information on the date may be included in the time difference information.

In addition, the content DB 101 may store and store the content information and the at least one original image for each of a plurality of different online cartoons (or for each of a plurality of different online comic contents).

Meanwhile, as shown in FIG. 4, the ROI extracting unit 110 may detect the boundary (or edge) by scanning the original image in the horizontal direction and the vertical direction when receiving the original image, (Or set) an ROI (Region Of Interest) for one meaningful scene (or cut or frame) in the original image according to the boundary detection.

As shown in the figure, the ROI extracting unit 110 may exclude the region of interest when the upper or lower end of the ROI is located at the upper or lower end of the original image, A second original image of the next or previous sequence is extracted from the content DB 101 and then successively scanned subsequent to the first original image to form a first region of interest that is in contact with the upper or lower end of the first original image, The second region of interest which is in contact with the upper or lower end of the image and which follows the first region of interest may be interconnected and selected (or set) as one region of interest.

3, when the plurality of original images constituting the online comic contents of the same number of times are plural as shown in FIG. 3, the interest area extracting unit 110 sequentially connects the plurality of original images according to the identification information of the original images, It can be created as a single image.

In this case, the identification information of the original image may be a file name of the original image.

As shown in FIG. 4, the ROI extracting unit 110 may detect the boundary line by scanning the single image in the horizontal direction and the vertical direction. The ROI extracting unit 110 may extract a single area, which is closed by the boundary line, Area can be selected.

At this time, the region of interest may include a single scene (or a single cut or a single frame) included in the original image, or a cartoon component such as a metabolism, a fingerprint, a simple word, and the like.

In addition, the ROI extracting unit 110 extracts an image corresponding to the ROI from the original image each time the ROI is detected in the original image, and provides (or transmits) the ROI-related image to the normalization unit 120, can do.

On the other hand, the on-line cartoon identification system extracts a feature point through HOG (Histogram of Oriented Gradient) based image analysis for the ROI, and uses the feature point as an identification means for on-line cartoon content. Hereinafter, the present invention will be described in detail with reference to the drawings.

5, the normalization unit 120 receives the ROI related image provided from the ROI extraction unit 110, and normalizes the ROI-related image with a predetermined normalization size, Images can be generated.

For example, the normalization unit 120 may normalize the original image according to a predetermined normalization size or a normalization ratio. For example, the normalization unit 120 may adjust the width of the original image to a predetermined number of pixels, The number of vertical pixels of the original image can be automatically adjusted according to the adjustment ratio of the width.

The normalization unit 120 provides (or transmits) the normalized image to the image analysis unit 130 and the image analysis unit 130 converts the normalized image into a predetermined size (or a first size) (Divided) into a plurality of setting areas.

In addition, the image analyzer 130 divides (divides) a plurality of blocks of a predetermined size (or a second size) for each unit setting area A, and generates a histogram of an Oriented Gradient ) Based image analysis can yield one or more dominant gradients.

6, the image analysis unit 130 divides (divides) a unit block into a plurality of cells having a predetermined size (or a third size), and divides (divides) The slope can be calculated for each cell.

In this case, the unit block may mean any one of the plurality of blocks.

In addition, the image analyzer 130 may calculate the magnitude of the gradient of each specific unit block by collecting the slope calculated for each cell belonging to the specific unit block. That is, the image analyzer 130 may calculate a plurality of slopes calculated for each cell belonging to a specific unit block, and calculate a slope-by-slope (or slope-dependent) size.

For example, the image analyzer 130 may divide a gradient magnitude by a predetermined gradient angle, which is divided into a predetermined angle interval according to a slope of each cell belonging to the specific unit block according to the HOG algorithm, ), And one or more dominant slopes having sizes equal to or larger than a predetermined reference size in the histogram can be calculated (or extracted) corresponding to the specific unit block.

At this time, the image analyzer 130 may calculate a histogram according to nine different tilt directions when different tilt directions are set at angular intervals of 40 degrees.

7, the feature point extracting unit 140 extracts a plurality of feature points corresponding to a predetermined unit block in association with the direction of each of the at least one governing slopes calculated in association with the specific unit block, (Or angles) of the gradient slope, and it is possible to binarize the gradient slopes according to the plurality of different angle ranges (or angles).

For example, a plurality of different angular ranges may be preset on the basis of a 40-degree angle (or an interval) in the feature point extracting unit 140, and a plurality of bit values for each angle range may constitute minutia point- Can be set in advance. Accordingly, when the angle range is divided by the angle of 40 degrees, the feature point extracting unit 140 can assign 9 bits to the feature point related data for one block.

The feature point extracting unit 140 may set the angular range in which the dominant inclination is present to 1 and the angular range in which the dominant inclination does not exist to 0 in accordance with whether the dominant inclination exists or not, Bit data of a predetermined unit block, and extract (or calculate) the data as minutiae of the specific unit block.

That is, as shown in the figure, the feature point extracting unit 140 sets the feature point extracting unit 140 to 1 because the feature point extracting unit 140 has a dominant inclination in an angular range exceeding 0 degrees and 40 degrees or less, which is set in advance for the block 3, Since there is no dominant slope in a preset angle range, it can be set to zero.

Accordingly, the feature point extracting unit 140 performs a binary operation according to the presence or absence of a dominant gradient for each of nine different angular ranges obtained by dividing 360 degrees at intervals of 40 degrees, thereby obtaining 9-bit feature points (or feature point information ) Can be calculated.

In this case, the reference angle for determining (dividing) the angular range that is preset in the feature point extracting unit 140 has been described as an example of 40 degrees. However, it is needless to say that such a reference angle can be changed and set to various angles For example, as the reference angle is lowered, the number of directions and the number of bits of the slope capable of expressing the characteristics of an image can be increased to increase the resolution of the feature points, thereby obtaining more precise feature point information.

According to the above-described configuration, the minutiae point extracting unit 140 can calculate (generate) 9-bit minutia information for each of a plurality of different blocks constituting the unit setting area A, The minutiae information of 36 bits can be calculated (generated) for the area A.

In addition, the minutiae point extracting unit 140 may calculate minutia information of 36 bits for each of a plurality of different setting areas constituting one interest area. If one interest area is composed of 9 setting areas, The feature point information of 324 bits can be generated for the region and the feature point for the region of interest can be extracted.

In this case, the feature point extracting unit 140 can divide the ROI into a plurality of setting areas and blocks, and it is also possible to binarize one or more governing slopes of each block (or convert binary ROI data into binary data) The feature point information of the value can be generated corresponding to the region of interest.

Through the above-described configuration, the minutiae point extracting unit 140 can generate minutiae point information having a capacity smaller than that of the existing minutiae point extraction method for the ROI, and through the use of the minutia information described below, The processing load can be lowered and the processing speed can be improved.

8, the feature point extracting unit 140 extracts feature points for a plurality of different blocks constituting the ROI, collects the feature point information generated for each block, And then provides (transmits) the minutia information to the control unit 150.

In addition, the controller 150 may extract one or more original images corresponding to the online comic contents of the specific times from the content DB 101 and then provide the extracted one or more original images to the interest area extraction unit 110, The feature point information generated in correspondence with the specific original image corresponding to the on-line cartoon contents of the specific times and the specific original image corresponding to the specific original image through the image processing unit 110, the normalization unit 120, the image analysis unit 130, And the content information corresponding to the online cartoon content extracted from the content DB 101 may be matched with each other and stored in the minutia DB 102.

At this time, the controller 150 identifies minutia information generated (or extracted) corresponding to the original image, sets minutia information corresponding to the original image as reference minutia information, and stores the minutia information in the minutia DB 102 And the reference minutia information generated for each region of interest selected from the at least one original image corresponding to a specific position can be matched with the content information and stored in the minutia DB 102. [

In the above-described configuration, it is needless to say that the content DB 101 and the minutia DB 102 may be configured as a single DB.

The controller 150 may include a micro controller unit (MCU) and various engines. The controller 150 may include a RAM, a ROM, a CPU, a GPU, and a bus. A RAM, a ROM, As shown in FIG. The CPU accesses the memory, performs booting using an O / S (Operating System) stored in the memory, and can perform various operations using various programs, contents, and data stored in the memory.

Also, the control unit 150 may receive an image to be identified from the outside.

For example, when the control unit 150 is configured in the server, the control unit 150 receives an image to be identified from an external device through a communication unit configured in the server supporting communication via a communication network with the external device, An additional DB for storing an identification target image may be included and extracted from the additional DB based on an external input through an input unit configured in the server and received.

The control unit 150 controls the ROI extracting unit 110, the normalizing unit 120, the image analyzing unit 130, and the feature point extracting unit 140 to generate feature point information And then compared with the reference minutia information stored in the minutia DB 102.

For example, the control unit 150 may provide the ROI extractor 110 with the identification image in place of (or in place of) the original image in the above-described configuration, and the ROI extractor 110 ), The normalization unit 120, the image analysis unit 130, and the feature point extraction unit 140, similar to the method of generating the feature point information for the ROI from the original image, And generate feature point information for each ROI corresponding to the ROI.

The control unit 150 may further include a feature point extracting unit 140 that extracts feature point information obtained from the ROI extracting unit 110, the normalizing unit 120, the image analyzing unit 130, and the feature point extracting unit 140, To the plurality of different reference minutia information stored in the minutia DB 102.

Accordingly, when the reference feature point information coinciding with the feature point information obtained in correspondence with the identification target image exists in the feature point DB 102, the control unit 150 generates a reference point corresponding to the feature point information of the ROI belonging to the identification target image Content information matching the minutia information can be extracted from the minutia DB 102 and provided as content information related to the identification target image.

That is, when the content information for the specific sequence of the specific on-line cartoon content matched with the reference feature point information coinciding with the feature point information of the identification target image is extracted, the control unit 150 determines whether the identification target image corresponds to the specific on- The image processing apparatus may determine that the image is generated by duplicating or clipping at least one or at least one of the at least one original image included in the content information corresponding to the specific rotation, And may provide the content information corresponding to the specific rotation of the content as original content associated with the identification target image.

At this time, the controller 150 determines that the minutia information of the identification target image and the reference minutia information coincide when the data are compared with each other by a predetermined threshold value or more when the minutia information of the identification target image and the reference minutia information are compared .

If the control unit 150 is configured in the server, the controller 150 may display the extracted content information corresponding to the identification target image through a separate display unit connected to the server, The content information may be displayed through a display unit included in the user terminal.

In addition, the server includes a server controller for performing functions of the interest region extracting unit 110, the normalizing unit 120, the image analyzing unit 130, the feature extracting unit 140, and the controller 150, And a terminal control unit for performing some other functions of the control unit 150 may be configured in a user terminal that communicates with the server through a communication network.

Accordingly, the terminal control unit configured in the user terminal may select an image to be identified, which is selected in accordance with user input through a user input unit configured in the user terminal among a plurality of different images stored in the memory unit of the user terminal, To the server.

Accordingly, the server control unit configured in the server may include a plurality of different constituent units including the ROI extraction unit 110, the normalization unit 120, the image analysis unit 130, and the feature point extraction unit 140 And controls the ROI extracting unit 110, the normalizing unit 120, the image analyzing unit 130, and the feature point extracting unit 140 when receiving the identification target image from the user terminal, Generates minutia information corresponding to the received identification target image, compares the minutia information with the reference minutia information previously stored in the minutia DB 102, and stores content information matched with the minutia information corresponding to the minutia information about the identification target image Extracted from the minutia DB 102 and transmitted to the user terminal.

In addition, the terminal control unit of the user terminal may display the corresponding content information on the display unit as content information corresponding to the identification target image when receiving the content information through the communication unit configured in the user terminal.

In addition, in the above-described configuration, the control unit 150 determines that the identification target image is illegal copy content illegally copied based on the content information based on the extracted content information corresponding to the identification target image, Information may be provided.

As described above, according to the present invention, as in the conventional method of extracting and comparing minutiae with respect to the entire area of the original image constituting the online comic contents, there is a mismatch between the size ratio of the original image and the identification target image, In order to solve the problem that it is not easy to compare the feature point of the image to be identified and the original image due to the presence of the image, a region of interest composed of one meaningful scene (or cut) in the image is identified, the region of interest is normalized, The feature points can be compared with each other through the method of extracting the feature points through the method of extracting feature points through the method of extracting feature points from the original image and the identification target image, Processing in the identification process The speed can be greatly improved. In addition, even if the identification target image is configured in various sizes, the feature points related to the HOG are not changed in the normalized ROI unit, so that the feature points of the original image and the identification target image can be accurately matched The identification accuracy can be greatly improved.

Also, in the method of extracting and comparing feature points based on the frequency characteristics according to the patent application filed by the same applicant, the feature points are extracted with respect to the entire region of the original image, and the number of feature points due to feature point extraction in the frequency domain is considerable The present invention can not only calculate the feature point information of the N-bit value by binarizing the governing slope extracted through the HOG-based image analysis algorithm, but also calculate the feature point information of the N-bit value, The number of minutiae points can be reduced as compared with the conventional method, and the minutiae point comparison is performed in units of a scene, thereby minimizing the influence of changes in image quality and resolution, thereby increasing the identification accuracy.

In the case where one region of interest is included in the image to be identified, the controller 150 controls the ROI extracting unit 110, the normalizing unit 120, the image analyzing unit 130, (140) to extract the content information matched with the reference minutia information corresponding to the minutia information about the region of interest of the identification target image from the minutia DB (102) and provide the extracted content information as content information corresponding to the identification target image have.

In addition, the controller 150 may determine whether there are a plurality of ROIs extracted from the identification target image in cooperation with the ROI extractor 110. If the ROI is a plurality of ROIs, the ROI extractor 110, The image analyzing unit 120, the image analyzing unit 130, and the feature point extracting unit 140 to identify the content corresponding to the identification target image in the following manner, and to provide the content information on the identified content.

First, when a plurality of different interest regions are selected from the identification target image through the ROI extracting unit 110, the controller 150 extracts the ROIs from the ROI extracting unit 110, the normalizing unit 120, The feature point extraction unit 140 and the feature point extraction unit 140 to generate feature point information for each of the plurality of different interest regions, thereby extracting feature points for each region of interest.

In addition, the controller 150 searches the minutia point DB 102 based on the plurality of minutia information for each region of interest generated corresponding to the identification target image from the minutia matching point extracting unit 140, Extracting content information matched with reference feature point information mutually matching with respect to each of the minutia information for each region of interest, and if the extracted minutia information corresponding to each of the minutia information of each region of interest of the identification target image coincides with each other, And can provide the content information extracted from the minutia DB 102 as the identified content corresponding to the target image.

That is, the controller 150 extracts specific content information for a specific sequence of the specific on-line cartoon content having the minutiae for each region of interest corresponding to the minutiae of the region of interest according to the minutia information for each region of interest of the identification target image, The identification target image may be an image obtained by duplicating the specific content information, and the specific content information may be provided as the original content (or the copy target content) of the identification target image.

9, the controller 150 stores content information matched with the reference minutia information corresponding to the minutia information with respect to each of the minutia information of each region of interest of the identification target image, from the minutia DB 102 Extracts a plurality of pieces of content information corresponding to the identification target image, compares the plurality of pieces of extracted content information with each other, and when the pieces of content information are partially different from each other, As content information.

For example, the control unit 150 extracts two pieces of first content information for one-time contents of a particular online comic corresponding to the minutia information of the region of interest of the identification target image, When the second and third contents information for each of the third-order contents are extracted one by one, the extracted contents information corresponding to the identification-object image is identified, and compared with each other, (Or identified) first content information as content information corresponding to the identification target image.

In addition, the controller 150 extracts one or more piece of content information corresponding to the partial feature point information from the feature point DB 102 on a part of a plurality of different pieces of feature point information generated corresponding to the identification target image , And may compare the one or more pieces of content information to provide the most detected (or identified) content information as content information corresponding to the identification target image.

In addition, the controller 150 may set various conditions for identifying the content that is the copy of the image to be identified, and may identify the original content corresponding to the image to be identified according to the condition and provide the same to the user.

FIG. 10 is a table showing a comparison of performance between an on-line cartoon identification system based on a region of interest and an existing feature point extraction method according to an embodiment of the present invention. As shown in FIG. 10, , The number of feature points is very small at 61 levels (average 2.87%) per one WebTune, which is an online cartoon content, and even when the feature point DB 102 of the WebTurn rotation is built at 85.6 times that of the existing algorithm, Compared to the existing algorithms, the feature extraction time and the identification time are 2.34 times faster than the average of 0.517 seconds, and the total identification speed is 0.83 seconds on average and 0.38 seconds less.

That is, according to the present invention, the number of minutiae points extracted from the contents of one turn constituting the online comic contents is configured to be smaller than that of the existing minicomputer, but the identification processing speed and accuracy can be greatly improved as compared with the existing method, Even if the amount is larger than the existing one, it can provide faster identification performance than the existing one.

FIG. 11 is a flowchart illustrating a method of identifying an on-line cartoon based on a region of interest according to an embodiment of the present invention. Referring to FIG. 11, The server including the DB which is matched and stored with the discrimination content information can receive the identification target image from the outside (S1).

At this time, the reference minutia information may be stored in the DB for each region of interest, and the minutia information may be stored in the DB by matching the content information of the specific minutia. In addition, the DB may be a minutia DB 102 or a minutia DB 102.

In addition, the server may extract a region of interest through boundary detection of the image to be identified (S2).

Thereafter, the server may generate a normalized image obtained by normalizing the region of interest to a predetermined normalization size (S3).

The server may divide the normalized image into a plurality of blocks having a predetermined size, and calculate one or more dominant slopes for each block based on Histogram of Oriented Gradient (S4).

Then, the server can generate N-bit minutia information obtained by binarizing the one or more governing slopes of the block constituting the ROI according to a preset reference (or algorithm) in association with the ROI S5).

The server compares the minutia information with the reference minutia information (S6). If the minutia information matches the minutia information, the server extracts content information matching the minutia information from the DB, The identification target image is determined as the content corresponding to the content information, and the corresponding content information may be provided to the user as the content associated with the identification target image (S7).

The various devices and components described herein may be implemented by hardware circuitry (e.g., CMOS-based logic circuitry), firmware, software, or a combination thereof. For example, it can be implemented utilizing transistors, logic gates, and electronic circuits in the form of various electrical structures.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or essential characteristics thereof. Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

110: ROI extraction unit 120: Normalization unit
130: Image analysis unit 140: Feature point extraction unit
150:

Claims (9)

An interest region extracting unit for extracting a region of interest through boundary detection from one or more original images constituting online cartoon contents;
A normalization unit for generating a normalized image obtained by normalizing the ROI with a predetermined normalization size;
An image analyzer for dividing the normalized image into a plurality of blocks having a predetermined size and calculating at least one dominant slope for each block based on Histogram of Oriented Gradient (HOG);
A minutiae point extracting unit for generating minutia information of N bit value obtained by binarizing the at least one governing slope for each block constituting the ROI according to a preset reference, corresponding to the ROI; And
Wherein the feature point information is set as reference feature point information to match and store content information of the on-line cartoon content corresponding to the original image, and upon receiving the identification target image, the ROI extraction unit, the normalization unit, And a controller for generating minutia information corresponding to the identification target image, comparing the minutia information with the reference minutia information, and providing content information matched with mutually matching reference minutia information in association with the identification target image,
The feature point extracting unit may divide a unit block into a plurality of cells of a predetermined number, compute a slope calculated for each unit cell on the basis of HOG to calculate a size of a gradient by a direction, And determining whether or not there is a dominant tilt for each of a plurality of predetermined angular ranges according to the direction of each of the at least one dominant tilt, and determining presence / absence of the dominant tilt for each of the plurality of angular ranges, An online comic book identification system based on interest area that processes binary data.
The method according to claim 1,
Wherein the content information includes a position information of a specific position of the on-line cartoon content.
The method according to claim 1,
Wherein the interest region extracting unit detects the boundary line by scanning the original image in the horizontal and vertical directions and generates a single image by sequentially connecting the original images when the source images constituting the online comic contents are plural, Wherein the region-of-interest-based on-line comics identification system detects the region-of-interest-based online comics.
The method of claim 3,
Wherein the ROI extracting unit extracts a ROI that is contiguous to the ROI from another original image that is contiguous with the original image if the ROI exists, Wherein the region of interest is connected to a specific region of interest and is set as a region of interest and extracted.
delete delete The method according to claim 1,
Further comprising a storage unit for storing the reference minutia information and the content information in a matching manner under the control of the control unit.
The method according to claim 1,
Wherein the control unit identifies content information according to reference minutia information matching the minutia information of each of the plurality of ROIs when the ROI includes a plurality of ROIs, And when the other content information is identified, the most-identified content information is provided as content information corresponding to the identification target image.
A method for identifying an on-line cartoon based on a region of interest of a server, the method comprising: a DB corresponding to each of a plurality of different online comic contents and storing reference minutia information about minutiae of each region of interest of the original image,
Receiving an identification target image and extracting a region of interest through boundary detection of the identification target image;
Generating a normalized image obtained by normalizing the ROI with a predetermined normalization size;
Dividing the normalized image into a plurality of blocks having a predetermined size, dividing the normalized image into a plurality of cells of a predetermined number, collecting slopes calculated on a per-cell basis based on HOG (Histogram of Oriented Gradient) Calculating one or more dominant slopes having a magnitude equal to or larger than a preset reference size for each block;
Generating minutia information of an N-bit value obtained by binarizing the one or more governing slopes of the block constituting the ROI according to a preset reference, corresponding to the ROI; And
Comparing the minutia information with the reference minutia information, extracting, from the DB, content information matched with the minutia information if the minutia information matches the minutia information, Judging the content to be a content,
The step of generating minutia information of the binarized N-bit values corresponding to the ROI may include determining whether a dominant slope exists in a plurality of predetermined angular ranges according to directions of the at least one dominant slopes for each block And performing binarization processing for each of the plurality of angle ranges according to the presence or absence of the dominant slope.

Images