JP2020501206A - Online cartoon identification system and method based on region of interest - Google Patents

Abstract

The present invention relates to an online cartoon identification system and method based on a region of interest. More specifically, a region of interest is detected from each of an original image constituting online manga content provided online and an identification target image obtained by copying a part or all of the original image, and extracted from the region of interest. The present invention relates to a system and method for online comics identification based on a region of interest that provides online comics content corresponding to the image to be identified based on the identified feature points.

Description

  The present invention relates to an online cartoon identification system and method based on a region of interest. More specifically, a region of interest is detected from each of an original image constituting online comic content provided via online and an identification target image obtained by copying a part or all of the original image, and extracted from the region of interest. The present invention relates to a system and method for identifying online comics based on a region of interest that provides online comic content corresponding to the image to be identified based on the identified feature points.

  Nowadays, the function of posting and sharing contents via online is activated through a network, and a copyright holder who produces and distributes contents through this function can use his / her own contents through various portal sites. Is uploaded and distributed, so that the user can easily obtain various content copyrighted works online.

  However, as it becomes easier to acquire content online, it becomes easier to duplicate content online, thereby creating duplicate content by downloading or capturing content posted through a portal site. However, the frequency of non-law cases in which this is illegally distributed online is increasing.

  This not only infringes much on the rights of copyright owners, but also hurt users who subscribe to content by providing legitimate costs for copyrighted works.

  Among these contents, contents related to online comics, such as web comics, are one of the most popular contents, and thus the situation where the most illegally copied contents are rampant. For this reason, techniques for identifying and preventing illegally copied content by applying various techniques for preventing such illegally copied content related to online comics have been developed.

  For example, a method has been proposed in which all the feature points of individual images that constitute illegally copied content obtained by copying online comic content are compared and compared in order. It should be preceded by identifying, from the illegally copied content, an area corresponding to a spatial area where the feature points extracted from the online comic content are located in the entire area of the content.

  However, illicit duplicators may change the size of the online comic content in a certain number of times to any size, or clip the part to generate illegally duplicated content. It is difficult to identify the spatial region corresponding to the extracted feature points from the illegally copied content, and even if all feature points are extracted, it is not only difficult to match them, but also to identify one image. Extraction of a number of feature point data is required, which greatly increases the amount of computation, and requires a considerable processing time to identify one of the illegally copied contents. Such a method has a problem that it is not only inefficient but also difficult to guarantee performance.

  In addition, when a server that posts online comic content on a website manages the original image that constitutes the online comic content by randomly dividing it into a plurality of images, it is necessary to specify the characteristics of the online comic content that is composed of a single image. When a scene is divided into a plurality of images, a situation may occur.In such a case, even if a feature point is extracted in a specific scene divided into each of a plurality of images constituting an original image, the feature point is not affected. A problem arises in that it is difficult to compare a part of a specific scene extracted from a single original image with an illegally duplicated image.

  At this time, the specific scene may be defined as a single cut, a single frame, a single panel, or a single frame.

  Recently, some scenes (or cuts) of an original image are cut out by a method such as capture, and are used for various purposes such as emoticon of a chat window and prevention of deletion. In addition, users who are interested in scene images (or cut images) used for such purposes use various image search-based search tools to search for content including the scene images. However, such scene images are also produced in various sizes, and as described above, it is not easy to detect through comparing feature points with the original image.

  Therefore, it is possible to easily identify the content copied from online comics posted online, thereby cutting off the distribution of illegally copied content in advance and linking it with the image of a specific scene in which the user is interested. There is a need for alternatives that provide information in the content to enhance the user's search convenience.

International Publication No. WO 2016/125997

  The present invention identifies a region of interest composed of a single meaningful scene (or cut) from an image, extracts the region of interest, normalizes the region of interest, and then extracts feature points through HOG-based image analysis. By comparing the feature points of the image to be identified and the original image that constitutes the online manga content in each region of interest through the method of extracting Helping to reduce the processing speed of content identification corresponding to the image to be identified and increasing the processing speed and clipping the original image into various shapes or linking it to the image to be identified manufactured in various sizes The purpose is to increase the accuracy of identification in content.

  An online cartoon identification system based on a region of interest according to an embodiment of the present invention includes a region-of-interest extraction unit that extracts a region of interest from one or more original images constituting online comic content through edge detection, and pre-determines the region of interest. A normalization unit that generates a normalized image normalized by the set normalization size; and divides the normalized image into a plurality of blocks of a predetermined size, based on a HOG (Histogram of Oriented Gradient). A video analysis unit that calculates one or more dominant gradients for each block, and a feature point of an N-bit value obtained by binarizing the one or more dominant gradients for each of the blocks constituting the region of interest according to a preset standard And a feature point extraction unit for generating information of the feature point corresponding to the region of interest. Set as point information, matched and stored with the content information of the online comic content corresponding to the original image, and when receiving the image to be identified, the region of interest extraction unit, the normalization unit, the video analysis unit and the features After controlling the point extraction unit to generate feature point information corresponding to the identification target image, the content information matching the information of the reference feature points that match each other is compared with the reference feature point information. A control unit that provides the image corresponding to the identification target image.

  As an example related to the present invention, the content information may include information on a specific time of the online comic content.

  As an example related to the present invention, the region of interest extracting unit scans the original image in a horizontal and vertical direction to detect edges, and when there are a plurality of original images constituting the online comic content, the original images are sequentially sorted. The edge may be detected after being connected and generated as a single image.

  As an example related to the present invention, the region of interest extraction unit, when there is a specific region of interest in contact with the upper end of the original image, or the lower end, excluding the specific region of interest, or other continuous with the original image A region of interest continuously connected to the specific region of interest may be connected to the specific region of interest from the original image and set as one region of interest and extracted.

  As an example related to the present invention, the feature point extracting unit divides the unit block into a plurality of cells of a preset number, combines the gradients calculated for each unit cell based on the HOG, and calculates the magnitude of the gradient for each direction. Is calculated, and one or more dominant gradients having a size equal to or larger than a preset reference size are calculated corresponding to the unit block.

  As an example related to the present invention, the feature point extracting unit determines whether or not there is a dominant gradient for each of a plurality of angular ranges set in advance according to each direction of the one or more dominant gradients, and determines the dominance for each of the plurality of angular ranges. Binary processing may be performed according to whether or not a gradient exists.

  According to an aspect of the present invention, the information processing apparatus may further include a storage unit configured to store the information of the reference feature point and the content information in a matching manner under the control of the control unit.

  As an example related to the present invention, when a plurality of regions of interest are included in the identification target image, the control unit may generate content information based on information of reference feature points that match information of feature points of each of the plurality of regions of interest. Identifying, and when a plurality of different pieces of content information are identified corresponding to the plurality of regions of interest, providing the most identified content information as content information corresponding to the identification target image. Can be.

  According to the embodiment of the present invention, information of reference feature points in feature points of each region of interest of the original image corresponding to each of a plurality of online comic contents having a difference according to the embodiment is stored in matching with content information of each of the instances. An online comics identification method based on a region of interest of a server including a DB includes receiving an image to be identified, extracting a region of interest through edge detection in the image to be identified, and normalizing the region of interest in advance. Generating a normalized image normalized by the size of, and dividing the normalized image into a plurality of blocks of a predetermined size, and dividing the normalized image into a plurality of blocks based on a HOG (Histogram of Oriented Gradient). Calculating the dominant gradient of, and said block for each said block constituting said region of interest Generating information on N-bit value feature points corresponding to the region of interest by binarizing at least one dominant gradient according to a preset reference, and comparing the information on the feature points with the information on the reference feature points. If there is reference feature point information that matches the feature point information, content information matching the reference feature point information is extracted from the DB, and the identification target image is extracted from the content corresponding to the content information. Determining.

  According to the present invention, a region of interest composed of one meaningful scene (or cut) is identified from an image, the region of interest is extracted and normalized, and then the region of interest is analyzed through HOG-based image analysis. By comparing the feature points of the original image and the image to be identified for each region of interest through the method of extracting feature points in the original image, the comparison can be performed in scene units excluding unnecessary background of the original image. As a result, not only can the processing speed in the process of identifying the online comic content associated with the image to be identified be greatly improved, but also the image to be identified can be normalized in various sizes. Compares by the unit of interest, and the feature of the feature point based on HOG does not change, and the feature point of the original image and the feature point of the image to be identified are accurately matched. Things can have the effect of improving greatly the accuracy of identification in the online cartoon content associated with the identification object image.

  In addition, the present invention not only can binarize a dominant gradient extracted through a video analysis algorithm based on HOG in the process of identifying online comic content associated with an image to be identified, and calculate feature points, By calculating the feature points in a preset block unit, not only can the number of feature points extracted for one image be reduced than in the existing feature point calculation method, but also the comparison of the feature points in scene units This has the effect of minimizing the effects of changes in image quality and resolution and increasing the accuracy of identification.

1 is a configuration diagram of an online cartoon identification system based on a region of interest according to an embodiment of the present invention; 5 is an illustration of a process of extracting a region of interest in the online cartoon identification system based on the region of interest according to an embodiment of the present invention. 5 is an illustration of a process of extracting a region of interest in the online cartoon identification system based on the region of interest according to an embodiment of the present invention. 5 is an illustration of a process of extracting a region of interest in the online cartoon identification system based on the region of interest according to an embodiment of the present invention. 5 is a diagram illustrating a process of extracting a feature point in a region of interest based on a HOG in an online cartoon identification system based on a region of interest according to an embodiment of the present invention. 5 is a diagram illustrating a process of extracting a feature point in a region of interest based on a HOG in an online cartoon identification system based on a region of interest according to an embodiment of the present invention. 5 is a diagram illustrating a process of extracting a feature point in a region of interest based on a HOG in an online cartoon identification system based on a region of interest according to an embodiment of the present invention. 3 is an operation example of content identification associated with an image to be identified in an online comic identification system based on a region of interest according to an embodiment of the present invention; 3 is an operation example of content identification associated with an image to be identified in an online comic identification system based on a region of interest according to an embodiment of the present invention; 5 is a table showing a comparison of performance between an online cartoon identification system based on a region of interest and an existing feature point extraction method according to an embodiment of the present invention. FIG. 4 is a flowchart illustrating an online comics identification method based on a region of interest according to an embodiment of the present invention.

  Extract the feature points of each continuous image from a plurality of images that make up the ill-scanned manga in the form of an image file created by illegitimately scanning an existing manga book composed of paper, and then construct the original manga In the case of technology that identifies illegal copying based on the matchability by comparing feature points extracted from each image in order, only the distinction of the page of the image composed of each of the illegally scanned manga and the original comic is clear In addition, the size ratio is also clear, and it is easy to compare the feature points extracted from the scanned cartoon and the original cartoon, thereby easily identifying the illegally scanned cartoon.

  However, in the case of online comic contents such as web toon, which is produced in the form of an image without being produced on paper, and is directly posted online, the recurring unit is not divided into pages. , And the size of the single image can also be configured differently for each round.

  In addition, the unauthorized person who illegally copies the online comic content captures the single image (original image) of each time, then divides the image into various sizes, and generates a plurality of image files. Alternatively, the plurality of image files may be combined into a single file to generate illegally duplicated content and distributed without permission.

  As a result, the size ratio of the original image constituting the online comic content and the duplicate image constituting the illegally duplicated content do not match each other, and is different from the scanned comic which is composed in units of pages and is easily identified. The duplicate image may be configured by dividing the original image into a plurality of images, or may be configured by combining a part of the images belonging to different repetitions. Since it is impossible to discriminate between the original image and the duplicate image, one-to-one matching through the extraction of feature points is not possible. There are difficulties in applying comics to detect illegally copied content.

  Also, recently, some scenes (or cuts) of the original image are cut out by a method such as capture and used for various purposes such as the use of emoticons in a chat window and prevention of deletion of writing. There is a demand to search for content that contains scene images (or cut images) used for such purposes, but these scene images are also produced in various sizes and original images It is not easy to detect through comparison of the feature points.

  According to Patent Application No. 10-2016-0152738, which has already been filed by the applicant, feature points are extracted based on the frequency characteristics of the image in the entire area of the original image that constitutes the online comic content, and based on the extracted feature points, By comparing feature points extracted from each of the copied content and the online comic content, all of the illegally copied content and the online comic content have the same frequency characteristics due to pixel changes, and the position and distribution of the feature points are different. No change was made, and the feature points of the illegally copied content and the feature points of the online comic content were accurately matched to improve the identification accuracy of the illegally copied content. While further reducing the identification processing time for illegal content In addition to guaranteeing more accurate identification performance than the existing ones, it is also easy to identify scene images obtained by cutting out some scenes from the original images that constitute online comic content, and to search for users who search for content based on the images Suggest a way to guarantee the convenience of

  Prior to the description, the term “scene” as used in the present invention may be defined as a single cut, a single frame, a single panel, or a single frame. Preferably, the region of interest described in the present invention may mean a cartoon image or an image region forming the single cut, the single frame, the single panel, or the single frame. Also, it should be noted that the original image can be composed of one or more cuts, frames, panels, or frames.

  At this time, a plurality of cuts, a plurality of frames, a plurality of panels, or a plurality of frames may be configured as one region of interest, depending on the cut configured in the original image or the layout of the frames.

Hereinafter, a detailed embodiment of the present invention will be described with reference to the drawings.
First, FIG. 1 is a block diagram of an online comics identification system based on a region of interest according to an embodiment of the present invention. As shown, a region of interest extracting unit 110, a normalizing unit 120, a video analyzing unit 130, A feature point extracting unit 140 and a control unit 150 can be included.

  In addition, at least one of the components constituting the online comic identification system may be included in a different component, and in addition to the above-described components, an additional component may be included in the online comic identification system. Of course, the system can be further configured.

  Also, each component constituting the online comic identification system is constituted by a server or at least a part of the component of the online comic identification system is constituted by each of a plurality of different devices capable of communicating with each other. And can communicate with each other.

  As an example, the region of interest extraction unit 110, the normalization unit 120, the video analysis unit 130, and the feature point extraction unit 140 are configured in a server, and the control unit 150 is configured in a terminal (for example, a user terminal) communicating with the server. Can be done.

  In addition, the online comic identification system may be configured with data related to an application executed on a server or a user terminal.

  Hereinafter, an example of an online comics identification system configured in a server will be described in detail with reference to the configuration of FIG. 1 with reference to the configuration of FIG.

  First, as shown in FIGS. 2 and 3, the region of interest extraction unit 110 can extract one or more original images constituting online comic contents for each time.

  At this time, the server may include one or more original images for each of the online comics and / or the content DB 101 in which the content information is stored by matching with the related content information of the online comic (or the online comic content). At this time, one or more original images may be stored in the content DB 101 corresponding to a specific time of the specific online comic content.

  In addition, the control unit 150 may extract an original image matching the specific content information from the content DB 101 and provide the extracted original image to the region of interest extracting unit 110, and the control unit 150 may correspond to the original image. When the feature point information is received, it may be matched with the specific content information extracted together with the original image and stored in the feature point DB 102 included in the server.

  Further, the content information may include content identification information corresponding to the online comic content, and repetition information in any one of a plurality of repetitions constituting the online comic content. The next information may include image identification information for one or more original images corresponding to (or matching with) any one of the plurality of times of difference.

  For example, corresponding to the specific online comic content, content information corresponding to each of a plurality of different times can be stored in the content DB 101 in advance, and a plurality of differences corresponding to the specific online comic content can be stored. The content information for each of the times can include the same content identification information and the different pieces of time information.

  In addition, the generation dates or the publication dates may be configured to be different between the original images of the different times, and the information on this may be included in the time information.

  In addition, the content DB 101 may store content information for each of a plurality of different online comics (or for each of a plurality of different online comic contents) and one or more original images in a mutually matching manner. it can.

  On the other hand, when receiving the original image as shown in FIG. 4, the region of interest extracting unit 110 may scan the original image horizontally and vertically to detect an edge. Selecting (or setting) a region of interest (ROI) in one significant scene (or cut or frame) from the original image by the edge detection. Can be.

  At this time, if the upper or lower end of the region of interest contacts the upper or lower end of the original image as shown in the figure, the region of interest extracting unit 110 removes the region of interest or removes the first original being scanned. Based on the identification information of the image, after extracting the next or previous second original image from the content DB 101, successively scan following the first original image, the upper end of the first original image, Alternatively, the first region of interest in contact with the lower end and the upper end of the second original image, or the second region of interest in contact with the lower end and following the first region of interest is interconnected to form one region of interest. It can be selected (or set) and extracted.

  Also, as shown in FIG. 3, when there are a plurality of original images constituting the same online comic content, as shown in FIG. The original image can be generated in one single image.

  At this time, the identification information of the original image may be a file name of the original image.

  In addition, as shown in FIG. 4, the region of interest extracting unit 110 may scan the single image in a horizontal direction and a vertical direction to detect an edge, and may close the edge through the edge detection. A single region can be selected as the region of interest.

  At this time, the ROI may include a single scene (or a single cut or a single frame) included in the original image, or a cartoon component such as dialogue, ground sentence, onomatopoeia, and the like. be able to.

  Also, every time a region of interest is detected from the original image, the region of interest extracting unit 110 extracts an image corresponding to the region of interest from the original image and provides a related image of the region of interest to the normalization unit 120 ( Or transmission).

  On the other hand, the online comic identification system can extract feature points through video analysis based on Histogram of Oriented Gradient (HOG) in the region of interest, and can use the characteristic points as means for identifying online comic content. The process of extracting the feature points will be described in detail with reference to the following drawings.

  First, as shown in FIG. 5, the normalizing unit 120 receives the related image of the region of interest provided from the region of interest extracting unit 110, and converts the related image of the region of interest into a predetermined normalization magnitude. Thus, a normalized image can be generated.

  For example, the normalization unit 120 may normalize the original image according to a preset normalization size or a normalization ratio. Is adjusted by a preset number of pixels, and the number of vertical pixels of the original image can be automatically adjusted according to the horizontal adjustment ratio.

  In addition, the normalization unit 120 provides (or transmits) the normalized image to the image analysis unit 130, and the image analysis unit 130 converts the normalized image into a predetermined size (or a first size). (Size) can be divided (divided) into a plurality of setting areas.

  In addition, the image analysis unit 130 divides (divides) a plurality of blocks of a predetermined size (or a second size) of a plurality of differences for each unit setting area A, and HOG (Histogram of HOG) for each block. One or more dominant gradients can be calculated through image analysis based on the Oriented Gradient.

  For example, as shown in FIG. 6, the image analysis unit 130 divides (divides) the unit block into a plurality of cells having a predetermined size (or a third size), and generates a HOG. The gradient can be calculated for each unit cell on the basis of the gradient.

  At this time, the unit block may mean any one of the plurality of blocks.

  In addition, the image analysis unit 130 may combine gradients calculated for cells belonging to a specific unit block to calculate a magnitude of a gradient in the specific unit block in each direction. That is, the image analysis unit 130 may combine a plurality of gradients calculated for each cell belonging to a specific unit block and calculate a magnitude for each gradient direction (or for each gradient).

  For example, the image analysis unit 130 may be divided at a predetermined angle interval according to a gradient of each cell belonging to the specific unit block according to the HOG algorithm, and the gradient may be determined according to a preset different gradient orientation. A histogram obtained by calculating a magnitude may be calculated, and one or more dominant gradients having a size equal to or greater than a predetermined reference size corresponding to the specific unit block may be calculated in the histogram. And can be calculated (or extracted).

  At this time, if different gradient directions are set at intervals of 40 degrees, the image analyzer 130 may calculate histograms based on nine different gradient directions.

  Also, as shown in FIG. 7, the feature point extraction unit 140 operates in conjunction with the video analysis unit 130 to calculate each of the one or more dominant gradients calculated corresponding to the specific unit block. The existence or nonexistence of the dominant gradient can be determined for each of a plurality of different angular ranges (or angles) set in advance according to the direction, and the existence of the dominant gradient can be determined for each of the plurality of different angular ranges (or each angle). Binary processing can be performed depending on the availability.

  As an example, in the feature point extracting unit 140, a plurality of different angle ranges are set in advance based on an angle of 40 degrees (or as an interval), and a bit value for each of the plurality of angle ranges is set as a feature of the unit block. It can be preset to constitute the relevant data of the point. Accordingly, when the angle range is divided based on the angle of 40 degrees, the feature point extracting unit 140 can allocate 9 bits of the feature point related data in one block.

  In addition, the feature point extracting unit 140 sets 1 to an angle range where a dominant gradient exists depending on whether or not a dominant gradient exists for each angle range, and sets 0 to an angular range where a dominant gradient does not exist. In the above, data having a size of 9 bits can be calculated, and the data can be extracted (or calculated) as feature points of the specific unit block.

  That is, as shown in the figure, the feature point extraction unit 140 sets 1 to 40 because the dominant gradient exists in the angle range exceeding 0 ° preset in block 3 and 40 ° or less. Since there is no dominant gradient in a preset angle range exceeding 80 ° or less, it can be set to 0.

  Accordingly, the feature point extracting unit 140 performs a binary operation according to the presence or absence of a dominant gradient for each of the nine different angle ranges obtained by dividing 360 degrees into intervals of 40 degrees, and performs a 9-bit feature point (in a unit block). Or, information on feature points) can be calculated.

  At this time, the reference angle for determining (dividing) the angle range preset in the feature point extraction unit 140 has been described as 40 degrees, but such a reference angle can be changed and set at various angles. Obviously, the lower the reference angle, the more the number of gradient directions and the number of bits that can be expressed in the image features are increased, and the resolution of the feature points can be increased. You can also get point information.

  With the above-described configuration, the feature point extraction unit 140 can calculate (generate) 9-bit feature point information for each of a plurality of different blocks constituting the unit setting area A, and is configured with four blocks. 36-bit feature point information can be calculated (generated) in the unit setting area A.

  In addition, the feature point extracting unit 140 can calculate 36-bit feature point information for each of a plurality of different setting regions constituting one region of interest, whereby one region of interest has nine setting regions. In this case, 324-bit feature point information in the region of interest can be generated to extract the feature point in the region of interest.

  At this time, it is needless to say that the feature point extracting unit 140 can divide the region of interest into various numbers of set regions and blocks, and through this, binarizes one or more dominant gradients for each block (or , Converted to binary data), N-bit feature point information can be generated corresponding to the region of interest.

  Through the above-described configuration, the feature point extraction unit 140 can generate information of feature points having a smaller capacity than the existing feature point extraction method in the region of interest, and through this, the features described below can be used. The load of the content identification processing and definition processing using the point information can be reduced, and the processing speed can be improved.

  On the other hand, as shown in FIG. 8, the feature point extracting unit 140 extracts feature points for each of a plurality of different blocks constituting the region of interest, and combines the information of the feature points generated for each of the blocks to combine the feature points. After generating the information of the feature points in the feature points of the region of interest, the information can be provided (transmitted) to the control unit 150.

  Further, the control unit 150 may extract one or more original images corresponding to a specific online comic content from the content DB 101 and provide the extracted original image to the region of interest extracting unit 110. The information of the feature points generated corresponding to the specific original image corresponding to the specific online comic content through the region extracting unit 110, the normalizing unit 120, the video analyzing unit 130, and the feature point extracting unit 140, and Content information corresponding to the specific online comic content extracted from the content DB 101 together with a specific original image can be mutually matched and stored in the feature point DB 102.

  At this time, the control unit 150 identifies the information of the feature points generated (or extracted) corresponding to the original image, and then uses the information of the feature points corresponding to the original image as the reference feature points. The information on the reference feature points generated for each region of interest selected from one or more original images corresponding to a specific round can be set as the content information. And stored in the feature point DB 102.

  From the above configuration, it goes without saying that the content DB 101 and the feature point DB 102 may be configured as a single DB.

  Further, the control unit 150 can be configured to include an MCU (Micro Controller Unit) and various engines. As an example, the control unit 150 includes a RAM, a ROM, a CPU, a GPU, a bus, and a RAM, a ROM, a CPU, GPUs and the like can be connected to each other through a bus. The CPU can access the memory and perform a boot using an O / S (Operating System) stored in the memory, and perform various operations using various programs, contents, data, and the like stored in the memory. Actions can be performed.

In addition, the control unit 150 may receive an image to be identified from outside.
For example, when the control unit 150 is configured as a server, the control unit 150 receives an image to be identified from an external device through a communication unit configured in the server that supports communication via a communication network with the external device. Alternatively, the server may include a separate additional DB in which the identification target image is stored, and extract the identification target image from the additional DB based on an external input through an input unit configured in the server. Can be received.

  Also, the control unit 150 controls the region of interest extraction unit 110, the normalization unit 120, the video analysis unit 130, and the feature point extraction unit 140 in the identification target image, and generates feature point information. , Can be compared with the reference feature point information stored in the feature point DB 102.

  For example, the control unit 150 may provide the identification target image to the region of interest extraction unit 110 instead of (or instead of) the original image in the above-described configuration, and The extraction unit 110, the normalization unit 120, the video analysis unit 130, and the feature point extraction unit 140 select a region of interest from the identification target image in the same manner as generating information of feature points in the region of interest from the original image. Then, it is possible to generate feature point information for each region of interest corresponding to the image to be identified.

  Also, the control unit 150 may extract feature point information obtained through the region of interest extraction unit 110, the normalization unit 120, the video analysis unit 130, and the feature point extraction unit 140 corresponding to the identification target image. The information can be compared with information of a plurality of different reference feature points stored in the feature point DB 102.

  Accordingly, when the information of the reference feature point that matches the information of the feature point obtained corresponding to the image to be identified exists in the feature point DB 102, the control unit 150 determines the region of interest belonging to the image to be identified. Content information that matches information of a reference feature point that matches information of a feature point can be extracted from the feature point DB 102 and provided as content information associated with the identification target image.

  That is, the control unit 150 may extract the identification target image when the content information in a specific turn of the specific online comic content matching the information of the reference feature point matching the information of the feature point of the identification target image is extracted. Is generated by duplicating or clipping at least a part or all of at least one of one or more original images included in content information corresponding to a specific turn of the specific online comic content. And providing content information corresponding to a specific time of the specific online comic content extracted corresponding to the identification target image as original content associated with the identification target image. be able to.

  At this time, when comparing the information on the feature points of the image to be identified with the information on the reference feature points, if the data matches the reference value or more, a feature point of the image to be identified may be used. It can be determined that the information of the reference point and the information of the reference feature point match each other.

  Further, when the control unit 150 is configured in the server, the control unit 150 displays the extracted content information corresponding to the identification target image through a separate display unit connected to the server. When configured in the user terminal, the content information can be displayed through a display unit configured in the user terminal.

  In addition, the server includes a server control unit that performs some functions of the region of interest extraction unit 110, the normalization unit 120, the video analysis unit 130, the feature point extraction unit 140, and the control unit 150, A terminal control unit that performs some other functions of the control unit 150 may be configured in a user terminal that communicates with a server via a communication network.

  Accordingly, the terminal control unit configured in the user terminal selects one of the plurality of different images stored in the memory unit of the user terminal by a user input via the user input unit configured in the user terminal. The identified image to be identified can be transmitted to the server through a communication unit included in the user terminal.

  Accordingly, the server controller configured in the server controls a plurality of different components including the region of interest extraction unit 110, the normalization unit 120, the video analysis unit 130, and the feature point extraction unit 140. When receiving the identification target image from the user terminal, the control unit controls the region of interest extraction unit 110, the normalization unit 120, the video analysis unit 130, and the feature point extraction unit 140, and receives the image from the user terminal. After generating the information of the feature points corresponding to the identified image to be identified, the information of the reference feature points corresponding to the information of the feature points in the identified image is compared with the information of the reference point stored in the feature point DB 102 in advance. Content information matching information can be extracted from the feature point DB 102 and transmitted to the user terminal.

  The terminal control unit of the user terminal, when receiving the content information through a communication unit included in the user terminal, displays the content information as content information corresponding to the identification target image through the display unit. Can be.

  Further, in the above-described configuration, the control unit 150 determines that the identification target image is the illegally copied content obtained by illegally copying the content based on the content information based on the content information extracted corresponding to the identification target image, This can also provide result information.

  As described above, according to the present invention, the size ratio of the original image and the image to be identified does not match due to the method of extracting and comparing the feature points in the entire area of the original image constituting the online comic content. Or a region of interest composed of one meaningful scene (or cut) from the image to improve the difficulty in comparing the feature points of the image to be identified and the original image by capturing some scenes After the region of interest is normalized and the feature point is extracted through HOG-based image analysis, the feature point can be compared between the original image and the image to be identified for each region of interest. With this support, the comparison is made on a scene-by-scene basis without unnecessary background of the original image, and the online Not only can the processing speed in the process of identifying the image content be greatly improved, but even if the identification target images are configured in various sizes, the comparison is performed in units of normalized regions of interest and based on the HOG. Since the characteristics related to the feature points do not change, the feature points of the original image and the identification target image can be accurately matched, and the accuracy of identification can be greatly improved.

  In addition, the method of extracting and comparing feature points based on frequency characteristics according to the applicant's existing patent application also extracts feature points for the entire area of the original image and extracts feature points in the frequency domain. Since the number of feature points is considerable, not only the amount of calculation is considerable, but also the change in image quality and resolution is weak, the present invention sets the dominant gradient extracted through the video analysis algorithm based on HOG to 2 By performing the evolution process and calculating the feature point information of the N-bit value, not only the number of feature points can be reduced than the existing one by calculating the feature points in a preset block unit, but also the feature unit can be calculated in a scene unit. Point comparisons can be made to minimize the effects of changes in image quality and resolution to increase the accuracy of identification.

  On the other hand, when one region of interest is included in the identification target image, the control unit 150 may control the region of interest extraction unit 110, the normalization unit 120, the video analysis unit 130, and the feature point extraction unit as described above. 140, and extracts from the feature point DB 102 content information that matches information on reference feature points that match information on feature points in the region of interest of the image to be identified, as content information corresponding to the image to be identified. Can be provided.

  Also, the control unit 150 can determine whether there is a plurality of regions of interest extracted from the identification target image in conjunction with the region of interest extraction unit 110, and if there are a plurality of regions of interest, , By controlling the normalization unit 120, the video analysis unit 130, and the feature point extraction unit 140 to identify the content corresponding to the identification target image in the following manner, and provide the content information of the identified content. be able to.

  First, when a plurality of different regions of interest are selected from the identification target image through the region of interest extraction unit 110, the control unit 150 may include the region of interest extraction unit 110, the normalization unit 120, and the image analysis unit 130. By controlling the feature points and the feature point extraction unit 140 to generate information on feature points for each of the plurality of different regions of interest, feature points for each region of interest in the identification target image can be extracted.

  Further, the control unit 150 searches the feature point DB 102 based on information of feature points for each of the plurality of different regions of interest generated from the feature point extraction unit 140 corresponding to the identification target image, and performs the identification. In each of the information of the feature points for each of the regions of interest of the target image, the content information matching the information of the reference feature points that match each other is extracted, and corresponding to each of the information of the feature points for each of the regions of interest of the identification target image. When all the extracted content information matches each other, the content information extracted from the feature point DB 102 as the content identified corresponding to the identification target image can be provided.

  That is, the control unit 150 may control the specific online comic content having a feature point for each region of interest that coincides with a feature point for each region of interest according to the information on the feature point for each region of interest in the identification target image in a specific round. It is possible to extract specific content information, determine that the identification target image is an image obtained by copying the specific content information, and provide the specific content information as original content (or copy target content) of the identification target image.

  Also, as shown in FIG. 9, the control unit 150 may include content information that matches information of a reference feature point that is mutually coincident with information of a feature point in each of information of feature points for each region of interest of the image to be identified. Can be extracted from the feature point DB 102, and if a plurality of pieces of content information extracted corresponding to the identification target image are partially compared with each other, the most detected among the plurality of pieces of content information is detected. The identified (or identified) content information can be provided as content information corresponding to the identification target image.

  For example, the control unit 150 may extract two pieces of first content information in the first-order content of the specific online comic corresponding to the information on the feature points of each region of interest of the identification target image, and When the second and third content information of each of the second and third comic contents is extracted one by one, the content information extracted corresponding to the identification target image is identified, and then the mutual content is extracted. The first content information that has been detected (or identified) most frequently corresponding to the identification target image in comparison can be provided as content information corresponding to the identification target image.

  In addition, the control unit 150 may correspond to the information of the partial feature points from the characteristic point DB 102 for a part of the information of the plurality of different characteristic points generated corresponding to the identification target image. Extracting one or more pieces of content information, comparing the one or more pieces of content information, and providing the most detected (or identified) content information as content information corresponding to the identification target image. You can also.

  In addition, the control unit 150 may set various conditions for identifying the content obtained by duplicating the identification target image. Can be provided.

  FIG. 10 is a table illustrating a comparison between the performance of an online comics identification system based on a region of interest according to an embodiment of the present invention and an existing feature point extraction method. As shown in FIG. Compared to the existing algorithm based on the frequency characteristic-based feature point extraction method, the number of feature points per web comic, which is online comic content, is about 61 (on average 2.87%), which is very small. Even when the feature point DB 102 of 85.6 times the web comic is constructed as compared with the conventional algorithm, the extraction time and the identification time of the feature points are 0.517 seconds on average and 2.34 times faster than the existing algorithm. In addition, it can be seen from the illustrated table that the overall identification speed requires an average of 0.83 seconds and less than 0.38 seconds.

  In other words, the present invention can reduce the number of feature points extracted from content in one round constituting online comic content, and can improve the identification processing speed and accuracy more than the existing method, while reducing the number of feature points less than the existing method. Instead, even if the amount of stored data stored in the DB is larger than the existing one, it is possible to provide a faster identification performance than the existing one.

  FIG. 11 is a flowchart illustrating a method of identifying an online comic based on a region of interest according to an exemplary embodiment of the present invention. The server including the DB in which the information of the reference feature point at the point is matched with the content information of each time can receive the identification target image from the outside (S1).

  At this time, the information of the reference feature points may be stored in the DB for each region of interest, and the information of the reference feature points may be stored in the DB in matching with the content information of a specific round. Can be Further, the DB can be configured to include the feature point DB 102 or the feature point DB 102.

  In addition, the server may extract a region of interest through edge detection in the identification target image (S2).

  Thereafter, the server may generate a normalized image in which the region of interest is normalized by a preset normalization size (S3).

  In addition, the server may divide the normalized image into a plurality of blocks of a predetermined size and calculate one or more dominant gradients for each block based on a HOG (Histogram of Oriented Gradient) ( S4).

  Thereafter, the server may convert the one or more dominant gradients of the blocks constituting the region of interest into N-bit feature points obtained by binarizing according to a predetermined reference (or algorithm). (S5).

  Accordingly, the server compares the information on the feature points with the information on the reference feature points (S6), and if there is information on the reference feature points that matches the information on the feature points, the information on the reference feature points It is possible to extract from the DB content information that matches the identification information, determine the identification target image as content corresponding to the content information, and provide the user with the content information as content associated with the identification target image ( S7).

  The various devices and components described herein may be embodied by hardware circuits (eg, CMOS logic), firmware, software, or a combination thereof. For example, the present invention can be implemented using transistors, logic gates, and electronic circuits in various electrical structures.

  The contents described above can be modified and varied without departing from the essential characteristics of the present invention, provided that those having ordinary knowledge in the technical field to which the present invention pertains. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical events of the present invention, but to explain them, and the scope of the technical events of the present invention is limited by such embodiments. It will not be done. The protection scope of the present invention should be construed according to the following claims, and all technical events falling within the equivalent scope should be construed as being included in the scope of the present invention.

101 Content DB
102 Feature point DB
110 region of interest extraction unit 120 normalization unit 130 video analysis unit 140 feature point extraction unit 150 control unit

Claims (9)

A region-of-interest extraction unit for extracting a region of interest from one or more original images constituting online comic content through edge detection;
A normalization unit that generates a normalized image in which the region of interest is normalized by a predetermined normalization size;
An image analyzer that divides the normalized image into a plurality of blocks of a predetermined size and calculates one or more dominant gradients for each block based on a Histogram of Oriented Gradient (HOG);
A feature point extraction unit that generates information of N-bit feature points obtained by binarizing the one or more dominant gradients for each of the blocks constituting the region of interest in accordance with a preset reference, corresponding to the region of interest; And setting the feature point information as reference feature point information, matching and storing the content information of the online manga content corresponding to the original image, and receiving the identification target image, the interest region extracting unit; The control unit controls the normalization unit, the video analysis unit, and the feature point extraction unit to generate feature point information corresponding to the identification target image, and compares the feature point information with the reference feature point information. A control unit for providing content information matching the information of the feature point in correspondence with the image to be identified. The online cartoon identification system based on a region of interest according to claim 1, wherein the content information includes information on a specific time of the online cartoon content. The region of interest extracting unit may scan the original image horizontally and vertically to detect edges, and when there are a plurality of original images constituting the online comic content, connect the original images in order to form a single image. The system of claim 1, further comprising detecting the edge after generating the image. The region of interest extraction unit, if there is a specific region of interest in contact with the upper end of the original image, or the lower end, excluding the specific region of interest, or from the other original image continuous with the original image, the specific region of interest The online cartoon identification system based on a region of interest according to claim 3, wherein a region of interest that is continuously connected to the region of interest is connected to the specific region of interest, and is set and extracted as one region of interest. The feature point extraction unit divides the unit block into a predetermined number of cells, combines the gradients calculated for each unit cell based on the HOG, calculates the magnitude of the gradient for each direction, and sets the gradient in advance. The online cartoon identification system based on a region of interest according to claim 1, wherein one or more dominant gradients having a size equal to or greater than the reference size are calculated corresponding to the unit block. The feature point extraction unit determines whether or not there is a dominant gradient for each of a plurality of angular ranges set in advance according to the direction of each of the one or more dominant gradients, and determines whether or not the dominant gradient exists for each of the plurality of angular ranges. The online cartoon identification system based on a region of interest according to claim 1, wherein the system performs an evolution process. The online cartoon identification system based on a region of interest according to claim 1, further comprising a storage unit that stores the information of the reference feature point and the content information in a matching manner under the control of the control unit. When a plurality of regions of interest are included in the identification target image, the control unit identifies content information based on information of a reference feature point that matches information of each feature point of the plurality of regions of interest. The interest according to claim 1, wherein when a plurality of different pieces of content information are identified corresponding to the region of interest, the most identified content information is provided as content information corresponding to the identification target image. Online cartoon identification system based on domain. For each of the different online comic contents, the information of the reference feature points in the feature points of each region of interest of the original image corresponding to each time is matched with the content information of each time of the server. In an online comics identification method based on a region of interest,
Receiving an image to be identified and extracting a region of interest via edge detection in the image to be identified;
Generating a normalized image in which the region of interest is normalized by a predetermined normalization size;
Dividing the normalized image into a plurality of blocks of a predetermined size, and calculating one or more dominant gradients for each of the blocks based on Histogram of Oriented Gradient (HOG);
Generating, corresponding to the region of interest, information of N-bit feature points obtained by binarizing the one or more dominant gradients for each of the blocks constituting the region of interest in accordance with a preset criterion; Point information is compared with the reference feature point information. If there is reference feature point information that matches the feature point information, content information matching the reference feature point information is extracted from the DB. Determining the image to be identified as the content corresponding to the content information.

Images