KR101541495B1 - Apparatus, method and computer readable recording medium for analyzing a video using the image captured from the video - Google Patents

Abstract

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving image analyzing apparatus using a captured image for collecting a screen image captured from a moving picture through a network and analyzing information related to the collected images, A video frame extracting unit for extracting a frame of the video frame; An extraction frame quantization unit for quantizing characteristics of each frame extracted through the video frame extraction unit; An image quantization unit for quantifying a characteristic of at least one collected object image to be searched; An image comparison search unit for comparing the search target image with at least one frame extracted from the plurality of frames by numerical values; And a search result processing unit for mapping the related information of the acquired search target image to the matching frame and storing the same in a database when there is a frame matching the search target image as a result of the comparison.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving image analyzing apparatus, a method, and a computer readable recording medium using a captured image.

The present invention relates to an apparatus for analyzing images captured from a moving image, and more particularly to a system and method for capturing a screen image captured from a moving image through a network and using a captured image for analyzing information associated with the collected images. A moving picture analyzing apparatus, a method, and a computer-readable recording medium.

In general, the Internet is an open network configured to freely connect and use a common protocol called TCP / IP to a remote computer to be accessed anywhere in the world and to anyone. It is an open network that not only transmits basic character information but also develops compression technology, And various services such as e-mail, file transfer, and World Wide Web (WWW), which are used for delivering the service, can be used.

As the use of the internet has rapidly increased in Korea and the world, the importance of the Internet has been rapidly increasing as a strategic tool for improving efficiency and productivity throughout the existing industries. As a result, new business opportunities through the Internet have been continuously created As the area is expanding, the number of Internet-based operators is also increasing.

Meanwhile, in recent years, a large amount of individual video contents (for example, UCC (User Created Contents)) has been produced due to the proliferation of digital devices such as smart phones and the like. TV broadcasts and movie contents are also transmitted through VOD It is becoming common to watch on the web.

In order to smoothly consume such video contents, various video meta information including information on a specific scene is required to enable searching and browsing of a desired content. For example, in the current video search or service, meta information of a text type related to a moving picture such as title, date, and number of views of a moving picture is mainly used, and the popularity of the moving picture is grasped by using the number of views of the moving picture.

However, as a method of searching for a moving picture by utilizing only the meta information in the form of text, it is impossible to know what kind of scene is actually a popular scene to be missed in the moving image. In the case of posting by linking to a moving image Except, you can not figure out how much you are talking about this video from outside (e.g., other websites).

In addition, users upload a specific screen image captured from a video, not a video itself, to the web, and use the SNS (Social Network Service) site such as Facebook, Twitter, I often talk to other users.

Particularly, it is difficult to post the content directly to a website or the like for content that is copyrighted or reproducible by login, and it is not easy to acquire a unique URL for a specific scene of a video to be talked about. Therefore, And captures only certain scenes of a movie and uses it as an article.

In this way, information including rich meta information related to a moving image that is distributed along with an image captured from a moving image can not be directly connected to the moving image.

On the other hand, as a technique for providing relevant information on such video contents, source information is generated from a moving picture production in Korean Patent Registration No. 10-0857381 entitled " Content Service System and Method (NHN Co., Ltd.) " A content service method capable of storing source data in a database and calculating a similarity with scene data and extracting a source.

However, these conventional methods merely provide meta information about a moving picture, and there is a limitation in that the main scenes can not provide relevant information about the captured image.

Accordingly, there is a need for a method for effectively utilizing various related information about each image captured from a moving image, thereby improving the utilization of the moving image.

[Patent Document 1] Korean Patent Registration No. 10-0857381 Content Service System and Method (NHN Co., Ltd.) 2008.09.01

An object of the present invention is to provide a moving image analysis apparatus and method using a captured image that can provide various meta information about a moving image by comparing each frame extracted from the moving image with captured images collected through an external web site, And a computer readable recording medium.

It is another object of the present invention to provide a capture apparatus and a capture apparatus capable of analyzing information such as news, a blog, an SNS post, and the like, by comparing each frame extracted from a moving image with a captured image collected through an external website, And a computer-readable recording medium.

It is another object of the present invention to provide a method and system for comparing a captured image obtained through a video from an external web site with a captured image, A method for analyzing a moving image using the captured image, and a computer-readable recording medium.

In order to achieve the above-described object of the present invention and to achieve the specific effects of the present invention described below, the characteristic structure of the present invention is as follows.

According to an aspect of the present invention, a moving image analyzing apparatus using a captured image includes: a video frame extracting unit extracting at least one frame from a moving image composed of a plurality of frames; An extraction frame quantization unit for quantizing characteristics of each frame extracted through the video frame extraction unit; An image quantization unit for quantifying a characteristic of at least one collected object image to be searched; An image comparison search unit for comparing the search target image with at least one frame extracted from the plurality of frames by numerical values; And a search result processing unit for mapping the related information of the acquired search target image to the matching frame and storing the same in a database when there is a frame matching the search target image as a result of the comparison.

Preferably, when the image difference between a plurality of frames among a plurality of consecutive frames extracted through the video frame extracting unit falls within a preset predetermined range, the apparatus selects a frame of a selected one of the plurality of frames, As a representative frame, and the extracted frame quantization unit digitizes the representative frame determined through the representative frame extraction unit and stores the numerical value in a database.

Preferably, the representative frame extractor determines a selected one of the plurality of frames as a representative frame when the difference value of the inter-frame gray histogram for the plurality of frames is less than a predetermined value.

Preferably, the representative frame extracting unit determines at least one frame as a representative frame every predetermined time.

Preferably, the extracted frame quantization unit quantizes the invariant points of each frame, and then expresses the invariant points of each frame by the frequency of visual words converted into cluster values that are defined in advance.

Preferably, the image quantization unit quantizes the invariant points of each image, and then expresses the constants by frequency numbers of visual words converted into cluster values defined in advance.

Preferably, the image comparison / search unit compares the visual word of each frame with the visual word of each image, and determines that the corresponding image matches the visual word if the number of visual words is equal to or greater than a predetermined number.

Preferably, the related information of the search target image is at least one selected from the source where the collected image is posted, the number of retrits on the social network service of the image, the number of comments, the recommendation number and the evaluation information.

Preferably, the apparatus further includes a result analysis processing unit for automatically calculating a statistical value for each frame of the specific moving image from the search result stored by the search result processing unit and generating an analysis result.

Preferably, the result analysis processing unit provides any one selected from news, a blog, a number of citations in an SNS site, and corresponding link information for each frame of a specific video in response to a result analysis request.

Preferably, the result analysis processing unit calculates and provides ranking of each frame from the number of citations in news, blog, and SNS site referring to the corresponding image in the video for each frame of the specific video according to the result analysis request.

According to another aspect of the present invention, there is provided a method of analyzing moving images using a captured image, comprising: extracting at least one frame from a moving image composed of a plurality of frames; Digitizing a feature of the extracted frame; Quantifying characteristics of at least one collected retrieval object image; Comparing the search target image with at least one frame extracted from the plurality of frames by numerical values; And mapping the related information of the acquired search target image to the matching frame if there is a frame matching the search target image as a result of the comparison, and storing the mapped related information in the database.

Preferably, if the image difference between the plurality of frames among the extracted plurality of consecutive frames falls within a preset predetermined range, the step of extracting the selected one of the plurality of frames, As a representative frame, and the step of digitizing the feature of the extracted frame digitizes the determined representative frame.

Preferably, when the difference value of the inter-frame gray histogram for the plurality of frames is equal to or smaller than a predetermined value, the step of determining as the representative frame determines the selected one frame among the plurality of frames as a representative frame.

Preferably, the step of determining as the representative frame determines at least one frame as a representative frame every predetermined time.

Preferably, the step of quantizing the characteristics of the extracted frame is performed by expressing the constants of each frame as numerical values and then expressing the frequency of the visual words converted into a cluster value defined in advance.

Preferably, the step of digitizing the feature of the image to be searched for is performed by expressing the invariant points of the respective images, and then expressing the constants of the images by the frequency of the visual words converted into the cluster values defined in advance.

Preferably, the comparison step compares the visual word of each frame with the visual word of each image, and determines that the corresponding image matches the corresponding visual word if the number of visual words is equal to or greater than a predetermined number.

Preferably, the related information of the search target image is at least one selected from the source where the collected image is posted, the number of retrits on the social network service of the image, the number of comments, the recommendation number and the evaluation information.

Preferably, the method further includes the step of automatically calculating a statistical value for each frame of a specific moving image from the stored search result to generate an analysis result after the step of storing in the database.

Preferably, the step of generating the analysis result may include the step of selecting one of the news, the blog, the number of citations in the SNS site, and the corresponding link information for each frame of the specific video, to provide.

Preferably, the step of generating the analysis result includes calculating a ranking of each frame based on the number of citations in the news, blog, and SNS site referring to the corresponding image in the video for each frame of the specific video according to the result analysis request do.

Meanwhile, the information for providing the moving image analysis method using the captured image may be stored in a recording medium readable by the server computer. Such a recording medium includes all kinds of recording media in which programs and data are stored so that they can be read by a computer system. Examples include ROMs (Read Only Memory), Random Access Memory, CD (Compact Disk), DVD (Digital Video Disk) -ROM, magnetic tape, floppy disk, optical data storage device, (For example, transmission over the Internet). Such a recording medium may also be distributed over a networked computer system so that computer readable code in a distributed manner can be stored and executed.

As described above, according to the present invention, it is possible to collect images captured from a moving image and accumulate various social meta information about the moving image using web contents linked to the collected images. In addition, by accumulating such social meta information, rankings and popular scenes for moving images can be distinguished.

In addition, according to the present invention, after extracting image frames embedded in a moving image, information on news, blog, and SNS post including the image is collected, Scene or the like can be detected. In addition, according to this analysis, there is an advantage that the social ranking can be analyzed as to which video contents are popular in the social network service and the like.

1 is a diagram illustrating a concept of moving image analysis using a captured image according to the present invention.
2 is a block diagram illustrating a detailed configuration of a moving image analyzing apparatus using a captured image according to an exemplary embodiment of the present invention.
3 is a graph showing a gray histogram of a specific frame for representative frame extraction according to an exemplary embodiment of the present invention.
4 is a diagram illustrating a frame comparison for selecting a representative frame according to an embodiment of the present invention.
5 is a diagram illustrating a method for selecting a representative frame according to an embodiment of the present invention.
6 is a diagram showing a concept of a method of digitizing representative frames according to an embodiment of the present invention.
7 is a flowchart illustrating a moving picture analysis procedure using a captured image according to an embodiment of the present invention.
8 to 10 are views showing an example of a moving image analysis result using a captured image according to an embodiment of the present invention.

The following detailed description of the invention refers to the accompanying drawings, which illustrate, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different, but need not be mutually exclusive. For example, certain features, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with an embodiment. It is also to be understood that the position or arrangement of the individual components within each disclosed embodiment may be varied without departing from the spirit and scope of the invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is to be limited only by the appended claims, along with the full scope of equivalents to which the claims are entitled, if properly explained. In the drawings, like reference numerals refer to the same or similar functions throughout the several views.

The present invention relates to a video capture image analyzing apparatus capable of providing various meta information about a main frame of a moving image and a moving image by comparing a screen image captured from a moving image with an external server or the like, .

That is, in the present invention, an image captured from a specific moving picture is collected from an external server (e.g., a social network service (SNS) site, a blog, a portal site, etc.) and compared with a plurality of frames extracted from the entire moving picture Thereby determining which frame of the moving image the collected image corresponds to.

Accordingly, when a frame matching the collected specific image is searched, related information (e.g., linked web content information) of the collected image is mapped and stored for the corresponding frame to perform various analyzes on the video content have. In this case, the related information may be a source of the content (that is, a posted web site), a number of the posted image, and the like, and the ranking of the specific image in the video or the video, , The number of citations for each source can be calculated.

In the meantime, the 'external server' in the following description is a server that provides various web services separately from the video capture image analysis apparatus that performs video analysis according to the present invention, and is a device physically or logically divided, And may be connected to the image analysis apparatus through a predetermined wired / wireless communication network.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, so that those skilled in the art can easily carry out the present invention.

Captured  Concept of video analysis using images

1 is a diagram illustrating a concept of moving image analysis using a captured image according to the present invention. Referring to FIG. 1, a plurality of frames are extracted for at least one moving image. Then, the extracted feature values of each frame are digitized and stored in the database. In this case, it is preferable that a representative frame is extracted from a plurality of frames according to an embodiment of the present invention, and only the extracted representative frame is digitized and stored.

On the other hand, a crawler collects postings attached with images captured from an external server (for example, a web server providing news, blog, Twitter, Facebook, etc.) and stores the postings as search candidates in a database. Then, a matching representative frame among the representative frames extracted from the moving image is retrieved for each of the collected images through a search engine.

And a post database for each scene referring to a topic-specific topic scene database and a topic scene according to the retrieved result. Further, based on this, various analyzes can be performed such as calculating a ranking of a moving image, selecting a topic scene in a moving image, and the like.

More specifically, when a representative frame corresponding to a specific image is searched for as a result of the searched result, various information collected together with the collected image (for example, the corresponding collected image is posted The number of references, the number of references to the representative frame, the degree of spread or preference of the image (e.g., the number of retrits (RT), the number of comments, the number of votes, etc.) are mapped and stored in the database.

Accordingly, as shown in FIG. 8 to FIG. 10, a scene of a topic (i.e., a frame of a topic) among the representative frames extracted from a specific moving picture, a list of posts that mention the scene of the topic, can do.

Hereinafter, an apparatus and method for analyzing a moving image using a captured image according to an embodiment of the present invention will be described in detail with reference to FIG. 2 to FIG.

Captured  Image analysis device using image

2 is a block diagram illustrating a detailed configuration of a moving image analyzing apparatus using a captured image according to an exemplary embodiment of the present invention. Referring to FIG. 2, the moving picture analyzing apparatus according to an embodiment of the present invention includes a video frame extracting unit 210, a representative frame extracting unit 220, an extracted frame quantizing unit 230, an image information collecting unit 240, A digitization unit 250, an image comparison / search unit 260, a search result processing unit 270, and a result analysis processing unit 280. The moving picture analysis apparatus may include at least one of a moving picture content database 291, a frame feature information database 292, an image feature information database 293, an image related information database 294 and an analysis result information database 295 Databases.

At least one moving image may be stored in the moving image content database 291, for example, in the form of a file. At this time, the moving picture may be configured to include a plurality of frames (e.g., 30 frames per second, etc.). On the other hand, the moving picture can be applied to the present invention in any kind and in any format. For example, any kind of moving picture such as a broadcast moving picture, a UCC moving picture, a movie moving picture, and the like can be applied to the present invention. In addition, various related information (e.g., meta information) about the moving image to be stored may be mapped and stored in the moving image content database 291. [ For example, information such as the source of the moving image, the playback time, the broadcast time, the broadcast channel, and the file format may be stored together.

The video frame extracting unit 210 selects a specific moving image stored in the moving image content database 291 and extracts a plurality of frames included in the selected moving image.

The representative frame extracting unit 220 selects a representative frame to be used as a main frame among a plurality of frames extracted through the video frame extracting unit 210 according to an embodiment of the present invention. That is, since frames extracted through the video frame extracting unit 210 have strong redundancy (or similarity) between adjacent frames, it is preferable to select and process a representative frame according to an embodiment of the present invention for database configuration and search efficiency desirable. However, the present invention is not limited to this, and the following procedure may be implemented for a whole frame in the extracted moving image or a frame matching a specific condition in the moving image.

In this case, the representative frame extraction method can be variously implemented. When the difference value of the inter-frame gray histogram is equal to or smaller than a predetermined value as in the embodiments described later, the frame is excluded because it is an image having high redundancy. Since the histograms may be different from each other with a small difference, the representative frames may be forcibly selected at predetermined time intervals. A detailed description thereof will be given later in the description of Figs. 3 to 5.

When the representative frame is determined by the representative frame extracting unit 220 in this way, the extraction frame quantization unit 230 digitizes the characteristics of each representative frame and stores it in the frame characteristic information database (282). Methods for digitizing the characteristics of each representative frame may be variously implemented, or a known image analysis method or the like may be used. In the present invention, an example using a visual words method is described as an example of a method of digitizing the characteristics of the representative frame, but the present invention is not limited thereto. A detailed description of the visual word method will be described later with reference to FIG.

The image information collecting unit 240 collects a post attached with an image (for example, an image captured from a specific moving image) from various external servers (e.g., portal site server, SNS server, blog server, etc.). At this time, the image information collecting unit 240 may be implemented using a crawler or the like, but the present invention is not limited thereto. The crawler is a type of software used by a site that operates a search engine to automatically search and index various information on the web, and is also called a spider, a bot, or an intelligent agent.

Meanwhile, the image information collecting unit 240 may collect the postings to which the image is attached when the collected images include the source, the diffusion or preference degree of the image (for example, the number of retrits (RT), the number of comments, Evaluation information (for example, recommendation number, positive / negative number (or positive / negative number), etc.), and the like. The image related information collected by the image information collecting unit 240 may be stored in the image related information database 294 in link with the corresponding image.

The image quantization unit 250 digitizes the characteristics of each image collected through the image information collection unit 240 and stores the quantities in the image feature information database 293. At this time, it is preferable to use the same method as that of the extraction frame quantization unit 230, and the present invention is not limited thereto. That is, various methods for digitizing the characteristics of each of the collected images may be implemented, and known image analysis methods may be used. In the present invention, an example of using the visual words method is described as an example of a method of digitizing the characteristics of the collected image, but the present invention is not limited thereto.

When the feature quantification for the representative frame is completed and the feature quantification process for the collected image is completed, the image comparison search unit 260 compares the digitized value of the specific image with the representative value of the representative frame stored in the frame feature information database 292 It is compared with the numerical value. As a result of the comparison, a representative frame having a matching value or a representative frame having the most similar value is searched. When the visual word method described below is used to quantify the feature, a representative frame having the largest number of visual words identical to the image is searched, or the number of times the same word appears is defined as a distance between images, The corresponding frame can be searched within the distance.

When a frame matching the specific image collected by the image comparison and search unit 260 is searched, the search result processing unit 270 transmits information collected in association with the matching image to the searched corresponding frame, (294), and stores it in the image analysis information database (295).

When a request for information on a moving image is received according to the embodiment of the present invention, the result analysis processing unit 280 extracts statistical values for each main frame for a specific moving image from the search results stored in the image analysis information database 295 Automatically generates the analysis result, and provides the generated result to the requesting user. For example, as shown in FIG. 8 to FIG. 10, various related information about a main image (or frame) of a specific moving picture (e.g., news referring to the corresponding image in the moving picture, A number of blogs, SNS sites, etc., and corresponding link information), and provide the generated information to the user.

Representative frame extraction method

3 is a graph showing a gray histogram of a specific frame for representative frame extraction according to an exemplary embodiment of the present invention. Referring to FIG. 3, a representative frame is extracted through a gray histogram difference value between frames according to an exemplary embodiment of the present invention.

That is, as shown in FIG. 3, a gray histogram for each frame is calculated, and when the difference value of the gray histogram between consecutive frames is equal to or smaller than a preset predetermined value, it is determined that the image has strong redundancy (or similarity). Therefore, only one frame is left as a representative frame, and the remaining one or more redundant frames are deleted. Accordingly, only a specific frame satisfying a condition among a large number of frames included in one moving picture is extracted and processed as a representative frame, thereby making it possible to efficiently perform database configuration and search.

The gray histogram can be calculated, for example, as shown in FIG. More specifically, the gray value of each pixel constituting the frame is calculated, and it is calculated whether or not it belongs to a previously-classified section, and the ratio is calculated. Accordingly, the distribution of gray values of the entire frame image can be calculated as shown in FIG.

4 and 5 show a method of selecting a representative frame using the gray histogram and the like. That is, referring to FIG. 4, it can be seen that the 410th frame and the 420th frame are consecutive frames, and the difference between the gray histograms is less than a preset value and the redundancy is strong. Therefore, the 420th frame is selected as the representative frame except the 410th frame. Similarly, it can be seen that the 430 and 440 frames are consecutive frames, and the difference between the gray histograms is less than a predetermined value and the image is highly redundant. Therefore, frame 440 except frame 430 is selected as the representative frame. If the gray histogram difference of three or more consecutive frames is equal to or less than a preset value in the same manner, only one frame may be left as a representative frame, and the remaining two frames may be excluded.

On the other hand, in a special case, the difference of the gray histograms as described above may be equal to or less than a predetermined value, but may be different images. Therefore, according to the embodiment of the present invention, even if the gray histogram difference is less than the predetermined value, it can be forcibly selected as the representative frame once every predetermined time (for example, 1 second). In other words, referring to FIG. 5, the gray histograms of all the frames 510 to 550 are shown below the preset values, but the images of the 520 and 530 frames may be different images. Therefore, according to the embodiment of the present invention, the representative frame can be necessarily selected every predetermined period.

Representative frame quantification method

4 and 5, the features of each selected representative frame are digitized and stored in a database. 6 is a diagram showing a concept of a method of digitizing representative frames according to an embodiment of the present invention.

That is, in order to compare the representative frame and the collected image according to the embodiment of the present invention, each representative frame and each of the collected images are digitized by their feature values, and stored and compared. A visual words method can be used as an example of a method of digitizing the image to a feature value.

The visual word method is a method in which an invariant point of an image or a frame is obtained, the invariant points are expressed numerically and converted into a predefined cluster value in order to facilitate the search.

More specifically, as shown in FIG. 6, each cluster is divided into a predetermined number using a plurality of training images in advance. At this time, these unit clusters are referred to as visual words. Next, an invariant point is first obtained for one image, and a descriptor is obtained as a statistic that can characterize the invariant point. Since this descriptor is a kind of vector, find the closest cluster to which this vector belongs and set it as the feature value of this point. By doing so, as shown, one image can be represented as a histogram of feature values that can be expressed as freqununcy values of a visual word.

By using this method, each representative frame included in the moving picture can be digitized and stored. The method is illustrated to facilitate understanding of the present invention, and the present invention is not limited to the method.

In FIG. 6, each cluster of the histogram representing the feature value may be regarded as a visual word representing an image (or a frame), and finally, one image may include at least one (preferably plural) visual words And the frequency of each visual word.

Accordingly, when a query image is acquired by collecting an image by the image information collecting unit 240, a visual word representative of the image is obtained by, for example, the above-described method, and compared with the visual words of each frame stored in the database . At this time, the frame having the largest number of the same visual words can be regarded as the same frame as the query image by borrowing the method used in the information search. In other words, by storing the database in which the movie is stored in the reverse file format in a manner similar to a general information search method, the most similar images can be retrieved in a short time with respect to the query image. In addition, the number of times the same visual word appears is defined as a distance between images, and only an image of a certain distance or less can be regarded as the same frame.

In this manner, when the same frame among the representative frames extracted from the moving image is retrieved for each of the collected images, various related information about the image collected together with the image in the retrieved frame (for example, (E.g., number of retrets (RT), number of comments, various evaluation information (e.g., recommendation number, positive / negative number (or positive / negative number), etc.), etc.) in the retrieved frame And stores them.

Captured  How to analyze videos using images

7 is a flowchart illustrating a moving picture analysis procedure using a captured image according to an embodiment of the present invention. Referring to FIG. 7, the representative frame is first extracted from the moving picture contents and stored in the database (S701) as described above. Then, the features of each extracted representative frame are digitized and stored in the database (S702).

In addition, postings to which images are attached from external servers are collected (S703), the characteristics of the collected images are digitized and stored in the database (S704).

Then, in step S705, a representative frame matching the collected image is searched for from each representative frame stored in the database according to an embodiment of the present invention.

If a matching representative frame is searched (S706), the related information of the collected image is mapped to the searched representative frame and stored in the database (S707).

Of video analytics Example

8 to 10 are views showing an example of a moving image analysis result using a captured image according to an embodiment of the present invention.

Referring to FIG. 8, a video capturing screen may be attached to a web, a news, an SNS, or the like, or a reference of the capturing screen may be provided according to the number of times mentioned.

For example, if a title of a TV broadcast such as a 'healing camp' is input 800 as a search term on a search portal site, statistical information referring to a specific scene captured in each day-by-day broadcast image according to an embodiment of the present invention, By providing ranking information, it is possible to know which scene of a certain broadcast date is the most talked about scene. That is, it is possible to capture a specific scene from the corresponding broadcast moving pictures, post it, refer to it, or provide the captured image 810 and related information 820 of the scene sequentially according to the number of times mentioned. For example, among the broadcasts of the program searched on June 9, 2012, images captured at 14 seconds are displayed at the top of the search results because they are the most talked about or referenced images. At this time, the number of times the scene is referred to (821 to 824) along with the captured image 810 can be displayed for each source (e.g., news, blog, twitter, facebook, etc.). Likewise, a captured image of the next broadcasted and referenced broadcast can be displayed below it.

The ranking determination method for each scene may be variously implemented, and may be determined by summing up the number of postings or referrals for each source, or may be determined by adding weights to each source and summing them.

On the other hand, if the 'video scene description' portion indicated by the related information on the specific captured image is selected on the search result screen, the source information on which the scene is referred to can be provided as shown in FIG.

That is, referring to FIG. 9, a list 900 referring to a corresponding scene in the moving image may be provided for each source. For example, each scene 910 in which the scene is mentioned in the news, each source 920 mentioned in the Twitter, each source mentioned in the blog, each source mentioned on Facebook, and the like may be provided. At this time, if the respective source information is selected, the corresponding scene may be directly connected to corresponding posts such as the referred news, blog, twitter, facebook, and the like.

As described above, according to the embodiment of the present invention, scenes that have become topics by a specific source (news, blog, cafe, SNS, etc.) are classified and social analysis of spread or preference degree (RT, comment, Provide popular scene ranking or. Or as meta information for rankings of search results as shown in FIG.

Meanwhile, as another embodiment of the present invention, scenes of a topic may be listed and provided for a specific moving picture as shown in FIG. That is, the topic information may be tagged in a moving picture, and the service may be provided by analyzing the moving picture based on the moving picture such as the <popular scene> or the <topic scene>.

For example, referring to FIG. 10, when the user selects a specific video replay service, the video replay player 1000 is displayed at the top and the scene ranking 1010 of the topic is displayed at the bottom, Can be provided. That is, the rankings for each scene in the moving image calculated as described above can be calculated according to a predetermined method, and the scene having the highest ranking can be displayed in that order. At this time, it is possible to provide the scene-specific scene information and the statistics for each scene and the posted list of the scene. Therefore, if the user selects the video play display 1020 in the information about the scene, the user can jump to the corresponding scene in the top moving image reproduction player 100 and reproduce the selected scene. In addition, if a specific source of a specific scene is selected from the scene link 1030, a list of posts and a link including capture of the scene at the source can be provided as shown in FIG. For example, if the twitter 1040 of the screen ranked in the third rank is selected, the twitter can provide a list and a link to which the corresponding scene is posted.

Also, as another embodiment of the present invention, it is possible to provide a moving image of a captured image captured by the acquaintance of the acquaintance, or to provide other processed related information. For example, the user can analyze a relationship of a specific user or a content subscription status to provide a scene posted by an acquaintance in the SNS, a scene mentioned in a news service to which the subscriber subscribes, and a link of a content related to the scene. In addition, it is also possible to collect evaluation information such as positive or negative of the content link, analyze the information, and analyze and provide the response of the corresponding scene.

The various operations and functions as described herein with respect to the various methods may be performed by any of a number of types of functions, such as a particular processing function and / or a processing function implemented therein, and / &Lt; / RTI &gt; For example, such functions may be described herein, as well as performing various operations and processes as described herein, or any other operations and functions as described herein, etc., or the like, And may perform such operations, processes, and the like.

In some embodiments, such functions (which may be implemented on the same device or on distinct devices) may be implemented in such a manner and in accordance with various aspects of the present invention, etc., and / Operations and functions, and their equivalents. In some embodiments, such processing is performed together by the first functional portion in the first device and the second functional portion in the second device. In other embodiments, such processing, operations, etc. are performed entirely by the processing units within one particular apparatus. Even in other embodiments, such processing and operations are performed using at least a first functional portion and a second functional portion within a single device.

Also, in the foregoing description, the terms &quot; system, &quot; &quot; device, &quot; and &quot; section &quot; may be a single processing device or a plurality of processing devices. Such a processing device may be a microprocessor, a microcontroller, a digital signal processor, a microcomputer, a central processing unit, a field programmable gate array, a programmable logic device, a state machine, a logic circuit, an analog circuit, a digital circuit, and / (Analog and / or digital) based on hard coding of operational instructions. The processing module, module, processing circuit, and / or processing unit may be implemented as a single memory device, a plurality of memory devices, and / or a combination of the processing modules, the processing circuitry, and / Lt; / RTI &gt; memory and / or an integrated memory element. Such a memory device may be any device that stores read only memory (ROM), random access memory (RAM), volatile memory, non-volatile memory, static memory, dynamic memory, flash memory, cache memory, and / . If the processing module, the module, the processing circuit, and / or the processing unit include one or more processing devices, the processing devices may be centrally located (i.e., directly or indirectly via a wired and / Connected), distributed (e.g., cloud computing via an indirect connection over a local and / or wide area network). If the processing module, module, processing circuit, and / or processing unit implements one or more of those functions through a static machine, analog circuitry, digital circuitry, and / or circuitry, memory and / It should further be noted that the corresponding operating instructions may be internal or external to the circuit including the static machine, analog circuitry, digital circuitry, and / or logic circuitry. The memory element may be stored and the processing module, module, processing circuitry, and / or processing unit may be configured to store a hard-coded and / or hard-coded data item corresponding to at least a portion of the steps and / or functions depicted in one or more of the figures It should be further noted that it carries out operational instructions. Such a memory device or memory element may be included within the components of the manufacture.

The invention has been described above with the aim of method steps illustrating the performance of certain functions and their relationships. The boundaries and order of these functional components and method steps have been arbitrarily defined herein for convenience of description. Alternative boundaries and sequences may be defined as long as the specific functions and relationships are properly performed. Any such alternative boundaries and sequences are therefore within the scope and spirit of the claimed invention. In addition, the boundaries of these functional components have been arbitrarily defined for ease of illustration. Alternative boundaries can be defined as long as certain important functions are properly performed. Likewise, the flow diagram blocks may also be arbitrarily defined herein to represent any significant functionality. For extended use, the flowchart block boundaries and order may have been defined and still perform some important function. Alternative definitions of both functional components and flowchart blocks and sequences are therefore within the scope and spirit of the claimed invention. Those skilled in the art will also appreciate that the functional components and other illustrated blocks, modules, and components herein may be implemented as illustrated or as separate components, such as semiconductor integrated circuits (ASICs) And the like, or any combination thereof.

The invention may also be described, at least in part, in the language of one or more embodiments. Embodiments of the invention are used herein to describe the invention, aspects thereof, features thereof, concepts thereof, and / or examples thereof. The physical embodiment of an apparatus, article of manufacture, machine, and / or process for implementing the invention may include one or more aspects, features, concepts, examples, etc., described with reference to one or more embodiments described herein . Moreover, in the entire drawings, embodiments may incorporate the same or similarly named functions, steps, modules, etc. that may use the same or different reference numerals, and so forth, Steps, modules, etc., may be the same or similar functions, steps, modules, etc., or the like.

In the meantime, the term " processing unit "is used in the description of various embodiments of the present invention. A module includes a functional block implemented via hardware to perform one or more functions, such as processing one or more input signals to produce one or more output signals. The hardware implementing the functional block may operate directly by combining software and / or firmware. As used herein, a module may include one or more sub-modules that are themselves modules.

While various combinations of features and specific combinations of features of the present invention are explicitly described herein, other combinations of these features and functions are likewise possible. The present invention is not limited to the specific examples disclosed herein, and explicitly incorporates these different combinations.

As described above, the present invention has been described with reference to particular embodiments, such as specific elements, and specific embodiments and drawings. However, it should be understood that the present invention is not limited to the above- And various modifications and changes may be made thereto by those skilled in the art to which the present invention pertains.

Accordingly, the spirit of the present invention should not be construed as being limited to the embodiments described, and all of the equivalents or equivalents of the claims, as well as the following claims, belong to the scope of the present invention .

210: video frame extracting unit 220: representative frame extracting unit
230: extraction frame quantification unit 240: image information collection unit
250: Image quantization unit 260: Image comparison search unit
270: search result processing unit 280: analysis result processing unit
291: Video content database
292: Frame characteristic information database
293: Image feature information database
294: Image related information database
295: Image analysis information database

Claims (23)

A video frame extracting unit for extracting at least one frame from a moving picture composed of a plurality of frames;
An extraction frame quantization unit for quantizing characteristics of each frame extracted through the video frame extraction unit;
An image quantization unit for quantifying a characteristic of at least one collected object image to be searched;
An image comparison search unit for comparing the search target image with at least one frame extracted from the plurality of frames by numerical values; And
And a search result processing unit for mapping the related information of the acquired search target image to the matching frame and storing the same in a database when there is a frame matching the search target image as a result of the comparison, .
The apparatus of claim 1,
When a difference in image between a plurality of frames among a plurality of consecutive frames extracted through the video frame extracting unit falls within a preset predetermined range, a representative frame, which determines a selected one of the plurality of frames as a representative frame, Further comprising an extracting section,
Wherein the extraction frame quantization unit digitizes the representative frame determined through the representative frame extraction unit and stores the numerical value in a database.
The apparatus of claim 2,
And determines a selected one of the plurality of frames as a representative frame when the difference value of the inter-frame gray histogram for the plurality of frames is less than a predetermined value.
The apparatus of claim 2,
And determines at least one frame as a representative frame every preset time.
The image processing apparatus according to claim 1,
Wherein an invariant point of each frame is digitized and then expressed as a frequency of a visual word converted into a cluster value defined in advance.
The image processing apparatus according to claim 1,
Wherein the invariant points of each image are digitized and then expressed as frequency numbers of visual words converted into cluster values that are defined in advance.
The image search apparatus according to claim 5 or 6,
And comparing the visual word of each frame with the visual word of each of the images to determine that the corresponding image matches the predetermined number of visual words if the number of visual words is equal to or greater than a predetermined number.
The image processing apparatus according to claim 1,
The number of comments, the number of comments, and the evaluation information on the social network service of the image, the source from which the collected image is posted, and the rating information on the social network service of the image.
The apparatus of claim 1,
And a result analysis processing unit for automatically calculating a statistical value for each frame of a specific moving image from the search result stored by the search result processing unit and generating an analysis result.
The system according to claim 9,
And provides the user with any one selected from the news, the blog, the number of citations in the SNS site, and the corresponding link information for each frame of the specific video according to the result analysis request.
The system according to claim 9,
Wherein ranking of each frame is calculated from the number of citations in news, blog, and SNS site referring to the corresponding image in the video for each frame of the specific video according to the result analysis request.
Extracting at least one frame from a moving picture composed of a plurality of frames;
Digitizing a feature of the extracted frame;
Quantifying characteristics of at least one collected retrieval object image;
Comparing the search target image with at least one frame extracted from the plurality of frames by numerical values; And
And mapping the related information of the acquired search target image to the matching frame and storing the same in a database when there is a frame matching the search target image as a result of the comparison.
13. The method of claim 12, wherein after extracting the frame,
Further comprising the step of determining a selected one of the plurality of frames as a representative frame when an image difference between a plurality of frames among the extracted plurality of consecutive frames falls within a preset predetermined range,
Wherein the step of digitizing the feature of the extracted frame is a step of digitizing the determined representative frame.
14. The method of claim 13,
And determining a selected one of the plurality of frames as a representative frame when the difference value of the inter-frame gray histogram for the plurality of frames is less than a predetermined value.
14. The method of claim 13,
And determining at least one frame as a representative frame every preset time.
The method of claim 12, wherein the step of quantizing the feature of the extracted frame comprises:
Wherein an invariant point of each frame is digitized and then expressed as a frequency of a visual word converted into a predefined cluster value.
The method according to claim 12, wherein the step of digitizing the feature of the search target image comprises:
Wherein an invariant point of each image is digitized and then expressed as a frequency of a visual word converted into a predefined cluster value.
The method of claim 16 or claim 17,
And comparing the visual word of each frame with the visual word of each of the images to determine that the corresponding image matches the predetermined number of visual words or the largest number of identical visual words.
13. The image search method according to claim 12,
The number of comments, the number of referrals, and the evaluation information on the social network service of the image, the source from which the collected image is posted, and the rating information on the social network service of the image.
13. The method of claim 12, further comprising: after storing in the database,
And automatically generating a statistical value for each frame of a specific moving image from the stored search result to generate an analysis result.
21. The method of claim 20, wherein generating the analysis results comprises:
Wherein the content providing unit provides any one selected from news, a blog, a number of citations in an SNS site, and corresponding link information for each frame of a specific video according to a result analysis request.
21. The method of claim 20, wherein generating the analysis results comprises:
Wherein ranking of each frame is calculated and provided from the number of citations in the news, blog, and SNS site referring to the image in the video for each frame of the specific video according to the result analysis request.
A program for executing the method according to any one of claims 12 to 17 and claims 19 to 22 is recorded.

Images