KR101551080B1 - Method and apparatus for providing video contents search service based on knowledge structure - Google Patents

Abstract

The present invention relates to a method and a system for searching video contents based on a knowledge structure. The method for searching video contents may include the steps of: selecting video contents corresponding to a query input by a user; obtaining a knowledge structure corresponding to each selected video content; calculating proximity between the video content and the query for each selected video content based on the knowledge structure; and determining the order of recommending the selected video contents based on the result of proximity calculation.

Description

[0001] METHOD AND APPARATUS FOR PROVIDING VIDEO CONTENTS [0002] SEARCH SERVICE BASED ON KNOWLEDGE STRUCTURE [

The present invention relates to a video content retrieval technology, and more particularly, to a video content retrieval method and system capable of creating and constructing a knowledge structure corresponding to video content and performing an image content retrieval operation using the constructed information structure.

Recently, the emergence of IPTV and smart TV, and the emergence of Web services that can view and search image contents, makes it easy to access image contents, and users are increasingly searching for desired contents.

However, the amount of content to be served increases exponentially, so that the keyword-based search used to search for image content results in a large amount of results and results that the user does not need are searched. Therefore, the search time and effort of the desired image contents are increased. To overcome this problem, research on video content recommendation and retrieval has been carried out.

In particular, in a service for providing image contents such as existing movies, considering the provision of various information (for example, genre, supervision, actor, keyword) of image contents by merely listing metadata of specific image contents, Based video content recommendation and search service are widely developed and used.

However, the meta data mapped to the image content has information representative of the corresponding content, but does not provide information related to the content content. In this case, based on the limited information included in the metadata mapped to the image content, the accuracy of the image search relatively decreases as the image search operation is performed.

On the other hand, in the case of video content such as a movie, the plot information mapped to the video content contains a simple content of the movie, unlike the metadata, such as genre, director, actor, It can deliver information representative of a specific movie.

Therefore, it is possible to search for and recommend a movie to the user through a story search in the movie metadata ontology, and it is possible to search for a movie containing similar contents although the representative metadata used when searching for the movie is different. However, even in this case, since the movie recommendation operation is performed by extracting the words included in the query word and analyzing the fact that the words are included in the story, the problem that the intention contained in the query word is not fully reflected still remains.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and it is an object of the present invention to provide a method and apparatus for generating and constructing a knowledge structure corresponding to image contents, And to provide a video content search method and system capable of performing a search operation.

The objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned can be clearly understood by those skilled in the art from the following description.

According to an aspect of the present invention, there is provided a method for searching a video content, the method comprising: selecting image contents corresponding to a query input by a user; Acquiring a knowledge structure corresponding to each of the selected image contents; Calculating proximity between image content and query terms for each of the selected image content based on the knowledge structure; And a step of determining a recommendation order of the selected image contents based on the proximity calculation result.

Acquiring the knowledge structure includes acquiring plot information corresponding to each of the selected image contents and extracting a core concept corresponding to each of the selected image contents based on the plot information; Acquiring a plurality of key concept pairs from the extracted core concept, and associating key concepts corresponding to each of the selected image contents with air information (Co-occurrence) corresponding to each of the plurality of core concept pairs Extracting; And generating a knowledge structure corresponding to each of the selected image contents based on the association between the key concepts.

The knowledge structure includes a plurality of nodes respectively corresponding to a plurality of core concepts and a plurality of links expressing the association between the plurality of nodes, and the inter-node distance is inversely proportional to the inter-node association.

Wherein the step of calculating the proximity between the image content and the query term comprises the steps of extracting words included in the query word and combining the extracted words to form a plurality of word pairs; Assigning each of the word pairs to the knowledge structure to acquire and collect word-to-word distances on a knowledge structure, and then average the words to obtain an average distance between words; And calculating the proximity between the image content and the query based on the average distance between the words.

"을 통해 상기 단어간 평균 거리(DistanceSore)를 획득하며, 상기 Q는 질의어, 상기 D는 영화 콘텐츠, 상기 (t_i, t_j)는 질의어에 포함된 단어쌍, 상기 N_t는 단어 개수, 상기 Distance(t_i, t_j)는 지식구조상에서 단어쌍(t_i, t_j)의 거리, 상기 maxDistance(D)는 지식구조의 핵심 개념간 거리의 최대값을 의미하는 것을 특징으로 한다.Wherein the step of acquiring the mean distance between words comprises:

(T _i , t _j ) is a word pair included in the query term, N _t is the number of words, and N is the number of words in the query word. Distance (t _i , t _j ) is the distance of the word pair (t _i , t _j ) in the knowledge structure, and maxDistance (D) is the maximum value of the distance between core concepts of the knowledge structure.

"을 통해 상기 영상 콘텐츠와 질의어간 근접성(Proximity)을 계산하며, 상기 a는 단어간 평균 거리와 영화와 질의어간 근접성을 반영하는 볼록 곡선의 볼록도를 조정하는 매개 변수인 것을 특징으로 한다. Wherein the step of calculating the proximity between the video content and the query term comprises "

Quot; is a parameter for adjusting the convexity of the convex curve reflecting the proximity between the word and the query distance, and the average distance between the words.

The step of selecting the image contents may include calculating a ranking score corresponding to each of a plurality of image contents through a ranking function, acquiring movie contents whose ranking score is larger than a preset value, as movies related to a query input by the user .

Wherein the step of determining the recommendation order of the selected image contents further includes a function of further considering the ranking score calculation result when determining the recommendation order of the selected image contents.

According to another aspect of the present invention, there is provided, as a means for solving the above problems, a content retrieval unit for selecting image contents corresponding to a query input by a user; An information structure managing unit for creating and managing a knowledge structure corresponding to each of a plurality of image contents; A proximity calculator for acquiring a knowledge structure corresponding to each of the selected image contents and calculating proximity between image contents and query terms for each of the selected image contents based on the acquired knowledge structure; And a movie recommendation unit for determining a recommendation order of the selected image contents based on the proximity calculation result and performing a movie recommendation operation.

In the present invention, a movie search operation is performed based on a knowledge structure that indicates what a core concept of an object to be analyzed is and how much weight is given to a core concept that indicates the association between the core concepts, .

Also, by extracting the words included in the query word and reflecting the relationship between the words, it is possible to grasp the proximity between the query word and the image content, so that the movie search result can better reflect the user intention.

FIG. 1 is a diagram illustrating a video content retrieval system according to an embodiment of the present invention. Referring to FIG.
2 is a diagram illustrating a video content search server according to an embodiment of the present invention.
FIG. 3 illustrates a method of providing a video content search service according to an embodiment of the present invention. Referring to FIG.
4 is a diagram for explaining the concept of a knowledge structure to be used in the present invention.
5 is a diagram for explaining a knowledge structure generated according to an embodiment of the present invention.
FIG. 6 is a diagram for explaining the knowledge structure obtaining step of the method of providing an image content search service according to an embodiment of the present invention in more detail.
FIG. 7 is a diagram for explaining the proximity calculation step of the image content search service providing method according to an embodiment of the present invention in more detail.
8 is a graph showing the correlation between the mean distance between words and the proximity of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other objects, features and advantages of the present invention will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which: FIG. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

The following terms are defined in consideration of the functions of the present invention, and these may be changed according to the intention of the user, the operator, or the like.

The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. These embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art to which the present invention pertains. Only. Therefore, the definition should be based on the contents throughout this specification.

FIG. 1 and FIG. 2 illustrate an image content retrieval system according to an embodiment of the present invention.

As shown in FIG. 1, the video content search system of the present invention includes a video content search server 10 for providing a movie search service, a video content search server 10 for accessing the video content search server 10, A plurality of user terminals 20, and the like.

As shown in FIG. 2, the image content search server 10 includes a screen configuration and providing unit 11 for constructing and providing a video search page, a video content related to a query input through a ranking function, A knowledge structure management unit 13 for generating and managing a knowledge structure corresponding to each of the image contents, a knowledge structure management unit 13 for acquiring knowledge structures corresponding to the selected image contents, The proximity calculation unit proximity calculation unit 14 for calculating the proximity between the image contents and the query terms for each of the selected image contents, and the recommendation order of the selected image contents based on the proximity calculation result, (E.g., metadata corresponding to the movie content, plot information, etc.) related to the image content, It may be provided with such a database 16 for storage.

Hereinafter, an operation method of the video contents retrieval system of the present invention will be described in detail with reference to FIG. 3 to FIG. In the following description, for convenience of explanation, the movie contents will be described as an example of the image contents. It is assumed that the movie content at this time is mapped with plot information that summarizes the contents of the movie.

FIG. 3 illustrates a method of providing a video content search service according to an embodiment of the present invention. Referring to FIG.

As shown in FIG. 3, the method of providing a video content search service of the present invention includes a query input step S10, a movie selection step S20, a knowledge structure acquisition step S30 of each movie, A query inter-word association relationship calculation step S40, a movie recommendation order determination step S50, a movie recommendation step S60, and the like.

In step S10, the video content search server 10 generates and provides a movie search page. If the user inputs at least one word into the query input window after accessing the query through his / her terminal 20, the query is acquired.

In step S20, the video content search server 10 analyzes all of the movie content stored in the database 16 through the ranking function to obtain a ranking score corresponding to each of the movie content, Only the movies related to the query input by the user are acquired.

As a reference, a ranking function is a function that allows the search engine to rank matching documents for a given query. For example, the Okapi BM25 algorithm exists. Okapi BM25 computes the ranking score (ie, BM25 score) by applying a stochastic model using the frequency of occurrence of words in each document. It calculates the score of the query document Q containing the words q ₁ , ..., q _n The ranking score of document D is calculated as follows.

[Equation 1]

Here, f (q _i , D) represents the frequency with which the word q _i appears in document D. | D | denotes the number of words in document D, and avgdl is the average number of words in the document group to be compared. k ₁ and b are free parameters, usually k ₁ = 2.0 and b = 0.75.

Hereinafter, for convenience of explanation, Okapi BM25 algorithm will be described as an example of a ranking function.

IDF (q _i ) is the inverse document frequency of the word q _i and is calculated as follows.

&Quot; (2) "

In the above equation, N is the number of documents in the document group to be compared, and n (q _i ) is the number of documents including the word q _i . By calculating the score of Okapi BM25 and filtering out documents below the threshold value, it is possible to obtain the effect of improving the performance by reducing the search target by excluding it from the depth test.

In step S30, the image content search server 10 acquires knowledge structures corresponding to the selected movies through step S20.

The knowledge structure is a systematic representation of associations based on key concepts and their associations that are generated when a learner learns through a certain document or medium. The lines connected between concepts are closely related to each other. . For example, if a learner has learned a document called "Composition of Computer" as shown in FIG. 4 and the core concepts in the document are Computer, CPU, Cache Memory, Main Memory and Hard Disk, Inside, the core concepts can be formed by interconnected structures, and this organizational structure can be the knowledge structure. In the present invention, the knowledge structure managing unit 13 analyzes the words included in the movie story to extract key words, identifies the association between key words using co-occurrences of key words, After creating the reflecting knowledge structure, it is used for movie search.

At this time, the knowledge structure can be represented by a plurality of nodes and a plurality of links as shown in Fig.

Each of the plurality of nodes corresponds to each of the main keywords included in the movie plot, and can be represented by various graphics (e.g., circles, squares, etc.) having a predetermined area. In addition, by changing the weight of the node (the area of the figure is proportional to the weight) in proportion to the frequency of occurrence of the keyword, the frequency of occurrence of the keyword can be easily grasped only by the weight of the node.

Each of the plurality of links represents an association between nodes, which can be represented by lines varying in thickness, color, and type according to the relationship (i.e., relevance, relevance) between the keywords linked to the link.

The distance between nodes is inversely proportional to the correlation between nodes. That is, as the distance between the nodes increases, the association between the nodes decreases.

In step S40, the proximity between the movie and the query word selected based on the acquired knowledge structure is calculated. That is, at least one word pair is created based on the words included in the query word, and then each word pair is substituted into the knowledge structure to obtain the word-to-word distance on the knowledge structure. After averaging these distances, convert them into proximity scores, and check the proximity between the movie and the query words.

In step S50, considering the proximity score calculated in step S40 or considering the BM25 score calculated in step S20 and the proximity score calculated in step S40, the recommendation order of movies selected through step S20 is determined.

In step S60, the movie contents are arranged in the recommended order determined in step S50, and the movie contents are guided through the movie search page.

FIG. 6 is a diagram for explaining a step of acquiring a knowledge structure of a method of providing an image content search service according to an exemplary embodiment of the present invention in more detail. The method of FIG. 6 is performed for each selected movie through step S20 .

In step S31, the knowledge structure corresponding to the selected movie is retrieved through step S20.

In step S32, the search result of the knowledge structure is confirmed. If the knowledge structure is found, the process proceeds to step S35; otherwise, the process proceeds to step S33.

In step S33, plot information corresponding to the movie content is obtained, morphological analysis is performed on the words included in the plot, and only nouns included in the plot information are selected. Then, based on the frequency of word usage, Extract the concept.

In step S34, a plurality of key concept pairs are acquired by grouping two words corresponding to the core concept, and the association between key concepts of the image content is extracted using the air information (Co-occurrence).

In the present invention, the air information is divided into sentence air information, which is the frequency of simultaneous occurrence of the two key concepts in the same sentence, and paragraph air information, in which the two concepts appear simultaneously in the same paragraph, Measure the relationship similarity.

In Equation (1), Sentence co-occurrence similarity (SS) is calculated using inter-sentence air information, and Equation (2) is used to obtain Paragraph co- occurrences Similarity (PS) .

[Equation 1]

&Quot; (2) "

In this case, the N _{s p} N, respectively, and the sentence number, the paragraph number in accordance with the order in which they appear in the document.

The similarity between words and words is normalized to the value between 0 and 1 by dividing the sum of the number of simultaneous appearances in each sentence or each paragraph by the maximum value of each document and paragraph air information shown in the document.

Although the similarity between words can be easily measured using the air information in the above equation, this method has a problem that the similarity relation with other words becomes higher as the word appears more frequently. To solve these problems, a method of measuring the degree of similarity of cosine, which is widely used in document clustering, is modified and used.

Sentence 1 Sentence 2 Sentence 3 ... Sentence N Wi 3 0 One ... One Wj 2 One 0 ... 2

As shown in Table 1, an ISV (Inverted Sentences Vetcor) consisting of the frequency of concepts appearing in each sentence is generated.

&Quot; (3) "

Then, using Equation (3), it is possible to measure the cosine similarity (SCS) between each concept from a single document.

In the same way, it is possible to measure the similarity of paragon cog- nocurrences cosine similarity (PSC) by replacing the sentence number in Table 1 with the paragraph number. In this way, regardless of the frequency of occurrences of words, , It is suitable for measuring the correlation between key concepts in a single document.

&Quot; (4) "

In step S35, the correlation (D _ij ) between the core concepts is converted into a 7-point scale (1: highly related, 7: not relevant at all) using Equation (5) By creating a table, applying a pathfinder algorithm, 7-scale score, etc., a knowledge structure is automatically created that connects the core concepts to each other at the shortest distance. Then, provide the generated knowledge structure to the next step.

&Quot; (5) "

In step S36, the knowledge structure retrieved through step S31 is provided to the next step.

However, when the knowledge structure is generated based on the storyline information, the accuracy of the knowledge structure may be changed according to the length of the storyline information. In other words, if the length of the plot information is too long, the length of the plot information is limited because the size of the knowledge structure becomes too large and the accuracy of the knowledge structure may deteriorate.

In addition, the plot information may include a name of an actor and a name of a play of a corresponding actor, and the user can perform a query operation using both an actor name and a play name. Accordingly, the present invention allows the user to perform a movie search operation regardless of the name of the actor and the name of the movie by mapping the actor name and the name of the play to each other.

FIG. 7 is a diagram for explaining the proximity calculation step of the image content search service providing method according to an embodiment of the present invention in more detail. The method of FIG. 7 is also performed for each selected movie through step S20.

In step S41, morphological analysis is performed on the query term to extract words included in the query term, and a plurality of word pairs are formed by combining the two words. For example, assuming that contain words of t _1, t _2, t ₃ to the query, in the present invention, _{(t 1, t 2),} (t 1, t 2), (t 1, t 2), _{(t 1, t 2),} (t 1, t 2), (t 1, t 2), (t 1, t 2), the word pairs of _{(t 1, t 2),} (t 1, t 2) Lt; / RTI >

In step S42, each of the word pairs is substituted into the knowledge structure to obtain the word-to-word distance on the knowledge structure. In step S43, the distance between words is obtained by collecting all the distances as shown in equation (6) and averaging them.

&Quot; (6) "

(T _i , t _j ) is the word pair contained in the query term, N _t is the word count, and Distance (t _i , t _j ) is the word pair (t _i , t _j ), and maxDistance (D) is the maximum value of the distance between core concepts of the knowledge structure.

In step S44, the proximity between the movie and the query corresponding to the average distance between the words acquired in step S43 is obtained by referring to the correlation between the average distance between the predefined words and the proximity.

FIG. 8 is a graph showing the correlation between the average distance between words and the proximity of the present invention. As shown in FIG. 8, when the average distance between words and the proximity between the movie and the query have a convex curve shape, It can be seen that there is a stronger association.

In the present invention, the average distance between words acquired through step S43 is converted into the proximity between the movie and the query word according to the following equation.

&Quot; (7) "

Where a is a parameter that adjusts the convexity of the convex curve that reflects the average distance between words and the proximity between the movie and the query.

  Note that the larger the value of a, the more convex function of the distance between words and proximity becomes, and the smaller the value of a, the closer the relationship between the two becomes. In particular, the best results can be obtained when a is between 0.6 and 0.8, which implies that the relationship between the mean distance between words and proximity has a function of exponential function (convex function).

Then, in step S50, it is possible to consider only the proximity score, or to determine the recommendation order of movies selected in step S20 by considering the proximity score and the BM25 score together based on the following mathematical expression.

&Quot; (8) "

KSRM (Q, D) = BM25 (Q, D) * Proximity (Q, D)

However, when the BM25 score is considered in addition to the proximity score, the motion recommendation order decision operation considering the frequency of the words is performed. This is because the BM25 score has a value proportional to the word frequency.

Although BM25 has been described as an example of the ranking function in the above description, all kinds of ranking functions capable of ranking related to query terms will be applicable. That is, the knowledge structure generation and re-ranking method applied here may be applied to searches based on other ranking functions.

The computer-readable recording medium on which the program commands are recorded may be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, Media storage devices.

The computer-readable recording medium on which the above-described program is recorded may be distributed to a computer apparatus connected via a network so that computer-readable codes can be stored and executed in a distributed manner. In this case, one or more of the plurality of distributed computers may execute some of the functions presented above and send the results of the execution to one or more of the other distributed computers, The computer may also perform some of the functions described above and provide the results to other distributed computers as well.

A computer capable of reading a recording medium on which an application, which is a program for driving a video content retrieval method based on the knowledge structure according to each embodiment of the present invention, is read may include not only a general PC such as a general desktop or a notebook computer, A mobile terminal such as a cellular phone, a tablet PC, a PDA (Personal Digital Assistants), and a mobile communication terminal. In addition to this, it should be interpreted as all devices capable of computing.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiments, it is to be understood that the invention is not limited to the disclosed embodiments. That is, within the scope of the present invention, all of the components may be selectively coupled to one or more of them. In addition, although all of the components may be implemented as one independent hardware, some or all of the components may be selectively combined to perform a part or all of the functions in one or a plurality of hardware. As shown in FIG. The codes and code segments constituting the computer program may be easily deduced by those skilled in the art. Such a computer program can be stored in a computer-readable storage medium, readable and executed by a computer, thereby realizing an embodiment of the present invention. As a storage medium of the computer program, a magnetic recording medium, an optical recording medium, or the like can be included.

It is also to be understood that the terms such as " comprises, "" comprising," or "having ", as used herein, mean that a component can be implanted unless specifically stated to the contrary. But should be construed as including other elements. All terms, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise defined. Commonly used terms, such as predefined terms, should be interpreted to be consistent with the contextual meanings of the related art, and are not to be construed as ideal or overly formal, unless expressly defined to the contrary.

The foregoing description is merely illustrative of the technical idea of the present invention, and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the scope of the present invention but to limit the scope of the technical idea of the present invention. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

Claims (11)

Selecting image contents corresponding to a query input by a user;
Acquiring a knowledge structure corresponding to each of the selected image contents;
Calculating proximity between image content and query terms for each of the selected image content based on the knowledge structure; And
And determining a recommendation order of the selected image contents based on the proximity calculation result,
The step of acquiring the knowledge structure
Acquiring plot information corresponding to each of the selected image contents and extracting a key concept corresponding to each of the selected image contents based on the plot information;
Acquiring a plurality of key concept pairs from the extracted core concept, and associating key concepts corresponding to each of the selected image contents with air information (Co-occurrence) corresponding to each of the plurality of core concept pairs Extracting; And
And generating a knowledge structure corresponding to each of the selected image contents based on the association between the key concepts. delete 2. The method of claim 1,
A plurality of nodes respectively corresponding to a plurality of key concepts and a plurality of links expressing an association between the plurality of nodes, wherein the inter-node distance is inversely proportional to the inter-node association. A method of searching a video content. 2. The method of claim 1, wherein calculating the proximity between the image content and query terms
Extracting words included in the query term, and combining the extracted words to form a plurality of word pairs;
Assigning each of the word pairs to the knowledge structure to acquire and collect word-to-word distances on a knowledge structure, and then average the words to obtain an average distance between words; And
And calculating the proximity between the image content and the query word based on the average distance between the words. 5. The method of claim 4, wherein obtaining the average distance between words comprises:
The method comprising: acquiring an average distance (DistanceSore) between words using the following formula:

(T _i , t _j ) is a word pair included in a query term, N _t is a word number, and Distance (t _i , t _j ) is a word pair t _i , t _j ), and maxDistance (D) is the maximum value of the distance between core concepts of the knowledge structure. 5. The method of claim 4, wherein calculating the proximity between the video content and the query terms
Wherein the proximity between the image content and the query word is calculated through the following equation: < EMI ID =

A is a parameter that adjusts the convexity of the convex curve reflecting the average distance between words and the proximity between the movie and the query word. The method of claim 1, wherein selecting the image content comprises:
Calculating a ranking score corresponding to each of a plurality of image contents through a ranking function and acquiring movie contents whose ranking score is larger than a predetermined value as movies related to a query inputted by a user, Based video contents retrieval method. 8. The method of claim 7, wherein determining the recommendation order of the selected video content comprises:
Further comprising a function of considering the ranking score calculation result when determining the recommendation order of the selected image contents. A computer-readable recording medium having recorded thereon a program for executing a method of searching for image contents based on the knowledge structure according to any one of claims 1 to 8. A content search unit for selecting image contents corresponding to a query input by a user;
An information structure managing unit for creating and managing a knowledge structure corresponding to each of a plurality of image contents;
A proximity calculator for acquiring a knowledge structure corresponding to each of the selected image contents and calculating proximity between image contents and query terms for each of the selected image contents based on the acquired knowledge structure; And
And a movie recommendation unit for determining a recommendation order of the selected image contents based on the proximity calculation result and performing a movie recommendation operation,
The knowledge structure management unit
Acquiring plot information corresponding to each of the selected image contents, extracting a core concept corresponding to each of the selected image contents based on the plot information, acquiring a plurality of key concept pairs from the extracted core concept Extracting a correlation between key concepts corresponding to each of the selected image contents using air information (Co-occurrence) corresponding to each of the plurality of key concept pairs, and based on the correlation between the key concepts, And generating a knowledge structure corresponding to each of the plurality of image contents. A video content search system including a video content search server based on the knowledge structure of claim 10.

Images