KR20000033410A - Image data retrieval method by partial result matrix and flexible attribute tree - Google Patents

Abstract

PURPOSE: An image data retrieval method is provided to group interrelated attributes into fixed and flexible attributes, structure a user's content based queries into a set of various attributes, simplify browsing structured flexible attribute trees by using a dictionary browser, and browse contents of a video file with an easily understandable method. CONSTITUTION: An image data retrieval method comprises steps of expressing dynamic information on contents of a video file as the flexible attributes, defining a flexible attribute tree on a flexible attribute A, structuring the flexible attribute tree TA as a set of the flexible attribute A and a plurality of children nodes(n1,n2,n3..), and letting each element of the set the flexible attribute have a unique value vi(1≤i≤p). The flexible attribute has following characteristics; 1) The node A is a root of a tree. 2) The node ni has more minute value vi than its pa rent node 3) If the node ni is an internal node, the domain of the node ni is vi 4) if the node ni is a leaf node, the node ni can have a minute value set(W) defined by a user and the domain of the node ni is a set of vi or a user defined value set(W) 5) A new node can be inserted into the tree, and a node can be deleted.

Description

Image data retrieval method using flow attribute tree and partial result matrix

The present invention relates to a method of retrieving image data, and more particularly, to a method of retrieving image data by using a flow attribute tree and a partial result matrix for retrieving images classified into fixed attributes and flow attributes.

Advances in computer, communication, and data compression technologies have enabled a variety of services using video data. Compared with other data types, even though compressed video data is compressed, it is difficult to manage video data because it is very large and requires continuous access in real time. In addition, in a continuous video stream, various pieces of information are represented without distinct boundaries. However, since video data can easily represent complex information, the use of video data is expected to increase. To support effective content-based retrieval, we define a unit of description that can adequately describe information about any part of a video file.

In addition, the description unit does not physically reconstruct the parts of a video file, but allows it to be physically separated but logically managed as one continuous bitstream.

There are two types of information about video data. One is information about the content that appears in the portion of the video file, and the other is general information about the video file itself or the objects that appear in that video file. The former is information representing the form and meaning of the object's position and movement appearing in the video file. The latter, on the other hand, is information that represents the physical properties of the video file and the general characteristics of the object. To distinguish this difference, the former is defined as dynamic information and the latter as static information. While most research on content-based retrieval in video databases has focused on how to effectively represent dynamic information, in general, queries can be expressed as a combination of static and dynamic information, so the organic relationship between static and dynamic information Should be able to express

The efficiency of content-based retrieval in a video database system depends not only on the structure of description (description) but also on the method of expressing dynamic information, and the form of dynamic information expressed according to the needs of the application field to be constructed may be changed. Therefore, it may be more efficient to fix the type of dynamic information required in a particular field. However, this method has limitations in using it as a general video database model because modeling elements change depending on the application. Therefore, most of the studies use keyword methods that can freely model various forms without the concept of attributes used in existing database management systems. This method can be implemented in a simple structure that can store keywords, but has the disadvantage that the search results of the query vary considerably depending on the words used in the annotation processing and the words used to express the query.

In the method of supporting the concept of attributes, a set of attributes describing the meaning of a scene can be classified into a fixed database schema and a dynamic configuration without a schema. However, there are the following disadvantages in expressing dynamic information with fixed schema.

The first is that there can be many types of movement of an object on one screen, so the method of fixing the shape of the movement in advance cannot be a general solution. Secondly, it is not a good idea to fix the type of objects or screens that are modeled because the type and amount of screens stored in the video database can be enormous and the importance of objects and screens can change according to changes in the user's perspective. Finally, the description (description) of the same dynamic information may be different depending on the user's point of view. For this reason, it is almost impossible to define a fixed set of modeling elements unless the video database is designed for a particular purpose.

Despite this demand of video database systems, no database management system manages attributes without a schema. Therefore, most studies provide a unique video data model with different query languages and index structures. To overcome this difficulty, the present invention efficiently manages a set of attributes and attribute values that are dynamically defined in the existing database model. In addition, unlike the existing database system, the video database requires the user to play the result video data in order to utilize the query result. However, if the number of query results is quite large, it is impossible to play all the videos one by one.

In general, a query for content-based retrieval has a property of extracting data having similar conditions rather than exact matching. Therefore, it is possible to effectively support the user's query expression by presenting the expected number of results for each condition constituting the query to the user before query processing. The present invention proposes a result browser that supports such a function so that the user can reconstruct the query conditions.

Various methods for supporting content-based retrieval for video databases have been conducted. Content-based retrieval consists of video file and symbol object composed of video class hierarchy and symbol class hierachy, respectively. Based on this hierarchy, the video information described (described) is maintained in the video_symbol_object table, and the query is expressed in a query language called CVQL.

The spatial or temporal relationship of the symbol object is defined by a predefined construction function. A class hierarchy called topical categories can be used to express domain specific information. Keywords are used to describe the video information and are stored in a specific schema of the domain, while queries are expressed in free-text form. The above two methods are suitable for limited applications because they use application domain specific information for annotation and indexing. Also, when the size of the semantic dictionary tree for the semantic words increases, the browsing method for the tree structure is not considered.

For content-based retrieval of television news, news information is described using text broadcasting. In this method, the query is expressed as a keyword, and the query results are displayed sequentially by sorting with a matching score indicating the degree of matching between the query statement and the text of the text. The method of expressing the query in the form of keywords and free phrases has a disadvantage in that it is difficult to express low-level information such as object position, size, and color.

The existing database management system is used to store the structure of the video data, and the content information of the video is annotated using keywords. Predefined functions are used to represent temporal relationships of video data and share video data.

Color histograms, colors, and average brightness are used to automatically extract the characteristics of the video stream, which can be used to find the keyframes needed to retrieve the video data. On the other hand, in order to structure the video data, video information about a specific screen is converted into a vector expression composed of a set of modeling elements, and the vector expression is used to detect the boundary of the screen and extract the neighboring frames and segments. Can be used to find similarities.

The OVID and Algibraic Video Model describe (dynamically) describe dynamic information of video data as dynamically defined attributes and attribute value pairs. The description unit in OVID is a video object and all property values used to describe content are kept in a generalized hierachy structure. Therefore, if you define a lot of attribute values, there is a disadvantage that the size of the tree increases. OVID retrieves video objects using Video SQL as its native query language. The description unit in the Algibraic Video Model is a video expression represented by a video algebraic operator. This structure has the advantage of being easily shared by other video expressions, but the video expression contains multiple algebraic operations. This complicates the relationship of identifying the actual video data corresponding to the video expression. In addition, the query is expressed by a combination of attribute and attribute value pairs, but since there is no semantic dictionary structure for browsing, the user must search each attribute and list of attribute values sequentially to express the definition.

In the existing database management system (DBMS), static information of video data can be expressed most effectively. However, most researches provide a unique query language to express content-based queries. Since these query languages are not compatible with SQL, it is difficult to express a query of video data as a combination of dynamic information and static information. Therefore, this method not only incurs a burden on the user to acquire a new query language, but also may cause compatibility problems when the video database needs to be integrated with an existing database.

Various temporal relationships can exist between two descriptive units. An interval inclusion relationship occurs when one of these sections is included in another section, and this relationship is used to inherit the dynamic description (description) in the OVID system. Attributes are categorized as inheritable and non-inheritable by the user, and inheritable attribute and attribute value pairs are inherited only when a merge or overlap operation is performed. The Algibraic Video Model, on the other hand, can dynamically inherit the described meaning between the included video expressions, but it is not optionally supported as in OVID.

A hierarchical temporal language is used for content-based retrieval of video data, and the spatiotemporal description (description) of video data can be expressed logically. Probabilistic retrieval model can be used to retrieve multimedia objects. Unlike other methods that provide search results based on exact matching between query and video clip content, this method expresses the relevance of the query as a probability. Modeling elements for video data can be defined differently according to the application of the video database, so the range of expressing a query is limited to the modeled elements.

The present invention is to solve the above conventional problems,

An object of the present invention is to provide a method for efficiently managing a set of attributes and attribute values that are dynamically defined in an existing database model.

Another object of the present invention is to provide a result browser that supports a function that allows a user to reconstruct query conditions.

It is still another object of the present invention to provide a method for presenting an expected number of results for each condition constituting a query to a user before query processing.

Another object of the present invention is to provide a method for expressing an organic relationship between static information and dynamic information.

It is yet another object of the present invention to provide a method in which parts in a video file can be physically separated but logically managed as one continuous bitstream.

In order to achieve the above object, the present invention defines a flexible attribute tree (FAT) for the flow attribute A in order to manage the flow attribute that can be interpreted in various abstraction steps, and puts a tree of the same concept We model with three different flow attribute trees or attribute values, and propose a schema structure to represent such information in a video database model in which various types of information are modeled. In addition, the dictionary browser, which simplifies the user's query expression and semantics, to support the user's effective browsing, and the result browser so that the user can analyze the query results for various combinations of query conditions. To provide. The result browser allows the user to reconstruct the query conditions.

1 shows an attribute type that can describe (explain) the content of video data.

Fig. 2 shows the flow attribute tree of the semantic attribute "behavior" and the flow attribute tree of the shape attribute "position" representing the position of the entity appearing on the x-y coordinates.

3 shows a method of browsing attribute values in a FAT.

4 shows the overall structure of a dictionary browser according to the present invention.

5 is a result viewer showing the result of a query.

Figure 6 shows the resulting browser according to the present invention.

Hereinafter, with reference to the accompanying drawings for an embodiment according to the technical idea of the present invention configured as described above in detail the operation and operation effects are as follows.

In the existing relational and object-oriented database models, static information of video data can be easily represented. Properties related to the general properties of an object are defined by a fixed schema when designing the editor base. For example, general properties of objects such as name, age and gender, or information about the video file itself, such as title, file format, size, and resolution, can be easily modeled in a fixed schema. On the other hand, dynamic information is difficult to model with fixed schema because it is impossible to define a common set of attributes when various users abstract the information about attributes with different stages from different perspectives. Therefore, as in the OVID system, dynamic information is represented by attribute and attribute value pairs. A set of attribute and attribute value pairs is defined as a DYNAMIC DESCRIPTION, and its elements can be added or removed dynamically as needed. Because of these fluid characteristics, existing methods of defining schemas cannot be used, and because it is not easy to clearly define attributes and domains of attributes, fluidly defined attributes and attribute values require special management.

Dynamic information can be classified into two phases: SYNNTACTIC LEVEL and semantic-level. Form level description (description) means the lowest level of information that appears in the description (description) unit. It is used to express the exact shape, color, or movement of an object as the size, position, or direction of continuous movement of the object as it appears in the video data. Thus, form-level descriptions are often used in physically continuous description (description) units. Many existing studies have been conducted on techniques for automatically annotating video data using only shape-level information.

Semantic level description (explanation) is a higher level abstraction and is used to model the conceptual meaning of the type level information that appears in the description (explanation) unit. For example, if the screen is a human reading, the shape level description (description) can model the position and size of each object appearing on the screen, the semantic level description (description) can model the screen as reading. Unlike the form-level description (description), the semantic description (description) can be abstracted in multiple stages and one or more small description (description) units as well as a continuous bitstream can be defined as one large description (description) unit. . In this case, the semantic level description (description), which consists of one, is described (explained) in an abstracted sense at a higher level than one or more other small description (description) units.

1 shows an attribute type that can describe (explain) the content of video data.

Attributes that represent dynamic information are defined as flow attributes, otherwise they are defined as fixed attributes.

Managing the set of flow attributes and attribute values is not easy because the names of flow attributes and attribute values can be expressed differently according to each user's point of view. For example, the position of an entity can be expressed as "position" or "center" and its attribute value can be expressed in x-y coordinates or r-θ coordinates.

A flexible attribute tree (FAT) is defined for flow attribute A to manage flow attributes that can be interpreted at various levels of abstraction. The flow attribute tree T _A is composed of a set of nodes N = {A, n ₁ , n ₂ , ..., n _p }, and each element has a unique value v _i (1 ≦ _i ≦ _p ). In addition, the flow attribute tree T _A has the following properties.

(i) Node A is the root of the tree.

(ii) Node n _i has a more detailed value v _i than its parent.

(iii) the domain of node n _i is an internal node n _i is _{D (n i) = v i} .

(iv) if node n _i is a leaf node,

-n _i may have a set of detailed values W = {w ₁ , w ₂ , ...., w _i } defined by the user.

The domain of -n _i is D (T _A ) = {v _i } ∪.

(v) D domain of T _A (T _A) = ∪

(vi) New nodes can be inserted into the tree and nodes in the tree can be deleted. When the node n _i is deleted node n _i child is a child of the parent node n _i.

The value of a node in the flow attribute tree. Certain values, such as proper nouns, cannot be used. Instead, they are managed as detailed values in the leaf node. For example, if one of the flow attribute tree buildings and one of its leaf nodes is a government agency, the Blue House would be a detailed value of the government agency. Fig. 2 shows the flow attribute tree of the semantic attribute "action" and the flow attribute tree of the shape attribute "position" representing the position of the entity appearing on the x-y coordinates.

Since the same concept tree can be modeled as two different flow attribute trees or attribute values, only an administrator can define a new flow attribute tree and use the structure of the flow attribute tree to use the flow attribute tree and node values consistently. You can change it. Based on the given flow attribute tree, the user can annotate the screen and express the content-based query.

As mentioned earlier, there are various types of information that are modeled in video databases. To represent this information in the existing database model, we propose the following schema structure.

Physical segment (P _id , file name, file location file size, file format, ..., physical property or static property)

Logical segment (L _id , P _id , start offset, end offset)

Logical Segment List (V _id , L _id , Sequence #)

Video clip (V _id , floating attribute name, attribute value, inheritance information, [object attribute name])

View (V _id , fixed property)

Flow attributes (flow attribute name, type, inheritance ?, FAT pointer)

Each video file inserted into the video database has a unique physical segment P _id and the logical segment L _id = (P _id = [s, e]) represents the interval from offset s to offset e in physical segment P _id , It is defined as. Since multiple logical segments can be defined in one physical segment, and vice versa, the relationship between a physical segment and a set of logical segments is 1: N.

Also define one or more logical segments as logical segment list [(P ₁ = [s ₁ , e ₁ ], P ₂ = [s ₂ , e ₂ ], ..., P _n = [s _n , e _n ]]] To define a video clip or view.

Video clips are units for describing content-based semantics, while views are used to organize logical video files. In other words, views are used to logically connect parts of several physical segments to form a single logical video file. A video clip or view can be defined as one or more logical segments, and the same logical segment can be shared by multiple views or video clips, so the video clip set or view set forms an M: N relationship with the logical segment set.

The dynamic meaning of a video clip is represented by selecting a node value in the flow attribute tree and linked to the actual video clip. You can create and remove video clips, but the logical segments and the logical segment list are maintained in the system. The fixed attributes of the physical segment and view schema and the object properties of the video clip schema are defined according to the application of the video database. Usually an object property represents the name of the object as the subject of the property and property value pairs. This allows you to describe each object independently, because more than one object can appear in a video clip at the same time.

Video clips v _i and v _j are represented by v _i = [l ₁ , l ₂ , ...., l _x ] and v _j = [l ₁ ', l ₂ ', ...., l _y '] La logical segment list to be defined, and, v if the video stream is represented by the _j v _{j i} ∠v containing all streams of v _i v each logical segment in the _{i l i = [p i,} (s i, e for _i)] v _i ∠ v _j is a logical segment _{l j '= [p j'} , (s j ', e j') which satisfy the following (i) and (ii)] should be present in the v _j do.

(i) p _i = p ' _j

(ii) s ' _j <s _i , e _i <e' _j

For a given shape attribute set A _syn and a semantic set A _sem , we define v _i ^syn and v _i ^sem for a video clip v _i .

v _i ^syn = {(attribute, attribute value) ｜ attribute∈ A _syn , attribute∈D (T _attr )}

v _i ^sem = {(attribute, attribute value) ｜ attribute∈ A _sem , attribute value∈D (T _attr )}

Therefore, the meaning described by the user for the video clip v _i is as follows.

v _i ^self = v _i ^syn ∪ v _i ^sem

In order to consider the description (explanation) by the section inclusion relation operation, a set of semantic attributes A _sem may be classified into an inheritable attribute (A _{I_sem} ) and a non-inheritable attribute (A _{NI_sem} ) as _follows .

A _sem = A _{I_sem} ∪ A _{NI_sem} , A _{I_sem} ∩ A _{NI_sem} = Ø

The schema manager determines whether any semantic attribute is inheritable or non-inheritable. Accordingly, the semantic description (description) of the video clip v _i is classified into two types: an inheritable description (description) (v _i ^I_sem ) and a non-inheritable description (description) (v _i ^NI_sem ).

v _i ^I_sem = {(attribute, attribute value) ｜ attribute∈ A _{I_sem} , attribute value∈D (T _attr )}

v _i ^NI_sem = {(attribute, attribute value) ｜ attribute∈ A _{NI_sem} , attribute∈D (T _attr )}

All forms of technology for the video clip (description) are subject to inherit the technique inherits the video clip v _i (description) comprises a v _i ∠ v _k all v inheritance of _k possible technique for satisfying the (description), as follows: do.

v _i ^inh = ∪ _k (v _k ^syn ∪ v _k ^I_sem )

Therefore, the meaning described in the video clip v _i including the inherited description is as follows.

v _i ^description = v _i ^self ∪ v _i ^inh

The name, flow attribute tree (FAT), and type (dynamic or semantic) of all dynamic attributes are stored in the flow attribute schema and maintain their inheritance. Semantic inheritance between video clips is expressed in the video clip schema. New form descriptions (descriptions) or inheritable semantic descriptions (descriptions) expressed by users in video clips v _s are expressed in the video clip schema as follows:

(v _s , attribute; attribute value; inheritance = no, [la])

Where (property; property value ') ∈ {(property, property value) │property∈A _syn or property∈A _I , property value ∈D (T _attr )} and [la]: object property value

At this time, a new tuple (v _t , property; property value; inheritance information = yes, [la]) is additionally added to the video clip schema for all video clips v _t satisfying v _i ∠ v _s . If there are many video clips that satisfy the containment relationship, the semantic inheritance processing takes a lot of time, but the inherited description (description) can be considered as a supplementary meaning of the video clip. It is more efficient to do it periodically than whenever it is done.

In inheriting the semantic description, a partially overlapped relationship occurs between two clips that partially share a segment, which is particularly important because the overlapping segment has an inclusion relationship with each video clip. It can be regarded as an interval inclusion relationship. This allows the system to define nested parts as new video clips, enabling content-based retrieval for parts not defined as video clips by any user.

Simple content-based queries can be expressed in graphical user interfaces (GUIs), but complex queries must be expressed in a query language based on the video data model. Most of the query languages used in previous studies are classified in three ways.

The first uses an ad-hoc solution that does not provide a formal method for expressing a query. It is used in systems that annotate keywords. The second uses a standard query language such as SQL. However, this method is limited to areas where modeling elements are fixed because the schema cannot be changed flexibly. The third is to provide a unique query language with a form suitable for the proposed system.

Suppose a politician video file is stored in a video database. If the flow attribute tree is as shown in Fig. 2, and you want to find the video clip that President Kim Young-Sam is speaking, the query is expressed as the following SQL.

Q1: select v _id

from video clips

where flow attribute = behavior and value = speech and individual name = Kim Young Sam

If the user wants to find all the video clips in which Kim Young-Sam appears in the range of x <X _max / 2 and y <Y _max / 2, the query looks like this:

Q2: (select v _id

from video clips

where flow attribute = position and value = x <X _max / 2 and individual attribute = Kim Young Sam)

intersect

(select v _id

from video clips

Where flow attribute = position and value = y <Y _max / 2 and individual name = Kim Young Sam)

Given the existence of schema politicians (name, age, gender, nationality, and work experience) in a relational database, the query to search for clips spoken by Korean male politicians is as follows.

Q3: select v _id

from video clips

where flow attribute = behavior and value = speech and

Objectnamein (select name

from Politicians

where nationality = Korea and gender = men)

In order to understand the inclusion relationship between video clips, it is necessary to confirm whether one section is included in another section. For example, if query Q4 retrieves the logical segment defined in the interval from t1 to t2 of physical segment p, and Q5 retrieves all segments of video clip v, then the query is as follows.

Q4: select v _id

from logical segment

where P _id = p and start offset ＞ t1 and end offset <t2

Q5: select L _id

from logical segment list

where V _id = v

order by Sequence #

If you want to search for video clips except inherited descriptions (description), you can use the inheritance? Express using attributes. For example, if the query Q2 is expressed as a query without inherited technology, it is as follows.

Q6: select v _id

from video clips

where flow attribute = position and value = x <X _max / 2 and entity attribute = Kennedy and inheritance information? = no)

intersect

(select V _id

from video clips

where flow attribute = position and value = y <Y _max / 2 and entity name = kennedy and inheritance? = no)

If the number of video clips and the description of the clips are enormous, clip search can take a lot of searching time, but the OVID or Algibraic Model does not provide a solution for this approach. As such, using an existing database management system, an access structure such as hash or B-tree can improve database search efficiency.

Because content-based queries can contain a variety of conditions for attributes, it is difficult for a user to express complex queries in a video database, and this problem can become more severe when fluidly managing attributes and attribute values. Also, even if the user has configured all the necessary query conditions, the user must play the result video of the query before using it.

The structure of the FAT is not suitable for direct user use. Because nodes are added and deleted, the FAT can be an unbalanced tree, making it difficult to show the overall tree structure. In addition, since one FAT represents attribute and attribute value pairs, most queries require more than one FAT in order to construct complex query conditions.

3 shows a method of browsing attribute values in a FAT. In order to show related attribute values in one FAT, the node next to the path of the currently selected node from the root of the FAT becomes the user's work target. When the user selects a search path in the tree, the browser only displays attribute values of the same level in the path. In the browser, the attribute value box represents the attribute value of a node in the FAT, and the level difference of the FAT causes the color of the attribute value box to be different for each level. The two arrows in the property box are controls that can be moved to the parent or child. When the child control of the attribute value box of FIG. 3 is selected, the attribute value box is replaced with the attribute value box corresponding to its own child node, and the parent control of the attribute value box has the opposite function as the child control.

The browser of the present invention can simultaneously browse more than one FAT. As shown in FIG. 3, three FATs can be searched in one workspace, ⁱ i, A ^j , and A ^k . Each column representing the flow attribute in the browser is defined as an attribute pallete. In the initial screen, the names of all the palettes defined in the system are provided in icon format, and after selecting the set of icons you want to search, the Properties palette appears in the browser.

The administrator can modify the structure of the FAT. If the size of the FAT sub-tree grows, the administrator can modify the contents of the video clip schema to create floating attribute fields of all tuples with the attribute values of the subtree to be defined as a new FAT. Convert to the name of the new FAT. If the number of FATs is too large to manage, they can be sub-grouped like a clipmap, depending on their similarity in meaning.

While browsing the FAT, the user can describe the meaning of the video clip. After selecting the desired flow attribute value from FAT, click the attribute value box to input basic query condition of attribute = attribute value. The user inputs the detailed attribute values of the leaf nodes in the text box representing the query condition of FIG. 3 (d), and inputs the individual attribute values in a similar manner. The information entered in this way is stored in the video clip schema.

In the existing method of expressing a query for video data, an interactive query is composed of a list of keywords or a combination of logical operators. Among them, we provided a unique interface for limiting query conditions or supporting color, texture, and spatial / temporal conditions. In the query mode, the dictionary browser can be used to express the query condition for the detailed attribute value in the same way as the aforementioned semantic description method. The query condition thus constructed is interpreted as a logical operation and converted into an SQL statement internally. Query conditions selected in the same palette mean OR between attribute values, and query conditions selected in other palettes mean AND. Therefore, the same property palette must be selected more than once to represent the AND condition between the values of the same FAT.

The overall structure of the dictionary browser is shown in FIG. 4 shows a browser after constructing a query by selecting four fixed attributes and five floating attributes. The two flow attribute and attribute value pairs, ACTION / handshake and POSITION / Xmax / 2, are represented by President Kim Young-sam. For the PLACE attribute, BLUE HOUSE is the detailed attribute value of the leaf node government organization. By selecting the Inh button provided for each query condition, the query result including the inherited meaning can be obtained. The query result is shown in the result viewer as shown in FIG. 5 and a representative frame of each video clip satisfying a given query is represented by an icon. This icon is the same as the micon, and the user can click the icon to play the video data corresponding to the representative frame, and can use the slide show function to sequentially view all the representative frames in a short time.

The result of the query is displayed as a list of matching scores in a sequential order, and a timebar structure is provided in the news broadcast to express the degree of matching with the query result. However, video databases need to provide more analytical capabilities to help users express their queries. If there are too many or too few query results, the user must reconfigure the query conditions to obtain a suitable search result. Since the number of results depends not only on the composition of the query, but also on the contents of the database, a special browser is needed to show the partial expected number of results of the user-configured query condition.

Figure 6 shows the resulting browser according to the present invention.

Let s _i , 1≤i≤c, be a query condition expressed by "attribute = attribute value" expressed by the user. For the partial result analysis of the query, we use the C × C matrix R called the partial result matrix. Each element of this matrix represents the number of video clips that satisfy the combination of query conditions.

(i) Number of video clips satisfying Diagonal elements R (i, i), 1≤i≤c: s _i

(ii) Video satisfying Upper-diagonal elements R (i, j), 1≤i≤c, 1≤j≤c: s _i ∧s _{i + 1} ∧ .... ∧s _j Number of clips

(iii) the number of video clips that satisfy the lower-diagonal elements R (i, j), 1≤i≤c, 1≤j≤c: s _i ∧s _j

For intuitive interpretation, the color brightness of each element in the partial result matrix indicates the number of result video clips.

The combination of meaningful query conditions provides enough information for the user to understand the degree of matching of the query. The order of elements in the matrix is an important factor in analyzing the results of a query, so the user can drag and drop to reorder them. The user can reconstruct the query based on the number of video clips in the result browser.

A structure browser is provided to support the browsing of the physical structure of one video file, and the configuration of the browser is similar to the result browser of FIG. In the structure browser, a two-dimensional structure is represented by a three-level tree structure representing representative frames of shots, scenes, and sequences. This feature is similar to a hierarchical browser. In other words, the tree represents the physical structure of the video file. In the Structure Browser, you can search for video clips defined in the video file.

As described above, according to the present invention, the user-defined content is defined as a fixed schema by grouping related properties, or by defining a property in a fluid manner without a fixed schema. You can construct a query based on a variety of attribute sets. The pre-browser simplifies the browsing of the tree, which is internally structured in a tree, and the structured browser provides a simple way to understand the content of a video file.

Claims (12)

In the video data search method, Represents dynamic information representing information on the content of the image file as a flow attribute, defines a flexible attribute tree (FAT) for the flow attribute _A , and the flow attribute tree T _A is a node N = {A , n ₁ , n ₂ , ..., n _p }, each element has a unique value v _i (1≤i≤p), and the flow attribute tree T _A has the following properties: An image data retrieval method using a flow attribute tree and a partial result matrix. (i) Node A is the root of the tree. (ii) Node n _i has a more detailed value v _i than its parent. (iii) the domain of node n _i is an internal node n _i is _{D (n i) = v i} . (iv) if node n _i is a leaf node, -n _i may have a set of detailed values W = {w ₁ , w ₂ , ...., w _i } defined by the user. The domain of -n _i is D (T _A ) = {v _i } ∪. (v) D domain of T _A (T _A) = ∪ (vi) New nodes can be inserted into the tree and nodes in the tree can be deleted. When the node n _i is deleted node n _i child is a child of the parent node n _i. The method of claim 1, wherein a specific value, such as a proper noun, is managed as a value of a node in the flow attribute tree as a detailed value in a leaf node. The method of claim 1, If the size of the sub-tree of the FAT becomes large, the contents of the video clip schema are modified to change the flow attribute fields of all tuples having the attribute values of the subtree to be defined as the new flow attribute tree. An image data retrieval method using a flow attribute tree and a partial result matrix, which can be converted into a name of a flow attribute tree. In the video data search method, Each video file inserted into the image database has a unique physical segment P _id , and logical segment L _id = (P _id = [s, e]) represents a section from offset s to offset e in physical segment P _id . , Denote one or more logical segments as a logical segment list [(P ₁ = [s ₁ , e ₁ ], P ₂ = [s ₂ , e ₂ ], ..., P _n = [s _n , e _n ]] And image data retrieval method using a flow attribute tree and a partial result matrix, which are used to define a video clip or a view. The schema of claim 4, wherein the schema structure of the image database is Physical segment (P _id , file name, file location file size, file format, ..., physical property or static property) Logical segment (L _id , P _id , start offset, end offset) Logical Segment List (V _id , L _id , Sequence #) Video clip (V _id , floating attribute name, attribute value, inheritance information, [object attribute name]) View (V _id , fixed property) Flow attributes (flow attribute name, type, inheritance ?, FAT pointer) Image data retrieval method using a flow attribute tree and a partial result matrix consisting of. 5. The method of claim 4, wherein the video clip set or view set forms an M: N relationship with the logical segment set. In the video data search method, In order to show related attribute values in a FAT, the target node is a node neighboring the path of the currently selected node from the root of the FAT. When the user selects a search path of the tree, the browser only displays attribute values of the same level in the path. In the browser, the attribute value box represents the attribute value of the node in the FAT, the level difference of the FAT changes the color of the attribute value box for each level, and the two arrows in the attribute value box move to the parent or child. An image data retrieval method using a flow attribute tree and a partial result matrix characterized in that the control. 8. The method of claim 7, wherein the two-dimensional structure is a three-level tree structure representing representative frames of shots, scenes, and sequences in order to support browsing of a physical structure of one video file. An image data retrieval method using a flow attribute tree and a partial result matrix characterized by providing a structure browser to represent. 8. The method of claim 7, wherein when the browser selects a child control of an attribute value box, the attribute value box is replaced with an attribute value box corresponding to its own child node, and the parent control of the attribute value box has a function opposite to that of the child control. An image data retrieval method using a flow attribute tree and a partial result matrix. 8. The method of claim 7, wherein after selecting a desired flow attribute value in the flow attribute tree, the attribute value box for inputting a basic query condition of attribute = attribute value is clicked to describe the meaning, and the leaf is displayed in a text box representing the query condition. A method for retrieving image data using a flow attribute tree and a partial result matrix, characterized by inputting detailed attribute values of a node and inputting object attribute values in a similar manner to describe the meaning of a video clip while browsing the flow attribute tree. 11. The method of claim 10, wherein the query result is displayed in the result viewer, and a representative frame of each video clip satisfying the given query is represented by an icon. 12. The method of claim 11, wherein a C × C matrix, R, is called a partial result matrix for partial analysis of the query, wherein each element of the matrix is a number of video clips that satisfy a combination of query conditions. The image data retrieval method using the flow attribute tree and the partial result matrix, characterized in that as follows. (i) Number of video clips satisfying Diagonal elements R (i, i), 1≤i≤c: s _i (ii) Video satisfying Upper-diagonal elements R (i, j), 1≤i≤c, 1≤j≤c: s _i ∧s _{i + 1} ∧ .... ∧s _j Number of clips (iii) the number of video clips that satisfy the lower-diagonal elements R (i, j), 1≤i≤c, 1≤j≤c: s _i ∧s _j