JP2001078091A - Device and method for displaying and editing media scene information and storage medium storing program related to the method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To more easily grasp the relevance between scene information by displaying extracted media scene information related to selected media scene information so as to identify it from the other media scene information on a picture. SOLUTION: Three (sub) windows are displayed in a main window 501 for displaying the scene information easily to recognize. The first sub-window is a time line display window 502, the second sub-window is an icon display window 504 and the third sub-window is a table display window 503. Not only one but also plural windows can be respectively arranged as well. When plural windows are arranged, one piece of scene information can be more versatilely grasped. A main menu 506 is provided on the main window 501 so that the read/write of the file of scene information, editing operation or retrieving operation can be instructed.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus and a method for displaying and editing media scene information, and a storage medium storing a program according to the method.

[0002]

2. Description of the Related Art In recent years, the price of large-capacity hard disks has been reduced, the processing performance of PCs has been improved, various video and audio compression technologies have appeared, and video-on-demand has spread.
Media data such as video data and audio data can be easily managed, searched, and viewed on a computer. In addition, there is an increasing need to efficiently manage, search, and view such media data by utilizing the characteristics of computers. Conventional media data management methods are based on information management in units of data volumes, such as searching for a media data file or a video / audio tape with a simple keyword, title, author name, or the like. However, the media data information desired by the user is not only volume information such as movie directors and tape titles, but also information in fine units such as when and where in the volume and who is talking. Often. In recent years, for such a reason, a system that can register and manage information not only in the entire media data but also in an internal unit called a scene in the media data has been desired.

[0003] In the present application, "media data"
The term refers to not only video data and audio data, but also entire data with time information, such as slide show data with time information added to still images and closed caption (subtitle) data with time information added to text data. Shall be. A portion obtained by cutting out a partial section of the media data is called a "scene", and information attached to a portion of the media data is called "media scene information" or simply "scene information". The “attribute” is used as a general name of additional information having no time information. In the embodiments of the invention described below, “subspace” is mainly used as a term representing a scene of media data in video format.

Hitherto, as a system that can handle data up to a scene unit in media data, in the case of video,
There are video editing software and automatic indexing systems that automatically analyze video content and register scene information.

Conventional representative video editing software includes:
Software that creates a new piece of media data by connecting a plurality of media data is known. In such software, a plurality of media data expressions resembling a movie film are displayed on the screen, and the user manipulates the film-like media data directly on the screen using a mouse or the like, so that the playback time of the video can be reduced. Or a special effect between two pieces of film media data can be instructed (Japanese Patent Laid-Open No. 07-046462).

As a typical example of an automatic indexing system, for example, when media data for which scene information is to be registered is designated, video contents are automatically analyzed, and the analysis result and accompanying time information are displayed in a table format. There is.

[0007]

In the above prior art,
In registering and managing scene information in media data with respect to given media data, the following problems remain.

[0008] In the above-mentioned video editing software, the temporal arrangement between a plurality of scenes can be easily grasped. However, since the software is a software for creating one piece of media data by combining a plurality of media data, the user already owns it. It is not possible to register or delete only the scene information later for the media data.

In the above automatic indexing system,
Although only scene information can be automatically extracted and registered from media data, user-unique scene information such as a character name or a person's age, which is difficult to automatically extract, cannot be freely added or deleted. In addition, since these data are mainly expressed in a table format, there is a problem that it is difficult to grasp a mutual relationship between a plurality of pieces of scene information. For example, in the case of a representation of scene information in a table format, a temporal correlation between scene information, such as a scene in which a plurality of people appear simultaneously,
Since the inclusion relation of scenes such as the scenes of the opening, the main story, the CM, and the ending in the drama cannot be simply displayed, it is difficult to grasp the scene information.

An object of the present invention is to make it possible to more easily grasp the relevance between scene information when registering information of a scene in the media data with respect to the media data. Is to be editable.

[0011]

In order to achieve the above object, the present invention employs the following constitution. First, as a data structure of media scene information, for example, a set of scenes, which are partial units of data (media data) having time information, a set of attribute information registered by a user, and a set of these scenes and attribute information A set of link information, which is an association between them, is defined as media scene information.

In the apparatus according to the present invention, first, media data storage means for storing media data itself and media scene information storage means for storing media scene information are prepared. In order to store information in these storage means, in addition to a method in which a user manually registers information using the system described in the embodiment of the present invention, a media data file stored in a video server or a file system is used. And a media scene information file.
Furthermore, media scene information extracted from a huge media scene information database according to a certain search condition may be stored.

Next, a display means for displaying the media scene information is provided. Next, there is provided a selecting means for selecting arbitrary media scene information from the displayed media scene information. Alternatively, an extraction condition specifying means for specifying conditions for extracting the media scene information is provided.
As a method of realizing the extraction condition specifying means, a fixed extraction condition may be determined at the time of starting the apparatus, or the extraction condition may be dynamically determined at the time of execution by an instruction from a user. With this media scene information extracting means, the user can specify only necessary data extraction conditions at a required time, and can perform more efficient content confirmation and editing of the media scene information. .

Further, there is provided media scene information extracting means for extracting media scene information associated with the selected media scene information or extracting media scene information according to designated conditions. By using this method, complex relationships between media scene information can be expressed in an easy-to-understand manner, and only the information that the user actually wants to edit from the vast amount of media scene information is displayed on the screen, making it difficult for media Editing of scene information can be facilitated. In the present invention, media scene information display control means is provided for displaying the media scene information extracted by the media scene information extraction means on a display means.
Thereby, the extracted media scene information is displayed so as to be distinguishable (distinguished) from other media scene information on the screen. Specifically, as in the embodiment of the invention described later, the user can change the color according to the selection state and display the image, or display the image in different colors for various search conditions specified by the user, so that the user When viewing the screen, the type of the media scene information can be confirmed at a glance. Further, a media scene information processing means capable of processing the extracted and displayed media scene information is provided.

In the present invention, the following three media scene information expression formats can be provided in order to more easily express the relationship between the media scene information.

The first is a timeline expression format for expressing a temporal correlation of media scene information. In this expression format, the media scene information is displayed at a screen position along the time axis direction defined on the display means according to the time information included in the media scene information.

The second is an icon display format for visually displaying a list of a plurality of components of media scene information. In this expression format, the components of the media scene information can be grasped at a glance by arranging each piece of media scene information in an easily understandable icon format on the screen.

The third is a table expression capable of displaying detailed contents of a plurality of pieces of media scene information.
By using this expression format, information such as attribute names and attribute values that could not be displayed in the above two display formats can be displayed. Also, if this display format is used, the attribute name and the attribute value can be directly selected, so that these values can be easily corrected.

Other features and methods of displaying and editing media scene information of the present invention will be apparent from the description in the specification.

[0020]

FIG. 1 is a block diagram showing an embodiment of the present invention.
This will be described with reference to FIGS. First, a computer device used in the present embodiment will be described with reference to FIG. Next, a data structure according to the present embodiment will be described with reference to FIGS. 2, 3, and 4. FIG. FIG. 5 and FIG. 11 are execution screens of the media scene information display / edit device according to the present embodiment. Finally, the processing method of the present embodiment will be described with reference to the structured flowcharts of FIGS.

In this embodiment, for the sake of simplicity, a video scene information display method for adding attribute information created by a user to video data will be described. The present invention can be easily applied to other types of media data having time information such as slide show data.

First, a computer device used in an embodiment of the present invention will be described with reference to FIG. Reference numeral 101 denotes a main body of the computer device, which has a CPU device 110 and a memory device 111 therein. In addition, the computer device body 1
01, an input device 115 that receives input from a user such as a keyboard and a mouse as external input / output means;
A display device 113 for displaying the content processed by the CPU device 110 to the user is prepared. In the present computer device, the auxiliary storage device 11
From 4, the media scene information used in the apparatus of the present embodiment can be read and stored.

In the present embodiment, the memory device 111 or the auxiliary storage device 114 is used to store media data and media scene information groups. Also, the CPU device 11
0 and a memory device 111, a media scene information extraction unit and a media scene information display control unit for extracting a media scene information group and displaying the same on the display device 113, a media scene information processing unit for processing the media scene information, and a user input. User instruction analysis means for changing the display according to the above.

FIG. 2 is a conceptual diagram of the data structure of the media scene information used in the present embodiment. The media scene information in the present embodiment is roughly divided into three parts. The first is a scene which is a partial unit of the video data 201 which is media data having time information, that is, data such as 211, 212, and 213 indicating a temporal existence position of a scene which is a part in the video 201. This is information that directly defines partial video data (scene) for which the user wants to register attribute information. The second is attribute information 221, 222, 223 that the user actually wants to register for a scene in the video data. The third is link information 231 to 234 for associating the time information 211, 212, 213 of the video data with the attribute information 221, 222, 223 created by the user. By using a data structure in which two pieces of information are linked by a link as described above, it is possible for a user to easily add attribute information to existing video data later.

In this embodiment, 211, 212,
Data related directly to video data (or media data in general) such as 213 is called a “subspace” (scene). This is because not only the part of the video data to which the attribute can be added by the method of the present invention but also the part separated by a temporal delimiter as shown in FIG. This is because a partial area in an XY coordinate space (in the case of three dimensions, an XYZ coordinate space) such as an area of one sphere which is a part of a three-dimensional CG image can be specified. In the present embodiment, a temporal portion of video data will be described as an example, and a temporal section of video data will be referred to as a “subspace”.

The symbols in FIG. 2 will be described briefly. The video data 201 is composed of three scenes: a person scene, a car scene, and a house and person scene. Conventionally, the only way to confirm the contents of these scenes is to reproduce the video data 201. Conventionally, there is no means for confirming scene information such as names and ages of persons appearing in these scenes.

In the system according to the present embodiment, a person A is added to such video data 201 as scene information, for example.
Can be registered later. Information 221 of person A
(Called “attribute information” in the present embodiment) records that the name of the person is “A” and the age of the person is 26 years old. In addition, this person is the video data 20
“Subspace 1” from 0:00 to 0:01 of 1
(Or scene 1) 211 and appear in “subspace 3” (or scene 3) from 0:05 to 0:06. For this reason, in the system of the present embodiment, the link information is provided between the attribute information “Mr. A” 221 and the “subspace 1” 211 and between the attribute information “Mr. A” 221 and the “subspace 3” 213. "Appearances" 231 and 232 are added.

As the attribute information, "A"
Other than the character string such as “c: / c” as in the attribute information 222.
ar.doc "and other external file link information. Other information such as car and background color information expressed in ternary color values (RGB values) of light, or special information that can be interpreted only by a certain application. It may be possible to embed various byte string data.

FIG. 3 is an example of a data structure obtained by converting the conceptual data structure of FIG. 2 so that the computer device 101 of FIG. 1 can actually handle the data structure. In the present embodiment, information on one subspace, one attribute information, etc.
+ Handled as object data used in programming languages. In the C ++ object data, a plurality of variables can be managed collectively. For example, information of the subspace 211 is expressed as object data 311.
The start time “0:00” of the subspace is assigned to the variable begin in the object, and the end time “0:01” is assigned to the variable e.
The information of the video data 201 that is the basis of the subspace is similarly expanded to the variable “file” “sample.mpg”.
Similarly, the subspaces 212 and 213 are also represented as objects 312 and 313 of the subspace.

The attribute information 221 includes the object data 3
It is converted to 31. However, since the attribute information 221 has a plurality of (attribute name, attribute value) pairs, in the present embodiment, “meta (attribute name, attribute value)” is included in the attribute set object.
It has a structure with multiple different objects. Similarly, the attribute information 222, 223 is converted into attribute information objects 332, 333.

In the link information of FIG. 2, one link object is prepared for one link for simplicity in this embodiment. The link object includes a link type type and reference information sr of the link source object.
c, Reference information dst of the link destination object can be registered. For example, the link 231 in FIG. 2 is converted to a link object 321. The name “appearance” of the link 231 is stored in a variable type in the link object 321. The variable src of the link object 321 includes an attribute information object 33 obtained by converting the attribute information 221 that is the link source of the link 231 into the object of FIG.
1 is registered. In this figure, the value of src is set to the variable id of the attribute information 331 in order to clearly show the link destination.
In practice, a value suitable for computer processing, such as the address value of the attribute information object 331 on the memory, is registered. Link object 32
Similarly, the variable dst of the subspace object 3 corresponding to the “subspace 1” shown at the end of the link 231
11 will be shown. Similarly, the links 232, 233, and 234 in FIG.
2,323,324.

Finally, in the system of this embodiment, a plurality of subspace objects 311 to 313,
The link objects 321 to 324 and the attribute objects 331 to 333 are stored in the set data 30 of the partial space.
1, link set data 302 and attribute set data 303. In order to realize such set data, a Set type included in the C ++ Standard Template Library or the like may be used. If you use these libraries, you can extract elements one by one from the set data,
Processing of set operations such as sum and difference can be easily realized.

The link realizing method shown in the conceptual diagram of FIG. 2 can be applied to methods other than the method of creating a link object according to the present embodiment. For example, a method may be used in which the attribute information object has an array of pointers that refer to the subspace objects. Similarly, a subspace object may have a dereference pointing to attribute information.

FIG. 4 is a diagram for explaining the specifications of the data structure shown in FIG. 3 in more detail, and is a diagram showing the relationship between object classes. The class here is the same as a term used in object-oriented programming such as a C ++ class. In addition, the class notation shown in FIG.
It is based on the ML (Uniformed Modeling Laguage) scheme.

In FIG. 4, the class of the subspace object in FIG.
It becomes 4. The class of the attribute information object is the class “Attr” 405. The class of the link object is the class “Link” 403. In this embodiment, for the sake of simplicity, a superclass “O” that can uniformly handle the Frag and Attr class objects is used.
object "402 is prepared. By providing the object, the set 301 of the subspace and the set 303 of the attributes in FIG. 3 can be easily handled as one set.
The object class 402 includes a member variable cl for distinguishing between Attr and Frag.
and a member variable id for naming the object.

In the present embodiment, a variable selectColor is also provided so that objects can be displayed in different colors depending on the selected state of the objects. In addition to the method of providing selectColor, a method of selecting a color according to the selected state when displaying an object by providing a selection state variable select or the like may be adopted. In the present embodiment, selectColo
By realizing the r variable as a list type, a plurality of colors can be simultaneously registered in one object. As a result, not only the color of the mouse selected state, but also the subspace and attribute information associated with the selected object are displayed on the screen in different colors, and a separate search condition is specified for various search conditions specified by the user. Colors can also be registered, and the user can confirm the type of the object at a glance when looking at the screen.

The member variables of the Frag class 404 include:
There are begin, end, and file described in FIG.
The class “Meta” 406 is for providing the Attr class 405 with a plurality of (attribute name, attribute value) information. The Meta class 406 includes a member variable name representing an attribute name and a member variable contents representing an attribute value.
There is. The Attr class 405 has a set-type member variable mlist in order to have a plurality of Meta class objects.

The member variables of the link class 403 include:
There are type, src, and dst described in FIG. sr
Both c and dst can refer to an Object class object.

Finally, the class "this system" 401 will be described. The class “this system” 401 includes
The scene information to be edited includes a member variable S which is a set of a subspace and attribute information and a member variable L which is a set of links.

FIG. 5 is an example of an execution screen of the system of this embodiment. In the main window 501, three (sub) windows for displaying scene information in an easy-to-understand manner are displayed. The first is a timeline display window 502 and the second is an icon display window 5
04 and the third is a table display window 503.
Note that, if there is a means other than these that can express scene information on the screen, these three types of windows need not be displayed. Conversely, not only one window but a plurality of windows may be provided. If a plurality of windows are provided, it is possible to grasp one scene information from multiple sides.

The main window 501 is provided with a main menu 506, by which reading and writing of a file of scene information, editing operation, search operation, and the like can be instructed.

Hereinafter, the timeline display window 5
02, the icon display window 504, and the table display window 503 in this order. afterwards,
A user operation method will be described.

In the timeline display window 502,
By displaying the scene information along the horizontal time axis on the screen, the temporal relationship between the scene information can be grasped at a glance. In the present embodiment, the video data 51 to be attributed is displayed in the timeline display window 502.
1, audio data 512 attached to the video, and user data 513, 514, and 515 in which information such as attribute information and partial space added by the user are displayed for each type of information. Note that the timeline display window 5
In 02, it is sufficient that only one user data is displayed on the time axis (for example, only 513), and the other data 51 is displayed.
1, 512, 514, 515, etc. need not be displayed. If the direction of the time axis is set within the screen, it may be not only horizontally but also vertically or in the depth direction during three-dimensional display. In FIG. 5, the time is 0:00.
From 0:06 to scene information of a time section. In the video data 511, a frame image in the time section is displayed. The audio data 512 displays an audio waveform graph during the time section.

"Partial space list" 51 of user data
In 3, the scene information stored in this system is
A plurality of subspace objects are represented on a time axis. For example, “subspace 1” 311 in FIG.
This is data representing information from 00 to 0:01, and this information is represented as a partial space bar 521 displayed from 0:00 to 0:01 on the screen in FIG.
Similarly, “subspace 2” 312 and “subspace 3” in FIG.
313 is represented as subspace bars 522, 523. Thus, the subspace object has the member variable b
It is displayed at a coordinate position on the screen according to the values of egin and end. This time coordinate conversion can be calculated by a simple proportional relationship from the position and width of the timeline display window 502, and the display start time and end time in the window.

It should be noted that the user data is stored in the “subspace list” 5
13, “Subtitle list” 514, and “Zoom list” 51
In order to classify and display the data as shown in FIG. 5, three pieces of scene information data for “subspace list”, “subtitle list”, and “zoom list” are prepared, and the corresponding timeline display windows are provided. It may be created separately. As a simple method of preparing scene information data separately, there are a method of opening three scene information files and a method of providing three parts in one scene information file. As another method, it is also possible to search and extract only information relating to the “subtitle list” from one piece of scene information data. In other words, when the scene information data is considered as a set as shown in FIG.
Subset, "subtitle list" subset, "zoom list"
This is equivalent to extracting and displaying a subset for use.

The plurality of subsets need not be mutually exclusive, and may have a common part. For example, if a “Ms. A list” subset and a “house list” subset are defined for the set of FIG.
Belongs to both subspaces. When the “A list” subset and the “house list” subset are separately displayed in the timeline display window, the subspace bar of “subspace 3” appears in both displays.

The icon display window 504 is a window for visually enumerating and displaying subspace objects or attribute objects included in scene information in an icon format. By using this window, a plurality of objects included in the scene information can be visually displayed in a list, and the user can grasp the contents of the scene information at a glance. In the timeline display, only data during a certain time section of the whole can be displayed, but in the icon display window, scene information according to certain conditions can be enumerated ignoring time, and many scenes can be displayed in less screen space View information.

In the present embodiment, the information of the three subspaces shown in the example of FIG. 3, ie, “subspace 1” 311, “subspace 2” 312, and “subspace 3” 313, is displayed in the icon display window. , Icons 541, 542, 5
43 is displayed. In the case of the present embodiment, the icon is composed of image data that visually represents the contents of the object, and a character string attached thereto. In the present embodiment, in the case of the icon of the partial space object, a frame image of the video data at the time position of the member variable begin is used as the image data. Thereby, the contents of the subspace can be visually grasped. In the case of an icon representing an attribute object or an icon representing a partial space for audio data, an image cannot be easily obtained from video data, and thus, fixed pattern data may be used.

The icon image need not be a single still image. For example, when a moving image is used for the icon, the video from the begin to end of the subspace may be constantly reproduced. In addition, it is also possible to arrange or switch about four arbitrary images from begin to end of the partial space to form one icon image.

The table display window 503 is a window for displaying the contents (contents = values of member variables) of the subspace objects, attribute objects, and link objects included in the scene information in a table format. With this window, more detailed attribute information that could not be displayed by the timeline display or the icon display can be displayed more. I will not explain the details of the processing method here,
Using this window, the user can not only check the contents but also add, delete, and change attribute values.

FIG. 5 shows that “subspace 3” 523 is
This is the screen at the time of selection at 05, and all information related to “subspace 3” is displayed in the table display window 503. For example, 531 indicates that “name is A” and “age is 26” as attributes of “A”. Reference numeral 532 represents attribute information of “house”.

When displaying the information of the subspace object instead of the attribute information in the window 503, the name of the subspace and the member variables begin, end, sr
A value such as c may be displayed. When displaying link information in this window 503, the member variables type, sr
The values of c and dst may be displayed. When the details of the subspace object and the link object are displayed, the table becomes large and difficult to understand. Therefore, an option for displaying only the attribute information in the table may be prepared.

Further, in order to make the table easy to see, a display of a tree structure may be added beside the table. For example, currently 5
In 31, “name” and “age” are displayed as they are below the line of “attr, A”, but in the case of a tree structure display, “attr, A” is the parent node,
If "name" and "age" are the child nodes, the display will be easier to understand. Similarly, if the link object is set as a parent node and the link destination data is displayed as a child node, the detailed contents of the link destination can be confirmed only by the operation of displaying the tree structure.

As a final explanation of FIG. 5, a method of operation by the user will be briefly described. Hereinafter, first, the selection operation at the time of clicking will be described, and then the editing operation method will be described.

The user can operate the visually represented scene information by using the mouse 505. For example, when the user clicks the timeline display (subspace bar) 523 corresponding to “subspace 3”, the subspace bar 523 is selectively displayed. At this time, in the present embodiment,
The same data 543 displayed in the icon display window 504 is also selectively displayed. At this time, in the present embodiment, the selected object itself and the attribute information object related thereto are displayed in the table display window 503.
Is displayed.

In the present embodiment, when an object is clicked to be in a selected state, an object related to the object (ie, a linked object) is expressed in a related selection display, a color different from the selection display. Is done. For example, “subspace 3” 523 and “subtitle 3” 5
Assuming that a link of a “containment” relationship has been established between the “subtitle 3” 551 and the “zoom 2” 552, the “subtitle 3” 523 is selected.
It is displayed in a different color from.

In the present embodiment, only scene information according to a certain condition can be expressed in another color. For example, the scene in which Mr. A appears is blue, only a certain CM image is green, the ending scene is hidden (transparent), and all the characters appearing in (associated with) a certain scene are red. , Respectively. This specific operation method and realization method will be described later in a structured flowchart.

As described above, in the present embodiment, complex related information in the scene information is displayed in different colors.
Scene information can be grasped more easily.

Next, an editing operation method in the system of the present embodiment will be briefly described. In the timeline display window 502, the user can add / delete a partial space by specifying an hour point in the window. The time position of the subspace bar can be changed by dragging the mouse.
Also, a link can be set by selecting a plurality of subspaces. In the icon display window 504, a subspace and attribute information can be added and deleted. In the table display window 503, the attribute value can be changed. When the partial space, attribute information, or link information is added or deleted, elements may be added or deleted from the sets 301, 302, and 303 shown in FIG. When changing the value of an object, a corresponding object set may be searched from the coordinate position selected by the user, and the value of the object may be changed. A more specific method of implementing the editing operation will be described with reference to the structured flowchart in FIG.

Next, a screen display processing method in the present embodiment will be described with reference to the structured flowcharts of FIGS. 6, 7, 8, 9 and 10. FIG. 6 shows a main process of the system according to the present embodiment. FIG. 7 shows a timeline display process. FIG. 8 shows an icon display process. FIG. 9 shows a table display process and a subroutine related thereto. FIG. 10 shows a selection process at the time of a user mouse click and a subroutine related thereto.

First, the main processing 600 of the system according to the present embodiment shown in FIG. 6 will be described.

When the system is started, first, step 6
01, Objectc which is a member variable of this system
set S0 of t (= set of subspaces 集合 set of attributes) and Li
The set L0 of nk is initialized. The method of this initialization is as follows. For new creation, S0 = L0 = φ
It may be initialized as When reading from a file, a file of the scene information designated by the user may be read and assigned to S0 and L0. As another method, the video search results of the DB are used as scene information for S0 and L0.
Or a video analysis library manufactured by Hitachi, Ltd.
Scene information of video data analyzed using Mediachef Component may be added to S0 and L0. In addition,
The sum of S0 and L0 is the entire media scene information in the present embodiment, and the memory area storing S0 and L0 is the media scene information storage means.

Next, in step 602, a mouse selected object set SS and a related object set SR used in screen display processing and mouse selection processing, and a registration condition list CL used in search processing are initialized with an empty set φ. I do.

Next, at step 700, a timeline display window 502 is created and a timeline display process is performed.
The timeline display window 502 is initially drawn by calling DrawTimeLine (S0, L0). Step 8
At 00, an icon display window 504 is created,
By calling the icon display process DrawIcon (S0, L0), the icon display window 504 is initially drawn. In step 900, the table display window 503 is created, and the table display window 503 is initially drawn by calling the table display process DrawTable (SS, L0). In the present embodiment, a set SS of scene information currently selected is passed as a parameter of DrawTable. Thereby, in the table display window 503,
Detailed contents can be displayed for only the currently selected scene information.

As described with reference to FIG. 5, a plurality of these three types of windows may be created, or may not be displayed when another display means is used.

Next, in a loop process 603, the user operation analysis process from step 621 is repeated until a user end instruction is issued. In the user operation analysis process, step 62
In step 1, the user waits for a user instruction from the input device 115, and then in step 622, a user operation process according to the user instruction is performed. When the user clicks the mouse or the like, a selection process is called in step 1000. When the user gives an instruction to add / delete / change an object such as a partial space, attribute information, or link information, in step 1500, an editing process is performed in accordance with the editing instruction. When the user selects the search menu, a search process is performed in step 632. Note that, regarding the flowchart of the search process 632, only the condition registration menu and the already registered condition menu among the search menus will be described later.

Next, a time line display process will be described with reference to FIG.
DrawTimeLine (S, L) (step 700) will be described. Dra
As an input parameter of wTimeLine (), O
b set of objects (= set of subspaces∪set of attributes)
And a set L of Link.

First, in step 701, all elements of the set S are examined to extract a set consisting of only the subspace from the set S of objects consisting of the subspace and the attribute information, and the member variable class of the element e of the set S is extracted. Is performed to extract only the element whose value is “frag” and add it to the set F. Note that the set S
May be divided into a set F of subspaces and a set of attributes. Next, for all elements e of the set F of subspaces,
A loop process 702 is performed to display the partial space bar of the timeline display.

In the loop process 702, first, in step 711, the time value of e.begin and the time value of e.end are converted into the coordinate position bx, ex on the time axis on the screen. To convert a certain time t to a coordinate x on the screen, coordinate x = time to coordinate conversion (t) = (t−start time of time axis on screen) ÷ (end time of time axis on screen−start) Time) x width of the time axis on the screen + start coordinate of the time axis on the screen.

Next, at step 712, a rectangle having upper left coordinates (bx, 0) and lower right coordinates (ex, 10) is drawn to draw the outer frame of the partial space bar. Next, the subspace name e.id is displayed in the subspace bar. The location is (b
x, 0). Next, in step 713, a small icon image is displayed at the head position of the partial space bar. As a result, it is possible to instantly confirm what kind of content the video data is by looking only at the partial space bar. The image is e.fi
Open the video file specified as le and the time position in it
Display the frame image in e.begin. The display position of the image is set to coordinates (bx, 30) and is displayed slightly below the preceding character string.

Finally, at step 714, step 71
The rectangle indicated by 2 is displayed using e.selectColor, which is the display color information of the object. For example, e.sel
The color of ectColor may be displayed as a square fill color as in the selected subspace 523 in FIG. 5, or may be displayed as a square frame color as in the related selected subspace 551 in FIG. You may. Note that multiple colors may be registered in selectColor at the same time.In this case, the subspace is surrounded by a plurality of different color frames, or a color icon such as a small flag is placed at the top of the subspace. What is necessary is just to take the method which can be distinguished on the screen, such as arranging by the number of colors.
In addition, when the color list of selectColor is empty, or when a predetermined transparent color or the like is set, a method of not displaying the corresponding partial space may be used. In addition, sele indicating the selection state and search result state of the object
At present, ctColor is multiplexed with objects on the screen using colors, but these identification states may be expressed using an expression format that can be identified on the screen other than colors. For example, the fill pattern of the partial space may be changed depending on the selected state, or a small flag-like icon using an identification number, an identification character string, an identification icon, or the like may be displayed on the screen for each identification state.

The time line display processing 700 shown in FIG.
In the above, only the partial space is subjected to the timeline display, but the timeline display can be applied to the attribute by the following method. To do so, first, a certain attribute a
After extracting a set of objects linked to, by a function GetLinkedObj (a, L0) described later (step 902), this set is set as a new S, and the step 701 and the loop processing 702 may be performed. In the case where the display target is a set including a plurality of attributes, the display processing of the one attribute may be repeated for all attributes.

Next, referring to FIG. 8, the icon display processing Draw
Icon (S, L) (step 800) will be described. DrawIcon ()
DrawTimeLine () in step 700
A set S of objects and a set L of links are passed.

First, in step 801, step 701 is executed.
A set F of subspaces is extracted from the set S by the same method as in. Next, in step 802, the display coordinates x, y of the icon are initialized to the upper left coordinates (0, 0). Next, a loop process 803 is performed to display icons for all elements e of the set F of the subspace.

In the loop processing 803, first, at step 811, a frame image at the time position of e.begin of the video data e.file is displayed at coordinates (x, y). Next, in step 812, the name e.id of the subspace and the start time e.begi
Displays n and end time e.end as character strings. Since the display position of the character string is below the image, the coordinates are (x, y + the height of the image). Next, in step 813, the place where the image and the character string are located is the color of the selected state of the object e. SelectCol
Display using or.

Next, a process of changing the display position of the icon is performed. In step 814, the width of the icon and the width of the space between the icons (gap width) are added to x in order to display the next icon on the right side. At this time, if x exceeds the width of the icon display window 504 (condition 815), in step 821, the position of x is returned to the left, and the display position is line-feeded. Add the height (the height of the image and text) and the height of the vertical space. of course,
If the y coordinate exceeds the height of the icon display window 504, a process for stopping the loop 803 may be included.

The icon display processing 800 according to the present embodiment
Although only the partial space is a display target, attribute information and link information can also be displayed as icons by adding the following processing. For example, after step 803, a process of extracting only the set A of attribute information from the set S of objects (A ← SF) is performed, and the icon display process of one attribute information is repeated for all extracted attribute information. good. When representing one attribute information as an icon,
As the icon image, a predetermined figure indicating an attribute may be used, and as the icon character string, meta information (= (attribute name, attribute value) set) included in the attribute information may be used.

At the end of the description of the display processing, table display processing DrawTable (S, L) (step 90) will be described with reference to FIG.
0) and its related subroutines will be described. The input parameters of the table display process DrawTable () include the same set of Objects S as in DrawTimeLine () in step 700.
And a set L of Link.

First, in step 901, the object associated with the current selected state S is assigned to the set R by using a function GetLinkedObjSet () for extracting all objects associated with the object set S. Next, in a loop process 902, a process of outputting the contents of all the elements e of the set R of the related objects as rows of the table is repeated.

In the loop processing 902, first, in a conditional step 911, table output processing is performed for each type of object. When the type of element e is attribute information (e.class = "a
In the case of "ttr"), the process of outputting the details of the attribute information is performed in step 921 and thereafter. First, in the loop process 921, the metadata set of (attribute name, attribute value) included in the attribute information e is output. The process of step 922 is performed on all elements m of e.mlist. In step 922, a row object of the table is created to add one meta object m to the table, and the “ID” column of the row is created. E.id, m.name in the “name” column of the row, m.co in the “value” column of the row
ntents, and finally this row object is added to the table. The table display window 50 shown in FIG.
In No. 3, the "ID" column is not displayed.

If the type of the element e is a partial space in the conditional step 911, three rows are added in a step 925. The row added to the table in step 925 includes a video file name (e.id, "file", e.file) and a start time (ei.
d, "begin", e.begin) and end time (e.id, "end", e.en
d).

In step 912, this time step 922
And 925 are displayed using the color e.selectColor of the selected state of the object.

Next, step 90 which is a subroutine function
The details of No. 1 GetLinkedObjSet (S, L) will be described. The input parameters of this function are a set S of objects and a set L of links, and the return value of this function is a set obtained by extracting all objects associated with the set S of objects. In this process, a set R, which is a return value, is initialized with an empty set φ in step 941, and a loop process 942 is performed on all elements e of the set S, thereby obtaining a result set R. In the loop processing 942, first, in step 951, a set of objects associated with the element e is obtained by GetLinkedObj (e, L), and is substituted into SL. In the next step 952, the obtained set SL is added to the result set R. Finally step 94
At 3, the set R of related objects determined for all elements is returned.

As the last part of FIG. 9, the subroutine GetLinke
Details of dObj (e, L) (step 951) will be described. The input parameter of this function is a set L of one object e and a link, and the return value of this function is a set obtained by extracting all objects associated with one object e. In this processing, first, step 971
Then, a set SL which is a return value is initialized with an empty set φ,
Next, a result set SL is obtained by performing a loop process 972 on all the elements link of the link set L. In the loop process 972, first, in step 981,
Check the contents of the link. If the link source of the link is the own object e, the link destination object link.dst is added to the result set SL (step 98).
3). If the link destination of the link "link" is the own object e, the link set object link.src is added to the result set SL.
Is added (step 985). Finally, in step 973, the set SL of related objects obtained for all link elements is returned.

FIG. 10 shows the details of the selection process (step 1000) in response to the user's click input. The input of this processing is the coordinate position (x, y) clicked by the user.

First, in step 1001, the current display format of the position clicked by the user is checked.

When the display format is the timeline display, a process for specifying the selected partial space is performed in step 1011 and thereafter. First, in step 1011, the coordinate value of the click position x is changed to a time value on the screen time axis, and is substituted for the selection time t. For this calculation, the inverse function of the time-to-coordinate transformation (t), which is the function described in step 711, may be used. Next, in step 1012, a function FindObject () is called to extract a subspace object including the selection time t from the object set S0, and the result is substituted into a new selection set SSnew. In the FindObject function, a condition function Within (*, t, t) indicating that a time position t clicked by the user is included as a search condition is specified as a parameter. This designation designates that an object including the section tt, that is, the object including the time point t is to be extracted. Details of how to use the conditional function and its definition will be described in a method of implementing search processing described later.

In step 1001, if the display format of the clicked position is an icon, step 10
At 14, the corresponding subspace object is obtained from the selected icon and is assigned to a new selection set SS.
Set to new. In order to realize this, when drawing the icon, the corresponding partial space object may be registered in the display icon information for remembering the display position and the image of the icon.

If the display format of the clicked position is a table in step 1001, step 10
At 17, only the clicked line is selected and displayed, and at step 1018, the main selection processing 1000 is terminated. The reason for terminating the function here is that if you change the selection state when clicking like other display formats, just click on the table and the display contents of your own table will be updated in the subsequent display update process This is because the information has been changed and the information cannot be operated correctly. Here, the selection state is not changed only when the table is clicked.

At step 1002, a set of objects associated with the set of objects SSnew selected by the user is obtained by GetLinkedObject (SSnew, L0), and is substituted into the related set SRnew. Finally, in step 1003, the selection state of the mouse is updated with SSnew and SRnew obtained by this function.

Update processing of mouse selection in step 1003
NewSelection will be described in detail. The parameters of this function are a new set of selected objects SSnew and a new set of related objects SRnew.

First, in step 1051, the mouse selection color and the mouse selection related color registered in the currently selected object are removed. Specifically, a loop is performed for all objects obj included in the global variables SS (set of selected states) and SR (set of related selected states), and a mouse selection color and a mouse are selected from the selection color list obj.selectColor. Delete the related selection color.

Next, in steps 1052 and 1053, the mouse selection color and the mouse-related color are registered in the new selected state object and the new related selected state object, respectively. Specifically, an element is extracted for each set of SSnew and SRnew, and color information corresponding to selectColor of the corresponding object is added.

Finally, in step 1054, the screen is redisplayed using the new color information. In step 1055, new selection information SSnew and SRnew are registered in global variables SS and SR.

Next, a search process in the system according to the present embodiment will be described with reference to FIGS. FIG.
Is an execution screen of the search process. 12 and 13
Is a structured flowchart of a search process. FIG.
9 is a table showing a method of implementing a search condition function used in search processing.

First, the execution screen of the search process will be described with reference to FIG. The search process is performed in the main menu 50
6 is performed by selecting a search menu. The search menu includes a menu 1101 for registering conditions for search and a menu 1102 for displaying a list of registered conditions. Condition registration menu 1
When 101 is selected, a registration screen for conditions as shown in 1105 is displayed.

On the condition registration screen 1105, a display method of search results and search conditions can be specified. After setting these pieces of information, a search process can be executed by pressing a search button 1151. The search result is displayed on the screen in the color specified on the condition registration screen. Specifically, of all the objects included in the media information, only the objects that satisfy the conditions specified on this condition registration screen are
The color information specified on the condition registration screen is displayed on the main window 501. The cancel button 115
Pressing 2 terminates the search process.

In the search result display method that can be specified on the condition registration screen 1105, the name 1111 of the search condition and the display color 1112 of the search result can be specified. Search condition name 11
Numeral 11 is a name displayed when this condition is registered in the search menu later. Also, the display color 1112 of the search result
Is a color used when displaying an object satisfying the search condition. This color selection can be performed, for example, such that colors determined by clicking on 1112 appear one after another. Further, as an application example, a transparent color may be specified in the selected color. For example, if a transparent color is specified, the search result will not be displayed, and the user will be able to hide redundant scene information on the screen by this search processing, making it easier to check and edit scene information. Can be more efficient.

As the search conditions that can be specified on the condition registration screen 1105, a method of searching within a certain time range, a method of searching only a certain attribute name, and a method of searching only a certain attribute value can be specified. These are a check box 1121 for time search, a check box 1131 for attribute name search,
Using the check box 1141 for attribute value search, a plurality of items can be specified at the same time. In this embodiment, when a plurality of conditions are specified, those conditions are AND
Think of them as being joined by a condition. When each check box is ON, the input area of the start position 1122 and the end position 1123 of the time section, the attribute name 1132, and the attribute value 1142 are enabled as the corresponding condition values, and the values can be set. Become like

As another method of specifying the start position 1122 and the end position 1123 of the time, a scroll bar for changing the display time position is attached on the screen of the timeline display 502 in FIG. A method of specifying a time range to be searched by an operation may be adopted. Conversely, the timeline display 50
2 may be considered as a user operation screen for retrieving and displaying scene information between the display start time and the display end time designated by the user. Similarly, the user selection processing described in FIG. 10 may be considered as a user operation method for dynamically searching for an object related to the object selected by the user.

The contents described here are one embodiment of the present invention, and various other conditions and setting items can be designated. For example, as a search condition, an OR of a plurality of search conditions, a complex search condition expression is input, only an object associated with a certain search result is extracted (using a subroutine 901 GetLinkedObjSet () in FIG. 9), It is possible to specify a related name at the time of search, and also to open a file of media data registered in a subspace and search for a scene including the moment when the corresponding building appears using image analysis, etc. You may do it. In such a case, it is desirable to display a list of search conditions and dynamically add search conditions from the viewpoint of realizing processing and operability.

In this embodiment, when a search is performed using the search button 1151, the searched object is
Is displayed in the color specified by, and at the same time, the mouse is selected as an editing operation target. The user can update or delete the search result as needed. Further, in the present embodiment, when a search is performed, search conditions for the past several cases are stored,
The search condition name is displayed on the search menu together with the search result display color. By looking at the search menu, the user can confirm, after the search result, which color object on the screen satisfies what condition. Further, by selecting this menu, the search process can be executed again using the search condition specified last time.

As an application example of the search operation method, a menu “to next search result object” may be added to the search menu. In this case, rather than displaying the search results at once, by selecting this menu many times, the user can perform the editing process while checking the objects of the search results one by one. This menu is also very effective when the aforementioned specific object is searched by using an image analysis method. This is because it is not necessary to analyze all the media data at the time of executing a search, and when the user desires the next search result, the image analysis may be partially applied one by one, and the waiting time of the user can be greatly reduced. Because you can.

FIG. 12 and FIG. 13 are structuralized flowcharts of the search processing in this embodiment. Here, the processing of the “register condition” menu 1101 and the registered condition menu 1102 of the search menu will be described.

Step 1200 is processing when the condition registration menu 1101 is selected. Step 121
In step 1, a search condition registration screen 1105 is displayed, and the user is caused to register search conditions. Next, step 1212
At step 1215, the search condition formula is assembled as a list of condition functions funcList, and at step 1202, a search is executed using the assembled condition function list funcList, and the search results are displayed in the designated color on the condition registration screen 1105. . Finally, in step 1216, information on the search condition specified this time (condition name, color, condition function list) and the like are added to the registration condition list CL, which is a global variable.
At 217, the current condition name and color information are added to the search menu of the main menu 506.

Next, a method of realizing and assembling a search condition expression will be described. The condition function registered in the condition function list funcList is represented in the memory by (function address, user parameter value). In the present embodiment, the start address of the condition function shown in FIG. 14 can be registered as the function address. To execute these condition functions, add the address of the media scene information object whose condition is to be determined together with the user parameter value specified here as a function parameter, and call the specified function address. good. The result of the function is defined to be returned to the calling process as a set value (φ if the condition is not satisfied, other than φ if the condition is satisfied).

Using the condition functions shown in FIG. 14, the processing for assembling the conditional function formulas in steps 1212 to 1215 will be described in detail. First, in step 1212, the condition function list funcList is emptied. Next, in step 1213,
It is checked whether or not the user has designated time search on the condition registration screen 1105.
At 21, the time condition function Within is added to the condition function list funcList.
Add (). At this time, the start time and the end time specified by the user are registered as user parameters of the conditional function Within. Step 1 also applies to the search for the attribute name.
At 214, it is determined whether or not there is a user's search instruction, and when performing an attribute name search, the condition function SameName () is registered in funcList at step 1223. The user parameter at this time is an attribute name specified by the user. Similarly, in the search for the attribute value, in step 1215, it is determined whether or not the user has issued a search instruction.
At 223, the condition function SameValue () is registered in funcList. The user parameter at this time is an attribute value specified by the user.

In the present embodiment, search conditions for time, attribute name, and attribute value have been described. However, other conditions can be similarly extended. Basically, when a search condition is added, an individual condition setting means and a condition function that inputs an object and a user parameter and returns a set value may be prepared for each search condition. In the search condition module, a setting screen may be provided as necessary so that the user can specify whether or not a corresponding condition is specified and set detailed condition values.

Next, the processing 1201 when the menu of the already registered search condition is selected from the search menu
Will be described. If the search condition is already registered, the information cond of the corresponding search condition is obtained from the search condition list CL using the menu number value designated by the user (step 1231). Since the search condition information cond includes a search condition list necessary for executing the search and color information necessary for displaying the search result, if the search execution process 1202 is called using these values, the user Can be executed again with the previously set conditions (step 1202).

Finally, the search execution processing 1202 will be described. The input parameters of this processing include a list of search condition functions set previously and a display color of the search result. First, in search, in step 1241, the search result set Sr is initialized with the search target set S0. In this example, to search the search condition list as a list of AND conditions, Sr is first set to a global variable S0 representing all objects.
Next, all search condition functions included in the search condition list funcList
A loop is performed for func (step 1242), and the search condition function fu is obtained using the FindObject () process of step 1300.
The result set Sr is narrowed down by nc. Loop processing 124
After the completion of step 2, AND for all search conditions
Is stored in Sr.

Next, the loop processing 1243 is performed on all the elements obj of the search result Sr, and the designated search result color
Is added to the selection color list selectColor of the object (step 1252). Finally, in step 1244, a mouse selection updating process is performed so that the search result can be handled in the same manner as the mouse selection state for later editing. Note that since the screen is redrawn in the mouse selection update processing, the search result color set in the loop processing 1243 is also displayed on the screen.

Next, referring to FIG.
FindObje: Extraction of Objects Satisfying the Search Conditions
ct () will be described. The input of this function is a set S of search target objects and one search condition function f ().
The output of this function is a set of objects in S that satisfy the search condition function f ().

In this function, first, a set Sr of search results is emptied (step 1310), and then, in a loop process 1311, conditions are determined for all the search target objects obj in S. To determine the condition, the object obj
Condition function f to check whether satisfies the search condition
(obj) is called (step 1321). At this time, the user parameters set in advance in step 1221 in FIG. 12 and the like are also passed at the same time. If the determination result of the condition function is true, an object set to be added to the search result set Sr is returned as a result of the condition function.
Add to Sr. When the judgment result is false, the result of the judgment function is assumed to be an empty set φ. Loop processing 1
Since the search result is stored in Sr at the stage when 311 ends, this is set as the return value of this processing, and this processing ends.

FIG. 14 is a table showing a method for realizing a search condition function used in the search processing.

First, the condition function Within () 1401 for determining an object including a certain time interval will be described. Within ()
Has a start time tb and an end time te as user parameters. As a result, the object specified as the function parameter is a subspace (object.class = "flag"), and the real number set [object.begin, object.end] and the real number set [t
b, te], returns a set {object} as the content to be added as an extraction result. That is, if the subspace object touches even a part of the section [tb, te], {object} is returned. If the object is attribute information, an empty set φ is returned.

Next, the condition function SameName () 1402 for determining whether the attribute names are the same will be described. SameName ()
Is the attribute name name. The condition that this function is satisfied is that the object specified as the function parameter is an attribute (object.class = "attr"), and the attribute name meta for any one meta of the Meta class list mlist of the attribute object If .name is equal to name, return {object} as the return value.

Similarly, in the case of the condition function SameValue () 1403 for determining whether the attribute values are the same, the user parameter is the attribute value contents, the object specified as the function parameter is the attribute, and the attribute value is meta.con
Returns {object} if there is at least one meta with tents = contents in the Meta class list. Note that SameName () and Sa
meValue () is used if the above condition is not satisfied, or
If the object specified as a function parameter is a subspace, it is an empty set φ.

Finally, as an application example of the conditional function, a conditional function GotIt () which returns a partial space at the moment when a person enters the screen will be described if the partial space is at the moment when a person enters the screen. In GotIt (),
If the object specified as a function parameter is a subspace, open the video file indicated in object.file, analyze the video using image processing, and recognize that a person has entered the screen, and then add a new Space object ne
Generate wObj and set the appearance time of the person to newObj.begin and newOb
Specify in j.end and return {newObj}. In order to realize this image processing, for example, an image recognition algorithm described in “Moving Object Detection / Extraction Apparatus and Method” in JP-A-8-221577 may be used. Of course, if a plurality of persons are detected, a set of elements of a plurality of subspaces may be returned.

FIG. 15 is a structural flowchart of the editing process according to the present embodiment. In the present embodiment, as an editing process, a process of deleting the selected object,
A description will be given of a time movement process of moving the time positions of all the selected subspaces by a predetermined time.

First, in step 1501, the content of the user's editing instruction is determined. If the instruction is to be deleted, the loop processing 1510 is performed on all the objects obj included in the set SS of the selected objects in the global variable, and the object obj is deleted from the set S0 of all objects and the set L0 of all links (step 151).
3). If the content of the editing instruction is time movement, first, in step 1520, the user is asked to specify a movement time amount dt. Thereafter, the loop processing 1521 is performed on all the selected objects SS, and if the object is a subspace (step 1530), the start time and the end time of the subspace are both moved by dt (step 153).
3).

Here, the object deletion 1513 will be described. First, in the loop processing 1550, all links are checked to find a link whose end or end is the object to be deleted obj, and the corresponding link link
Is found (condition determination step 1560),
delete link from link set L (step 156)
3). When the loop processing 1550 ends, the corresponding object obj is finally deleted from the set S of all objects (step 1551).

[0122]

According to the present invention, the operation of registering and managing scene information in media data with respect to given media data can be performed efficiently as follows. First, the user can easily register the attribute information for the media data already possessed by the user later, and can easily register the user-specific information without changing the media data itself, unlike conventional video editing software. Can be registered. Since complicated time information, association information, and attribute information between media scene information can be visually represented on a screen, it is significantly easier to grasp the contents of the media scene information and to process the media scene information. In particular,
The temporal context of the media scene information can be clearly displayed, for example, in a timeline display format as described in the embodiment. Further, it is possible to perform a display in which association information between media scene information, which has not been conventionally displayed on the screen, and an existing position of attribute information can be identified at a glance by using colors or the like. In addition, by using a table format or icon format, depending on the amount of information that the user wants to check,
The display density of the media scene can be changed. Further, according to the present invention, media scene information can be searched, selected, and edited by directly operating on these visually understandable expression formats, so that editing can be performed more easily than in the past.

[Brief description of the drawings]

FIG. 1 is a view showing an example of realizing a media scene display / editing apparatus.

FIG. 2 is a conceptual diagram of a data structure of media scene information used in the embodiment.

FIG. 3 is a view showing an example of an actual data structure corresponding to the conceptual data structure of FIG. 2;

FIG. 4 is a diagram showing details (classes) of the data structure of FIG. 3;

FIG. 5 is a diagram showing an execution screen according to the embodiment.

FIG. 6 is an exemplary view showing main processing of the system according to the embodiment.

FIG. 7 is an exemplary view showing a timeline display process according to the embodiment;

FIG. 8 is a view showing an icon display process according to the embodiment.

FIG. 9 is a view showing a table display process according to the embodiment;

FIG. 10 is a view showing a user selection process of the embodiment.

FIG. 11 is a view showing a search processing execution screen according to the embodiment;

FIG. 12 is a view showing search menu processing according to the embodiment.

FIG. 13 is a view showing search condition execution processing according to the embodiment.

FIG. 14 is a view showing a list of search condition functions according to the embodiment;

FIG. 15 is a diagram showing an editing process according to the embodiment.

[Explanation of symbols]

501: Main window, 502: Timeline display window, 503: Table display window, 504: Icon display window, 505: Mouse pointer, 506: Main menu, 511: Video timeline display, 512: Audio timeline display, 513 to 515: timeline display of user data, 521, 522: bar display of partial space, 523: partial space bar in selected state, 531,532 ... attribute information related to selected partial space, 541,542 ... icon display of partial space, 543 ... Selected subspace icons, 551 and 552... Subspaces related to the selected subspace, 1101... Search condition registration menu, 1102... Registered search conditions and their display colors, 1105.

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Akira Nagasaka 1-280 Higashi-Koigakubo, Kokubunji-shi, Tokyo Inside the Central Research Laboratory, Hitachi, Ltd. (72) Inventor Junichi Hatooka 5030 Totsuka-cho, Totsuka-ku, Yokohama-shi, Kanagawa Prefecture F term (reference) 5C023 AA02 AA14 AA38 CA01 CA04 CA05 DA04 DA08 5C053 FA14 HA29 HA40 JA16 KA04 KA05 KA24 LA06 LA11

Claims (13)

[Claims] 1. A media scene information display / edit device for displaying and processing media scene information which is data in which attribute information is associated with a scene which is a partial unit of media data which is data having time information. Data storage means for storing media scene information, media scene information storage means for storing media scene information, display means for displaying the media scene information, and selection means for selecting arbitrary media scene information from the displayed media scene information And media scene information extracting means for extracting media scene information associated with the selected media scene information; and media for displaying the extracted media scene information so that it can be identified on the screen from other media scene information. Scene information display control means Media scene information display editing apparatus according to claim the door. 2. A media scene information display and editing apparatus for displaying and processing media scene information which is data in which attribute information is associated with a scene which is a partial unit of media data which is data having time information. , Media scene information storing means for storing media scene information, display means for displaying the media scene information, and conditions for extracting media scene information from the media scene information storing means. Extracting condition specifying means, and media scene information satisfying the specified extracting condition,
Media scene information extracting means for extracting from the media scene information storing means; and media scene information display controlling means for displaying the extracted media scene information so that it can be distinguished from other media scene information on a screen. A media scene information display / editing device characterized by the above-mentioned. 3. The media scene information display editing device according to claim 1, wherein the display means displays a time axis,
A media scene information display / editing device, wherein a position at which each media scene information is displayed is displayed on a timeline display screen that displays the position on a time axis corresponding to a start time of the information included in the media scene information. 4. The media scene information display / editing apparatus according to claim 1, wherein said display means displays the media scene information on an icon display screen for displaying icons. . 5. The media scene information display and editing device according to claim 3, wherein when any one of the media scene information is selected from the media scene information displayed on the timeline display screen or the icon display screen, the selection is performed. A media scene information display / editing device comprising a table display screen for displaying a list of contents of media scene information related to the selected media scene information in a table format. 6. The media scene information display / editing device according to claim 2, wherein the extraction condition designation means displays a screen for registering conditions for extracting media scene information on the display means, and displays the screen by the user. A media scene information display / editing device, wherein a condition is set by the user. 7. The media scene information display / edit device according to claim 2, wherein the extraction condition specifying means specifies a search character string included in the media scene information, and the media scene information extracting means A media scene information display / editing device for extracting media scene information including a designated search character string. 8. The media scene information display / editing device according to claim 2, wherein the media scene information extracting means associates the media scene information with the target media scene information for determining whether or not to extract the media scene information. A media scene information display / editing device which directly analyzes media data and determines whether or not the media scene information satisfies or does not satisfy a condition based on the result. 9. The media scene information display / editing apparatus according to claim 1, further comprising: media scene information processing means for processing the extracted media scene information. Display editing device. 10. A media scene information display and editing method for displaying and processing media scene information which is data in which attribute information is associated with a scene which is a partial unit of media data which is data having time information, Displaying media scene information; selecting arbitrary media scene information from the displayed media scene information; extracting media scene information associated with the selected media scene information; Displaying the media scene information so as to be distinguishable from other media scene information on a screen. 11. A media scene information display and editing method for displaying and processing media scene information which is data in which attribute information is associated with a scene which is a partial unit of media data which is data having time information, Displaying the media scene information; specifying conditions for extracting the media scene information; extracting media scene information according to the specified conditions; and extracting the extracted media scene information to other media scene information. And displaying the image so as to be identifiable on a screen. 12. A storage medium storing a media scene information display editing program for displaying and processing media scene information which is data in which attribute information is associated with a scene which is a partial unit of media data which is data having time information. A medium, wherein the media scene information display / editing program includes a step of displaying media scene information, a step of selecting arbitrary media scene information from the displayed media scene information, and a program associated with the selected media scene information. Storage medium characterized by comprising a step of extracting media scene information that has been extracted, and a step of displaying the extracted media scene information so as to be distinguishable from other media scene information on a screen. . 13. A storage device for storing a media scene information display / editing program for displaying and processing media scene information which is data in which attribute information is associated with a scene which is a partial unit of media data which is data having time information. A medium, wherein the media scene information display / editing program includes a step of displaying media scene information; a step of specifying conditions for extracting the media scene information; and a step of extracting media scene information according to the specified conditions. And displaying the extracted media scene information so that it can be distinguished from other media scene information on a screen.