JP4660824B2 - Information storage device for storing attribute information of media scene, information display device, and information storage method - Google Patents

Description

The present invention relates to an information storage device that stores attribute information of a media scene , an information display device, and an information storage method .

  In recent years, media data such as video data and audio data has been made easy on computers by reducing the price of large-capacity hard disks, improving the processing performance of PCs, introducing various video and audio compression technologies, and penetrating video on demand. It has become possible to manage, search and watch. In addition, there is an increasing need to efficiently manage, search, and view such media data by taking advantage of the features of computers. The conventional media data management method is based on information management in units of data volume, such as searching by adding simple keywords, titles, author names, etc. to media data files and video / audio tapes. However, the media data information that the user wants is not just volume information such as movie directors and tape titles, but also information in small units such as when and where in the volume or who is talking. There are many cases. In recent years, for such reasons, there is a demand for a system that not only registers information in the entire media data but also registers and manages information in internal units called scenes in the media data.

  In this application, “media data” is not only video data and audio data, but also slide show data with time information attached to still images, closed caption (caption) data with time information attached to text data, etc. It refers to the entire data with time information. A piece of media data cut out is called a “scene”, and information associated with a piece of media data is called “media scene information” or simply “scene information”. The “attribute” is used as a general name of additional information that does not have time information. In the embodiments of the invention described later, “subspace” is mainly used as a term representing a scene of media data in video format.

  Up to now, there are video editing software and automatic indexing system that automatically analyzes video contents and registers scene information as a system that can handle data up to scene units in media data. .

  As conventional representative video editing software, software for creating a new piece of media data by connecting a plurality of media data is known. In such software, a plurality of media data representations that resemble movie films are displayed on the screen, and the film-like media data is operated on the screen directly by the user using a mouse or the like, so that the playback time of the video is displayed. Or a special effect between two pieces of film-like media data can be instructed (Japanese Patent Laid-Open No. 07-046462, etc.).

  As a typical automatic indexing system, for example, when media data for which scene information is desired to be registered is designated, video contents are automatically analyzed, and the analysis result and accompanying time information are displayed in a table format.

  In the above prior art, the following problems remain in registering and managing scene information in media data for given media data.

  In the above video editing software, it is possible to easily grasp the temporal arrangement between a plurality of scenes. However, since the software is a software that creates a piece of media data by combining a plurality of media data, the media already owned by the user It is not possible to register or delete scene information for the data later.

  In the above automatic indexing system, only scene information can be automatically extracted and registered from media data, but user-specific scene information such as the names of characters and people's age that are difficult to automatically extract can be freely added or deleted. Can not. Further, since these data are mainly expressed in a tabular format, there is a problem that it is difficult to grasp the mutual relationship between a plurality of pieces of scene information. For example, in the case of tabular scene information representation, temporal correlations between scene information such as scenes in which multiple persons appear simultaneously, and scenes such as opening, main part, CM, and ending are included in the drama. It is difficult to grasp the scene information because the inclusive relation of the scene such as being cannot be displayed concisely.

  An object of the present invention is to make it possible to more easily grasp the relationship between scene information and to edit scene information more easily when registering scene information in the media data with respect to the media data. Is to make it.

  In order to achieve the above object, the present invention adopts the following configuration. First, as the data structure of media scene information, for example, a set of scenes that are partial units of data (media data) having time information, a set of attribute information registered by the user, and these scenes and attribute information A set of link information that is an association between them is defined as media scene information.

The apparatus according to the present invention includes media data storage means for storing media data itself, media scene attribute information, identification information of the stored media data, and the time location of the media scene in the media data. Information storage means for storing the subspace objects having information so as to be associated with each other so as to be uniformly referred to by the superclass object is prepared. In order to store information in these storage means, in addition to a method in which a user manually registers information using the system described in the embodiment of the present invention, a media data file stored in a video server or a file system is used. Alternatively, a media scene information file may be read, and further, media scene information extracted according to a certain search condition from a huge media scene information database may be stored. The characteristic point of the present invention is in the way of holding data. The data holding method is a data holding method that allows metadata to be added to an arbitrary position later on a piece of already created media data. In the present invention, attribute information is provided separately from media data, and attribute information can be given to a part of the media data later, which is a feature. Conventionally, although a related partial space can be traced based on attribute information, it is not possible to trace from a partial space to an attribute. Therefore, the method of the present invention for selecting a partial space of one piece of media data and displaying related attribute information is novel.

Next, display means for displaying information related to the subspace object together with the time axis at a position on the time axis corresponding to the time location information of the stored subspace object is provided. A selection unit that selects a media scene indicated by the information about the displayed subspace object; an information extraction unit that extracts attribute information associated with the selected media scene information in the storage device; There is provided information display control means for displaying the attribute information so as to be distinguishable from other media scene information on the screen. Is an extraction condition may be set to fixed extraction condition at the time of activation of the device, or may be dynamically set the extraction conditions according to an instruction from the user at runtime. As a result , the user can designate data extraction conditions as much as necessary at a required time, and can perform more efficient confirmation and editing of media scene information.

Further, a link object indicating the association between the attribute information and the subspace object is stored in the information storage means, and the information extraction means performs extraction based on the association information possessed by the link object. This makes it easy to understand the complex relationships between media scene information, and displays only the information that the user wants to edit from the vast amount of media scene information on the screen. Work can be facilitated.

In the present invention, the subspace object further associates the subspace object by storing reference information for the subspace object.

  In the present invention, the following three media scene information expression formats can also be provided in order to express the relationship between the media scene information more easily.

  The first is a timeline expression format for expressing the temporal correlation of media scene information. In this representation format, media scene information is displayed at a screen position according to the time information of the media scene information along the time axis direction defined on the display means.

  The second is an icon display format for visually displaying a list of components of a plurality of media scene information. In this representation format, each piece of media scene information is arranged on the screen in a visually easy-to-understand icon format, so that the components of the media scene information can be grasped at a glance.

  The third is a table expression capable of displaying detailed contents of a plurality of media scene information. By using this expression format, it is possible to display information such as attribute names and attribute values that could not be displayed in the previous two display formats. Also, if this display format is used, attribute names and attribute values can be directly selected, and these values can be easily corrected.

  Other characteristic media scene information display / editing method and apparatus of the present invention will be apparent from the description in the specification.

  By using the present invention, it is possible to efficiently perform the operation of registering and managing scene information in media data for given media data as follows. First, the user can easily register the attribute information later on the media data already owned by the user, and the user-specific information can be easily changed without changing the media data itself as in conventional video editing software. Can be registered. Since complicated time information, association information, and attribute information between media scene information can be visually expressed on the screen, it is greatly facilitated to grasp the contents of the media scene information and process the media scene information. Specifically, the temporal context of the media scene information can be clearly displayed by the timeline display format as described in the embodiment, for example. In addition, it is possible to display the association information between the media scene information that has not been displayed on the screen and the presence position of the attribute information at a glance using colors or the like. In addition, by using a table format or an icon format, the display density of the media scene can be changed according to the amount of information that the user wants to confirm. Furthermore, in the present invention, media scene information can be searched, selected, and edited by direct operation on these visually easy-to-understand expression formats, so that editing can be performed more easily than before.

  Hereinafter, embodiments of the present invention will be described with reference to FIGS. First, a computer apparatus used in this embodiment will be described with reference to FIG. Next, the data structure in this embodiment will be described with reference to FIGS. 2, 3, and 4. 5 and 11 are execution screens of the media scene information display editing apparatus according to this embodiment. Finally, the processing method of this embodiment will be described with reference to the structured flowcharts of FIGS.

  In this embodiment, for the sake of simplicity, a video scene information display method for adding user-created attribute information to video data will be described. However, the present system is not only video data but also audio data, slide show data, etc. It can be easily applied to other types of media data having time information.

  First, a computer apparatus used in an embodiment of the present invention will be described with reference to FIG. Reference numeral 101 denotes a main body of a computer apparatus, which includes a CPU device 110 and a memory device 111 therein. Further, the computer apparatus main body 101 is provided with an input device 115 for receiving input from the user such as a keyboard and a mouse and a display device 113 for displaying the contents processed by the CPU device 110 to the user as external input / output means. Has been. The computer apparatus can read and store media scene information used in the apparatus of the present embodiment from the auxiliary storage device 114 as necessary.

  In the present embodiment, the media data and the media scene information group are stored using the memory device 111 or the auxiliary storage device 114. The CPU 110 and the memory device 111 extract media scene information groups and display them on the display device 113. Media scene information extracting means, media scene information display control means, and media scene information processing means for processing media scene information. , And user instruction analyzing means for changing the display according to the user input.

  FIG. 2 is a conceptual diagram of a data structure of media scene information used in this embodiment. The media scene information in the present embodiment is roughly divided into three parts. The first is data such as 211, 212, and 213 indicating the time location of a scene that is a partial unit of video data 201 that is media data having time information, that is, a scene that is a part in video 201. This is information that directly defines partial video data (scene) for which the user wants to register attribute information. The second is attribute information 221, 222, and 223 that the user wants to actually register for the scene in the video data. The third is link information 231 to 234 for associating the time information 211, 212, 213 of the video data with the attribute information 221, 222, 223 created by the user. By using a data structure in which two pieces of information are linked in this way, the user can easily add attribute information to existing video data later.

  In the present embodiment, data directly related to video data (or generally media data) such as 211, 212, and 213 is referred to as a “partial space” (scene). This is because the portion of the video data to which the attribute can be added according to the method of the present invention is not only the portion divided by the time division as shown in FIG. 2, but also the coordinate position of the person shown in a part of the screen of the video data This is because a partial region in the XY coordinate space (XYZ coordinate space in the case of three dimensions), such as a region of one sphere that is a part of a three-dimensional CG image, can also be specified. In this embodiment, the temporal part of the video data is described as an example, so the temporal section of the video data is referred to as a “partial space”.

  The symbols in FIG. 2 will be briefly described. The video data 201 is composed of three scenes: a human scene, a car scene, and a house and human scene. Conventionally, the only way to check the contents of these scenes is to reproduce the video data 201. Conventionally, there has been no means for confirming scene information such as names and ages of persons appearing in these scenes.

  In the system of this embodiment, the information 221 of the person A can be registered later in such video data 201 as scene information, for example. The person A information 221 (referred to as “attribute information” in this embodiment) records that the name of the person is currently “A” and the age of the person is 26 years old. In addition, in the video data 201, this person includes “partial space 1” (or scene 1) 211 from 0:00 to 0:01 and “subspace 3” (or scene) from 0:05 to 0:06. Appears in 3). For this reason, in the system of the present embodiment, link information between the attribute information “Mr. A” 221 and “partial space 1” 211 and between the attribute information “Mr. A” 221 and “subspace 3” 213 is linked information. “Appearance” 231 and 232 are added.

  As attribute information, in addition to the character string such as “Mr. A”, “c: /car.doc” or the like and link information of an external file may be embedded as in attribute information 222. Alternatively, information expressing the color information of the car or background with light ternary color values (RGB values) or special byte string data that can be interpreted only by a certain application may be embedded.

  FIG. 3 is an example of a data structure obtained by converting the conceptual data structure of FIG. 2 so that the computer apparatus 101 of FIG. 1 can actually handle it. In this embodiment, information on one partial space, one attribute information, and the like are handled as object data used in a C ++ programming language or the like. With C ++ object data, a plurality of variables can be managed together. For example, the information of the partial space 211 is expressed as object data 311. The subspace start time “0:00” is the variable begin in the object, the end time “0:01” is also the variable end, and the information of the video data 201 that is the base of the subspace is the variable file “sample. expanded to mpg ". Similarly, the subspaces 212 and 213 are also expressed as subspace objects 312 and 313.

  The attribute information 221 is converted into object data 331. However, since the attribute information 221 has a plurality of (attribute name, attribute value) pairs, in the present embodiment, the attribute set object has a structure having a plurality of different objects “meta (attribute name, attribute value)”. Yes. Similarly, the attribute information 222 and 223 are converted into attribute information objects 332 and 333.

  Since the link information in FIG. 2 is simple in the present embodiment, one link object is prepared for one link. In the link object, the link type type, the link source object reference information src, and the link destination object reference information dst can be registered. For example, the link 231 in FIG. 2 is converted into a link object 321. The name “appearance” of the link 231 is stored in a variable type in the link object 321. In the variable src of the link object 321, an attribute information object 331 obtained by converting the attribute information 221 that is the link source of the link 231 into the object of FIG. 3 is registered. In this figure, in order to illustrate the link destination in an easy-to-understand manner, the value of src is “Mr. A”, which is the variable id of the attribute information 331. Register the appropriate value. Similarly, the variable dst of the link object 321 indicates the partial space object 311 corresponding to the “subspace 1” indicated at the tip of the link 231. Similarly, the links 232, 233, and 234 in FIG. 2 are converted into link objects 322, 323, and 324, respectively.

  Finally, in the system of this embodiment, there are a plurality of subspace objects 311 to 313, link objects 321 to 324, and attribute objects 331 to 333, subspace set data 301, link set data 302, and attributes, respectively. Are stored in the collective data 303. To realize these set data, the Set type included in the C ++ Standard Template Library may be used. By using these libraries, it is possible to easily realize processing for extracting elements one by one from the set data and processing of set operations such as sum and difference.

  Note that the link realizing method shown in the conceptual diagram of FIG. 2 can be applied to methods other than the method of creating a link object of the present embodiment. For example, a method in which the attribute information object has a pointer that refers to the subspace object as an array may be used. Similarly, a subspace object may have a reverse reference that points to attribute information.

  FIG. 4 is a diagram for explaining the specification of the data structure shown in FIG. 3 in more detail, and is a diagram showing the relationship between object classes. The class here is the same as a term used in object-oriented programming, such as a C ++ class. The class notation method shown in this figure is based on the UML (Uniformed Modeling Language) diagram.

  In FIG. 4, the class of the subspace object in FIG. 3 is the class “Frag” 404 in this embodiment. The class of the attribute information object is class “Attr” 405. The class of the link object is class “Link” 403. For the sake of simplicity in this embodiment, a super class “Object” 402 that can handle these Flag and Attr class objects in a unified manner is prepared. By providing an object, the subspace set 301 and the attribute set 303 of FIG. 3 can be easily handled as one set. The Object class 402 includes a member variable class for distinguishing whether the object class is Attr or Flag, and a member variable id for naming the object.

  In the present embodiment, a selectColor variable is also provided so that each object can be displayed with a color depending on the selection state of the object. In addition to the method of providing selectColor, a method of selecting a color according to the selection state at the time of displaying an object by providing a selection state variable select or the like may be adopted. In the present embodiment, a plurality of colors can be simultaneously registered in one object by realizing the selectColor variable as a list type. As a result, not only the color of the mouse selection state but also the partial space and attribute information associated with the selected object can be displayed on the screen in different colors, or different search conditions specified by the user can be displayed separately. Colors can also be registered, and the type of object can be confirmed at a glance when the user looks at the screen.

  The member variables of the Flag class 404 include begin, end, and file described with reference to FIG. The class “Meta” 406 is used to cause the Attr class 405 to have a plurality of pieces of information (attribute name, attribute value). The Meta class 406 includes a member variable name representing an attribute name and member variable contents representing an attribute value. The Attr class 405 has a set type member variable mlist in order to have a plurality of Meta class objects.

  The member variable of the Link class 403 includes type, src, and dst described with reference to FIG. Both src and dst can refer to the Object class object.

  Finally, the class “present system” 401 will be described. The class “present system” 401 includes, as scene information to be displayed / edited, a member variable S that is a set of partial spaces and attribute information, and a member variable L that is a set of links.

  FIG. 5 is an example of an execution screen of the system of this embodiment. In the main window 501, three (sub) windows for displaying scene information in an easy-to-understand manner are displayed. The first is a timeline display window 502, the second is an icon display window 504, and the third is a table display window 503. Note that these three types of windows need not be displayed if there is a means for expressing scene information on the screen other than these. Conversely, a plurality of windows may be provided instead of only one. If a plurality of windows are provided, it is possible to grasp one scene information in a multifaceted manner.

  The main window 501 is provided with a main menu 506, which can instruct reading / writing of scene information files, editing operations, search operations, and the like.

  Hereinafter, the contents will be described in the order of the timeline display window 502, the icon display window 504, and the table display window 503. Then, a user's operation method is demonstrated.

  In the timeline display window 502, by displaying the scene information along the horizontal time axis on the screen, the temporal relationship between the scene information can be grasped at a glance. In the present embodiment, the video data 511 to be attributed, the audio data 512 attached to the video, and attribute information and partial space added by the user are displayed in the timeline display window 502 for each type of information. The displayed user data 513, 514, 515 is displayed. In the timeline display window 502, only one user data needs to be displayed on the time axis (for example, only 513), and other data 511, 512, 514, 515, etc. need not be displayed. Further, as long as the direction of the time axis is set in the screen, not only the horizontal direction but also the vertical direction or the depth direction during the three-dimensional display may be used. In FIG. 5, scene information of a time section from 0:00 to 0:06 is displayed as time. In the video data 511, a frame image in the time interval is displayed. In the voice data 512, a voice waveform graph during the time interval is displayed.

  In the “subspace list” 513 of user data, a plurality of subspace objects are represented on the time axis in the scene information stored in the system. For example, “subspace 1” 311 in FIG. 3 is data representing information from time 0:00 to 0:01, but this information is displayed from 0:00 to 0:01 on the screen in FIG. It is expressed as a subspace bar 521. Similarly, “partial space 2” 312 and “partial space 3” 313 in FIG. 3 are represented as partial space bars 522 and 523. Thus, the subspace object is displayed at the coordinate position on the screen according to the values of the member variables begin and end. This time coordinate conversion can be calculated by a simple proportional relationship from the position and width of the timeline display window 502 and the display start time and end time in the window.

  In order to classify and display user data such as “partial space list” 513, “subtitle list” 514, and “zoom list” 515, “subspace list”, “subtitle list”, “ Three pieces of scene information data for the “zoom list” are prepared, and a timeline display window corresponding to each is prepared separately. As a simple method for preparing the scene information data separately, there are a method of opening three scene information files and providing three parts in one scene information file. As another method, only information related to “subtitle list” may be searched and extracted from one scene information data. In other words, when the scene information data is considered as a set as shown in FIG. 3, a “subspace list” subset, a “subtitle list” subset, and a “zoom list” subset are extracted and It corresponds to displaying.

  Note that the plurality of subsets do not have to be mutually exclusive and may have a common portion. For example, if a “Mr. A list” subset and a “house list” subset are defined for the set of FIG. 3, “subspace 3” 313 belongs to both subspaces. If the “A-san list” subset and the “house list” subset are displayed separately in the timeline display window, the subspace bar of “subspace 3” will appear in both displays.

  The icon display window 504 is a window for visually listing and displaying subspace objects or attribute objects included in the scene information in an icon format. By using this window, a plurality of objects included in the scene information can be visually displayed as a list, and the user can grasp the contents of the scene information at a glance. In the timeline display, only the data for a certain time interval can be displayed, but if it is an icon display window, scene information according to certain conditions can be enumerated ignoring time, and more scenes can be saved with less screen space. Can browse information.

  In the present embodiment, information on the three partial spaces shown in the example of FIG. 3, that is, “partial space 1” 311, “partial space 2” 312, and “subspace 3” 313, is displayed in the icon display window. , 542, 543. In the case of this embodiment, the icon is composed of image data that visually represents the contents of the object and a character string that accompanies the image data. In the present embodiment, in the case of an icon of a partial space object, a frame image of video data at the time position of the member variable begin is used as the image data. Thereby, the contents of the partial space can be visually grasped. In the case of an icon representing an attribute object or an icon representing a partial space with respect to audio data, an image cannot be easily acquired from video data, and therefore certain symbol data may be used.

  The icon image does not have to be a single still image. For example, when a moving image is used for the icon, the video from the begin to the end of the partial space may be always reproduced. In addition, it is also possible to arrange or switch any four images from begin to end of the partial space to form one icon image.

  The table display window 503 is a window that displays the contents (contents = member variable values) of the subspace object, attribute object, and link object included in the scene information in a table format. With this window, it is possible to display more detailed attribute information that could not be displayed by timeline display or icon display. Although details of the processing method will not be described here, using this window, the user can not only confirm the contents but also add, delete, and change attribute values.

  FIG. 5 shows a screen when “partial space 3” 523 is selected with the mouse 505, and all the information related to “partial space 3” is displayed in the table display window 503. For example, 531 indicates that “name is A” and “age is 26” as attributes of “Mr. A”. Reference numeral 532 denotes attribute information of “house”.

  Note that when displaying the information of the subspace object instead of the attribute information in the window 503, the name of the subspace and the values of the member variables begin, end, src, etc. may be displayed. When link information is displayed in this window 503, the values of member variables type, src, dst, etc. may be displayed. If the details of the subspace object and the link object are displayed, the table becomes large and difficult to understand. Therefore, an option for displaying only the attribute information on the table may be prepared.

  In order to make the table easier to see, a tree structure display may be added to the side of the table. For example, in the current 531, “name” and “age” are displayed as they are under the “attr, Mr. A” line. However, if the tree structure is displayed, “attr, Mr. A” is the parent node. If “name” and “age” are its child nodes, the display becomes easier to understand. Similarly, if the link object is displayed as a parent node and the link destination data is displayed as a child node, the detailed contents of the link destination can be confirmed only by a tree structure display operation.

  As a final description of FIG. 5, a user operation method will be briefly described. In the following, the selection operation at the time of clicking will be described first, and then the editing operation method will be described.

  The user can operate the visually expressed scene information using the mouse 505. For example, when the user clicks the timeline display (partial space bar) 523 corresponding to “partial space 3”, the partial space bar 523 is selectively displayed. At this time, in the present embodiment, the same data 543 displayed in the icon display window 504 is also selectively displayed. At this time, in this embodiment, the selected object itself and the attribute information object related thereto are displayed in the table display window 503.

  In this embodiment, when an object is clicked to be in a selected state, an object related to the object (that is, a link is established) is expressed in a color different from the selection display, that is, a related selection display. For example, assuming that “subspace 3” 523, “subtitle 3” 551, and “zoom 2” 552 are linked to “include”, “subtitle 3” 551 and “zoom 2” 552 are selected. "Partial space 3" 523 is displayed in a different color.

  In the present embodiment, it is also possible to express only scene information according to a certain condition with another color. For example, the scene where Mr. A appears is blue, only a certain CM video is green, the ending scene is hidden (transparent), and all the people appearing in (related to) a scene are red , Each can be displayed. The specific operation method and realization method will be described later in the structured flowchart.

  As described above, in the present embodiment, the scene information can be more easily grasped by displaying the complicated related information in the scene information in different colors.

  Next, an editing operation method in the system of this embodiment will be briefly described. In the timeline display window 502, the user can add / delete a partial space by designating one time point in the window. The time position of the subspace bar can be changed by dragging the mouse. A plurality of partial spaces can be selected to set a link. In the icon display window 504, a partial space and attribute information can be added / deleted. In the table display window 503, the attribute value can be changed. When a partial space, attribute information, or link information is added / deleted, elements may be added / deleted from the sets 301, 302, 303 shown in FIG. When changing the value of an object, it is only necessary to find the corresponding object set from the coordinate position selected by the user and change the value of the object. A more specific method for realizing the editing operation will be described with reference to the structured flowchart of FIG.

  Next, a screen display processing method according to the present embodiment will be described with reference to the structured flowcharts of FIGS. 6, 7, 8, 9, and 10. FIG. 6 shows the main processing of the system of this embodiment. FIG. 7 shows a timeline display process. FIG. 8 shows icon display processing. FIG. 9 shows a table display process and a subroutine related thereto. FIG. 10 shows a selection process when the user mouse is clicked and a subroutine related thereto.

  First, the main processing 600 of the system of this embodiment shown in FIG. 6 will be described.

  When the system is activated, first, in step 601, a set of objects S0 (= a set of subspaces and a set of attributes) which are member variables of the system and a set of links L0 are initialized. The initialization method is as follows. In the case of new creation, initialization may be performed as S0 = L0 = φ. When reading from a file, a scene information file designated by the user may be read and assigned to S0 and L0. Other methods include substituting the video search result of DB into S0 and L0 as scene information, or scene information of video data analyzed using a video analysis library Mediachef Component manufactured by Hitachi, Ltd. in S0 and L0. It may be added. Note that the sum of S0 and L0 is the entire media scene information in this embodiment, and the memory area for storing S0 and L0 is the media scene information storage means.

  Next, in step 602, the mouse selection object set SS and the related object set SR used in the screen display process and the mouse selection process and the registration condition list CL used in the search process are initialized with the empty set φ.

  Next, in step 700, a timeline display window 502 is created, and the timeline display window 502 is initially drawn by calling the timeline display process DrawTimeLine (S0, L0). In step 800, an icon display window 504 is created, and the icon display window 504 is initially drawn by calling icon display processing DrawIcon (S0, L0). In step 900, a table display window 503 is created and the table display window 503 is initially drawn by calling the table display process DrawTable (SS, L0). In the present embodiment, the currently selected scene information set SS is passed as a parameter of the DrawTable. As a result, the table display window 503 can display detailed contents only for the currently selected scene information.

  As described in the explanation of FIG. 5, a plurality of these three types of windows may be created, or may not be displayed when another display means is used.

  Next, in a loop process 603, the user operation analysis process after step 621 is repeated until a user end instruction is issued. In the user operation analysis process, in step 621, a user instruction from the input device 115 is awaited, and in step 622, a user operation process according to the user instruction is performed. If the user clicks the mouse or the like, the selection process is called in step 1000. If the user gives an instruction to add, delete, or change an object such as a partial space, attribute information, or link information, in step 1500, an editing process is performed according to the editing instruction. If the user selects the search menu, search processing is performed in step 632. In addition, regarding the flowchart of the search process 632, only the condition registration menu and the already registered condition menu in the search menu will be described later.

  Next, the timeline display process DrawTimeLine (S, L) (step 700) will be described with reference to FIG. As input parameters of DrawTimeLine (), a set S of objects to be displayed (= set of subspaces∪set of attributes) and a set L of links are passed.

  First, in step 701, in order to extract a set consisting only of a subspace from a set S of objects consisting of a subspace and attribute information, all elements of the set S are examined, and the value of the member variable class of the element e of the set S is determined. Only the element that is “frag” is taken out and added to the set F. Note that the set S may be divided into a subspace set F and an attribute set in advance in order to improve processing efficiency. Next, a loop process 702 is performed to display timeline display partial space bars for all elements e of the set F of partial spaces.

In the loop processing 702, first, in step 711, the time value of e.begin and the time value of e.end are converted to coordinate positions bx, ex on the time axis on the screen. To convert a time t to a coordinate x on the screen,
Coordinate x = time to coordinate transformation (t)
= (T-time axis start time on the screen) /
(Time axis end time on screen-start time) x
The time axis width on the screen + the time axis start coordinate on the screen may be used.

  Next, in step 712, a rectangle with upper left coordinates (bx, 0) and lower right coordinates (ex, 10) is drawn to draw the outer frame of the partial space bar. Next, the subspace name e.id is displayed in the subspace bar. The location is (bx, 0). Next, in step 713, a small icon image is displayed at the top position of the partial space bar. As a result, even if only the partial space bar is viewed, it is possible to instantly confirm the content of the video data. For the image, the video file specified in e.file is opened, and the frame image at the time position e.begin is displayed. The display position of the image is set to coordinates (bx, 30) and is displayed slightly below the previous character string.

  Finally, in step 714, the square displayed in step 712 is displayed using e.selectColor, which is the display color information of the object. For example, the color of e.selectColor may be displayed as a rectangular fill color as in the selected partial space 523 in FIG. 5, or a rectangular frame as in the related selected partial space 551 in FIG. 5. It may be displayed as a color. Note that selectColor may register multiple colors at the same time. In this case, enclose the partial space with multiple separate color frames, or place a small flag-like color icon at the top of the partial space. What is necessary is just to take an identifiable method on the screen, such as arranging the number of colors. In addition, when the color list of selectColor is empty or a certain transparent color is set, a method of not displaying the corresponding partial space may be used. In addition, selectColor, which indicates the object selection status and search result status, is currently displayed in multiples with objects on the screen using colors, but in addition to colors, an expression format that can be identified on the screen is used. These identification states may be expressed. For example, the filling pattern of the partial space may be changed depending on the selection state, or a small flag-like icon using an identification number, an identification character string, an identification icon, or the like may be displayed on the screen for each identification state.

  In the timeline display processing 700 of FIG. 7, only the partial space is the target of timeline display, but the timeline display can also be applied to the attributes by the following method. For this purpose, first, a set of objects linked to a certain attribute a is extracted by a function GetLinkedObj (a, L0) described later (step 902), and this set is set as a new S, and step 701 and loop processing 702 are performed. Just do it. If the display target is a set of a plurality of attributes, the display process for the one attribute may be repeated for all attributes.

  Next, the icon display process DrawIcon (S, L) (step 800) will be described with reference to FIG. As input parameters of DrawIcon (), a set S of objects and a set L of links similar to DrawTimeLine () in step 700 are passed.

  First, in step 801, a set F of subspaces is extracted from the set S by the same method as in step 701. In step 802, the display coordinates x, y of the icon are initialized to the upper left coordinates (0, 0). Next, loop processing 803 is performed to display icons for all elements e of the set F of subspaces.

  In the loop processing 803, first, in step 811, the frame image at the time position of e.begin of the video data e.file is displayed at the coordinates (x, y). In step 812, the subspace name e.id, start time e.begin, and end time e.end are displayed as character strings. Since the display position of the character string is below the image, the coordinates (x, y + image height) are used. Next, in step 813, the place where the image and the character string are present is displayed using the color e.selectColor of the selection state of the object.

  Next, processing for changing the display position of the icon is performed. In step 814, in order to display the next icon on the right side, the width of the icon and the width of the space between the icons (gap width) are added to x. At this time, if x exceeds the width of the icon display window 504 (condition 815), in step 821, the position of x is returned to the left, and the display position is changed to a new line. Add the height of the image and text and the height of the vertical space. Of course, when the y coordinate exceeds the height of the icon display window 504, processing for stopping the loop 803 may be included.

  Although the icon display processing 800 of the present embodiment is intended to display only a partial space, attribute information and link information can be displayed as icons by adding the following processing. For example, after step 803, if only the attribute information set A is extracted from the object set S (A ← S−F) and the icon display process of one attribute information is repeated for all the extracted attribute information. good. When one attribute information is represented as an icon, a default graphic representing the attribute may be used as the icon image, and meta information (= a set of (attribute name, attribute value)) included in the attribute information may be used as the icon character string.

  At the end of the description of the display process, the table display process DrawTable (S, L) (step 900) and a subroutine related thereto will be described with reference to FIG. As the input parameters of the table display process DrawTable (), the same object set S and link set L as the DrawTimeLine () in step 700 are passed.

  First, in step 901, an object related to the current selection state S is substituted into the set R by using a function GetLinkedObjSet () that extracts all objects associated with the object set S. Next, in a loop process 902, the process of outputting the contents of all elements e of the related object set R as a table row is repeated.

  In the loop processing 902, first, in a condition step 911, a table output process is performed for each object type. When the type of the element e is attribute information (when e.class = "attr"), processing for outputting details of the attribute information is performed in step 921 and subsequent steps. First, in a loop process 921, in order to output a meta set of (attribute name, attribute value) possessed by the attribute information e, the process of step 922 is performed for all elements m of the meta set e.mlist. In step 922, in order to add one meta object m to the table, a table row object is created, the “ID” column of the row is e.id, the “name” column of the row is m.name, and the “ The value column is m.contents, and finally this row object is added to the table. In the table display window 503 in FIG. 5, the “ID” column is not displayed.

  In condition step 911, if the type of the element e is a partial space, three lines are added in step 925. The line added to the table in step 925 includes the video file name (e.id, “file”, e.file), start time (e.id, “begin”, e.begin), and end time (e .id, "end", e.end).

  In step 912, the row of the table added in steps 922 and 925 this time is displayed using the color e.selectColor of the object selection state.

  Next, details of GetLinkedObjSet (S, L) in step 901, which is a subroutine function, will be described. The input parameters of this function are an object set S and a link set L, and the return value of this function is a set obtained by extracting all objects associated with the object set S. In this process, first, a set R as a return value is initialized with an empty set φ in step 941, and then a loop process 942 is performed on all elements e of the set S to obtain a result set R. In the loop process 942, first, in step 951, a set of objects associated with the element e is obtained by GetLinkedObj (e, L) and substituted into SL. In the next step 952, the obtained set SL is added to the result set R. Finally, in step 943, a set R of related objects obtained for all elements is returned.

  Details of the subroutine GetLinkedObj (e, L) (step 951) will be described at the end of FIG. An input parameter of this function is a set L of one object e and a link, and a return value of this function is a set obtained by extracting all objects associated with one object e. In this process, first, in step 971, a set SL as a return value is initialized with an empty set φ, and then a loop process 972 is performed on all the elements link of the link set L to obtain a result set SL. Go. In the loop processing 972, first, in step 981, the contents of the link link are examined. When the link source of the link link is the self object e, the link destination object link.dst is added to the result set SL (step 983). When the link destination of the link link is the self object e, the link source object link.src is added to the result set SL (step 985). Finally, in step 973, the related object set SL obtained for all the link elements is returned.

  FIG. 10 shows the details of the selection process (step 1000) for the user's click input. The input of this process is the coordinate position (x, y) clicked by the user.

  First, in step 1001, the current display format of the position clicked by the user is checked.

  When the display format is timeline display, processing for specifying the selected partial space is performed in step 1011 and subsequent steps. First, in step 1011, the coordinate value of the click position x is changed to a time value on the screen time axis, and is substituted for the selection time t. For this calculation, an inverse function of time-to-coordinate transformation (t), which is the function described in step 711, may be used. Next, in step 1012, in order to extract a subspace object including the selection time t from the object set S0, the function FindObject () is called and the result is substituted into the new selection set SSnew. In the FindObject function, a condition function Within (*, t, t) indicating that the time position t clicked by the user is included as a search condition is specified as a parameter. This designation designates that an object including the sections t to t, that is, the time t is to be extracted. Details of how to use the condition function and its definition will be described later in the method for realizing search processing.

  In step 1001, if the display format of the clicked position is an icon, in step 1014, a corresponding subspace object is acquired from the selected icon, and set to a new selection set SSnew. In order to realize this, it is only necessary to register a corresponding partial space object in the display icon information for remembering the icon display position and image when drawing the icon.

  If the display format of the clicked position is a table at step 1001, only the clicked line is selected and displayed at step 1017, and the selection processing 1000 is terminated at step 1018. The reason for terminating the function here is that if the selection state is changed when clicked, as with other display formats, the display content of its own table will be displayed in the subsequent display update process simply by clicking the table. This is because the information has been changed and the information cannot be correctly operated. Here, the selection state is not changed only when the table is clicked.

  In step 1002, a set of objects associated with the set of objects SSnew selected by the user is obtained by GetLinkedObject (SSnew, L0) and substituted into the related set SRnew. Finally, in step 1003, the selection state of the mouse is updated with SSnew and SRnew obtained by this function.

  The mouse selection update process NewSelection in step 1003 will be described in detail. The parameters of this function are a new selection object set SSnew and a new related object set SRnew.

  First, in step 1051, the mouse selection color and the mouse selection related color registered in the currently selected object are removed. Specifically, a loop is performed for all the objects obj included in the global variables SS (set of selected states) and SR (set of related selected states), and the mouse selection color and the mouse are selected from the selection color list obj.selectColor. Delete related selection color.

  Next, in steps 1052 and 1053, the mouse selection color and the mouse related color are registered in the new selection state object and the new related selection state object, respectively. Specifically, an element is extracted for each set of SSnew and SRnew, and color information corresponding to selectColor of the corresponding object is added.

  Finally, in step 1054, the screen is displayed again using the new color information, and in step 1055, new selection information SSnew and SRnew are registered in the global variables SS and SR.

  Next, search processing in the system of this embodiment will be described with reference to FIGS. FIG. 11 is a search processing execution screen. 12 and 13 are structured flowcharts of the search process. FIG. 14 is a table showing a method for realizing a search condition function used in search processing.

  First, a search processing execution screen will be described with reference to FIG. The search process is performed by selecting a search menu from the main menu 506. In the search menu, a menu 1101 for registering conditions for search and a menu 1102 for displaying a list of already registered conditions are displayed. When the condition registration menu 1101 is selected, a condition registration screen as shown in 1105 is displayed.

  On the condition registration screen 1105, a search result display method and search conditions can be designated. After setting these pieces of information, the search process can be executed by pressing the search button 1151. The search result is displayed on the screen in the color specified on the condition registration screen. Specifically, among all the objects included in the media information, only the objects that satisfy the condition specified on the condition registration screen are displayed on the main window 501 with the color information specified on the condition registration screen. The Note that when the cancel button 1152 is pressed, the search process is terminated.

  In the search result display method that can be specified on the condition registration screen 1105, a search condition name 1111 and a search result display color 1112 can be specified. The search condition name 1111 is a name displayed when this condition is registered in the search menu later. The search result display color 1112 is a color used when an object satisfying the search condition is displayed. This color selection can be performed so that, for example, colors determined by clicking 1112 appear in order. As an application example, a transparent color may be designated as the selected color. For example, if a transparent color is specified, the search result will not be displayed, and the user can hide redundant scene information on the screen by this search process, making it possible to check and edit scene information more. Increase efficiency.

  As a search condition that can be specified on the condition registration screen 1105, a method for searching within a certain time range, a method for searching for only a certain attribute name, and a method for searching for only a certain attribute value can be specified. A plurality of these can be specified simultaneously using a time search check box 1121, an attribute name search check box 1131 and an attribute value search check box 1141. In this embodiment, when a plurality of conditions are specified, it is considered that these conditions are combined with an AND condition. When each check box is ON, the input areas of the start position 1122 and the end position 1123 of the time section, the attribute name 1132, and the attribute value 1142 are valid as corresponding condition values, and values can be set. It becomes like this.

  As another method for specifying the start position 1122 and the end position 1123 of time, a scroll bar for changing the display time position is attached on the screen of the timeline display 502 in FIG. A method of specifying a time range to be searched may be adopted. Conversely, the timeline display 502 may be considered as a user operation screen for searching and displaying scene information sandwiched between the display start time and display end time specified by the user. Similarly, the user selection process described with reference to FIG. 10 may be considered as a user operation method for dynamically searching for an object related to the object selected by the user.

  The content described here is one embodiment of the present invention, and various other conditions and setting items can be specified. For example, as a search condition, OR of a plurality of search conditions, a complex search condition expression is input, only an object associated with a certain search result is extracted (using subroutine 901 GetLinkedObjSet () in FIG. 9), You can specify a related name at the time of search, or even open a media data file registered in a partial space and search for a scene containing the moment when the corresponding building appears using image analysis. You may do it. In such a case, it is desirable from the viewpoint of operability and operability to display a list of search conditions so that the search conditions can be dynamically added.

  In the present embodiment, when a search is performed using the search button 1151, the searched object is displayed in the color specified in 1112 and at the same time, the mouse is selected as an editing operation target. The user can update or delete the search results as necessary. In the present embodiment, when a search is performed, the search conditions for the past several items are stored, and the search condition name is displayed on the search menu together with the search result display color. By looking at the search menu, the user can confirm which color object on the screen satisfies which condition after the search result. Also, by selecting this menu, the search process can be executed again using the previously specified search condition.

  As an application example of the search operation method, a menu “to next search result object” may be added to the search menu. In this case, instead of displaying the search results at once, by selecting this menu many times, the user can perform an editing process while checking the objects of the search results one by one. Further, this menu is very effective when searching for the specific object described above using the image analysis technique. This is because it is not necessary to analyze all media data at the time of search execution, and it is sufficient to apply partial image analysis step by step when the user desires the next search result, which can greatly reduce the user's waiting time. Because it can.

  12 and 13 are structured flowcharts of the search processing in this embodiment. Here, the processing of the “registration condition” menu 1101 and the registered condition menu 1102 in the search menu will be described.

  Step 1200 is processing when the condition registration menu 1101 is selected. In step 1211, the search condition registration screen 1105 is displayed first, and the user is made to register the search condition. Next, in steps 1212 to 1215, the search condition formula is assembled as a condition function list funcList. In step 1202, a search is executed using the assembled condition function list funcList, and the search result is specified in the condition registration screen 1105. Display in the selected color. Finally, in step 1216, information on the search conditions specified this time (condition name, color, condition function list) and the like are added to the registration condition list CL, which is a global variable, and in step 1217, the search menu of the main menu 506 is added. Add the current condition name and color information.

  Next, a method for realizing the search condition formula and an assembly method will be described. The conditional functions registered in the conditional function list funcList are expressed by (function address, user parameter value) on the memory. In the present embodiment, the start address of the conditional function shown in FIG. 14 can be registered as the function address. Also, to execute these conditional functions, add the address of the media scene information object whose condition you want to determine, along with the user parameter value specified here as a function parameter, and call the specified function address. good. The function result is defined to be returned to the calling process as a set value (φ if the condition is not satisfied, and other than φ if the condition is satisfied).

  The process of assembling the conditional function expressions in steps 1212 to 1215 will be described in detail using the conditional function shown in FIG. First, in step 1212, the conditional function list funcList is emptied. Next, in step 1213, it is checked whether or not the user has specified time search on the condition registration screen 1105. If time is specified, the time condition function Within () is added to the condition function list funcList in step 1221. To do. At this time, the start time and end time specified by the user are registered as user parameters of the conditional function With. Similarly, in the attribute name search, in step 1214, it is determined whether or not there is a user search instruction, and if the attribute name search is to be performed, the conditional function SameName () is registered in funcList in step 1223. The user parameter at this time is an attribute name designated by the user. Similarly, in the attribute value search, whether or not there is a user search instruction is determined in step 1215, and if an attribute value search is performed, the conditional function SameValue () is registered in funcList in step 1223. The user parameter at this time is an attribute value designated by the user.

  In the present embodiment, the search conditions for time, attribute name, and attribute value have been described, but other conditions can be similarly extended. Basically, when a search condition is added, an individual condition setting unit and a condition function that inputs an object and a user parameter and returns a set value may be prepared for each search condition. In the search condition module, a setting screen may be provided as necessary so that the user can set the presence / absence of a corresponding condition and set detailed condition values.

  Next, a description will be given of processing 1201 when a menu of search conditions that have already been registered is selected from the search menu. If the search condition is already registered, the information cond of the corresponding search condition is acquired from the search condition list CL using the menu number value specified by the user (step 1231). Since the search condition information cond includes a search condition list necessary for executing the search and color information necessary for displaying the search result, if the search execution process 1202 is called using these values, the user The search can be re-executed while maintaining the previously set conditions (step 1202).

  Finally, the search execution process 1202 will be described. The input parameters of this process consist of the search condition function list set in advance and the display color of the search results. First, in the search, in step 1241, the search result set Sr is initialized with the search target set S0. In this example, since the search condition list is searched as an array of AND conditions, Sr is first set as a global variable S0 representing all objects. Next, a loop is performed for all the search condition functions func included in the search condition list funcList (step 1242), and the result set Sr is narrowed down by the search condition function func using the FindObject () process in step 1300. At the stage where the loop processing 1242 is completed, a set of objects that satisfy the conditions obtained by ANDing all the search conditions is stored in Sr.

  Next, the loop processing 1243 is performed for all the elements obj of the search result Sr, and the specified search result color color is added to the object selection display color list selectColor (step 1252). Finally, in step 1244, the mouse selection is updated so that the search result can be handled in the same way as the mouse selection state for later editing. Since the screen is redrawn in the mouse selection update process, the search result color set in the loop process 1243 is also displayed on the screen.

  Next, the object extraction process FindObject () satisfying the search condition in step 1300 will be described with reference to FIG. The input of this function is a set S of search target objects and one search condition function f (). The output of this function is a set of objects that satisfy the search condition function f () in S.

  In this function, first, the search result set Sr is emptied (step 1310), and then in a loop process 1311, conditions for all search target objects obj in S are determined. In determining the condition, a condition function f (obj) is called to check whether the object obj satisfies the search condition (step 1321). At this time, it is assumed that the user parameters set in advance in step 1221 of FIG. If the determination result of the conditional function is true, an object set to be added to the search result set Sr is returned as a result of the conditional function, and is added to Sr. When the determination result is false, an empty set φ is returned as the result of the determination function. Since the search result is stored in Sr when the loop processing 1311 is completed, this is set as a return value of this processing, and this processing ends.

  FIG. 14 is a table showing a method for realizing a search condition function used in search processing.

  First, the condition function Within () 1401 for determining an object including a certain time interval will be described. Within () has a start time tb and an end time te as user parameters. The result is that the object specified as a function parameter is a subspace (object.class = "flag"), and the real number set [object.begin, object.end] and real number set [tb, te] overlap. In this case, the set {object} is returned as the contents to be added as the extraction result. That is, {object} is returned if the subspace object touches even a part of the interval [tb, te]. If the object is attribute information, an empty set φ is returned.

  Next, the condition function SameName () 1402 for determining whether or not the attribute names are the same will be described. The user parameter of SameName () is the attribute name name. The condition that this function is satisfied is that the object specified as the function parameter is an attribute (object.class = "attr"), and the attribute name meta for any one meta of the meta class list mlist that the attribute object has Assume that .name is equal to name, and return {object} as a return value.

  Similarly, in the case of the condition function SameValue () 1403 for determining whether the attribute values are the same, the user parameter is the attribute value contents, the object specified as the function parameter is the attribute, and the attribute value meta.contents Returns {object} if there is more than one meta with = contents in the Meta class list. Note that SameName () and SameValue () are assumed to be an empty set φ when the above condition is not satisfied or when the object specified as the function parameter is a partial space.

  Finally, as an application example of the conditional function, a conditional function GotIt () that returns a partial space at the moment when a person enters the screen will be described. In GotIt (), if the object specified as a function parameter is a subspace, the video file indicated in object.file is opened and the video is analyzed using image processing to recognize that a person has entered the screen. Then, a new subspace object newObj is generated, the appearance time of the person is specified in newObj.begin and newObj.end, and {newObj} is returned. In order to realize this image processing, for example, an image recognition algorithm shown in “Moving object detection / extraction apparatus and method” of JP-A-8-221577 may be used. Of course, if a plurality of persons are detected, a set of elements of a plurality of subspaces may be returned.

  FIG. 15 is a structured flowchart of editing processing in this embodiment. In the present embodiment, as the editing process, a process for deleting a selected object and a time movement process for moving the time positions of all the selected partial spaces by a predetermined time will be described.

  First, in step 1501, the content of the user's editing instruction is determined. If the instruction content is deletion, loop processing 1510 is performed for all objects obj included in the set SS of selected objects in the global variable, and the object obj is deleted from the set S0 of all objects and the set L0 of all links (step 1513). ). If the content of the editing instruction is time movement, first, in step 1520, the user specifies the movement time amount dt. Thereafter, loop processing 1521 is performed for all selected objects SS. If the object is a partial space (step 1530), the start time and end time of the partial space are both moved by dt (step 1533).

  Here, the object deletion 1513 will be described. First, in the loop processing 1550, all the links are examined, and a link whose head or tail is the object to be deleted obj is found. If a corresponding link is found (condition determination step 1560), the link is linked on the spot. Are deleted from the set L (step 1563). When the loop processing 1550 ends, the corresponding object obj is finally deleted from the set S of all objects (step 1551).

The figure which shows the implementation example of a media scene display edit apparatus. The conceptual diagram of the data structure of the media scene information used by this embodiment. The figure which shows the example of the actual data structure corresponding to the conceptual data structure of FIG. The figure which shows the detail (class) of the data structure of FIG. The figure which shows the execution screen of embodiment. The figure which shows the main process of the system of this embodiment. The figure which shows the timeline display process of this embodiment. The figure which shows the icon display process of this embodiment. The figure which shows the table display process of this embodiment. The figure which shows the user selection process of this embodiment. The figure which shows the search process execution screen of this embodiment. The figure which shows the search menu process of this embodiment. The figure which shows the search condition execution process of this embodiment. The figure which shows the search condition function list | wrist of this embodiment. The figure which shows the edit process of this embodiment.

Explanation of symbols

  501 ... Main window, 502 ... Timeline display window, 503 ... Table display window, 504 ... Icon display window, 505 ... Mouse pointer, 506 ... Main menu, 511 ... Video timeline display, 512 ... Audio timeline display, 513 to 515: User data timeline display, 521,522 ... Partial space bar display, 523 ... Selected partial space bar, 531,532 ... Attribute information related to selected partial space, 541,542 ... Subspace icon display, 543 ... subspace icon in selected state, 551,552 ... subspace related to the selected subspace, 1101 ... search condition registration menu, 1102 ... already registered search conditions and their display colors, 1105 ... search condition registration screen.

Claims (6)

An information storage device that stores media scene attribute information that is a partial unit of media data that is data having time information,
Media data storage means for storing media data;
Media class attribute information and subspace objects having identification information of the stored media data and information on the time existence position of the media scene in the media data can be referred to each other uniformly by a super class object. And an information storage means for storing the information in association with the information storage device. An information display device comprising the information storage device according to claim 1,
An information display device comprising: display means for displaying information on the subspace object together with the time axis at a position on the time axis corresponding to the time location information of the stored subspace object. The information display device according to claim 2,
Selecting means for selecting a media scene indicated by the information about the displayed subspace object;
Information extracting means for extracting attribute information associated with the selected media scene information in the storage device;
An information display device comprising: information display control means for displaying the extracted attribute information so as to be distinguishable from other media scene information on the screen. The information display device according to claim 3,
The information storage means further stores a link object indicating an association between the attribute information and the subspace object,
The information display device is characterized in that the information extraction means performs the extraction based on association information of the link object. The information storage device according to claim 1,
The information storage device, wherein the subspace object associates the subspace object by storing reference information for the subspace object. An information storage method for storing attribute information of a media scene that is a unit of media data that is data having time information,
Storing media data in the media data storage means;
In the information storage means, the media class attribute information, and the sub-space object having the identification information of the stored media data and the time location information of the media scene in the media data are unified by a super class object. An information storage method comprising the steps of: storing them in association with each other so that they can be referred to each other.

Images