JP2003323434A - Image multimedia data relating device, method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To relate the multimedia data by every object composing an image. <P>SOLUTION: The conformity of a text describing the characteristic of an object of an image (image object) and a keyword extracting the characteristic of the multimedia data is checked by a matching processing means 25, the data processing is performed on the data having the conformity by using a relating means 28 and an identifier application means 29 to acquire the multimedia data from the image object, and the relating of the multimedia data can be performed by a unit of object. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video multimedia data associating device, a video multimedia data associating method, and a program used therefor.

[0002]

2. Description of the Related Art In recent years, multimedia contents in which video is associated with additional information have been frequently created. Such multimedia content is, for example, "An Annotation Engine for Supporting Video Databa
se Population (Multimedia Tools and Application 5,2
33-5,258 (1997)) ”, it is known that the contents can be efficiently created by using the content creation tool. The multimedia content created by using such a content creation tool is a screen that is composed of several scenes of video content, that is, for each sequence, that is, for each sequence, related video, audio, and text. Multimedia data composed of data including at least one of the above is added.

As a method of creating multimedia contents using a network environment, CORBA (Common O
bject Request Broker Architecture) using the naming service function. The naming service function is to give a name for identifying an object and manage the object by this name. A user who wants a specific object specifies the name given to the object by the client. The server providing the naming service function performs matching processing using the name sent from the client as an identifier, extracts the corresponding object, and sends it to the client.

[0004]

However, when multimedia contents are created using the contents creation tool, the multimedia data can be added only in units of video sequences. Therefore, in order to increase the added value of multimedia contents, it has been desired to develop a technique in which multimedia data can be associated with each object forming an image.

Further, when multimedia contents are created by using the naming service function of CORBA, it is required that the names given to the data in the matching process be exactly the same, so that the mutual contents are beforehand performed. Since information must be shared, it is difficult to associate a plurality of data in an environment where the other party is unknown, such as in a network environment.

[0006] Therefore, the problem to be solved by the present invention is to associate multimedia data with each object constituting a video. It is also necessary to ensure the creation of multimedia contents even in a network environment.

[0007]

The invention according to claim 1 of the present invention for solving the above-mentioned problems provides an object included in an image and at least one or more of an image, an audio and a text in a network environment. A video multimedia data associating device for associating with multimedia data including: first data associated with an object for identifying an object; and second data associated with the multimedia data for identifying a multimedia data. In the network environment of the multimedia data related to the matching data matched by the matching processing means for acquiring the data and extracting the matching data which is the result of matching the first data and the second data. An identifier for identifying the storage location is added to the consistent data, and the video And the identifier providing means for the associated data between the objects and multimedia data, a storage means for storing associated data, and video multimedia data association device comprising a.

The invention according to claim 3 is that an object included in a video and a video in a network environment,
A video multimedia data associating method for associating multimedia data including at least one of audio and text, comprising: first data associated with an object for identifying an object; A matching step of acquiring the second data associated with the multimedia data and extracting the matching data that is the result of matching the first data and the second data, and a multi-step related to the matching data matched in this matching step. An identifier assigning step of assigning an identifier for identifying a storage location of the media data in the network environment to the matching data to make the associated data between the video object and the multimedia data, and a storing step of storing the associated data in the storage means. And including -Media was the data associated with the method.

Further, the invention according to claim 4 specifies a device for associating an object included in a video with multimedia data including at least one of video, audio, and text in a network environment. To obtain first data associated with the object to identify the multimedia data and second data associated with the multimedia data to identify the multimedia data,
Matching processing means for extracting matching data that is the result of matching the first data and the second data, and for specifying the storage location in the network environment of the multimedia data related to the matching data matched by this matching processing means A video multimedia data associating program is provided which causes an identifier to be added to the matching data and serves as an identifier giving means for making the related data between the video object and the multimedia data and a storage means for storing the related data.

Data that characterizes the object is added as first data to the video object, and data that characterizes the multimedia data is added as second data to the multimedia data. The first data and the second data are matched, the matched data is extracted as the matched data, and the matched data is managed using the identifier, whereby the multimedia data is associated with each object. Claim 1 relates to a device for performing such an association process, and claim 3 relates to a method for such an association process. Further, claim 4 relates to a program for performing such an association process.

According to a second aspect of the present invention, in the video multimedia data associating apparatus according to the first aspect, the matching processing means uses a video including an object in addition to the first data and the second data. The matching data is extracted by using the information of.

This video multimedia data associating device performs the matching process in consideration of the user's information, thereby performing the associating process according to the user's taste and purpose. It can also be used to manage the user's access right to multimedia data.
Here, the user information is acquired as text, numbers, or symbols.

[0013]

BEST MODE FOR CARRYING OUT THE INVENTION Details of a video multimedia data associating apparatus according to an embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a video multimedia data associating system according to an embodiment of the present invention. FIG. 2 is a diagram for explaining processing performed in the video multimedia data associating system of FIG.

In this embodiment, an object video which is a video including an object representing a subject and a data related to the object video (hereinafter, referred to as a multi-media) by using the video multimedia associating system 1 shown in FIG. This is referred to as media data). The video multimedia association system 1
The object video management device 10 for managing the object video, the multimedia data management device 14 for managing the multimedia data, and the video multimedia associating device 2 for associating the object video and the multimedia data are network-connected to each other. ing.
Note that the types of multimedia data include video, audio, and text stored in the multimedia data management device 14, and the multimedia data is configured to include at least one of these.

The object video management apparatus 10 has an input means 19 for inputting an object video, an object extracting means 21 for extracting a specific object from the input object video, and an extracted object (hereinafter, referred to as "object").
(A video object) and characteristic data assigning means 23 for assigning characteristic information as first characteristic data (metadata) which is the first data, and the object video is stored in the video storing means 24, and the video object and the feature are stored. The data is configured to be stored in the object storage means 22. If the first feature data is data that can specify the content or type of the video object,
It may be any data such as an explanation text or a color feature amount. In this embodiment, a case where text data is used as shown in FIG. 1 will be described.

Such an object video management device 10
For example, a computer device that has a communication control device (not shown) and can be connected to a network can be used. In this case, the input means 19 is a device for reading the object video from the storage medium, a device for receiving the object video via the network, or the like. The video storage means 24 and the object storage means 22 serve as a storage device such as a hard disk. Further, the object extracting means 21 is a means realized by causing a control device having a CPU (Central Processing Unit) or the like to start a predetermined program, and Japanese Patent Application No. 2000-385276 filed by the applicant of the present application. You may use the moving object extraction apparatus described in. The characteristic data adding means 23 is a means for the operator to manually input with a mouse or keyboard (not shown), but is a means for recognizing the contents of the video object and automatically adding appropriate data as the first characteristic data. May be. An example of automatically adding the characteristic data is a means for recognizing the face of the person and adding the first characteristic data when the video object is composed of a video of a person.

The multimedia data management device 14 includes a multimedia data storage means 11 for storing multimedia data, and keyword extraction for extracting second characteristic data (metadata) which is second data characterizing the multimedia data as keywords. And means 13 and. As such a multimedia data management device 14, for example, there is a computer device which has a communication control device (not shown) and can be connected to a network. In this case, the multimedia data storage means 11 is a storage device such as a hard disk, and the keyword extraction means 13 is realized by causing the control device of the computer device to start a predetermined program. If the multimedia data is stored in advance in association with the second feature data, it is not necessary to provide the keyword extracting means 13.

The second characteristic data may be any data as long as it can specify the content, type, storage location, etc. of the multimedia data. It is desirable that the format of the second feature data is the same as the format of the first feature data in order to easily perform the matching process described later. Therefore, the second feature data in this embodiment is data in text format. Further, the reason why the second characteristic data is described as a keyword is to facilitate the distinction from the first characteristic data.

The video multimedia associating device 2 correlates the first characteristic data and the second characteristic data, which are matched with each other, with the matching processing means 25 for performing the matching processing between the first characteristic data and the second characteristic data. An associating unit 28 for creating one piece of data, and an identifier assigning unit 29 for assigning a name identifier, which will be described in detail later, to the data created by associating to create related data,
It is configured to include a storage unit 27 that stores related data. An example of such a video multimedia associating device 2 is a computer device that can be connected to a network. Matching processing means 25 in this case
And associating means 28 and identifier assigning means 29
Is realized by causing a control device of a computer device to start a predetermined program. Also, the storage means 27
Is a storage device such as a hard disk.

Next, the processing in the video multimedia associating system 1 will be described with reference to FIGS.
Will be explained. Note that FIG. 2 shows a work procedure for associating a multimedia data group with a video object existing in the object video.

First, the object video management device 10
From the object video #V shown in FIG. 2, the video object # V ¥ # P for that video is extracted. This extraction processing is performed by the object extraction means 21 of FIG. Then, the plurality of extracted video objects # V ¥
The text that is the first characteristic data is added to #P. Since the text given here can include a plurality of information, the first feature data is regarded as a set of texts, and the object text set # V ¥ # P ¥
It will be expressed as {text}.

On the other hand, the multimedia data management device 14
If the stored multimedia data including video, audio, text, etc. does not have the second characteristic data, the keyword that is the second characteristic data is extracted from such multimedia data. For example, the data stored in the multimedia data storage means 11 shown in FIG. 1 is converted into multimedia data Data as shown in FIG.
1 to Data # m (m: positive integer), the keyword extracting means 13 (see FIG. 1) for each of them.
Extracts multiple keywords. Where keyword D
ata1 \ {text} is multimedia data D
It means a set of keywords for data1, and the keyword Data # m ¥ {text} means a set of keywords for multimedia data Data # m. Note that the number of keywords may be one.

Then, the video multimedia associating device 2 obtains the object text set # V ¥ # P ¥ {text} from the object video managing device 10 via the network, and the multimedia data managing device 14 receives the keyword Data1 ¥. {Text} to Data #
m ¥ {text} and the keyword Data1 ¥
The keyword Dat for specifying the storage location of the multimedia data Data1 to Data # m giving {text} to Data # m \ {text} on the network.
a1 \ {text} to Data # m \ {text} not shown name identifiers Name1 to Name
#M is acquired via the network. The name identifier is an identifier for uniquely identifying the location of data on the Internet, and is an XML (eXtensive Marku).
Namespaces provided in p Language). The information used for the namespace is URL (Uniform Reso
urceLocator) and so on. In other words, Name1
By referring to, the keyword Data1 ¥ {tex
The storage location of the multimedia data Data1 giving t} can be known.

The matching processing means 25 of the video multimedia associating device 2 has the object text set # V ¥ # P ¥ {text} which is the acquired first characteristic data.
And the keyword Data1 ¥ {t which is the second feature data
The matching process is performed using ext} to Data # m \ {text}. The matching process in this case is text matching because both the first feature data and the second feature data are in text format. The text included in the first characteristic data and the keyword included in the second characteristic data are compared, and if there is a match, it is considered that the text and the keyword are matched.

As a result of the matching processing, for example, when there are k pieces of matched data (k: a positive integer), FIG.
Associated data Data # L (1) to D in which the associating means 28 shown in FIG.
The associating unit 28 creates ata # L (k). Also, this data Data # L (1) to Data # L
To (k), the identifier assigning means 29 assigns name identifiers Name # L (1) to Name # L (k), respectively.
In the name identifiers Name # L (1) to Name # L (k), matched keywords Data1 \ {text}
~ Name identifier of Data # m \ {text} Name1
~ Name # m.

Then, the video multimedia data associating device 2 gives the video object # V ¥ which gives the first feature data (text) matched by the matching process.
#P and name identifiers Name # L (1) to Name # L
Related data Data # L (1) to D added with (k)
The naming service conversion table 32 shown in FIG. 2 is created by associating ata # L (k). The naming service conversion table 32 is created by the naming service correspondence conversion table creation process 31 and stored in the storage means 27. For example, when reproducing the object video #V, the name service function is used with the video object # V # P included in the object video #V. The object video management device 10 refers to the naming service conversion table 32 built in the storage means 27 of the video multimedia data associating device 2, and determines the storage location of the multimedia data related to the video object # V # P. Get by name identifier. Then, the obtained storage location is accessed to obtain the related multimedia data. The association in the present embodiment may be performed by storing the positional relationship between data so that specific multimedia data Data1 to Data # m can be acquired from the storage location using the video object # V # P. , It is not always necessary to give an identifier. In short, any process that can convert the data (feature space) freely created by the user or the like for the object video #V into the unique data (name space) that can specify the related data on the network may be used.

In this way, the video object # V # P and the keyword Data1 ¥ {te
By associating xt} to Data # m \ {text}, it becomes possible to associate multimedia data in a finer unit than in the case of associating data in a sequence unit. In particular, this associating process includes the video object # V # P and the keyword Da as described above.
Since it is automatically performed after acquiring ta1 ¥ {text} to Data # m ¥ {text}, it is possible to reduce the load of the association work and improve the accuracy of the process.

It is also possible to add the external information acquired from the user to perform the matching process. The processing in this case will be described with reference to FIG.
When performing a process of associating multimedia data, the user asks the object video management device 10 to send an I
Identification data (ID) using a D card or keyboard
Enter. This identification data is a text that expresses the use, taste, information that can identify the person when the user handles the video, and together with the object text set # V ¥ # P ¥ {text}, the video multimedia association is performed. Sent to device 2. The video multimedia associating device 2 performs the matching process with the object text set # V ¥ # P ¥ {text} that is the first feature data, the identification data, and the keyword group that is the second feature data. The matching processing means 25 extracts a keyword that matches the text and outputs a keyword that matches the identification data from among the extracted keywords. The output result is the conforming data Data # L
(1), ..., Data # L (k). After that, after assigning a name identifier as above,
The naming service conversion table 32 is created.

In this way, by adding information from the user in addition to the text characterizing the video object and performing the process of associating the multimedia data, the association according to the taste of the user becomes possible.
The amount of data stored in association with the naming service conversion table 32 can be reduced, and the amount of data traffic in the network can be reduced. For the user as well, it is possible to eliminate multimedia data that does not match the user's taste and multimedia data that is less relevant.

Although the identification data is text data in the above description, it may be a classification identifier composed of a number or a code indicating the department to which the user belongs, and the same action and effect as above can be obtained. You can As processing in this case, the video multimedia associating device 2
The classification identifier is acquired, and the user information is acquired as text data according to the table defined in the storage means 27 and used for the matching process. Or, add a classification identifier to the keyword of multimedia data,
During the matching process, the text of the video object and the keyword of the multimedia data are text matched, and the classification identifier obtained from the user and the classification identifier attached to the keyword are matched to perform text matching and classification identifier matching. It is also possible to handle the data obtained by both of them as consistent data. Especially, in these cases, it is effective when a video multimedia associating system is built in-house and it is desired to set an access right to data prepared as multimedia data according to a job type and a job system.

According to the above, the following effects can be obtained in this embodiment. With the associating device of this embodiment, it is possible to automatically associate a video and data related to the video in a fine unit at the object level of the video in a network environment. As a result of performing the matching process in the feature space of the associated data, there is flexibility that it is not necessary to be aware of the other party. Once the associated data is automatically registered in the naming service, the work of each other is automatically updated thereafter. It goes without saying that this is useful for associating data in a network environment that changes from moment to moment.

The present invention is not limited to the above embodiment and can be widely applied. For example, a CD that records a program that causes a computer device to function as the video multimedia association device 2, the object video management device 10, or the related data management device 14.
It can be said that putting the recording medium such as a ROM or the Internet into a downloadable state is also an implementation of the present invention.

Further, as shown in FIG. 1, the video multimedia associating device 2, the object video management device 10,
Although the related data management device 14 has been described as an independent device, it may have a single device configuration or two devices may have a single device configuration.

[0034]

According to the invention described in any one of claims 1, 3, and 4, it becomes possible to perform data association for each video object in a network environment. In particular, since the association is automatically performed, it is not necessary to share information with each other in advance without being aware of the other party.

According to the second aspect of the present invention, in the network environment, when the data association is performed for each video object, the data association according to the user's taste and use and the access right management are performed. It will be possible.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a video multimedia data associating system according to an embodiment of the present invention.

FIG. 2 is a diagram for explaining a process in the video multimedia data associating system of FIG.

FIG. 3 is a diagram for explaining processing in the video multimedia data associating system of FIG. 1.

[Explanation of symbols]

1 Video multimedia association system 2 Video multimedia association device 10 Object video management device 11 Multimedia data storage means 13 Keyword extraction means 14 Multimedia data management device 21 Object Extraction Means 23 means for adding characteristic data 25 Matching processing means 27 storage means 28 associating means 29 identifier assigning means

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Chung Wen             1-10-11 Kinuta, Setagaya-ku, Tokyo, Japan             Broadcasting Association Broadcast Technology Institute (72) Inventor Toshihiko Misu             1-10-11 Kinuta, Setagaya-ku, Tokyo, Japan             Broadcasting Association Broadcast Technology Institute (72) Inventor Yasuaki Kinji             4-3441-8 Higashi Tamagawa Gakuen, Machida City, Tokyo F term (reference) 5B075 ND16 NK02                 5C052 AC08 DD04                 5C053 FA14 GB06 HA30 JA21 KA05                       LA11 LA14

Claims (4)

[Claims] 1. A video multimedia data associating device for associating an object included in a video with multimedia data including at least one of video, audio and text under a network environment, wherein the object is specified. For obtaining the first data associated with the object and the second data associated with the multimedia data for identifying the multimedia data, and matching the first data with the second data. Matching processing means for extracting the matching data as a result of the matching processing, and an identifier for identifying the storage location in the network environment of the multimedia data related to the matching data matched by this matching processing means is added to the matching data, The video object and the multimedia data Related data and identifier assigning means for, the relevant data video multimedia data association device, characterized in that it comprises a storage means for storing. 2. The matching processing means extracts matching data using information of a user who uses a video including the object in addition to the first data and the second data. 1. The video multimedia associating device according to 1. 3. A video multimedia data associating method for associating an object included in a video with multimedia data including at least one of video, audio and text under a network environment, wherein the object is specified. For obtaining the first data associated with the object and the second data associated with the multimedia data for identifying the multimedia data, and matching the first data with the second data. A matching step for extracting matching data as a result of the matching, and an identifier for specifying the storage location in the network environment of the multimedia data related to the matching data matched in this matching step is added to the matching data, Objects and the multimedia And identifier assigning step to relevant data and over data, a storing step of storing the associated data in the storage means,
A method for associating video multimedia data, comprising: 4. A device for associating an object included in a video with multimedia data including at least one of video, audio and text under a network environment, the device being associated with the object to identify the object. First data and second data associated with the multimedia data to identify the multimedia data, and obtain consistent data that is a result of aligning the first data and the second data. Matching processing means for extracting, the identifier for identifying the storage location in the network environment of the multimedia data related to the matching data matched by the matching processing means is added to the matching data, and the video object and the multimedia data Identifier assigning means to be related data with, A video multimedia data associating program characterized by causing it to function as a storage means for storing related data.