JP2003032616A - Image information producing device, image editing device, image information production program and image editing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image information producing device for recognizing a person in an image material by image recognition techniques to associate the image with the recognition results; and an image editing device for editing images with reference to information on a person appearing on an image program when producing the image program. SOLUTION: The arrangement comprises: an object detecting means 11 for detecting an object from an image signal; an object feature database 12 for accumulating an image feature of the object; and an object information producing means 13 for associating temporally the image signal with the detected information of the object detecting means 11 to produce object information for specifying the object in the image signal.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video information generating apparatus for recognizing a subject appearing in a video material from a video material used in producing a video program and generating subject information. Further, the present invention relates to a video editing device which edits a video based on the subject information when a video program is produced. Furthermore, the present invention relates to a video information generation program and a video editing program.

[0002]

2. Description of the Related Art In recent years, video programs have been distributed not only by conventional broadcasting but also through a plurality of media such as distribution via the Internet, and an opportunity to use the same video material multiple times. Is increasing.

However, there are restrictions on the use of video material such as the appearance fee of the performer caused by the distribution of the video program and the payment destination, consideration for portrait rights, and the usage conditions of the video exchanged with the interviewee. Since it is managed separately from the video material, it is necessary to compare the actual video with the constraint conditions when the video program is produced.

Therefore, as disclosed in the specification of Japanese Patent Application No. 9-338856, "Intellectual Property Information Management System and Computer-Readable Recording Medium Recording Its Program", There is one that stores and uses images in association with intellectual property information.

[0005]

However, Japanese Patent Application No. 9-3
In the intellectual property information management system described in the specification of No. 38856, it is necessary to identify the person shown in the video.
The user of the intellectual property information management system had to make a visual judgment. In addition, the work of associating the video with the intellectual property information on a time-by-time basis had to be done manually.

Further, conventionally, the concealing process of a person's face in consideration of the portrait right of the person and the work of changing the person's voice have been performed manually during image editing.

For this reason, the clerical processing for paying the appearance fee for the performers who appear in the video program and the processing for matters in which the portrait right of the performers should be taken into consideration are complicated.

The present invention has been made in view of the problems of the intellectual property right management at the time of producing the conventional video program as described above, and recognizes the person in the video material by the image recognition technique,
An object of the present invention is to provide a video information generating device that associates a video with a recognition result, and a video editing device that provides information of a person who appears in the video program when the video program is produced and edits the video.

[0009]

DISCLOSURE OF THE INVENTION The present invention is provided to achieve the above object. First, the video information generating apparatus according to claim 1 is characterized by the image characteristics of a subject included in a video signal. On the basis of the above, in the video information generation device that detects the subject from the video signal and generates detection information of the subject, the video signal and the detection information are associated with each other on a time axis, and the subject is detected in the video signal. It is configured to include subject information generation means for generating subject information to be specified.

According to such a configuration, in the video information generating device, the detection information of the subject is generated from the video signal based on the image feature, and the subject information generating means associates the video signal with the detection information on the time axis. Then, subject information for identifying the subject in the video signal is generated.

According to a second aspect of the present invention, there is provided the video information generating apparatus according to the first aspect, further comprising embedding means for embedding the subject information in the video signal as a digital watermark.

According to this structure, in the video information generating apparatus, the embedding means embeds the subject information in the video signal as a digital watermark, and generates a video signal having the subject information as an internal signal.

The subject information generating apparatus according to a third aspect is the video information generating apparatus according to the first or second aspect, wherein the subject information generating means is at least a person when the subject is a person. The face position and size are generated as subject information.

According to this structure, in the video information generating device, the subject information generating means generates the information for specifying the subject from the video signal based on at least the position and size of the face of the person.

A video editing apparatus according to a fourth aspect is
Among the video signals included in the video material, a subject information database that stores subject information that identifies a subject, a subject-related information database that stores relevant information related to the subject, and the subject-related information based on the subject information. The information control means for outputting at least one of the related information of the database and the statistical information obtained by statistics of the related information is provided.

According to this structure, in the video editing apparatus, subject information for identifying the subject in the video signal is stored in the subject information database, and related information relating to the subject is stored in the subject related information database. The information control means outputs at least one of the relevant information of the subject-related database and the statistical information in which the relevant information is statistically based on the subject information.

The video editing apparatus according to a fifth aspect is
From a video signal included in a video material, a digital watermark detecting means for detecting subject information for specifying a subject, which is embedded in the video signal as a digital watermark, a subject-related database that stores relevant information relating to the subject, and Based on the subject information, an information control unit that outputs at least one of the relevant information of the subject-related database and the statistical information that statistics the relevant information is configured.

According to this structure, in the video editing apparatus, the digital watermark detecting means detects the subject information for identifying the subject, which is embedded in the video signal as a digital watermark, and the subject-related database stores the relations related to the subject. The information is accumulated, and the information control means outputs at least one of the relevant information of the subject-related database and the statistical information obtained by statistics of the relevant information, based on the subject information.

Further, the video editing apparatus according to claim 6 is
The video editing apparatus according to claim 4 or 5, further comprising an editing unit that edits the video signal in units of frames, and the information control unit updates the statistical information based on an editing result of the editing unit. Then, it is configured to output.

According to this structure, in the video editing apparatus, the editing means edits the video signal on a frame-by-frame basis, and the information control means statistics the information for each frame based on the editing result of the editing means. Output information.

The video editing apparatus according to claim 7 is
The video editing apparatus according to claim 6, wherein the subject information in the subject information database is updated based on the editing result of the editing means.

According to this structure, the video editing apparatus updates the subject information in the subject information database based on the contents of the subject information that needs to be updated by the editing by the editing means.

Further, the video editing apparatus according to claim 8 is
7. The video editing apparatus according to claim 6, further comprising: a digital watermark embedding unit that embeds the subject information as a digital watermark in a video signal, and based on an editing result of the editing unit,
The subject information is updated and embedded in the video signal of the edited result as a digital watermark.

According to this structure, the video editing device updates the subject information based on the editing result of the editing means, embeds the subject information corresponding to the editing result in the video signal as a digital watermark, and internally stores the subject information. Generates a video signal held as a signal.

The video editing apparatus according to claim 9 is
The video editing apparatus according to any one of claims 4 to 8, wherein a video special effect is added to the video signal based on related information related to a subject extracted from the subject related database. It is configured to have an addition means.

According to this structure, the video editing device adds the video special effect to the video signal by the video special effect adding means based on the special effect content of the related information related to the subject extracted from the subject related database.

The video editing apparatus according to a tenth aspect is the video editing apparatus according to the ninth aspect, further comprising video display means for outputting a video signal to which the video special effect is added by the video special effect adding means. The editing means stops the output of the video signal to the video display means in each frame unit and selects whether or not to add the video special effect.

According to this structure, in the video editing apparatus, the editing means stops the display of the video signal in frame units, and further selects in each frame unit of the video signal whether or not to add the video special effect.

The video editing apparatus according to an eleventh aspect is the video editing apparatus according to any one of the fourth to tenth aspects, in which the relevant information related to the subject extracted from the subject-related database is added. Based on this, a configuration is provided that includes a sound special effect adding unit that adds a sound special effect to the sound signal.

According to this structure, in the video editing apparatus, the audio special effect adding means adds the audio special effect to the audio signal based on the special effect content of the related information related to the subject extracted from the subject related database.

According to a twelfth aspect of the present invention, there is provided a video information generation program for detecting the subject from the video signal based on image characteristics of the subject included in the video signal and generating detection information. The video signal and the detection information are associated with each other on the time axis, and the video signal and the detection information are made to function as a subject information generating unit that generates subject information for identifying a subject in the video signal.

According to this structure, in the video information generation program, the detection information of the subject is generated from the video signal based on the image feature, and the subject information generation means associates the video signal with the detection information on the time axis. Is
Subject information for identifying a subject in the video signal is generated.

A video editing program according to a thirteenth aspect is a video editing program for editing a video material recorded on a recording medium, wherein the video signal included in the video material specifies a subject. At least one of a subject information database that stores information, a subject-related information database that stores relevant information related to a subject, and related information in the subject-related database and statistical information that statistics the related information based on the subject information. Is configured to function as an information control unit that outputs

According to this structure, in the video editing program, the subject information database stores the subject information for specifying the subject in the video signal, and the subject related information database stores the relevant information related to the subject. The information control means outputs at least one of the relevant information of the subject-related database and the statistical information in which the relevant information is statistically based on the subject information.

[0035]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described in detail below with reference to the drawings. (First embodiment: video information generation device)

FIG. 1 is a block diagram showing the overall configuration of a video information generating apparatus according to the first embodiment of the present invention. As shown in FIG. 1, the video information generation device 1 includes a subject detection unit 11 that detects a subject from a video signal, and a subject feature database 12 that stores image features of the subject.
And subject information generation means 13 for associating the video signal with the detection information of the subject detection means 11 by time and generating subject information for specifying the subject in the video signal.

The video information generating apparatus 1 detects a subject from the video signal based on the image characteristics of the subject registered in advance, associates the detection result with the video signal in time, and This is a device that outputs the appearance status of the included subject as subject information.

The subject detecting means 11 performs a matching process on the subject appearing in each frame of the video signal by using the image feature of each subject registered in the subject feature database 12 described later and the image of the frame to determine the subject. The detection is performed, and the identification number of the subject and the position and size of the subject are output to the subject information generating means 13 as detection information.

The subject feature database 12 is a database in which the image features of the subject appearing in each frame of the video signal are registered. This subject feature database 1
In the item 2, image features are registered in advance for each identification number of a subject, and the image features are input in advance from an input device (not shown).

For example, Japanese Patent Application No. 11-206764 "Subject recognition device and recording medium", Japanese Patent Application No. 2000-3199.
15 Detecting a subject based on the technique described in "Personal Face Image and Feature Database and Storage Medium", and Japanese Patent Application No. 2000-59502 "Device and Program Storage Medium for Estimating Face Direction in Image". Is possible. Here, the feature points of the face of the subject are registered in the subject feature database 12 in advance. Next, the subject detection means 11
, The subject is estimated from the face feature points registered in the subject feature database 12 and each frame of the video signal,
As the detection information, the identification number of the subject and the position and size of the face of the subject are output.

The subject information generating means 13 determines, based on the detection information generated by the subject detecting means 11, subject information at which time and in which frame the subject appears in the entire video signal as subject information 101a. Output. At this time, the subject information generating means 13 associates, for each frame of the video signal, the number of persons appearing in the frame and the position and size of the face of each person. By associating the person with the frame, the person in the video signal is associated with the time axis. Note that the subject information 101a is stored in the subject information database 101, which is an external storage unit.

The subject information database 101 stores, for example, the subject information 101a shown in FIG. 3, the subject in the video signal is associated with the time axis, and the subject detection information is stored.

In the subject information 101a of FIG. 3, "Tape No." indicates the identification number of the tape in which the video material for creating the video program is stored, and "the number of cuts" is the continuous number included in the tape. The total number of cuts when the video scene is one cut is shown, and “cut No.” indicates an identification number for identifying each cut. The "cut start time" and "cut end time" indicate the start time and end time of the one cut. The above information can be created by notifying the subject information generating means 13 in advance by an input means (not shown).

The "frame number" indicates the total number of frames in the cut, and the "frame No." is a continuous identification number assigned to identify the frame.

The "number of persons" is the frame number. Is the number of subjects appearing in one frame. "Person ID"
Is an identifier for identifying the subject in the video signal, and has the same ID for the same subject in all frames.

"X coordinate of face position", "Y coordinate of face position", "face size (width)", "face size (height)"
Are respectively the detection results of the subject's face detected by the subject detecting means 11.

As described above, the data from the "person ID" to the "face size (height)" of the person is repeatedly accumulated for the "number of persons".

Data from the "frame No." to the "face size (height)" of the person at the end of the frame is repeatedly accumulated for the "frame number". In addition, the data from the “cut number” to the “face size (height)” of the person at the end of the cut is repeatedly accumulated for the “cut number”.

As described above, the video information generating apparatus 1 according to the first embodiment detects a subject from the video signal based on the image feature of the subject registered in advance, and the detection result and the video. By relating to the signal with time,
The appearance status of the subject included in the video signal can be output as the subject information 101a. (First embodiment: video editing apparatus)

FIG. 2 is a block diagram showing the overall structure of the video editing apparatus according to the first embodiment of the present invention. As shown in FIG. 2, the video editing device 2 includes an editing unit 20 for editing video material including a video signal and an audio signal.
A special effect adding means 30 for adding a special effect to a video signal and an audio signal, a subject information database 101 generated by the video information generating apparatus 1 of FIG. 1, and related information of a subject in advance (in the present embodiment, It comprises a subject-related information database 102 in which the appearance fee information 102a and portrait right information 102b) are accumulated, and information control means 40 for extracting information about the subject in the video signal.

Further, the video editing apparatus 2 includes an information display means (not shown) such as a CRT for displaying the relevant information of the subject,
Instruction means (not shown) such as a keyboard for inputting instruction information for video editing, and video display means (not shown) such as CRT for displaying edited video are connected to the outside.

Here, an example of the subject related information database 102 will be described with reference to FIG. The subject-related database 102 is a database that stores subject-specific information relating to a subject, and stores, for example, information about a person's appearance fee and information about a person's portrait right.

FIG. 4A shows an example in which the appearance fee information 102a is used as a database as the subject-related information. Here, the "number of persons" indicates the number of persons registered in the performance fee information 102a, the "person ID" is an identifier for identifying each performer, and the "person name" and "contact information" are appearances. The person's name and contact address are shown. In addition, "casting fee payee" and "casting unit price" indicate the payee of the casting fee paid to the performers and the casting fee per hour.

As described above, the data from the "person ID" to the "appearance unit price" are repeatedly accumulated for the "number of persons".

Further, FIG. 4B is an example in which the portrait right information 102b of the subject is used as a database as the subject-related information. Here, the “number of persons” indicates the number of persons registered in the portrait right information 102b, and the “person ID” is an identifier for identifying each person, and the “person name” and “contact information”.
Indicates the name and contact address of a person. Also,
The “whether face can be used” and “whether voice can be used” indicate whether the face may be displayed or the voice may be used as it is when the person appears as a video program. . This "whether face can be used" and "whether voice can be used" can be set directly by a value, or the condition can be described, for example, if a face or voice is used after a specified date. It is also possible to specify the condition that it is acceptable. In addition, "whether face can be used" and "whether voice can be used" are
A flag indicating whether to make an inquiry and a database destination may be described in an external database (not shown), and the availability information may be described in the database.

As described above, the data from the "person ID" to the "availability of voice" is repeatedly stored for the "number of persons".

Also, the subject-related information database 102
The copyright information may be converted into a database and, for example, a character string code displayed on the screen when a certain performer appears may be incorporated in the database, but the present embodiment is not limited to this. Absent.

Now, returning to FIG. 2, the description of the video editing apparatus 2 will be continued. The video editing device 2 is a device capable of performing video editing by outputting information regarding a subject appearing in the video signal and adding a special effect to the video signal and the audio signal during the video editing.

The editing means 20 edits / processes the video material composed of the input video signal and audio signal by an instruction means (not shown) such as a keyboard or the like.
01, a special effect addition control unit 202 for instructing addition of a special effect of video or audio to the special effect adding unit 30 described later, and an edited video / audio output unit 203 for outputting an edited result. .

The editing means 20 includes an editing / processing section 201.
Thus, in order to create a video program for a certain specified time from the video material, videos can be partially extracted and joined together. In this case, since the frame order / configuration is changed by this editing, the changed information is notified to the information control means 40 described later as the editing result information.

When adding a special effect of video or audio, the editing means 20 notifies the special effect adding means 30 to be described later by the special effect addition control section 202 of the effect instruction signal indicating the effect contents, and the special effect is added. The edited video / audio output unit 203 outputs the edited video signal and the edited audio signal based on the effect video signal and the effect audio signal to which the special effect is added by the adding means 30.

Further, the editing means 20 can display the editing / processing work on the video display means (not shown) for each frame of the video signal, and determine for each frame whether or not to perform the special effect. .

The special effect adding means 30 is composed of a video special effect adding section 301 and an audio special effect adding section 302.

The special effect adding means 30 is the editing means 2
Based on the 0 effect instruction signal, an effect video signal obtained by adding a special effect to the video signal is output. For example, the editing means 20
Is used as the effect instruction signal. By designating the designated area and the special effect (mosaic), the video special effect adding unit 301 in the special effect adding unit 30 can apply mosaic to the designated video area. Further, for example, as a special effect, it is possible to perform an effect such as displaying a character on the image.

Further, the special effect adding means 30 outputs an effect sound signal obtained by adding a special effect to the sound signal based on the effect instruction signal of the editing means 20. For example, the editing means 2
0 is the effect instruction signal, and the start frame No. for performing voice processing. And by instructing time and special effects (onomatopoeia),
Voice special effect adding section 302 in special effect adding means 30
However, the onomatopoeia can be output for a specified time.

The information control means 40 includes a database reading section 401 for reading information in an external database and a database writing section 402 for writing information in the external database.
And a subject information calculation unit 403 that calculates numerical information read from an external database, and a subject information output unit 4 that outputs the calculation result or the contents of the database.
04 and.

The information control means 40 includes the subject information 101a of the subject information database 101 generated by the image information generating apparatus 1 and the subject related information database 102.
The performance information 102a is read, and the information related to the performance time, such as the performance fee, is calculated as the subject information calculation unit 403.
Then, the subject information output unit 404 outputs the calculation result or the relevant information of the subject appearing in the video signal (the appearance fee information 102a, the portrait right information 102b).

The information output here is the editing means 2
It is possible to superimpose it on the video signal of the video display means (not shown) used for 0 and display it, or to display it on another display means (not shown).

Further, the information control means 40 gives the editing instruction information to the editing means 20 when the subject needs a special effect by the video signal or the audio signal by the portrait right information 102b of the subject related information database 102. Add special effects to signals or audio signals.

Further, the information control means 40 includes the editing means 20.
Based on the editing / processing result of the
2 is the subject information 10 of the subject information database 101.
Update 1a.

For example, when a frame is deleted by the editing of the editing means 20, the frame information of the subject information 101a is deleted and the "number of frames",
By updating the "cut start time" and the "cut end time", the subject information database 101 corresponding to the newly created video program can be constructed. (First Embodiment: Operation of Video Editing Device)

Next, the operation of the video editing apparatus 2 according to the first embodiment will be described with reference to FIG.

First, the video editing apparatus 2 has the subject information database 101 generated by the video information generating apparatus 1.
Information of the subject appearing in the video signal is read from the subject information 101a (step S1).

Next, the "person ID" of the subject information 101a
Read the performance fee information 102a and portrait right information 102b from the subject-related information database 102 (step S
2, step S3).

Next, the "person I" of the subject information 101a is described.
The appearance time of the "person ID" is calculated based on the "cut start time" and "cut end time" of "D" and the number of frames in which the person appears, and the "appearance unit price" of the appearance fee information 102a is used to calculate the appearance time. Calculate the appearance fee of a person (step S
4).

The information about the subject such as the person name and contact information of the "person ID", the information that changes with the video time such as the appearance fee and the appearance time, and the portrait right such as whether or not the face or voice can be used The information is displayed on the information display means (not shown) (step S5).

Next, it is judged whether or not the face can be used by the "whether or not the face can be used" information of the portrait right information 102b (step S
6) If the face cannot be used, a video special effect such as mosaic processing is added to a region corresponding to the position and size of the face of the subject information 101a (step S7).

Next, it is determined whether or not a voice can be used by the "whether voice can be used" information of the portrait right information 102b (step S
8) If the voice cannot be used, the voice is changed by adding a voice special effect (step S9).

The above operation is performed by the subject information database 1
01 and the contents of the subject-related information database 102, or it is possible to edit while confirming the contents for each frame of the video signal. (Second embodiment: video information generation device)

FIG. 6 is a block diagram showing the overall structure of a video information generating apparatus according to the second embodiment of the present invention. As shown in FIG. 6, the video information generating apparatus 1A is configured by adding a digital watermark embedding means 14 to the video information generating apparatus 1 shown in FIG. The components other than the digital watermark embedding unit 14 are designated by the same reference numerals as those shown in FIG.

The video information generating apparatus 1A uses the video signal to
The subject is detected based on the image characteristics of the subject registered in advance, the detection result and the video signal are associated with each other by time, and the appearance situation of the subject included in the video signal is used as subject information by a digital watermark. It is a device that outputs a video signal in which subject information is embedded.

The digital watermark embedding means 14 embeds the subject information in the video signal by a digital watermark by using the appearance information of the subject in the entire video signal generated by the subject information generating means 13 as the subject information.

At this time, the information to be embedded in the video signal is the same as the content of the subject information 101a shown in FIG. However, instead of embedding all the information in one frame,
For example, from "Tape No." to "Number of frames" and the first
Information of subjects corresponding to the number of persons in the frame included in the frame is embedded in the signal of the first frame as a digital watermark,
After the second frame, it is also possible to embed information of subjects corresponding to the number of persons in the frame included in the frame as a digital watermark in the signal of the frame.

The digital watermark embedding method can be performed based on the existing technique described in, for example, Japanese Patent Application No. 10-144237 "Digital watermark embedding device and digital watermark extraction".

As described above, the video information generating apparatus according to the second embodiment detects a subject from the video signal based on the image feature of the subject registered in advance, and the detection result and the video signal are detected. And associated with time,
By using the appearance status of the subject included in the video signal as subject information, a subject information-added video signal can be output by a digital watermark. Here, it is assumed that the video signal with subject information by the digital watermark is stored as the video material 601a in the external video storage device 601 together with the audio signal. (Second embodiment: video editing device)

FIG. 7 is a block diagram showing the overall structure of the video editing apparatus according to the first embodiment of the present invention. As shown in FIG. 7, in the video editing apparatus 2A, the information control means 40 of the video editing apparatus 2 shown in FIG. 2 is replaced with the information control means 40A whose function is changed, the subject information database 101 is lost, and a new electronic device is provided. A watermark detecting / embedding unit 50 is added. Configurations other than the function of the information control unit 40A and the additional configuration of the digital watermark detection / embedding unit 50 are denoted by the same reference numerals as those shown in FIG.

Further, the video editing device 2A inputs the video storage device 601 generated in FIG. 6, information display means (not shown) such as a CRT for displaying relevant information of the subject, and video editing instruction information. An instruction means (not shown) such as a keyboard, and a video display means (not shown) such as a CRT for displaying an edited video are connected to the outside.

Here, the video material 601a of the video storage means 601 of FIG. 6 holds the subject information 101a embedded in the video signal by a digital watermark. The subject information 101a, the subject-related information database 102, and the appearance fee information 102 in the subject-related information database 102.
Since the a and portrait right information 102b are the same as those in FIGS. 3 and 4, description thereof will be omitted.

The digital watermark detecting / embedding means 50 includes a digital watermark detecting section 501 for detecting an information signal from a video signal in which an information signal is embedded as a digital watermark, and a digital watermark embedding for embedding the information signal in the video signal as a digital watermark. And a section 502.

The digital watermark detecting / embedding means 50 is
The digital watermark detection unit 501 detects the subject information 101a from the video signal in which the subject information 101a is embedded with a digital watermark, and separates the video signal and the subject information 101a. A video signal in which the subject information 101a is embedded with a digital watermark is a video material 601 generated by the video information generation device 1A of FIG. 6 and stored in the video storage device 601.
a can be used.

Further, the digital watermark detecting / embedding means 50
The digital watermark embedding unit 502 digitally watermarks the subject information 101a to the edited video signal from the edited video signal and the edited audio signal generated by the editing means 20 and the subject information 101a changed by the information control means 40A. Embedded and output as video signal. By updating the video material 601a of the video storage device 601 with this signal,
The video material 601a resulting from the video editing can be used again in the video editing device 2A.

The digital watermark detecting / embedding means 5 is used.
For 0, the input signal of the audio signal is output as it is, but the subject information 101a may be embedded in the audio signal instead of the video signal by a digital watermark.

The information control means 40A adds a digital watermark control section 405 instead of the database writing section 402 of the information control means 40 shown in FIG.
5 reads the subject information 101a detected by the digital watermark detection / embedding unit 50. Also, the subject information 101a
Is updated, the electronic watermark control unit 405 updates the subject information 1 to the electronic watermark detection / embedding unit 50.
By notifying 01a, the digital watermark embedding unit 502 of the digital watermark detecting / embedding unit 50 causes the subject information 101
A is embedded in the video signal as a digital watermark.

The method of detecting and embedding a digital watermark performed here can be performed based on the existing technique described in, for example, Japanese Patent Application No. 10-144237 "Digital watermark embedding apparatus and digital watermark extracting apparatus".

The functions other than the functions of the digital watermark detecting / embedding means 50 and the information control means 40A described above are the same as those of the video editing apparatus 2 of FIG.

The present invention has been described above based on the embodiment, but the present invention is not limited to this.

[0097]

As described above, the video information generating apparatus and the video editing apparatus according to the present invention have the following excellent effects.

According to the first aspect of the present invention, the video information generating device can detect the subject from the video signal based on the image feature and generate the detection information of the subject,
The subject information generation means can generate subject information that identifies the subject in the video signal by associating the video signal with the detection information on the time axis.

Thus, the subject included in the video signal can be automatically detected, and the subject information including the position and size of the subject appearing in the video signal, the appearance time of the subject, and the appearance frame can be manually detected. Can be generated without going through.

According to the second aspect of the present invention, the video information generating apparatus can embed the subject information in the video signal as a digital watermark by the embedding means, and generate the video signal having the subject information as an internal signal. Can be generated.

Since the subject information is embedded in the video signal, it is not necessary to divide and manage the video signal and the subject information appearing in the video signal.

According to the third aspect of the present invention, the video information generating device generates, by the subject information generating means, information for identifying the subject from the video signal based on at least the position and size of the face of the person. be able to.

As a result, the person in the video signal can be detected by the face, and the size and position of the face can be used as subject information.

According to the fourth aspect of the present invention, in the video editing device, the subject information database can store subject information for identifying the subject in the video signal, and the subject-related information database can be used as the subject information database. Related information related to the subject information can be accumulated, and the information control means can output at least one of the relevant information of the subject related database and the statistical information that is a statistical information of the relevant information based on the subject information. it can.

Thus, it is possible to check the cast information at any time during video editing. Further, for example, the appearance fee of the person appearing in the video signal can be automatically calculated according to the appearance time, so that the payment of the appearance fee can be easily managed.

According to the fifth aspect of the invention, in the video editing apparatus, the digital watermark detecting means can detect the subject information for specifying the subject, which is embedded in the video signal as a digital watermark, and the subject-related information can be obtained. The database is
Relevant information related to the subject can be accumulated, and the information control means outputs at least one of the relevant information in the subject-related database and the statistical information that statistics the relevant information based on the subject information. You can

With this, since the video signal in which the subject information is embedded can be edited, it is not necessary to mistakenly recognize the performer in the video signal. You can also check the performer information at any time during video editing.
Also, for example, there is no mistake in the cast, and
Since the appearance fee of the person appearing in the video signal can be automatically calculated according to the appearance time, the performance fee payment management can be performed reliably and easily.

According to the invention described in claim 6, in the video editing apparatus, the editing means can edit the video signal on a frame-by-frame basis, and the information control means calculates the frame based on the editing result of the editing means. Information can be statistically calculated for each time and the statistical information can be output.

As a result, even if the video editing is performed, the subject information can be updated accordingly. For example, even if the video editing is performed, the performer's performance fee is updated according to the appearance time associated with the editing. can do.

According to the invention described in claim 7, the video editing apparatus stores the subject information in the subject information database,
It can be updated based on the content of the subject information that needs to be updated by the editing of the editing means.

As a result, when the edited video is edited again, the updated subject information database can be reused, and the work efficiency of video editing can be improved.

According to the invention described in claim 8, the video editing apparatus updates the subject information based on the editing result of the editing means, and embeds the subject information corresponding to the editing result in the video signal as a digital watermark, A video signal having the subject information as an internal signal can be generated.

Thus, when the edited video is edited again, the updated video signal can be reused, so that the work efficiency of the video editing can be improved. Further, since the subject information is embedded in the video signal, it is not necessary to divide and manage the video signal and the subject information appearing in the video signal, which facilitates management.

According to the invention described in claim 9, in the video editing apparatus, the video special effect adding means adds the video special to the video signal based on the special effect content of the related information related to the subject extracted from the subject related database. The effect can be added.

Thus, for example, when the face of the performer cannot be displayed due to the portrait right or the like, editing such as applying mosaic to the face of the performer can be easily performed, and the portrait right is taken into consideration. Editing can be done easily.

According to the tenth aspect of the invention, in the video editing apparatus, the editing means can stop the display of the video signal in frame units, and further add a video special effect in each frame of the video signal. You can choose to do it or not.

As a result, it is possible to confirm the editing on a frame-by-frame basis. For example, it is easy to change the size of the mosaic on the performer's face or to specially incorporate character information into the video. Can be done.

According to the eleventh aspect of the present invention, in the video editing apparatus, the audio special effect adding means adds the audio special to the audio signal based on the special effect content of the related information related to the subject extracted from the subject related database. The effect can be added.

Thus, for example, when the voice of the performer cannot be used because of the portrait right, it is possible to perform editing such as performing voice processing on the voice of the performer, and the edit in consideration of the portrait right. Can be done easily.

According to the twelfth aspect of the present invention, the video information generation program can detect the subject from the video signal based on the image feature and generate the detection information. A signal and the detection information can be associated with each other on a time axis to generate subject information that identifies a subject in the video signal.

Thus, the subject included in the video signal can be automatically detected, and the subject information including the position and size of the subject appearing in the video signal, the appearance time of the subject, and the appearance frame can be manually detected. Can be generated without going through.

According to the thirteenth aspect of the present invention, in the video editing program, the subject information database can store subject information for identifying the subject in the video signal, and the subject-related information database can be used as the subject information database. Related information related to the subject information can be accumulated, and the information control unit can output at least one of the relevant information of the subject related database and the statistical information that is the statistical information of the relevant information based on the subject information. it can.

With this, it is possible to check the cast information at any time during the video editing. Further, for example, the appearance fee of the person appearing in the video signal can be automatically calculated according to the appearance time, so that the payment of the appearance fee can be easily managed.

[Brief description of drawings]

FIG. 1 is a block diagram showing an overall configuration of a video information generation device according to a first embodiment of the present invention.

FIG. 2 is a block diagram showing the overall configuration of a video editing apparatus according to the first embodiment of the present invention.

FIG. 3 is a diagram illustrating an example of a subject information database (subject information).

FIG. 4 is a diagram illustrating an example of a subject related information database (casting person information and portrait right information).

FIG. 5 is a flowchart illustrating the operation of the video editing apparatus according to the first and second embodiments of the present invention.

FIG. 6 is a block diagram showing an overall configuration of a video information generation device according to a second embodiment of the present invention.

FIG. 7 is a block diagram showing an overall configuration of a video editing device according to a second embodiment of the present invention.

[Explanation of symbols]

1, 1A ... Image information generating device 2, 2A ... Video editing device 11 ... Subject detection means 12 ... Subject feature database 13 ... Subject information generating means 14 ... Digital watermark embedding means 20 ... Editing means 201 …… Editing / Processing Department 202 ... Special effect addition control unit 203 ... Editing video / audio output section 30 ... Means for adding special effects 301: Image special effect adding section 302 ... Voice special effect adding section 40 ... Information control means 401 ... Database reading unit 402 ... Database writing section 403 ... Subject information calculation unit 404 ... Subject information output section 405 ... Digital watermark control unit 50 ... Digital watermark detection / embedding means 501 ... Digital watermark detection unit 502 ... Digital watermark embedding section 101 ... Subject information database 101a ... subject information 102 ... Subject related information database 102a …… Performance information 102b ... Portrait right information 601 ... Video storage device 601a ... Footage

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) H04N 5/91 Z (72) Inventor Koichiro Arita 4-14 Kamiyamacho, Shibuya-ku, Tokyo Daiichi Kyodo Building Co., Ltd. NPK Computer Service (72) Inventor Mutsuo Yamaga 1-10-11 Kinuta, Setagaya-ku, Tokyo Inside Broadcasting Technology Laboratories, Japan Broadcasting Corporation (72) Takayuki Ito 1-10-1 Kinuta, Setagaya-ku, Tokyo Japan Broadcasting Association Broadcasting Technology Research Institute (72) Inventor Clipping del Simon 1-10-11 Kinuta, Setagaya-ku, Tokyo Japan Broadcasting Corporation Broadcasting Technology Research Institute (72) Inventor Takahiro Mochizuki 1-10-11 Kinuta, Setagaya-ku, Tokyo No. Japan Broadcasting Corporation Broadcasting Technology Research Institute F-term (reference) 5C023 AA06 AA09 AA11 BA02 BA11 CA02 CA04 CA05 5C053 FA13 GB06 GB11 JA01 JA12 JA16 JA21 KA24 5C063 AA01 AB03 AB07 AC01 AC05 AC10 CA03 CA23 CA29 CA36 DA03 DA07 DA13 DB09 5L096 FA00 HA08 JA11

Claims (13)

[Claims] 1. A video information generation apparatus for detecting the subject from the video signal and generating detection information of the subject based on image characteristics of the subject included in the video signal, the video signal and the detection information. And a subject information generating unit that generates subject information for identifying a subject in the video signal by associating with each other on a time axis. 2. The video information generation apparatus according to claim 1, further comprising an embedding unit that embeds the subject information in the video signal as a digital watermark. 3. The video according to claim 1, wherein the subject information generating unit generates at least the position and size of the face of the person as subject information when the subject is a person. Information generation device. 4. A video editing apparatus for editing video material recorded on a recording medium, comprising: a subject information database that stores subject information for identifying a subject in a video signal included in the video material; A subject-related information database that accumulates relevant related information; and an information control unit that outputs at least one of the relevant information of the subject-related database and statistical information that statistics the relevant information based on the subject information. A video editing device characterized by being provided. 5. A video editing apparatus for editing video material recorded on a recording medium, wherein subject information for identifying a subject, which is embedded as a digital watermark in the video signal from the video signal included in the video material, is provided. At least one of digital watermark detection means for detecting, a subject-related database that stores relevant information related to a subject, and related information in the subject-related database and statistical information that statistically collects the related information based on the subject information. An image editing apparatus comprising: an information control unit for outputting the. 6. An editing means for editing the video signal frame by frame, wherein the information control means updates and outputs the statistical information based on an editing result of the editing means. The video editing device according to claim 4 or claim 5. 7. The video editing apparatus according to claim 6, wherein the subject information in the subject information database is updated based on the editing result of the editing means. 8. A digital watermark embedding unit for embedding the subject information as a digital watermark in a video signal, and updating the subject information based on an editing result of the editing unit,
7. The video editing apparatus according to claim 6, wherein the video signal of the edited result is embedded as a digital watermark. 9. A video special effect adding means for adding a video special effect to the video signal based on related information related to a subject extracted from the subject related database. Any one of item 8
The video editing device according to the item. 10. A video display means for outputting a video signal subjected to the video special effect in said video special effect adding means, wherein said editing means stops the output to said video display means for each frame of the video signal. However, it is possible to select whether or not to add the video special effect.
The video editing device described in. 11. A sound special effect adding unit for adding a sound special effect to a sound signal based on related information related to a subject extracted from the subject related database. Any one of ten
The video editing device according to the item. 12. A video information generation program for detecting the subject from the video signal and generating detection information based on image characteristics of the subject included in the video signal, wherein the video signal and the detection information are time-based. And a video information generation program that causes the video information to function as a video information generation unit that generates video information to identify a video object in the video signal. 13. A video editing program for editing video material recorded on a recording medium, comprising a subject information database in which subject information for identifying a subject in a video signal included in the video material is accumulated, Associated information database for accumulating associated information, and information control means for outputting at least one of associated information in the associated subject database and statistical information that statistics the associated information based on the subject information. Editing program.