JPWO2008050718A1 - Rights information extraction device, rights information extraction method and program - Google Patents

Abstract

The credit recognition means 100 includes a credit information superimposition section detection means 400 for detecting a credit information superimposition section in which credit information such as a music name and a character name is superimposed in content such as video, and the credit information superimposition section. Credit information reading means 600 for reading credit information and outputting the result as credit candidate information.

Description

  The present invention relates to a rights information extraction device, a rights information extraction method, and a program, and more particularly, to a rights information extraction device, a rights information extraction method, and a program that extract information related to rights such as copyright and copyright right from content such as a program. .

  Conventionally, for example, Patent Document 1 discloses a rights management system for managing rights such as copyrights attached to content.

  The rights management system described in Patent Document 1 collectively manages rights such as copyrights on a content management server, and responds to requests from content users by cooperating with a contract management server, billing server, authentication server, and the like. Automatic contracts and secure distribution of content.

  On the other hand, it is assumed that the right information is manually registered by an intermediary. That is, the present situation is that the maintenance of the database of rights information relies on humans, and in order to be able to handle the contents produced in the past with a system as described in Patent Document 1, somebody has rights information. Must be extracted and registered in the database through an intermediary.

  However, in the case of past contents, there are many cases where details of contract information and the like do not remain, and it is first necessary to clarify who has the rights to the contents. Conventionally, these operations are registered while being confirmed manually, and there has been a problem that a very large number of man-hours must be spent on this process. This has also been a cause of not having excellent content such as dramas broadcast on the secondary distribution market.

JP 2002-109254 A

  The first problem is that identification of right information related to content such as video is not automated. For this reason, when using the past contents, it is necessary to manually extract the right information in order to identify the right holder of the contents, which takes a lot of time and effort.

  The second problem is that the accuracy is not sufficient when this is extracted by a technique such as telop recognition. The reason is that normal telop recognition does not specialize in rights information, and therefore has a problem of low accuracy. In addition, there is a lot of information not related to rights, and there is a problem that it is difficult to specify rights information.

  Accordingly, the present invention has been invented in view of the above problems, and it is an object of the present invention to provide a rights information extraction apparatus and a rights information extraction method that can automatically extract information about rights from content such as video.

  The present invention for solving the above-described problems is characterized by comprising credit information recognition means for reading credit information related to rights from content and outputting the result as credit candidate information.

  The present invention for solving the above-mentioned problems is a rights information extraction device for extracting rights information related to rights from content, the credit information recognition means for reading credit information about rights from content and outputting the result as credit candidate information, Analyzing the content, recognizing the object related to the right in the content, and outputting the result as object identification information, integrating the credit candidate information and the object identification information, And integrating means for outputting as information.

  The present invention for solving the above-mentioned problems is a rights information extraction device for extracting rights information related to rights from content, the credit information recognition means for reading credit information about rights from content and outputting the result as credit candidate information, Object recognition means for referring to credit candidate information, analyzing the content to recognize an object related to rights in the content, and outputting the result as object identification information; the credit candidate information and the object identification And integrating means for integrating the information and outputting the information as right information.

  The present invention for solving the above-mentioned problems is a rights information extraction method characterized by reading credit information about rights from content and outputting the result as credit candidate information.

  The present invention for solving the above-mentioned problems is a right information extraction method for extracting right information related to rights from content, a process of reading credit information related to rights from content and outputting the result as credit candidate information, and analyzing the content A process for recognizing an object related to the right in the content and outputting the result as object identification information; and a process for integrating the credit candidate information and the object identification information and outputting as rights information. It is characterized by having.

  The present invention for solving the above-mentioned problems is a right information extraction method for extracting right information related to rights from content, a process of reading credit information related to rights from content and outputting the result as credit candidate information, and the credit candidate information And analyzing the content, recognizing the object related to the right in the content, and outputting the result as the object identification information, integrating the credit candidate information and the object identification information, And processing for outputting as right information.

  The present invention that solves the above problems is a program that causes an information processing apparatus to execute a process of reading credit information about a right from a content and outputting the result as credit candidate information.

  The present invention for solving the above-described problems is a process of reading credit information related to rights from content, outputting the result as credit candidate information, analyzing the content, recognizing an object related to rights in the content, and obtaining the result. A program that causes an information processing device to execute a process of outputting as object identification information and a process of integrating the credit candidate information and the object identification information and outputting as rights information.

  The present invention that solves the above-described problems is a process of reading credit information related to rights from content and outputting the result as credit candidate information, and referring to the credit candidate information, analyzing the content, and an object related to rights in the content And processing the information processing apparatus to output the result as object identification information and the process of integrating the credit candidate information and the object identification information and outputting as right information. It is a program.

  According to the present invention, since the credit information related to the right is read from the content and the result is output as the credit candidate information, the information related to the right related to the content can be automatically extracted with high accuracy.

  In addition, it is possible to save labor in processing for extracting information related to rights such as copyrights from the content. The reason is that it is possible to automatically extract copyright information such as copyright from the content.

FIG. 1 is a block diagram of the credit recognition means 100. FIG. 2 is a diagram showing the configuration of the credit information superimposition section detecting means 400. FIG. 3 is a diagram showing a specific configuration of the theme song detection means 410. FIG. 4 is a diagram showing another specific configuration of the theme song detection means 410. FIG. 5 is a diagram showing another specific configuration of the theme song detection means 410. FIG. 6 is a diagram showing another specific configuration of the theme song detection means 410. FIG. 7 is a diagram showing another specific configuration of the theme song detection means 410. FIG. 8 is a diagram showing another specific configuration of the theme song detection means 410. FIG. 9 is a diagram showing another configuration of the credit information superimposition section detecting means 400. In FIG. FIG. 10 is a diagram showing another configuration of the credit information superimposition section detecting means 400. FIG. 11 is a diagram showing another specific configuration of the credit information superimposition section detecting means 400. FIG. 12 is a diagram showing an example of the credit information reading unit 600. FIG. 13 is a diagram showing another configuration of the credit information reading means 600. FIG. 14 is a diagram showing the configuration of the theme song background video generation means 620. FIG. 15 is a diagram showing another configuration of the credit information reading means 600. FIG. 16 is a diagram showing the configuration of the second exemplary embodiment of the present invention. FIG. 17 is a diagram showing the configuration of the third exemplary embodiment of the present invention. FIG. 18 is a diagram showing a configuration example of the music work recognizing means 101. FIG. 19 is a diagram showing another configuration of the music work recognizing means 101. FIG. 20 is a diagram showing the configuration of the music work collating means 822. FIG. 21 is a diagram showing the configuration of the character recognition means 102. FIG. 22 is a diagram showing another configuration of the character recognition means 102. FIG. 23 is a diagram showing another configuration of the third exemplary embodiment of the present invention. FIG. 24 is a diagram showing another configuration of the third exemplary embodiment of the present invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Credit information recognition means 103 Integration means 105 Object recognition means 120 Integration means 130 Integration means 400 Credit information superimposition area detection means 410 Theme song detection means 420 Image | video cutout means 430 Sound feature-value extraction means 431 Sound feature-value collation means 432 Theme song Acoustic feature quantity database 435 Thematic song acoustic feature quantity database 440 Acoustic feature quantity extraction means 441 Acoustic feature quantity verification means 442 Visual feature quantity extraction means 443 Visual feature quantity verification means 445 Acoustic feature quantity verification means 450 Continuous acoustic section extraction means 451 Thematic song Section determination means 452 Thematic song candidate section determination means 470 Continuous telop detection means 480 Roll telop detection means 481 Selection means 600 Credit information reading means 610 Telop reading means 620 Thematic song background video generation Stage 630 Theme song background difference video generation means 640 Ticker reading means 650 Ticker reading result comprehensive judgment 700 Background video generation means 710 Corresponding frame calculation means 720 Visual feature quantity extraction means 800 Music candidate extraction information output means 801 Candidate acoustic feature quantity selection means 802 Music work collation means 803 Music acoustic feature database 820 Music related production information extraction means 821 Music work collation parameter selection means 822 Music work collation means 823 Music work collation parameter database 900 Performer candidate information extraction means 901 Candidate person characteristics Quantity selection means 902 Performer collation means 903 Person feature quantity database 920 Performer affiliation group extraction means 921 Performer collation parameter selection means 922 Performer collation means 923 Person collation parameter database 9 0 audio superimposing determining means 951 acoustic feature checker means

<First Embodiment>
A first embodiment will be described.

  In the first embodiment, a credit information recognition unit 100 that analyzes content, reads credit information, and outputs information that is a candidate for credit information will be described.

  FIG. 1 is a block diagram of the credit recognition means 100.

The credit information recognizing means 100 extracts a section (hereinafter, this section is referred to as a credit information superimposition section) that is likely to contain credit information from the content. Next, video and audio included in the credit information superimposition section are analyzed, and telop information and audio are read as credit information from the content. Then, the result is output as credit candidate information.
Further, when the credit information recognition unit 100 reads the credit information, it may be configured to read not only the section where the possibility that the credit information is superimposed is high but also the low section. Furthermore, the credit information to be read may be such that credit information not related to the content right is also read.

  Here, the content is composed of video, audio, and the like, for example, a television program, a radio program, a movie, and the like, and is not necessarily limited to broadcast or released, but stored in a recording medium such as a DVD. Also included.

  Credit information is a telop or audio that contains information such as the original author, screenwriter, performer, theme song, cooperating organization, provider, etc., superimposed on the theme song or the last part of the content. is there.

  The content input to the credit information recognition unit 100 may be input in a compressed format such as MPEG, or may be input after being already decrypted. When input as a compressed video, analysis is performed while decoding the video in the credit information recognition means. The program video may be a video for a specific one-time broadcast, or multiple times of the same program (for example, from the first episode to the tenth episode of the drama) at the same time. It may be configured to input.

  Further, the credit candidate information may include a recognized character string, its time information, and position information in the image (coordinates in the frame). In addition, it may include an index representing the probability of recognition of telop or voice. Further, the credit candidate information may output one piece of information for each recognized character string, or may output a plurality of candidate character strings. If the telop cannot be read, information specifying the spatio-temporal position in the video including the telop may be included in the credit candidate information and output. Furthermore, the video information itself at this spatiotemporal position may be included in the credit candidate information and output.

  Next, each configuration of the credit information recognition unit 100 shown in FIG. 1 will be described. In the following description, a program video will be described as an example of content.

  Referring to FIG. 1, the credit information recognition unit 100 includes a credit information superimposition section detection unit 400 and a credit information reading unit 600.

  The credit information superimposition section detection means 400 receives a program video as an input, and its output is connected to the credit information reading means 600. The credit information reading unit 600 receives the credit information superimposed section video data output from the credit information superimposed section detecting unit 400 and outputs credit candidate information.

  Next, the operation of the embodiment of the credit information recognition unit 100 shown in FIG. 1 will be described.

  First, the program video is input to the credit information superimposition section detecting means 400. The credit information superimposition section detecting means 400 specifies a credit information superimposition section using features such as visual feature quantity, acoustic feature quantity, or telop appearance pattern. Details of this method will be described later. Then, the video data of the specified time interval is output as credit information superimposed interval video data.

  The credit information superimposed section video data is input to the credit information reading means 600. In the credit information reading means 600, telop recognition is performed on an input video or a video obtained by processing it. The obtained recognition result is output as credit candidate information.

  As described above, the credit information recognition unit 100 according to the present invention identifies a portion where credit information is superimposed and identifies the section with priority, thereby simply applying telop recognition to a video. In comparison, it is possible to extract credit information efficiently and accurately.

<Specific Configuration of Credit Information Superimposing Section Detection Unit 400>
1. Detecting credit information superimposition section focusing on the theme song A specific configuration of the credit information superimposition section detection means 400 will be described. The credit information superimposing section detecting means 400 described below is a specific example using the fact that credit information is often superimposed on the theme song portion of the sound included in the content. In addition, although the theme song was used as an example of sound, if it is similar to this, it is not restricted to a theme song.

  FIG. 2 is a diagram showing the configuration of the credit information superimposing section detecting means 400, and the credit information superimposing section detecting means 400 includes a theme song detecting means 410 and a video cutout means 420.

  The theme song detection means 410 receives the program video as input, and the section designation time information as the output is connected to the video cutout means 420. The video cutout unit 420 receives the program specified video and the section designation time information output from the theme song detection unit 410, and outputs credit information superimposed section video data.

  Next, the operation of the credit information superimposition section detecting means 400 shown in FIG. 2 will be described.

  The program video is first input to the theme song detection means 410. The theme song detection means 410 extracts a section including the theme song from the video. This is because in most cases, such as dramas, credit information is superimposed on the theme song. That is, the time section of the theme song can be regarded as a credit information superimposition section. Details of the method for extracting the theme song time section from the program video will be described later. The time information specifying the extracted theme song portion is output as the section designation time information.

  The section designation time information is input to the video cutout unit 420 together with the program video. The video cutout means 420 identifies video data designated by the section designation time information from the program video and outputs it as credit information superimposed section video data. Here, the specified video may be actually cut out from the original program video and output. Or, do not actually cut out, just get the information to jump to the beginning and end of the section (for example, the number of bytes from the beginning of the program), so that you can immediately head to the specified section But you can. Also in this case, since the head of the designated section can be accessed immediately, the subsequent processing can be performed in the same manner as when the cut is actually performed.

  As described above, the credit information superimposition section detecting means shown in FIG. 2 can obtain the credit information superimposition section with high accuracy by detecting the theme song.

(1) Specific configuration example 1 of the theme song detection means 410
A specific configuration of the theme song detection unit 410 will be described.

  Referring to FIG. 3, the theme song detection means 410 includes an acoustic feature quantity extraction means 430, an acoustic feature quantity verification means 431, and a theme song acoustic feature quantity database 432. The acoustic feature quantity extraction means 430 receives the program video, and the output acoustic feature quantity is input to the acoustic feature quantity verification means 431. The acoustic feature quantity matching means 431 receives the acoustic feature quantity output from the acoustic feature quantity extraction means 430 and the acoustic feature quantity from the theme song acoustic feature quantity database 432, and outputs section designation time information.

  Next, the operation of the theme song detection means 410 shown in FIG. 3 will be described.

  The program video is first input to the acoustic feature quantity extraction means 430. The acoustic feature quantity extraction means 430 analyzes the acoustic signal of the program video and extracts the acoustic feature quantity. The extracted acoustic feature amount is output to the acoustic feature amount matching unit 431. The acoustic feature quantity matching unit 431 compares the acoustic feature quantity of the program video input from the acoustic feature quantity extraction unit 430 with the theme song acoustic feature quantity in the theme song acoustic feature quantity database. Then, time information (such as the start point, end point, and time length of the section) that specifies the time section corresponding to the collated portion in the program acoustic feature amount is output as the section designation time information.

  Here, the theme song acoustic feature quantity is an acoustic feature quantity extracted in advance from the theme song. This is registered in advance and used for theme song detection. At this time, if the theme song is known in advance, an acoustic feature can be extracted from the sound source (such as a CD) of the theme song and used. Alternatively, in the case of a series drama or the like, the section of the theme song can be specified by a certain video, and the acoustic feature of this part can be used to detect the theme song of another drama video.

  Since the above-described theme song detection means 410 performs matching using the feature quantity registered in the theme song acoustic feature quantity database, the theme song part can be reliably extracted.

(2) Specific configuration example 2 of the theme song detection means 410
Another specific configuration of the theme song detection unit 410 will be described.

  Referring to FIG. 4, another specific configuration example of the theme song detection unit 410 is shown, which includes an acoustic feature amount extraction unit 440 and an acoustic feature amount comparison unit 441. The acoustic feature quantity extraction means 440 receives the program video, and the output acoustic feature quantity is input to the acoustic feature quantity verification means 441. The acoustic feature quantity matching unit 441 receives the acoustic feature quantity output from the acoustic feature quantity extracting unit 440 and outputs section designation time information.

  Next, the operation of the theme song detection means 410 shown in FIG. 4 will be described.

  The program video is first input to the acoustic feature quantity extraction means 440. Here, the program video is not a single-time program video, but a plurality of times of program video are input together. For example, in the case of a series drama, it is assumed that drama videos for several episodes are input together. The acoustic feature quantity extraction means 440 extracts an acoustic feature quantity for each of the plurality of drama videos. The extracted acoustic feature amount at each time is output to the acoustic feature amount matching unit 441.

  The acoustic feature amount matching unit 441 performs matching between acoustic feature amounts of a plurality of input programs. At this time, collation is not performed for the entire program of each time, but between sections of an arbitrary length cut out from the program. As a result, a section in which the acoustic feature amounts coincide with each other is obtained. Of the sections obtained in this way, those having a certain section length or more are considered to be highly likely to correspond to the theme song. Therefore, the time information for designating a section longer than the predetermined section length obtained above is output as the section designation time information. Or you may determine using the positional information on an area further. That is, the section of the theme song may be specified using the fact that the theme song often flows at the beginning or end of the program. This information is output for each program.

  Even if the theme song detection means 410 shown in FIG. 4 does not know what the theme song is, the theme song portion can be specified by finding a place having the same acoustic pattern using a plurality of images. That is, a database storing theme song feature quantities is not required.

  If the theme song part is specified in the first few times and the feature quantity of the theme song is extracted, the theme song part is extracted in the same manner as in FIG. 3 in the subsequent times using the feature quantity of the extracted theme song. It can also be specified. This makes it possible to reliably identify the theme song portion with a smaller amount of computation than in the case where a common portion is found in a plurality of moving images as a whole.

(3) Specific configuration example 3 of the theme song detection means 410
Another specific configuration of the theme song detection unit 410 will be described.

  Referring to FIG. 5, an example of another specific configuration of the theme song detection unit 410 is shown, which includes a continuous sound segment extraction unit 450 and a theme song segment determination unit 451. The continuous sound segment extraction means 450 receives a program video as input, and the continuous sound time information as the output is input to the theme song segment determination means 451. The theme song section determination means 451 receives the continuous sound time information output from the continuous sound section extraction means 450 and outputs the section designated time information.

  Next, the operation of the theme song detection means 410 shown in FIG. 5 will be described.

  The program video is first input to the continuous sound segment extraction means 450. Here, the continuity (sustainability) of the sound is analyzed from the sound signal in the video. And when a continuous acoustic section is found, the time information is output to the theme song section determination means 451 as continuous acoustic time information.

  For example, a continuous sound analysis may be performed by finding a silent section from the power of an audio signal of a program video and setting a section between the silent sections as a continuous acoustic section. At this time, it is possible to analyze the acoustic feature amount to determine the musicalness of the acoustic signal and output it as a continuous acoustic section only when it is high. For this analysis, an acoustic discriminator such as a support vector machine learned based on acoustic data can be used.

  The theme song section determination means 451 selects a time section corresponding to the theme song from the input continuous sound time information and outputs it as section designated time information. At this time, the theme song section is determined using conditions such as that the sound of the theme song portion lasts for a long time and that the theme song exists near the beginning or end of the program.

  In this way, in the theme song detection means 410 of this description, there are many theme songs and BGM portions where the sound continues continuously. In particular, the theme song portion has a long acoustic signal (several tens of seconds to several seconds). Min)) to identify the theme song part. This makes it possible to easily identify the theme song portion without actually performing detailed acoustic analysis.

(4) Specific configuration example 4 of the theme song detection means 410
Another specific configuration of the theme song detection unit 410 will be described.

  Referring to FIG. 6, an example of another specific configuration of the theme song detection unit 410 is shown, which includes a visual feature amount extraction unit 442 and a visual feature amount comparison unit 443. The visual feature quantity extraction unit 442 receives a program video as an input, and the program visual feature quantity as an output is input to the visual feature quantity collating unit 443. The visual feature amount matching unit 443 receives the visual feature amount output from the visual feature amount extraction unit 442 and outputs the section designation time information.

  Next, the operation of the theme song detection means 410 shown in FIG. 6 will be described.

  The program video is first input to the visual feature quantity extraction means 442. Here, as in the case of FIG. 4, it is assumed that the program video is input a plurality of times. The visual feature amount extraction means 442 extracts a visual feature amount for each of the plurality of program videos. The extracted visual feature amount at each time is output to the visual feature amount matching unit 443.

  The visual feature amount matching unit 443 performs matching between visual feature amounts of a plurality of input programs. At this time, collation is not performed for the entire program of each time, but between sections of an arbitrary length cut out from the program. Thereby, the section where visual feature-values correspond in each time is obtained. Of the sections obtained in this way, those having a certain section length or more are considered to be highly likely to correspond to the theme song. Therefore, the time information for designating a section that is equal to or greater than the predetermined section obtained above is output as the section designation time information. This information is output for each program.

  Similar to the case of FIG. 4, the theme song detection means 410 shown in FIG. 6 also finds a subject having the same visual pattern using a plurality of images without knowing what the theme song is. The song part can be specified.

  Note that the visual feature amount may be calculated from the entire screen, or may be extracted from only a part of the screen. In the latter case, it is possible to cope with the case where the main video is superimposed on a part of the theme song background.

  Furthermore, the collation result based on the visual feature quantity in FIG. 6 and the collation result based on the acoustic feature quantity in FIG. 4 can be combined. As a result, the theme song section can be detected with higher accuracy. In particular, the appearance order of the background video may be changed every time, but by combining the acoustic feature amounts, the theme song section can be reliably specified even in such a case. Further, even when the sound of the main part overlaps with the theme song and it is difficult to specify all sections by sound, it can be complemented by combining the results of collation by visual feature amount.

(5) Specific configuration example 5 of the theme song detection means 410
Another specific configuration of the theme song detection unit 410 will be described.

  Referring to FIG. 7, an example of another specific configuration of the theme song detection unit 410 is shown. The acoustic feature amount extraction unit 440, the acoustic feature amount comparison unit 445, the acoustic feature amount extraction unit 430, and the acoustic feature amount are illustrated. It consists of collation means 431 and a theme song acoustic feature database 435.

  The acoustic feature quantity extraction unit 440 receives a program video as an input, and the program acoustic feature quantity as an output thereof is connected to the acoustic feature quantity verification unit 445. The acoustic feature quantity matching means 445 receives the program acoustic feature quantity output from the acoustic feature quantity extraction means 440 as an input, outputs the theme song acoustic feature quantity to the theme song acoustic feature quantity database 435, and outputs section designation time information. To do. The theme song acoustic feature quantity database 435 receives the theme song acoustic feature quantity output from the acoustic feature quantity collating means 445 and outputs it to the acoustic feature quantity collating means 431. The acoustic feature quantity extraction unit 430 receives the program information as an input and outputs the output acoustic feature quantity to the acoustic feature quantity verification unit 431. The acoustic feature quantity matching means 431 receives the theme song acoustic feature quantity output from the theme song acoustic feature quantity database 435 and the acoustic feature quantity output from the acoustic feature quantity extraction means 430, and outputs a matching result.

  Next, the operation of the theme song detection means shown in FIG. 7 will be described.

  It is assumed that the program video is a video composed of a plurality of times. The operation of the acoustic feature quantity extraction means 440 is the same as that in FIG. The operation of the acoustic feature quantity matching unit 445 is similar to the operation of the acoustic feature quantity matching unit 441 in FIG. 4, but further outputs the detected theme song acoustic feature quantity to the theme song acoustic feature database 435. The theme song acoustic feature quantity database 435 stores the theme song acoustic feature quantity output from the acoustic feature quantity collating means 445 and outputs it to the acoustic feature quantity collating means 431.

  Of the program video, the remaining multiple times of video are input to the acoustic feature quantity extraction means 430. The operations of the acoustic feature quantity extraction unit 430 and the acoustic feature quantity verification unit 431 are the same as those in the case of FIG.

  This makes it possible to reliably identify the theme song portion with a smaller amount of computation than in the case where a common portion is found in a plurality of moving images as a whole. In addition, although the configuration in the case where the acoustic feature amount is used is described in FIG. 7, the theme song section can be detected in exactly the same manner when the visual feature amount or the acoustic feature amount and the visual feature amount are used.

(6) Specific configuration example 6 of the theme song detection means 410
Next, another specific configuration of the theme song detection unit 410 will be described.

  Referring to FIG. 8, there is shown another specific configuration of the theme song detection unit 410, which is a continuous acoustic segment extraction unit 450, a theme song candidate segment determination unit 452, an acoustic feature quantity extraction unit 433, and an acoustic feature quantity collation. A means 431 and a theme song acoustic feature database 432; The continuous sound section extraction means 450 receives the program video as an input, and outputs continuous sound time information as an output to the theme song candidate section determination means 452. The theme song candidate section determination means 452 receives the continuous sound time information output from the continuous sound section extraction means 450 and outputs the theme song candidate section time information as the output to the acoustic feature quantity extraction means 433. The acoustic feature quantity extraction unit 433 receives the program video and the theme song candidate section time information output from the theme song candidate section determination unit 452 as input, and outputs the output acoustic feature quantity to the acoustic feature quantity verification unit 431. The acoustic feature quantity collating means 431 receives the acoustic feature quantity output from the acoustic feature quantity extracting means 433 and the theme song acoustic feature quantity output from the theme song acoustic feature quantity database 432, and outputs section designation time information.

  Next, the operation of the theme song detection means 410 shown in FIG. 8 will be described.

  The program video is input to continuous sound segment extraction means 450. The operation of the continuous sound section extraction unit 450 is the same as that in FIG. 5, and the obtained continuous sound time information is output to the theme song candidate section determination unit 452.

  The operation of the theme song candidate section determination means 452 is basically the same as that of the theme song section determination means 451 in FIG. 5, but here it is not necessary to completely specify the theme song section, and the candidate section is selected. Since it only needs to be extracted, a criterion that is looser than in the case of FIG. 5 may be used. The obtained theme song candidate section time information is output to the acoustic feature quantity extraction means 433.

  A program video is also input to the acoustic feature amount extraction means 433, and an acoustic feature amount is extracted. However, here, the acoustic feature quantity is extracted only for the section specified by the theme song candidate section time information. The extracted acoustic feature amount is output to the acoustic feature amount matching unit 431.

  The operations of the acoustic feature quantity matching means 431 and the theme song acoustic feature quantity database 432 are the same as in the case of FIG.

  In the theme song detection means 410 shown in FIG. 8, since the acoustic feature quantity is extracted and collated only for the theme song candidate section, the processing amount can be reduced as compared with the case where the feature quantity is extracted for the entire program. Such narrowing down can also be applied to the theme song detection means 410 shown in FIGS. 4, 6, 7, etc., and the amount of processing can be reduced.

2. Detecting a credit information superimposing section by paying attention to the characteristic that telops appear continuously. Another specific configuration of the credit information superimposing section detecting means 400 will be described. The credit information superimposing section detecting means 400 described below is a specific example using the characteristic that in the content, telops on which credit information is superimposed appear continuously.

  Referring to FIG. 9, an example of an embodiment of the credit information superimposing section detecting means 400 is shown, and it comprises a continuous telop detecting means 470 and a video clipping means 420. The continuous telop detection means 470 receives a program video as input, and the section designation time information as the output is connected to the video cutout means 420. The video cutout unit 420 receives the program video and the section designation time information output from the continuous telop detection unit 470, and outputs credit information superimposed section video data.

Next, the operation of the credit information superimposition section detecting means 400 shown in FIG. 9 will be described.
The program video is first input to the continuous telop detection means 470. The continuous telop detection means 470 extracts sections where telops appear continuously. This is based on the characteristic that telops appear continuously in sections where credit information is superimposed as telops in drama and variety programs. Then, this time section is output as section designation time information.
Specifically, telop detection is performed on the program video, and if the telop is detected, the process for obtaining the start time and end time is repeated. Next, the start time and the end time are analyzed, and a time interval in which a plurality of telops appear one after another with almost no gap is obtained. Alternatively, instead of analyzing the time interval between different telops, the telop occupation area in one screen may be obtained, and the section designation time information may be obtained as a section in which the occupation area of a certain region or more is intermittently continued. The section designation time information is input to the video cutout unit 420 together with the program video. The operation of the video cutout means 420 is the same as in the case of FIG.

  As described above, the credit information superimposing section detection means shown in FIG. 9 can obtain the credit information superimposing section using only the pattern information of the telop appearance without performing complicated processing such as analysis for acoustic features. To do. This is particularly effective for programs in which credit information is displayed in a stationary telop.

3. Detecting credit information superimposition section by paying attention to the characteristic that credit information appears continuously on roll telop Referring to FIG. 10, another example of credit information superimposition section detection means 400 is shown. 480 and image cutout means 420. The roll telop detection means 480 receives the program video as input, and the section designation time information as the output is connected to the video cutout means 420. The video cutout means 420 receives the program video and the section designation time information output from the roll telop detection means 480, and outputs credit information superimposed section video data.

  Next, the operation of the credit information superimposition section detecting means 400 shown in FIG. 10 will be described.

  The program video is first input to the roll telop detection means 480. The roll telop detection unit 480 detects a roll telop that scrolls in the horizontal direction or the vertical direction, and outputs a section where the roll telop exists as section designation time information. This is based on the fact that credit information is often displayed while scrolling horizontally or vertically in a drama or a variety program.

  This type of credit is often referred to as an end roll because it is usually the end of a program. For this reason, when detecting a roll telop, time information may also be used in combination, and roll telop detection may be performed on a portion near the end of the video. Thereby, compared with the case where roll telop detection is performed on the entire program video center, the processing amount can be greatly reduced.

  As a specific roll telop detection method, motion estimation is performed between frames to search for a region in which a uniform linear motion is performed in the horizontal or vertical direction. And when this constant velocity linear motion continues for a fixed time interval, it detects as a roll telop. For motion estimation, for example, block matching or generalized Hough transform can be used.

  The obtained section designation time information is input to the video cutout means 420 together with the program video. The operation of the video cutout means 420 is the same as in the case of FIG.

  Thus, the credit information superimposition section detection means shown in FIG. 10 can detect the credit superimposition section without using the acoustic signal by detecting the roll telop. This is particularly effective when the credit information scrolls vertically or horizontally at the end of the content, such as a movie or drama. Further, even when there is no theme song such as variety and acoustic information cannot be used, a credit superimposition section can be obtained by detecting a roll telop.

4). Detection of credit information superimposed section by combination of the above-described configurations Next, another specific configuration of the theme song detection means 400 will be described. Referring to FIG. 11, there is shown another specific configuration of the credit information superimposition section detecting means 400, which includes a theme song detecting means 410, a roll telop detecting means 480, a continuous telop detecting means 470, a selecting means 481, and a video cutout. Means 420. The theme song detection means 410, the roll telop detection means 480, and the continuous telop detection means 470 all receive the program video and output the section designation time information to the selection means 481. The selection unit 481 obtains the section designation time information output from the theme song detection unit 410, the section designation time information output from the roll telop detection unit 480, and the section designation time information output from the continuous telop detection unit 470. As an input, the section designation time information is output to the video cutout means 420. The video cutout unit 420 receives the program video and the section designation time information output from the selection unit 481 and outputs credit information superimposed section video data.

  Next, the operation of the credit information superimposition section detecting means 400 shown in FIG. 11 will be described. The program video is input to the theme song detection means 410, the roll telop detection means 480, and the continuous telop detection means 470. The operations of the theme song detection means 410, the roll telop detection means 480, and the continuous telop detection means 470 are the same as those described above. The section designation time information output from these is input to the selection means 481. The selection means 481 selects and outputs a probable piece of input section designation time information. If only one of the inputs is input with the section specified time information, the section specified time information is output. On the other hand, when a plurality of section designation time information overlaps (for example, when a roll telop appears in the theme song), overlapping section designation time information is output. However, since each detection means may be able to detect only partly, the section designation time information may be obtained by taking the entire OR. The obtained section designation time information is output to the video cutout means 420.

  The operation of the video cutout means 420 is the same as that in FIG.

  The credit information superimposition section detecting means in FIG. 11 has an advantage that it can adaptively cope with various credit appearance patterns. Moreover, the detection accuracy of a credit superimposition area can be improved by using a plurality of sources.

<Specific Configuration of Credit Information Reading Unit 600>
(1) Specific configuration example 1 of the credit information reading unit 600
Next, an example of a specific configuration of the credit information reading unit 600 will be described.

  FIG. 12 is a diagram showing an example of the credit information reading unit 600, and the credit information reading unit 600 includes a telop reading unit 610. The telop reading means 610 receives credit information superimposed section video data as input, and outputs credit candidate information.

  The operation of the credit information reading means 600 in FIG. 12 will be described.

  The credit information superimposed section video data is input to the telop reading means 610. The telop reading means 610 performs telop recognition on the input video and outputs the recognition result as credit candidate information. Here, the identification rate can be improved by customizing the telop recognition for credit. For example, it is possible to use a dictionary that focuses on important words frequently used in credit information, such as “screenplay” and “theme song”. Alternatively, by registering such a specific word in advance and determining whether or not the word has appeared, the word can be extracted with higher accuracy. It also learns such specific words for multiple fonts, estimates the font when a character string appears, selects a customized telop recognition dictionary for each font, and stores other credit information You may make it read. Also, the names of people who may appear in credits are registered in the database according to their attributes (for example, information such as screenwriters, actors, etc.). By searching for and identifying the name, the identification accuracy of the person name can be dramatically improved. Furthermore, by using this personal name database, it is possible to narrow down candidates efficiently even when a part of the personal name cannot be read. There is also a certain degree of regularity in the order and pattern in which credit information appears (for example, information on screenwriters and original authors is likely to appear earlier in time than the information on performers, or is displayed alone) Therefore, the accuracy can be further improved by identifying the telop by reflecting such information. Hereinafter, such parameters used for telop recognition are referred to as telop recognition parameters.

  Since such credit information reading means 600 is configured using only the telop reading means, it can be easily configured. Further, since the credit information superimposing section video data on which the telop is superimposed is input to the credit information reading means 600, compared with the case where the telop is read for the entire program, an extra process is not performed. , Can reduce the processing. That is, the telop can be read by analyzing the portion where the telop is superimposed more efficiently and in detail than when the telop is read for the entire program. For this reason, the reading algorithm can be specialized for telop reading, and the reading accuracy of credit information can be improved.

(2) Specific configuration example 2 of the credit information reading unit 600
An example of another specific configuration of the credit information reading unit 600 will be described. This example is an example in which credit information is read by paying attention to a video in which a theme song is flowing among sounds included in content.

  Referring to FIG. 13, an example of an embodiment of the credit information reading means 600 is shown, which comprises a theme song background video generation means 620, a theme song background difference video generation means 630 and a telop reading means 640.

  The theme song background video generation means 620 receives the credit information superimposed section video data as input, and outputs the theme song background video to the theme song background difference video generation means 630. The theme song background difference video generation means 630 receives the credit information superimposed section video data and the theme song background video output from the theme song background video generation means 620, and outputs the theme song background difference video to the telop reading means 640. . The telop reading unit 640 receives the theme song background difference video output from the theme song background difference video generation unit 630 and outputs credit candidate information.

Next, the operation of the credit information reading unit 600 in FIG. 13 will be described.
First, the credit information superimposed section video data is input to the theme song background video generation means 620. Here, it is assumed that the credit information superimposed section video data includes a plurality of times of video.

  The theme song background video generation means 620 associates frames having the same background (portion other than the credit information) among a plurality of videos. Image processing is performed between the associated frames to create a background video of the theme song and output it to the theme song background difference video generation means 630. Details of the image processing will be described later.

  The theme song background difference image generation means 630 obtains a difference between the input theme song background image and the credit information superimposed section image data, and generates a theme song background difference image based on this value. Specifically, the original image is used as it is for pixels having a large difference, and the pixel value is set to 0 for pixels that are not. Thereby, the theme song background difference image in which only the credit portion remains can be generated. The theme song background difference video is output to the telop reading means 640.

  The telop reading unit 640 performs telop recognition on the input video and outputs the recognition result as credit candidate information.

  In the credit information reading means 600 of FIG. 13, the influence of the background is eliminated in the telop recognition, so that the reading accuracy can be improved.

  Here, the theme song background video generation means 620 will be described.

  Referring to FIG. 14, an example of an embodiment of the theme song background video generation means 620 is shown. The visual feature quantity extraction unit 720 receives the credit information superimposed section video data as input, and outputs the theme song background visual feature quantity to the corresponding frame calculation unit 710. Corresponding frame calculation means 710 receives the theme song background visual feature quantity output from visual feature quantity extraction means 720 and outputs frame correspondence information to background video generation means 700. The background video generation means 700 receives the credit information superimposed section video data and the frame correspondence information output from the corresponding frame calculation means 710, and outputs the theme song background video.

  Next, the operation of the theme song background video generation means 620 in FIG. 14 will be described.

  First, the credit information superimposed section video data is input to the visual feature amount extraction means 720. Here, the credit information superimposed section video data is credit information superimposed section video for a plurality of times of video. For example, in the case of a series drama, it is assumed that videos corresponding to a drama for several episodes are input together. The visual feature amount extraction means 720 extracts a visual feature amount from each video. The extracted visual feature value is output to the corresponding frame calculation means 710 as the theme song background visual feature value.

  Corresponding frame calculation means 710 performs collation between the inputted visual feature quantities each time. At this time, the collation is not performed for the entire feature amount of each time, but is performed between sections of an arbitrary length cut out from the credit information superimposed section video of each time. As a result, a section in which the video feature amount matches each time is obtained. When a section is obtained, the correspondence between video frames is also obtained. Here, there is a possibility that the association of the sections may be shifted around several frames, so a mechanism for compensating for this deviation may be added. For example, a method may be considered in which an inter-frame difference is taken between the frames before and after the correspondence and a frame having the smallest difference is selected, or a frame having the maximum number of pixels that can be matched is selected. The correspondence information between the frames obtained in this way is output to the background video generation means 700 as frame correspondence information.

  The background video generation means 700 generates the theme song background video from the input credit information superimposed section video data each time and the frame correspondence information output from the corresponding frame calculation means 710. It is generated by performing statistical processing on the pixel value at the corresponding position of each frame associated from the frame correspondence information.

によって算出する。一方、分散σ(i,j)が大きい場合には、テロップが重畳されている可能性が高いと考えられる。この場合にそのまま単純平均を出すと、テロップの影響が背景に含まれ、背景映像がうまく生成できないという問題がある。そこで、分散σ(i,j)が大きいときには、例えば、Fn,m_n(i,j) (n=1、…、N)のメディアン値をBm(i,j)とする。これにより、テロップが載っている回の方が少ない場合には、背景画像生成でのテロップの影響を排除できる。Next, details of this algorithm will be described. Here, Fn, m (i, j) is a pixel value at the position (I) of the mth frame of the nth video. Also, assume that the number of input programs is N, and the m_nth frame of the nth video is a corresponding frame. Also, the pixel value at the position (i, j) of the mth frame of the background video to be generated is represented by Bm (i, j). At this time, the value of Bm (i, j) is calculated from Fn, m_n (i, j) (n = 1,..., N).
First, in each pixel (i, j), a variance σ (i, j) of Fn, m_n (i, j) (n = 1,..., N) is obtained. If this is sufficiently small, it is considered that there is no telop at this position. Therefore, the value of Bm (i, j) can be calculated by simply averaging. That is,

Calculated by On the other hand, when the variance σ (i, j) is large, it is considered that there is a high possibility that the telop is superimposed. In this case, if a simple average is calculated as it is, the effect of the telop is included in the background, and there is a problem that the background video cannot be generated well. Therefore, when the variance σ (i, j) is large, for example, the median value of Fn, m_n (i, j) (n = 1,..., N) is set to Bm (i, j). Thereby, when the number of times the telop is placed is less, the influence of the telop in the generation of the background image can be eliminated.

  However, it is considered that there are positions (i, j) where there are more times when the telop is placed. Therefore, in each image Fn, m_n (i, j) (n = 1,..., N), an index indicating the possibility that the target pixel (i, j) is included in the telop is defined. Decrease the weight and take the weighted average. Thereby, the weight of the pixel value of the video without the telop is increased, and the influence of the telop on the background video can be reduced.

ここで、g(x)は非負の値を返すxに対する単調減少関数である。このようにして、テロップの影響が少ない背景映像を生成することが可能である。Rn,m_n(i,j)としては、例えば、位置(i,j)近傍のエッジの多さや勾配の大きさなどを用いることができる。あるいは、テロップらしいパターンを学習したニューラルネットなどの識別器を用いて、テロップらしさを判定するようにしてもよい。An index representing the telop-likeness is represented by Rn, m_n (i, j). However, Rn, m_n (i, j) has a non-negative value, and it is assumed that the larger the value, the higher the possibility of inclusion in the telop. Using this, the pixel value Bm (i, j) of the background video is calculated by the following equation.

Where g (x) is a monotonically decreasing function for x that returns a non-negative value. In this way, it is possible to generate a background video that is less affected by the telop. As Rn, m_n (i, j), for example, the number of edges near the position (i, j), the magnitude of the gradient, and the like can be used. Alternatively, the telop-likeness may be determined using a discriminator such as a neural network that has learned a telop-like pattern.

  By using the background video obtained in this way, it is possible to generate a video consisting only of a credit portion without the influence of the background, which contributes to improving the accuracy of telop recognition connected to the subsequent stage.

(3) Specific configuration example 3 of the credit information reading unit 600
Referring to FIG. 15, another example of the credit information reading unit 600 is shown. The theme song background video generation unit 620, the theme song background difference video generation unit 630, the first telop reading unit 610, and the second telop are illustrated. A reading unit 640 and a telop reading result integrating unit 650 are included. The first telop reading unit 610 receives the credit information superimposed section video data as input, and outputs the first credit candidate information to the telop reading result integration unit 650. The theme song background video generation means 620 receives the credit information superimposed section video data as input, and outputs the theme song background video to the theme song background difference video generation means 630. The theme song background difference image generation means 630 receives the credit information superimposed section image data and the theme song background image output from the theme song background image generation means 620 as input, and uses the theme song background difference image as the second telop reading means 640. Output to. The second telop reading means 640 receives the theme song background difference video output from the theme song background difference video generation means 630 and outputs the second credit candidate information to the telop reading result integration means 650. The telop reading result integration unit 650 receives the first credit candidate information output from the first telop reading unit 610 and the second credit candidate information output from the second telop reading unit 640 as input, and credit candidate information Is output.

  Next, the operation of the credit information reading unit 600 in FIG. 15 will be described. The operation of the first telop reading unit 610 is the same as that of the telop reading unit 610 of FIG. 12, and the first credit candidate information is output to the telop reading result integration unit 650. The operations of the theme song background video generation means 620 and the theme song background difference video generation means 630 are the same as those in FIG. The operation of the second telop reading unit 640 is the same as that of the telop reading unit 640 of FIG. 13, and the second credit candidate information is output to the telop reading result integration unit 650.

  The telop reading result integration unit 650 integrates the first credit candidate information and the second credit candidate information, and generates and outputs credit candidate information. There are several possible integration methods, for example, a method for outputting candidate information by combining the candidate information of both, a method for outputting candidate information with higher reliability of telop recognition, of both, There is a method of outputting all information having reliability higher than a certain standard as candidate information. Other than this, any method may be used as long as both methods are integrated to generate an output.

  The credit information reading means 600 of FIG. 15 only needs to correctly read either normal image data or image data of the theme song background difference, so that the recognition accuracy is higher than when used alone as shown in FIGS. Can be improved.

  The credit information reading unit 600 of the present description selects and merges one of the reading results obtained by the first reading unit and the reading result obtained by the second reading unit with higher reliability. The reading accuracy can be improved as compared with the case where only one is used. For example, when the same credit information is superimposed at the same position in the background theme song every time, the credit character string cannot be extracted by the background difference, so it is better to read the telop directly. On the other hand, when the telop superposition position and content are different each time, even when the background is complicated and normal telop reading is impossible, the credit information can be read by reading the background difference. In this way, the accuracy of credit reading can be improved by merging both.

<Second Embodiment>
A second embodiment of the present invention will be described in detail with reference to the drawings.

  Referring to FIG. 16, an example of the second embodiment of the present invention is shown, which includes a credit information recognition unit 100, an object recognition unit 105, and an integration unit 103.

  The credit information recognition unit 100 receives a program video as an input, and its output is connected to the integration unit 103. The object recognition unit 105 receives a program video as an input, and the output is connected to the integration unit 103. The integration unit 103 receives the output of the credit information recognition unit 100 and the output of the object recognition unit 105 as inputs, and outputs right information.

  Next, the operation of the second embodiment will be described.

  The program video is input to the credit information recognition unit 100 and the object recognition unit 101.

  The operation of the credit information recognition unit 100 is the same as that of the first embodiment or example described above, and the credit candidate information is output to the integration unit 103.

  The object recognition means 105 is a means for recognizing an object related to a right in the content, and the object is a music work or a character in the content.

  For example, when the target object is a music work, an acoustic feature amount is extracted from a program video and collated with an acoustic feature amount already registered in the database. At this time, the collation is performed by permitting the collation of only a part of the section, not the entire music work. As a result of the collation, if it is determined that the music is the same as the music in the database, music identification information for specifying the music (for example, an ID given to the music) is output. A plurality of sound sources are registered in the database even for the same music piece, and when one of them is collated, information for specifying the sound source may be included. Further, when a part of the music is collated rather than the whole, the music identification information may include information for specifying the collation section. Further, it may also include an index representing the probability of music identification. Further, only one piece of music identification information may be output for each music piece, or a plurality of candidates may be output. Also, if the extracted acoustic feature quantity does not match any acoustic feature quantity that has been verified, the music identification information may include information for specifying a video section including the acoustic feature quantity. Good. Furthermore, the acoustic signal of this section may be output together. The music identification information obtained in this way is output to the integration unit 103.

  When the target object is a character, the person feature amount of the character appearing in the video is extracted and verified. That is, a person feature is extracted from the video information and collated with a person feature already registered in the database. As a result of the collation, if it is determined that the person is the same as the person in the database, person identification information for identifying the person (for example, an ID given to the person) is output. In addition, it may also include an index representing the probability of person identification. Further, only one person identification information may be output for each character, or a plurality of candidates may be output. In addition, when the extracted person feature quantity does not match any of the person feature quantities that have been verified, the person identification information includes information that specifies the video section including the person feature quantity or the spatio-temporal position in the video. May be output. Further, the video information itself at the spatiotemporal position may be output together. The person identification information obtained in this way is output to the integration unit 103. Here, the person feature amount may be a feature amount describing a face, or a feature amount of a human voice. Or the feature-value which combined these may be sufficient, and the other feature-value which can be used for a person's identification may be sufficient.

  The integration unit 103 integrates the credit candidate information output from the credit information recognition unit 100 and the object identification information output from the object recognition unit 105, and outputs them as right information.

  As an integration method of the integration unit 103, a method of simply outputting the credit candidate information output from the credit information recognition unit 100 and the object identification information output from the object recognition unit 105 can be considered.

  As another method of integration, a method is conceivable in which credit candidate information output from the credit information recognition unit 100 and object identification information output from the object recognition unit 105 are collated, grouped and output. . At this time, priorities may be given according to the reliability. Alternatively, the one with the highest reliability may be selected, or one with a certain degree of reliability may be selected.

  As a matching method, when the object is music, a theme song / theme song is selected from the lengths of continuous sounds from among the identified songs. The title of the selected song and its attribute information (lyricist, composer, singer, performer name) are collated with the music information in the credit candidate information, and if the degree of coincidence exceeds a certain level, they are considered to be the same There is a way. Also, considering the appearance time in the program, determine whether it is a theme song (ie, determine whether it appears near the beginning or near the end of the program), and determine overlap based on the same criteria as above There is also a way to do it.

  Furthermore, when the object is a person, the name obtained as a result of person identification is simply compared with the name of the performer obtained from the credit candidate information, and the person is regarded as the same person when the degree of coincidence is above a certain level. There is. For example, if the number of characters is equal to or greater than a certain value, or if the similarity of character shapes is greater than or equal to the entire name, the characters are regarded as the same person. At this time, the similarity may be determined in consideration of the versatility of the matched characters. For example, since “Takuya” is easier to identify a person with the same two characters than “Kimura”, the latter may be determined to have a higher degree of similarity. For this determination, a method such as the TF / IDF method can be used. In addition, based on the results of person identification, the person identification information determined to be the same person is grouped to calculate or estimate the appearance frequency or appearance time, determine whether it is the leading role, the order of appearance in the credit candidate information, the appearance Considering the pattern (multiple supporting roles are displayed together, the leading role is displayed alone, or in the case of a roll telop, the leading role is displayed with an interval before and after) There is also a method for determining whether or not they are the same person.

  Using such a collation method, credit candidate information and object identification information are grouped and output as right information.

  Next, effects of the second exemplary embodiment of the present invention will be described.

  In the second embodiment, the credit information recognition means and the object recognition means are moved independently, and these results are integrated, so that information about the rights can be obtained more accurately than in the case of only credit candidate information. be able to.

<Third Embodiment>
A third embodiment will be described.

  The third embodiment is a combination of the first embodiment and the second embodiment described above, and the object recognition unit 105 uses credit candidate information from the credit information recognition unit. It is characterized by that. In the following description, examples of the music work recognizing means 101 and the character recognizing means 102 are shown as examples of the object recognizing means.

  Referring to FIG. 17, the third embodiment of the present invention includes a credit information recognition unit 100, a music work recognition unit 101, a character recognition unit 102, and an integration unit 103. The credit information recognition unit 100 receives a program video as an input, and its output is connected to the music work recognition unit 101, the character recognition unit 102, and the integration unit 103. The music work recognizing means 101 receives the program video and the output of the credit information recognizing means 100 as inputs, and the output is connected to the integrating means 103. The character recognition means 102 receives the program video and the output of the credit information recognition means 100 as inputs, and the output is connected to the integration means 103. The integration unit 103 is connected to the outputs of the credit information recognition unit 100, the music work recognition unit 101, and the character recognition unit 102, and outputs the right information.

  Next, the operation of the embodiment of FIG. 17 will be described.

  The program video is input to the credit information recognizing means 100, the music work recognizing means 101, and the character recognizing means 102. Of these three means, first, the program information is analyzed by the credit information recognition means 100.

  The credit information recognition means 100 analyzes the input program video, reads the credit information superimposed in the video, and outputs information that is a candidate for credit information.

  Here, the credit information is, as described above, a telop or sound that describes information such as the original author, screenwriter, performer, or theme song that is superimposed on the theme song or the last part of the program. The program video may be input in a compressed format such as MPEG, or may be input after being already decoded. When input as a compressed video, analysis is performed while decoding the video in the credit information recognition means. Note that the program video may be a video for a specific one-time broadcast, or may be configured to simultaneously input a plurality of times of video of the same program.

  The credit information recognizing means 100 extracts an information superimposition section in which credit information is superimposed from the program video. Next, the video included in the credit information superimposition section is analyzed, and the telop information is read from the video. Then, the result is output as credit candidate information. The credit candidate information may include a recognized character string, its time information, and position information in the image (coordinates in the frame). In addition, it may include an index representing the probability of telop recognition. Further, the credit candidate information may output one piece of information for each recognized character string, or may output a plurality of candidate character strings. The credit candidate information obtained in this way is output to the integration unit 103 and also output to the music work recognition unit 101 and the character recognition unit 102.

  Note that the credit information recognition means 100 can use any of the specific configurations described above.

  The music work recognizing means 101 analyzes the input program video and credit candidate information, and extracts and collates music information used in the video. That is, first, an acoustic feature amount is extracted from a program video, and then collated with an acoustic feature amount already registered in the database. At this time, the collation is performed by permitting the collation of only a part of the section, not the entire music work. In addition, information related to music is extracted from the credit candidate information and used for controlling a database used for collation or adjusting parameters during collation. As a result of the collation, if it is determined that the music is the same as the music in the database, music identification information for specifying the music (for example, an ID given to the music) is output. A plurality of sound sources are registered in the database even for the same music piece, and when one of them is collated, information for specifying the sound source may be included. Further, when a part of the music is collated rather than the whole, the music identification information may include information for specifying the collation section. Further, it may also include an index representing the probability of music identification. Further, only one piece of music identification information may be output for each music piece, or a plurality of candidates may be output. The music identification information obtained in this way is output to the integration unit 103.

  The character recognizing means 102 analyzes the input program video and credit information, and extracts and collates character feature amounts of the characters appearing in the video. That is, first, a person feature amount is extracted from video information, and then collated with a person feature amount already registered in the database. At this time, information related to the characters is extracted from the credit candidate information, and is used to control a database used for matching or to adjust parameters during matching. And this collation result is output as person identification information. As a result of the collation, if it is determined that the person is the same as the person in the database, person identification information for identifying the person (for example, an ID given to the person) is output. In addition, it may also include an index representing the probability of person identification. Further, only one person identification information may be output for each character, or a plurality of candidates may be output. The person identification information obtained in this way is output to the integration unit 103.

  Here, the person feature amount may be a feature amount describing a face, or a feature amount of a human voice. Or the feature-value which combined these may be sufficient, and the other feature-value which can be used for a person's identification may be sufficient.

  The integration unit 103 integrates the credit candidate information output from the credit information recognition unit 100, the music identification information output from the music work recognition unit 101, and the person identification information output from the character recognition unit 102 to obtain rights information. Output as.

  For the integration method, the method described in the integration unit 103 in the second embodiment described above may be used, and based on the relationship between the recognized character string and its position, the original object, screenwriter, performer, etc. Each type may be associated and output as right information.

  In the case of a music work, the recognized music title and music identification information are output for each song. Alternatively, when the database storing the copyright information of each musical piece can be accessed, the right information accompanying the music may be obtained from the music identification information and output. For the person, the person identification information may be output as it is, or the person name may be output together with it. Note that these rights information may not be finally narrowed down to one, but all candidates may be output, and finally, a person may confirm. This facilitates correction when the recognized right information is incorrect.

<Example of Configuration of Musical Work Recognition Unit 101>
(1) Specific configuration example 1 of the music work recognition means 101
Referring to FIG. 18, a configuration example of the music work recognizing means 101 is shown. From the music candidate extracting means 800, the candidate acoustic feature quantity selecting means 801, the music work collating means 802, and the music acoustic feature quantity database 803. Become. The music candidate extraction unit 800 receives credit candidate information as input, and outputs the music candidate information to the candidate acoustic feature quantity selection unit 801. The candidate acoustic feature quantity selection unit 801 selects a music acoustic feature quantity from the music acoustic feature quantity database 803 based on the music candidate information output from the music candidate extraction unit 800, and the candidate acoustic feature quantity is stored as a music work collating unit 802. Output to. The music work collating means 802 receives the program video and the candidate acoustic feature quantity output from the candidate acoustic feature quantity selecting means 801 and outputs music identification information.

  Next, the operation of the music work recognition means 101 in FIG. 18 will be described.

  The credit candidate information is input to the music candidate information extraction unit 800. The music candidate information extraction means 800 extracts candidate information related to music used in the program, such as a theme song and an insertion song, from the credit candidate information. For example, keywords related to music such as “theme song”, “insert song”, “theme song” are registered, and when these keywords are detected, they are displayed side by side or displayed in succession. The character string recognition result is extracted as music candidate information. The information obtained here includes the title of the song, the names of the singer and performer, the names of the lyrics and composers, and so on. Then, the obtained music candidate information is output to the candidate acoustic feature quantity selection means 801.

  The candidate acoustic feature quantity selection means 801 selects from the music acoustic feature quantity database 803 a feature quantity of a song that matches or is similar to the obtained song candidate information. Then, the selected acoustic feature amount data is output to the music work collating means 802 as a candidate acoustic feature amount.

  The music work collation means 802 first extracts an acoustic feature quantity from the program video. This acoustic feature quantity and the candidate acoustic feature quantity output from the candidate acoustic feature quantity selection means 801 are collated, and if they are collated, the identification information of the music is output. At this time, the collation is performed by permitting the collation of only a part of the section, not the entire music work.

  In this way, the music copyright recognizing means 101 in FIG. 18 extracts the copyright information regarding the music by collating the actually used music with the feature quantities even if the credit information is not completely read. Accuracy can be improved.

(2) Specific configuration example 2 of the music work recognition means 101
Referring to FIG. 19, there is shown another configuration example of the music work recognizing means 101, which includes a music related production information extracting means 820, a music work collation parameter selection means 821, a music work collation means 822, and a music work. It consists of a collation parameter database 823 and a music acoustic feature database 803. The music-related production information extraction unit 820 receives the credit candidate information as input, and outputs the music-related production information to the music work collation parameter selection unit 821. The music work collation parameter selection means 821 receives the music related production information output from the music related production information extraction means 820, selects a music copyright collation parameter from the music work collation parameter database 823, and performs music work collation. Output to means 822. The music work collation means 822 receives the program information, the music work collation parameters output from the music work collation parameter selection means 821, and the music acoustic feature quantity stored in the music acoustic feature quantity database 803, and performs music identification. Output information.

Next, the operation of the music work recognition means 101 in FIG. 19 will be described.
The credit candidate information is input to the music related production information extracting means 820. The music related production information extracting means 820 extracts music related production information from the credit candidate information. Here, the music-related production information is music-related information in program production, such as a person in charge of music, a record company that cooperated in music, a person in charge of music selection, and the like. Similarly to the music candidate information extraction unit 800 described above, keywords such as “music” and “music selection” are registered, and if these keywords are detected, are they displayed alongside them? Or the recognition result of the character string displayed continuously is extracted as music related production information. Then, the extraction result is output to the music work collation parameter selection means 821 as music production related information.

  The music work collation parameter selection means 821 selects a parameter used for collation of music works stored in the music work collation parameter database 823 in accordance with the input music-related production information. Alternatively, the music work collation parameter is controlled based on the selected information. For example, when the character string extracted as music-related production information is the name of a record company, the music work collation parameter is adjusted so that the music held by the record company is preferentially selected. Alternatively, if music acoustic feature quantities are stored in the music acoustic feature quantity database 803, grouped for each record company or divided into databases, information for designating the group or database is used as music works. Select as a verification parameter. Alternatively, when the music-related production information is the name of a person or group involved in the selection of music such as BGM, the music work collation parameter is adjusted according to the person's past music use history. It may be. The music work collation parameter selected in this way is input to the music work collation means 822.

  The operation of the music work collating means 822 is basically the same as the operation of the music work collating means 802 of FIG. The difference is that a music work collation parameter is further input from the music work collation parameter selection means 821 so that the collation parameters can be adjusted. The result of collation is output as music identification information.

  By adjusting the collation parameter by the music work recognizing means 101 of FIG. 19, it is possible to improve the recognition accuracy.

  Here, the music work collating means 822 in the music work recognizing means 101 in FIG. 19 will be described in more detail.

  Referring to FIG. 20, an example of an embodiment of a music work collating unit 822 is shown, which includes an audio superimposition determining unit 950 and an acoustic feature amount collating unit 951. The voice superimposition determination unit 950 receives the program video and outputs the voice superimposition section time information to the acoustic feature amount matching unit 951. The acoustic feature amount matching unit 951 receives the program video, the sound superimposition section time information output from the sound superimposition determination unit 950, and the music work collation parameter, and outputs the sound identification information.

  Next, the operation of the music work collating means 822 in FIG. 20 will be described.

  The program video is input to the audio superimposition determination means 950. The sound superimposition determination means 950 analyzes the acoustic signal and determines whether or not the sound is superimposed. For example, the frequency analysis of an acoustic signal is performed, and when it has the characteristic close | similar to a human voice, it determines with the audio | voice being superimposed. In addition to this, any method can be used as long as it can determine whether audio is superimposed. When it is determined that the voice is superimposed, the acoustic feature amount matching unit 951 uses the time information (section start point, end point, section time length, etc.) of the section in which the voice is superimposed as the voice superimposed section time information. Output to.

  The acoustic feature amount matching unit 951 extracts an acoustic feature amount from the input video program and compares it with the candidate acoustic feature amount. At this time, the collation method is adjusted using the voice superimposition section time information output from the voice superimposition determination unit 950. For example, a method may be considered in which collation is performed by omitting a section in which speech is superimposed, or in a section in which speech is superimposed, collation is performed by lowering the weight of the speech frequency band. In addition, a music work collation parameter is also input, and the collation method may be adjusted using this parameter. The result of collation is output as music identification information.

  The music work collating means 822 in FIG. 20 can suppress the influence even when the voice information is applied to the BGM, and can improve the recognition accuracy.

<Configuration Example of Character Recognition Unit 102>
(1) Specific configuration example 1 of the character recognition means 102
Referring to FIG. 21, an example of an embodiment of the character recognition unit 102 is shown. The performer candidate information extraction unit 900, the candidate person feature amount selection unit 901, the performer collation unit 902, and the person feature amount database 903 are shown. It consists of. The performer candidate information extraction unit 900 receives the credit candidate information as input, and outputs the performer candidate information to the candidate person feature amount selection unit 901. Candidate person feature quantity selection means 901 receives performer candidate information output from performer candidate information extraction means 900, selects candidate person feature quantities from person feature quantity database 903, and outputs them to performer verification means 902. . The performer collating unit 902 receives the program image and the candidate person feature quantity output from the candidate person feature quantity selecting unit 901, and outputs person identification information.

  Next, the operation of the character recognition unit 102 in FIG. 21 will be described.

  The credit candidate information is input to the performer candidate information extraction unit 900. The performer candidate information extraction unit 900 extracts a portion corresponding to the performer from the credit candidate information and outputs it as performer candidate information. Specifically, the person name written together with the name estimated as the cast name is extracted, the person name displayed at the same time as or following the word “appearance” is extracted and written in the credit. A method may be considered in which the names of persons who are not clearly appearing in the program, such as screenwriters and producers, are determined from the names of the persons who are present, and the names of persons excluding these names are extracted. The extracted performer candidate information is output to the candidate person feature quantity selection means 901.

  The candidate person feature quantity selection unit 901 selects a feature quantity of a person whose person names match or are close to each other from the person feature quantity database 903. At this time, it is not necessary to output one person feature amount in association with one recognized person name candidate, and the person feature amount of a person having a plurality of similar names may be output. . The selected candidate person feature amount is output to the performer verification unit 902.

  The performer collating means 902 first extracts a person feature amount from the input program video. For example, when the person feature amount is a face feature amount, face detection is performed on the video, and then the face feature amount of the detected area is calculated. Alternatively, when the person feature amount is a voice feature amount, first, a section including the voice is extracted, and then, a voice feature amount of this section is extracted. The extracted person feature amount is collated with each of the candidate person feature amounts input from the candidate person feature amount selecting unit 901. If it is determined as a result of the collation, information for identifying the person is output as person identification information.

  In the character recognition means 102 shown in FIG. 21, even if the credit information cannot be completely accurately recognized or is ambiguous with the same name and the same name, the actual character feature amount is collated to perform the performer. Allows information to be extracted correctly.

(2) Specific configuration example 2 of the character recognition means 102
Referring to FIG. 22, an example of an embodiment of the character recognition unit 102 is shown. The performer affiliation group extraction unit 920, the performer collation parameter selection unit 921, the performer collation unit 922, and the person feature amount database 903 are shown. And a person verification parameter database 923.

  The performer affiliation group extraction unit 920 receives the credit candidate information as input, and outputs the performer affiliation related information to the performer collation parameter selection unit 921. Performer collation parameter selection means 921 receives performer affiliation related information output from performer affiliation group extraction means 920, selects performer collation parameters from person collation parameter database 923, and outputs them to performer collation means 922. To do. The performer verification unit 922 receives the program video and the performer verification parameter output from the performer verification parameter selection unit 921 and the person feature amount stored in the person feature amount database 903, and outputs person identification information.

  Next, the operation of the character recognition unit 102 in FIG. 22 will be described.

  The credit candidate information is input to the performer affiliation group extraction means 920. The performer affiliation group extraction means 920 extracts information related to the affiliation of the performer, such as theatrical company name and entertainment production name, from the credit candidate information. Specifically, it can be extracted by preparing a dictionary of performer affiliation information and collating it with names registered in this dictionary. The extracted result is output to the performer collation parameter selection unit 921 as performer affiliation related information.

  The performer verification parameter selection unit 921 selects a performer verification parameter from the person verification parameter database 923. For example, when the person feature quantity is stored in the person feature quantity database 903 by grouping for each affiliated organization or by dividing the database, information for designating the group or database is used as a performer verification parameter. select. The selected performer verification parameter is output to the performer verification unit 922.

  The operation of the performer verification unit 922 is basically the same as the operation of the performer verification unit 902 of FIG. The difference is that the performer collation parameter output from the performer collation parameter selection means 921 is further input, and the collation parameter can be adjusted accordingly. The result of collation is output as person identification information.

  The character recognition means 102 in FIG. 22 makes it possible to efficiently extract who appeared in the group even if only the group name such as theatrical company name is described in the credit. .

  The effect of the 3rd Embodiment of this invention is demonstrated.

  In the present embodiment, since credit overlap sections are obtained from video and telop recognition is performed, it is possible to directly obtain important credit information as rights information. In addition, since the processing is limited to only the overlapping section, the calculation load can be reduced as compared with the case where telop recognition is performed on the entire program.

  In addition, since this credit information is used for identifying music works, the accuracy of identification can be improved compared to ordinary music identification. In addition, since credit information is used for the identification of the characters, the identification accuracy can be improved as compared with the single person identification.

  In the above description, examples of the music work recognition means 101 and the character recognition means 102 are shown as examples of the object recognition means. However, the present invention is not limited to this example. For example, FIGS. It is good also as a structure which uses only either one like this. Further, the object recognition means may be used in combination with the specific configurations described above.

This application claims the priority on the basis of Japanese Patent Application No. 2006-291442 for which it applied on October 26, 2006, and takes in those the indications of all here.

Claims (37)

  A right information extraction apparatus comprising credit information recognition means for reading credit information related to a right from content and outputting the result as credit candidate information. A rights information extraction device that extracts rights information related to rights from content,
Credit information recognition means for reading credit information about rights from content and outputting the result as credit candidate information;
Analyzing the content, recognizing an object related to the right in the content, and outputting the result as object identification information;
An rights information extraction device comprising: an integration unit that integrates the credit candidate information and the object identification information and outputs the information as rights information. A rights information extraction device that extracts rights information related to rights from content,
Credit information recognition means for reading credit information about rights from content and outputting the result as credit candidate information;
Object recognition means for referring to the credit candidate information, analyzing the content and recognizing an object related to a right in the content, and outputting the result as object identification information;
An rights information extraction apparatus comprising: an integration unit that integrates the credit candidate information and the object identification information and outputs the information as rights information. The credit information recognizing means includes a credit information section detecting means for detecting a credit information section including credit information in the content,
The right information extracting device according to any one of claims 1 to 3, further comprising credit information reading means for reading credit information from the credit information section and outputting the result as credit candidate information.   5. The credit information section detection means detects a video section in which credit information is superimposed on a video from content, and outputs credit information section video data that is video data of the video section. 5. The right information extraction device according to 5. The credit information section detecting means detects a sound section from the content, and outputs sound detecting section information as sound section information;
5. The right information extracting apparatus according to claim 4, further comprising means for outputting the content section specified by the acoustic section information as the credit information section. The acoustic detection means includes
Continuous acoustic time measuring means for measuring the duration of continuous acoustic information in the content and outputting as continuous acoustic time;
The right information extraction device according to claim 6, further comprising: a sound section determination unit that determines a sound section using the continuous sound time and outputs the sound section as sound section information. The acoustic detection means includes
An acoustic feature quantity extracting means for extracting an acoustic feature quantity for each of a plurality of times constituting the content and outputting the acoustic feature quantity;
The acoustic feature quantity is collated between the acoustic feature quantities, an acoustic feature is detected by specifying a section having a common acoustic feature quantity, and output as acoustic section information. The right information extracting device according to claim 6 or 7. The credit information section detecting means is
Continuous telop detection means for detecting video sections in which telop candidate areas continuously appear from the content and outputting the video sections as continuous telop appearance section information;
6. The right information extracting apparatus according to claim 5, further comprising means for outputting a video section of the program video specified by the continuous telop appearance section information as the credit information section video data. The credit information section detecting means is
Roll telop detection means for detecting roll telop from the content and outputting time information of the video section as roll telop section information;
6. The right information extracting apparatus according to claim 5, further comprising means for outputting a video section of the content specified by the roll telop section information as the credit information section video data.   6. The right information extracting apparatus according to claim 5, wherein the credit information reading means includes performing telop recognition on the credit information section video data and outputting the credit candidate information. The credit information reading means includes
A plurality of times of credit information section video data, and an acoustic background video generation means for generating and outputting an acoustic background video having characteristics common to the plurality of credit information section video data;
Acoustic background difference video generation means for generating and outputting an acoustic background difference video by subtracting the acoustic background video from the credit information section video data;
6. The right information extraction apparatus according to claim 5, further comprising: a telop reading unit that applies telop recognition to the sound background difference video to acquire and output the credit candidate information. The credit information reading means includes
First telop reading means for obtaining and outputting first credit candidate information by applying telop recognition to the credit information section video data;
A plurality of times of credit information section video data, and an acoustic background video generation means for generating and outputting an acoustic background video having characteristics common to the plurality of credit information section video data;
Acoustic background difference video generation means for generating and outputting an acoustic background difference video by subtracting the acoustic background video from the credit information section video data;
A telop reading unit that applies telop recognition to the acoustic background difference video to obtain and output second credit candidate information;
6. The right information according to claim 5, further comprising: a telop reading result integrating unit that integrates the first credit candidate information and the second credit candidate information to obtain and output credit candidate information. Extraction device. The acoustic background video generation means includes
A visual feature amount extracting means for extracting a visual feature amount for each time of the credit information section video data, and outputting it as an acoustic background visual feature amount;
A corresponding frame calculation means for collating visual feature amounts between the acoustic background visual feature amounts, associating video frames having a common background, and outputting as frame correspondence information;
A background image generation means for performing a statistical process of pixel values between each frame associated with the frame correspondence information to calculate a value of each pixel of the acoustic background, generating an acoustic background image, and outputting it; The right information extracting device according to claim 12 or 13, characterized in that   15. The right information extraction apparatus according to claim 14, wherein the background video generation unit uses a median as the statistical processing when a variation in pixel value between corresponding frames is large.   The background video generation means is an indicator that indicates that the pixel value corresponds to a telop area from the pixel value information in the vicinity of the pixel value as the statistical processing when the variation of the pixel value between corresponding frames is large. 15. The right information extracting apparatus according to claim 14, wherein statistical processing is performed by applying a greater weight to the smaller index. The credit information recognition means includes
17. If there is a credit candidate area that could not be read, information specifying a spatio-temporal position in a video including the credit candidate area is output together with the credit candidate information. The right information extraction device according to any one of the above.   The object recognizing means analyzes the acoustic feature amount of the content, recognizes the music work of the content based on the acoustic feature amount and the credit candidate information, and outputs the result as music identification information 5. The right information extracting device according to claim 2, wherein the right information extracting device is a recognition unit.   The target object recognition means analyzes the character feature amount of the content, recognizes the character of the content based on the character feature amount and the credit candidate information, and outputs the result as performer identification information 5. The right information extraction apparatus according to claim 2, wherein the right information extraction apparatus is a means. The music work recognizing means is:
Music candidate information extraction means for extracting candidate information of used music from the credit candidate information and outputting it as used music candidate information;
A candidate acoustic feature quantity selecting unit that selects an acoustic feature quantity of music close to the use song candidate information from an acoustic feature quantity database and outputs it as a candidate acoustic feature quantity;
A music work collating unit that collates the candidate acoustic feature quantity with the acoustic feature quantity extracted from the content and outputs the music identification information when it is determined that they match each other. Item 21. The right information extracting device according to Item 18. The music work recognizing means is:
Music-related production information extracting means for extracting information on the person, group, or record producer involved in music production from the credit candidate information and outputting it as music-related production information;
Music work collation parameter selection means for selecting collation parameters such as variables used for music collation and music database selection information according to the music-related production information;
A music work that compares the acoustic feature quantity in the music acoustic feature quantity database with the acoustic feature quantity extracted from the content using the matching parameter, and outputs the music identification information when it is determined that they match. 19. The right information extracting device according to claim 18, further comprising an object collating unit. The music work collating means is:
Analyzing the content to determine a section including sound, and outputting the time information of the section as the sound superimposition section time information;
An acoustic feature amount is extracted from the content, a voice non-superimposed section that is a section in which no voice is superimposed is detected based on the voice superimposition section time information, and the music work collation parameter is detected only in the voice non-superimposed section. The right information extraction unit according to claim 20 or 21, further comprising: an acoustic feature amount matching unit that performs matching with the candidate acoustic feature amount by using and outputs a matching result as the music identification information. apparatus. The music work collation means is:
Audio superimposition determination means for analyzing the content and determining an audio superimposition section including audio, and outputting time information of the section as audio superimposition section time information;
An acoustic feature amount is extracted from the content, a voice non-superimposed section that is a section in which no voice is superimposed is detected based on the voice superimposition section time information, and the music work collation parameter is set in the voice non-superimposed section. The candidate acoustic feature is used for collation, and in the voice superimposition section, the influence of the signal of the voice frequency band is suppressed, and the music work collation parameter is used for matching with the candidate acoustic feature, The right information extraction device according to claim 20 or 21, further comprising an acoustic feature amount matching unit that outputs a matching result as the music identification information. The music work recognizing means is:
When the acoustic feature quantity extracted from the program video does not match any acoustic feature quantity that has been tried to collate, information for identifying a video section including the acoustic feature quantity is output together with the music identification information. The right information extraction device according to any one of claims 18 to 23. The character recognition means is:
Performer candidate information extracting means for extracting candidate information of a person related to the performer from the credit candidate information and outputting it as performer candidate information;
A candidate person feature quantity selecting means for selecting a person feature quantity of a person close to the performer candidate information from a person feature quantity database and outputting as a candidate person feature quantity;
A performer collating unit that collates the candidate person feature amount with a person feature amount extracted from the content and outputs the person identification information when it is determined that the candidate feature amount is matched. 19. The right information extraction device according to 19. The character recognition means is:
Performer affiliation organization extraction means for extracting information about the affiliation organization of the performer from the credit candidate information and outputting as performer affiliation related information;
Performer collation parameter selection means for selecting collation parameters according to the performer affiliation related information;
Using the matching parameter, the person feature quantity in the person feature quantity database is collated with the person feature quantity extracted from the content, and if it is determined that they match, the performer collation that outputs the person identification information 20. The right information extracting apparatus according to claim 19, wherein the right information extracting apparatus is a means.   27. The right information extracting apparatus according to claim 19, 25 or 26, wherein the person feature amount includes at least a face feature amount of a person.   27. The right information extraction apparatus according to claim 19, 25 or 26, wherein the person feature amount includes at least a feature amount of a person's voice.   If the character feature extracted from the content does not match any of the character features that have been verified, the character recognition means identifies a video section including the character feature or a spatio-temporal position in the video. 29. The right information extracting apparatus according to claim 19, wherein the information to be output together with the person identification information is output.   A right information extraction method comprising: reading credit information related to a right from content and outputting the result as credit candidate information. A rights information extraction method for extracting rights information related to rights from content,
A process of reading credit information about rights from the content and outputting the result as credit candidate information;
Processing for analyzing the content, recognizing an object related to rights in the content, and outputting the result as object identification information;
A right information extraction method comprising: processing for integrating the credit candidate information and the target object identification information and outputting the information as right information. A rights information extraction method for extracting rights information related to rights from content,
A process of reading credit information about rights from the content and outputting the result as credit candidate information;
A process of referring to the credit candidate information, analyzing the content, recognizing an object related to a right in the content, and outputting the result as object identification information;
A right information extraction method comprising: processing for integrating the credit candidate information and the target object identification information and outputting the information as right information. The process of outputting the credit candidate information is as follows:
A process of detecting a credit information section in which credit information is superimposed in the content;
33. The right information extraction method according to claim 30, further comprising: processing for reading credit information from the credit information section and outputting the result as credit candidate information.   A program for causing an information processing apparatus to execute a process of reading credit information related to rights from content and outputting the result as credit candidate information. A process of reading credit information about rights from the content and outputting the result as credit candidate information;
Processing for analyzing the content, recognizing an object related to rights in the content, and outputting the result as object identification information;
A program for causing an information processing apparatus to execute a process of integrating the credit candidate information and the object identification information and outputting them as right information. A process of reading credit information about rights from the content and outputting the result as credit candidate information;
A process of referring to the credit candidate information, analyzing the content, recognizing an object related to a right in the content, and outputting the result as object identification information;
A program for causing an information processing apparatus to execute a process of integrating the credit candidate information and the object identification information and outputting them as right information. The process of outputting the credit candidate information is as follows:
A process of detecting a credit information section in which credit information is superimposed in the content;
38. The program according to claim 34, further comprising a process of reading credit information from the credit information section and outputting the result as credit candidate information.

Images