WO2009119063A1

WO2009119063A1 - Program information display device and program information display method

Info

Publication number: WO2009119063A1
Application number: PCT/JP2009/001274
Authority: WO
Inventors: 信裕神戸
Original assignee: パナソニック株式会社
Priority date: 2008-03-24
Filing date: 2009-03-23
Publication date: 2009-10-01
Also published as: JP2009232250A

Abstract

A program information display device that displays, with good accuracy and little noise, the names of performers in a program by associating them with the faces on the screen. Device (100) is equipped with a program acquisition unit (101) that acquires information comprising a moving-image program, and the names of performers, a related image-acquisition unit (107) that acquires related images from a network (200), a feature-value calculation unit (108) that calculates feature values for images of persons extracted from the related images, a representative feature value determination unit (109) that determines a representative feature value and reliability thereof from the feature values and the source from which the related images were obtained, a performer information management unit (104) that associates the determined values with the names of performers and manages the same, a display image acquisition unit (102) that acquires a display image, a feature value calculation unit (111) that calculates feature values for images of persons extracted from the display image, a lookup unit (105) that retrieves the name of the performer for whom the degree of similarity of the representative feature value is the greatest, a display information generation unit (106) that generates display information based on the reliability or the degree of similarity with the area of a second extracted image of a performer, and a display unit (103) that displays the display image as well as the display information.

Description

Program information display apparatus and program information display method

The present invention relates to a program information display device and a program information display method for displaying program information, for example, a program information display for displaying or reproducing a moving image, such as a television receiver, a DVD (Digital Versatile Disc) player, a hard disk recorder, etc. The present invention relates to an apparatus and a program information display method in such an apparatus.

Conventionally, when a performer of a program being viewed on a TV or the like is unknown, a request to know the name of the performer is general. As a method for obtaining information on this unknown performer, for example, it is common to obtain the name of a performer from a television column of a newspaper or EPG (Electrical Program Guide).

In such a method, the name and photo of the performer are not always posted, so it is difficult to match the face and name. Even if the user is able to know the name of the performer from the program information, the name can be matched with the face by accessing the Internet and searching for an image with the name of the performer, for example. It may become. However, it is troublesome for the user.

As a method for solving this situation, there is an information display method for extracting a face image from a moving image, searching for a pre-registered face image and unique information, and displaying the unique information in the vicinity of the face image (for example, Patent Documents). 1).

As another method, an image list and an image are periodically acquired from an external device in order to update the image feature database to be searched for images based on the image feature to the latest information. There is an image search device (for example, Patent Document 2).
JP 2006-293912 A JP 2006-185320 A

However, such a conventional program information display device has the following problems.

In the information display method described in Patent Document 1, the image feature database is fixed. In a situation where new performers appear in various programs every day, it is not realistic to hold the image feature amounts for all the performers.

By using the above-mentioned Patent Document 2 as this solution, it is possible to periodically update the image feature amount database to be searched. However, the image search device described in Patent Document 2 is a database for searching for an image feature quantity from a keyword and searching for a similar image based on the image feature quantity. ) Does not search. Further, when the image search apparatus searches the network for images acquired based on keywords, many images (noise) that are related but do not indicate a performer are also searched. However, since the search method is different, no consideration is given to noise.

Moreover, even if the above methods are combined, there is a problem that the name of the performer is not displayed until it is determined that the image feature amounts match.

The purpose of the present invention is to perform a search with less noise and high accuracy in response to a request to know a performer name in association with a face on the screen, such as when a character does not match a performer name while watching a program. It is an object to provide a program information display device and a program information display method that can be used.

The program information display device of the present invention includes a program acquisition unit that acquires a program that is a moving image and program information including a performer name, and a related image that acquires a related image from a network based on the performer name. An acquisition unit; a first feature amount calculation unit that calculates a feature amount of a first extracted image obtained by cutting a person's region from the related image; and a feature amount of the first extracted image and a source of the related image Based on the information, a representative feature amount determining unit that determines a representative feature amount and its reliability, and the determined representative feature amount and reliability are managed as performer information in association with the performer name. A performer information management unit, a display image acquisition unit that acquires a display image from a frame that constitutes the moving image, and a second feature that calculates a feature amount of a second extracted image obtained by extracting a person's region from the display image A quantity calculation unit and the second feature A similarity between the feature amount of the second extracted image calculated by the calculation unit and the representative feature amount held in the performer information management unit is calculated, and the representative feature amount that maximizes the similarity A search unit for acquiring a performer name associated with the at least one of the reliability or the similarity, the performer name acquired by the search unit, and the region of the second extracted image. A configuration is provided that includes a display information generation unit that generates display information based on the display information and a display unit that displays the display image and the display information.

The program information display method of the present invention includes a step of acquiring a program that is a moving image and a program including a performer name, a step of acquiring a related image from a network based on the performer name, and the related Based on a first feature amount calculating step for calculating a feature amount of a first extracted image obtained by cutting out a person's region from the image, and on the feature amount of the first extracted image and the information on the source of the related image, A step of determining a representative feature amount and its reliability, a step of managing the determined representative feature amount and reliability as performer information in association with the performer name, and a frame constituting the moving image A display image is obtained from the display image; a second feature value calculating step for calculating a feature value of a second extracted image obtained by cutting out a person region from the display image; and a second feature value calculating step. Calculating the degree of similarity between the feature quantity of the second extracted image and the retained representative feature quantity, and obtaining a performer name associated with the representative feature quantity having the maximum similarity Generating display information based on at least one of the reliability or the similarity, the performer name acquired by the search unit, and the region of the second extracted image, and the display Displaying an image and the display information.

According to the present invention, it is not necessary to prepare a performer's face image database in advance, and intuitively determine the correctness of the performer's determination result when displaying the performer's name in association with the performer's area. The effect that can be obtained.

In addition, when specifying a performer from the image feature value obtained by extracting a face image from a moving image, the search is performed with less noise and high accuracy by reflecting the reliability based on a database that is dynamically searched. Can be realized.

The figure which shows the structure of the program information display system provided with the program information display apparatus which concerns on Embodiment 1 of this invention. The figure which shows an example of the electronic program guide of the program information display apparatus which concerns on this Embodiment 1. The figure which shows an example of the performer information hold | maintained at the performer information holding part of the program information display apparatus which concerns on this Embodiment 1. FIG. The flowchart which shows the processing operation until the representative feature-value of the program information display apparatus which concerns on this Embodiment 1 is determined. The flowchart which shows the processing operation until the display of the performer name of the program information display apparatus which concerns on this Embodiment 1. The figure which shows the example of display information generation at the time of paying attention only to the font size of the display information of the program information display apparatus which concerns on this Embodiment 1. FIG. The figure for demonstrating the example of calculation of the representative feature-value of the program information display apparatus which concerns on this Embodiment 1. FIG. The figure explaining the display area and display image in case the display part of the program information display apparatus which concerns on this Embodiment 1 is a display part of a television. The figure which shows the example of a display of the search result displayed by the display part of the program information display apparatus which concerns on this Embodiment 1. FIG. The figure which shows the structure of the program information display system provided with the program information display apparatus which concerns on Embodiment 2 of this invention. The flowchart which shows the process until the display of the performer name of the program information display apparatus which concerns on this Embodiment 2. The figure which shows the structure of the program information display system provided with the program information display apparatus which concerns on Embodiment 3 of this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(Embodiment 1)
FIG. 1 is a diagram showing a configuration of a program information display system provided with a program information display device according to Embodiment 1 of the present invention. The program information display apparatus according to this embodiment is an example in which the present invention is applied to a television receiver capable of receiving digital broadcast radio waves.

1, the program information display system includes a program information display device 100, a network 200, an image search device 210, an image server 220, a program information server 230, a program guide server 240, and a broadcast station 250.

The program information display device 100 is a digital broadcast receiver that reproduces program information transmitted from the broadcast station 250. The detailed configuration of the program information display device 100 will be described later.

The network 200 is a communication network composed of the Internet or a dedicated line. More specifically, the network 200 is a network composed of a mobile communication network, a public telephone network, a LAN, the Internet, or the like. The network 200 may be either a wired system or a wireless system, and the type of protocol is not particularly limited. Further, as an access line of the network 200, a large capacity line such as FTTH (Fiber To The Home), HFC (Hybrid Fiber Coax), and ADSL (Asymmetric Digital Subscriber Line) can be used.

The image search device 210 is an image search site. More specifically, the image search device 210 is an image search site that collects and holds images held by various sites, their description information, and URLs (Uniform Resource Locator) of images via the network 200. The sites to be searched by the image search device 210 include a plurality of sites including the image server 220 and the program information server 230. The image search device 210 searches for information on the collected image based on the keyword, and outputs the search result.

The information about the image includes the image itself and the URL of the image. In addition, the image server 220 is included in the collection destination of the information related to the image of the image search apparatus 210. However, since the information posted on the image server 220 may not be appropriate, the image search device 210 attempts to improve search accuracy by devising a search algorithm.

The image server 220 is a general site that publishes web pages to be posted. More specifically, the image server 220 is a site that is managed by an unspecified number of general users or companies, and holds some images and publishes them to the network 200. Public sites do not always represent information properly. However, there are far more public sites than, for example, an official site of a specific broadcast program, and the network 200 as a whole has a huge amount of information.

The program information server 230 is an official site for programs broadcast by the broadcast station 250 (hereinafter referred to as “programs” as appropriate). More specifically, the program information server 230 is an official program site, and is generally managed by a television station or a program production company that is the right holder of the program. Therefore, it can be said that the official site is one of the sites that appropriately express the contents of the program. However, depending on the program, the official website may not exist, and even if it exists, the level of content fulfillment varies. Here, the official site is described as a program site, but the official site of a program performer can also be handled in the same manner because it is a site that appropriately expresses information about the performer.

The program guide server 240 is a site that publishes a program guide of programs of each station including the broadcast station 250. More specifically, the program guide server 240 is a site that provides, via the network 200, a net program guide that is a list of detailed program information of the corresponding program based on the date and time or the region. The detailed program information includes a program name, a performer name, a URL of an official site, and the like.

The

servers

220, 230, and 240 are connected via a network 200 to a control unit that includes a computer that controls the entire server. The

servers

220, 230, and 240 are each composed of, for example, a communication interface that transmits and receives data accessed by a URL on the Internet, and a database (DB) that stores data.

The program information display device 100 has the following configuration.

The program information display apparatus 100 includes a program acquisition unit 101, a display image acquisition unit 102, a display unit 103, a performer information management unit 104, a search unit 105, a display information generation unit 106, a related image acquisition unit 107, and a feature amount calculation unit 108. (First feature value calculating unit), representative feature value determining unit 109, performer information holding unit 110, and feature value calculating unit 111 (second feature value calculating unit). Although not shown, the program information display device 100 includes an interface for connecting various devices such as an input device that accepts user input operations and an external recording device that records information. The input device is a keyboard or a remote control device including numeric keys, cross keys, and the like.

The program acquisition unit 101 is a tuner that receives broadcasts of programs and program information and decodes the programs and program information. The program consists of moving images. Program information is, for example, various types of information related to programs included in an electronic program guide such as EPG. The electronic program guide is composed of a program name, a channel, a broadcast date, a broadcast start time and end time, a performer name, and the like. Hereinafter, it is assumed that the program acquisition unit 101 extracts an electronic program guide from a broadcast signal and acquires program information from the extracted electronic program guide.

The program acquisition unit 101 has a memory (not shown) inside, and stores the acquired program information in this memory. For example, when the program acquisition unit 101 acquires program information from EPG or the like, the name described as a performer in the electronic program guide is a target to be acquired as the performer name of the program. It is not subject to acquisition as a person's name.

FIG. 2 is a diagram showing an example of an electronic program guide. As shown in FIG. 2, this electronic program guide has set values for program names, synopsis, performers, and URL field names. In the example of FIG. 2, “Good Morning Tokyo” is described as the program name, “Takeshi (Taro Matsuyama), Hiroko (Hanako Takeda), Yuri (Ryoko Umekawa)” as the performers, and the official website of the program as the URL The address is listed.

The display image acquisition unit 102 extracts a frame constituting a moving image to be displayed from the program acquired by the program acquisition unit 101, and an image of the program to be displayed on the TV screen (hereinafter referred to as “display image”) from the extracted frame. To get.

The display unit 103 is a display that displays a moving image (display image) in units of frames extracted by the display image acquisition unit 102. The display unit 103 displays a display image and display information at the same time.

The performer information management unit 104 extracts information such as the performer name and the URL of the official site from the program information stored in the program acquisition unit 101. The performer information management unit 104 stores the representative feature amount and reliability related to the related image obtained from the representative feature amount determination unit 109 in the performer information holding unit 110 in association with the performer name.

Here, the related image is an image related to the performer of the program, and is an image that is likely to include the face image of the performer. The representative feature amount is a value representing the image feature amount of a face image extracted from a plurality of related images related to the same performer (hereinafter referred to as “related extracted image”). This is an image feature amount that is highly likely to match the image feature amount. The degree of reliability is the degree of reliability that the representative feature amount matches the image feature amount of the actual face image of the performer.

The search unit 105 searches for a performer name related to a face image similar to the face image extracted from the display image (hereinafter referred to as “display extracted image”) from the related extracted images, and temporarily stores the search result. To do. Specifically, the search unit 105 uses the feature amount calculation unit 111 to obtain a region of the display extracted image and an image feature amount (hereinafter referred to as “extracted image feature amount”). And the search part 105 acquires the representative feature-value of a related extraction image by the performer information management part 104. FIG. Then, the search unit 105 calculates the similarity between the extracted image feature quantity and the representative feature quantity, and acquires the performer name associated with the representative feature quantity having the maximum similarity. That is, the search part 105 acquires the performer name corresponding to the related extraction image similar to a display extraction image.

The display information generation unit 106 generates display information to be displayed on the display unit 103 from at least one of the reliability or similarity of the representative feature amount, the performer name acquired by the search unit 105, and the display extracted image area. . An example of display information generation by the display information generation unit 106 will be described later with reference to FIG.

The related image acquisition unit 107 acquires a related image by communicating with the network 200. The related image acquisition unit 107 is connected to the network 200 and acquires one or more related images related to the performer based on the program information.

Feature

amount calculation units

108 and 111 calculate the image feature amount of an extracted image obtained by cutting out a human face area from an image. The algorithm for calculating the image feature amount is the same between the feature amount calculation unit 108 and the feature amount calculation unit 111, but the processing target is different. The feature amount calculation unit 108 calculates an image feature amount for a related extracted image obtained by cutting out a human face area from a related image acquired from the network 200. The feature amount calculation unit 111 calculates an image feature amount for a display extracted image obtained by cutting out a human face area from a display image displayed on the TV screen. Here, when a plurality of persons are shown in the related image or the display image, the feature

amount calculation units

108 and 111 calculate the image feature amount for each of the plurality of persons.

The representative feature amount determination unit 109 compares the weighted value of the image feature amount of the related extracted image to determine the representative feature amount and the reliability. Specifically, the representative feature amount determination unit 109 performs weighting on the image feature amount of the related extracted image (face region). As a weighting method, for example, when the source of the base related image is an official site, there is a method of calculating the representative feature amount in a state where the number of the related extracted images is increased. According to this method, the number of image feature quantities of the relevant extracted image is larger than the number of image feature quantities of the other related extracted images, so that the image feature quantities of the related extracted images can be weighted. . Further, as a representative feature amount determination method, a method using an arithmetic average or a method using a value that maximizes the number of all the image feature amounts of the relevant extracted image can be considered.

The performer information holding unit 110 is configured by a hard disk or a memory, and the representative feature amount determined by the representative feature determination unit 109 in association with the performer name extracted from the program information by the performer information management unit 104 and its performer Maintain confidence. Hereinafter, the performer name, the representative feature quantity, and the reliability are collectively referred to as “performer information” as appropriate.

FIG. 3 is a diagram illustrating an example of performer information held in the performer information holding unit 110. The performer names shown in FIG. 3 are representative feature amounts of the performer names (titles) “Taro Matsuyama, Hanako Takeda (Hiroko), Ryoko Umekawa (Yuri)” of the performers in the electronic program guide of FIG. And its reliability. In the case of FIG. 3, the reliability of the representative feature amount of the performer name (title) “Hanako Takeda” is higher than the reliability of the representative feature amounts of the other two people.

The performer information management unit 104, the search unit 105, the display information generation unit 106, the feature amount calculation unit 108, the representative feature amount determination unit 109, and the feature amount calculation unit 111 include a CPU and the like, and execute program information display processing. Control of the entire device including The CPU includes a ROM, a RAM, and an EEPROM (electrically erasable programmable) ROM, which is an electrically rewritable nonvolatile memory, or a flash ROM. The memory stores various data such as programs, communication control data, and terminal identification codes. The memory stores the performer information of the performer information holding unit 110.

Hereinafter, the operation of the program information display apparatus 100 configured as described above will be described.

4 and 5 are flowcharts showing the operation of the program information display apparatus 100. FIG. FIG. 4 shows processing until the representative feature amount is determined. FIG. 5 shows processing until the name of the performer is displayed after the representative feature amount is determined. The flow shown in FIG. 4 is basically executed while the user is viewing the corresponding program. The flow shown in FIG. 5 is basically executed with a user instruction as a trigger. As described above, since the execution conditions are different, the description will be divided into two flows. The flow shown in FIGS. 4 and 5 is executed as a program information display program by the CPU as described above. In the figure, the symbol “S” indicates each step of the flow.

First, processing until the representative feature amount determination unit 109 determines a representative feature amount will be described with reference to the flowchart of FIG.

In step S11, the program acquisition unit 101 acquires program information and a moving image of a program to be displayed. The easiest way to acquire the program information is to acquire an electronic program guide (EPG) and acquire it from the acquired electronic program guide. However, it is possible to recognize character strings, audio, or closed captions in moving images. The method of performing and acquiring from the recognition result may be employ | adopted. Further, the electronic program guide may include the URL of the official site of the performer on the Internet and the URL of the official site of the program. An example of the electronic program guide is shown in FIG.

In step S12, the performer information management unit 104 repeats the process between the start and end of the following loop for “all performer names”. The performer information management unit 104 extracts a performer name from the performer field of the electronic program guide based on the program information acquired by the program acquisition unit 101, and for all the performer names obtained thereafter, Process.

In step S13, the performer information management unit 104 checks whether or not the performer name is held in the performer information holding unit 110. However, it is assumed that no performer name is registered in the performer information holding unit 110 at the start of the process.

In step S14, when the performer name is held in the performer information holding unit 110, the performer information management unit 104 moves the process to the next performer. On the other hand, if the performer name is not held in the performer information holding unit 110, the performer management information unit 104 proceeds to step S15.

In step S15, the related image acquisition unit 107 acquires a related image based on the performer name. The related image acquisition unit 107 searches for related images in the network 200 including the Internet using the performer name as a keyword. Then, the related image acquisition unit 107 acquires and holds, as a search result, information such as the related image related to the performer name or the URL of the acquisition destination from the image search device 210, for example.

In step S16, the feature amount calculation unit 108 repeats the process between the start and end of the following loop for “all related images”.

In step S 17, the feature amount calculation unit 108 cuts out a person's face area as a related extracted image from all the related images acquired for each performer by the related image acquisition unit 107, and sets the image feature amount for each extracted related extracted image. Ask for. Then, the feature amount calculation unit 108 temporarily stores the obtained image feature amount in association with the performer name corresponding to the base related image. As a method of cutting out a person's face area, a method of extracting an edge from color or luminance information and performing a pattern recognition process of a face outline and a face part using the extracted edge is used. The image feature amount is a value obtained by digitizing the color tone, the arrangement position of the face parts, and the like according to a predetermined reference.

In step S18, the representative feature amount determination unit 109 determines a representative feature amount and reliability for each performer name from the image feature amounts of the related extracted images obtained by the feature amount calculation unit 108.

And in step S19, the performer information management part 104 hold | maintains the performer name, representative feature-value, and reliability in the performer information holding | maintenance part 110 in the linked | related state. An example of information held in the performer information holding unit 110 is shown in FIG. The image feature amount differs depending on the algorithm, accuracy, and storage format of the feature amount calculation unit 108, and this example does not indicate a general expression of the image feature amount.

Next, processing of the representative feature amount determination unit 109 will be described.

Prior to the processing of the representative feature amount determination unit 109, the feature amount calculation unit 108 performs noise removal processing on the image feature amount obtained from the related extracted image. Here, the noise is a characteristic value that greatly differs among the feature values constituting the image feature amount associated with the same performer name, that is, the feature value of the image feature amount of the performer's face image. Points that are not likely. Or noise refers to an image from which such a characteristic value is acquired, that is, an image that is not likely to be a face image of the performer.

The program information display device 100 searches for related images from the image search device 210 via the network 200 using the performer name as a keyword. The program information display device 100 extracts related images from the program information server 230 or the image server 220 according to the search algorithm of the image search device 210. Such an image search is generally performed by a method of searching for an image related by a keyword. In addition, among the images managed by the image server 220, there are images that are not face images of performers only by matching the keywords. Therefore, the search result may include an image that is not a face image related to the performer name.

The image feature amount is obtained according to the algorithm of the feature amount calculation unit 108. The image feature amount represents, for example, the arrangement position of the face part by a combination of a plurality of feature values (here, the feature value is 0 or more). For each performer, the feature amount calculation unit 108 obtains a statistical distribution of feature values of a plurality of related images related to the performer, and obtains a representative feature amount from the distribution. Specifically, the feature amount calculation unit 108 identifies feature values that are within a predetermined threshold from the average of all of the plurality of feature values, recalculates the average of the specified feature values, and displays the recalculation result as a representative feature. Adopt to quantity. Thereby, the feature amount calculation unit 108 excludes feature values (noise) whose values are greatly different from the average. A specific example of the feature amount calculation by the feature amount calculation unit 108 will be described later with reference to FIG.

By such noise removal processing, the feature amount calculation unit 108 excludes all image feature amounts obtained from images other than the target person's face screen as noise. When an image search is performed using a so-called search engine, an irrelevant image is searched, or an image in which a person other than the person is shown is often searched. Since it is considered that the accuracy increases as the number of searches increases and the number of feature amount calculation processes increases, it is possible to increase the accuracy by switching a plurality of search engines and repeating the search several times. For example, while the program is being received after the program has started, the related image search process may be constantly performed by background processing. If there is a program reservation, it is possible to execute the flow of FIG. 4 in advance prior to the start of the program. In this way, more related images can be searched, and the reliability of the representative feature amount can be increased.

Here, the performer information management unit 104 compares the URL of the program official site held by the program acquisition unit 101 with the URL of the related image acquired by the related image acquisition unit 107, and if they are the same domain, , It is determined that the acquired related image is of the official website. Then, the representative feature amount determination unit 109 weights the feature amount of the related image to reflect the high reliability of the related image in the calculated representative feature amount. For example, the reliability weighting is calculated as follows.

Here, it is assumed that the image feature amount F (x) for each performer calculated by the feature amount calculation unit 108 is expressed by the following equation (1) including m feature values. Here, the variable x represents a performer. At this time, the representative feature amount F _t is obtained by the following equation (2), where n is the number of related images after noise elimination (number of image feature amounts) and s _i is a coefficient indicating whether or not the site is an official site. .

Further, the reliability vector C _v is defined as following equation (3), defined as the following equation (4) the reliability C.

That is, the reliability vector C _v is the characteristic value of the group, a vector value that indicates how much held together (or not vary) among a plurality of related images, reliability C is a scalar of the vector value This is a value obtained by normalizing the quantity by the number of feature values. As the ratio of the correct image (the target performer's face image) in the plurality of related extracted images for which the representative feature amount F _t is calculated becomes higher, the variation in the feature value becomes smaller and the reliability C becomes higher. The value gets bigger. Therefore, the reliability C indicates the high reliability that the representative feature amount F _t matches the image feature amount of the actual face image of the performer.

The coefficient s _{i for} determining whether or not the site is an official site is set to 2 for an image of an official site, for example, and 1 for an image from another site.

Next, a specific example of feature amount calculation performed by the feature amount calculation unit 108 will be described.

The image feature amount is a combination of m feature values as shown in the above equation (1). Individual image feature amounts can be obtained from color information and position information of each part of the face. For example, the feature amount calculation unit 108 calculates a hue or the like based on the color information, and uses the calculation result as a feature value. Alternatively, the feature amount calculation unit 108 calculates the aspect ratio or relative position of each part of the face, and uses the calculation result as a feature value. The position information of the facial parts can be obtained, for example, by extracting the contour in the image from the brightness, hue, etc. obtained from the color information.

Hereinafter, a specific example of a representative feature amount calculation method will be described. For simplicity, m = 3.

FIG. 7 shows an example of the feature amount calculated by the feature amount calculation unit 108. In the case of FIG. 7, the image feature amount has three feature values 1 to 3 for each feature amount number. An example of calculating a representative feature amount will be described using this feature amount. Here, the feature quantity number is an identification number of each feature quantity of the related images acquired corresponding to the same performer.

The feature amount calculation unit 108 calculates an arithmetic mean and variance in order to remove noise and to calculate reliability. Variance means the degree of variation of feature values. The feature amount calculation unit 108 calculates the above-described threshold value based on the variance, and removes noise by excluding the feature value exceeding the calculated threshold value from the representative feature amount calculation target. In addition, the aspect which makes the value of dispersion | distribution a reliability may be sufficient.

FIG. 7 shows an example where n = 5. In this case, the feature amount calculation unit 108 obtains the arithmetic mean and variance using the five related images. As described above, the related image is an image searched based on a certain keyword (performer name). Therefore, a plurality of related images are searched for one keyword (performer name). The feature amount calculation unit 108 obtains a representative feature amount for the keyword (performer name) using the plurality of related images.

Here, the related image searched by the keyword (performer name) includes an unrelated image (noise). In some cases, the retrieved related image may not be an image of a target person. The average value obtained with noise included is unlikely to be a reasonable value. Therefore, the feature amount calculation unit 108 recalculates the average value after removing the noise.

In order to obtain a representative feature amount from a plurality of related images, the feature amount calculation unit 108 performs the following processes (a) to (e). (A) A temporary average value is obtained from a population including noise. (B) The variance is obtained from the provisional average value. (C) Remove noise based on dispersion. (D) The average of the population excluding noise is obtained. This is recalculation. (E) The average obtained is used as the representative feature amount.

First, the feature quantity calculation unit 108 obtains an arithmetic average of five feature values corresponding to the feature quantity numbers 1 to 5 for each of the feature values 1 to 3 in FIG. At that time, the feature amount calculation unit 108 weights the feature amount of the official site by a predetermined multiple (for example, five times) over the other feature amounts. In the case of FIG. 7, the feature amounts 1 to 3 with the feature amount number 1 are the feature amounts acquired from the official site, and thus are weighted five times as much as the feature amounts 1 to 3 with the feature amount numbers 2 to 5, respectively. . As a result, the number of populations has increased by four.

The feature value calculation method will be described by taking the case of the feature value 1 among the feature values 1 to 3 shown in FIG. 7 as an example.

The arithmetic average f _1m of the feature value 1 is expressed as the following equation (5). The variance σ ² is expressed as the following equation (6).

f _1m = ((2 ^* 5) + 3 + 2 + 8 + 1) / (5 + 4) ≈2.67 (5)
σ ² = (2 ^* 2 ^* 5 + 3 ^* 3 + 2 ^* 2 + 8 ^* 8 + 1 ^* 1) / (5 + 4) − (f _1m ) ²
≒ 10.89-7.13 = 3.76 (6)

When it is assumed that the feature value 1 has a normal distribution and the threshold value th is previously defined as ± standard deviation σ, the threshold value th is expressed by the following equation (7) in the arithmetic mean f _1m of the feature value 1.

Th ≒ ± √ (3.76) = ± 1.94 (7)

That is, the values f _{1e that} can be originally taken by the feature value 1 are expressed by the following equations (8) and (9).

(F _1m −th) ≦ f _1e ≦ (f _1m + th) (8)
0.73 ≦ f _1e ≦ 4.61 (9)

From the above equation (9), the feature value 1 of the feature number 4 shown in FIG. 7 does not correspond to the value f _1e that the feature value 1 can originally take, and can be regarded as noise. For example, the feature amount calculation unit 108 recalculates f _{1m according} to the following equation (10), excluding the data of feature amount number 4 and considering weighting. However, noise removal by excluding other feature values is ignored.

f _1m = ((2 ^* 5) + 3 + 2 + 1) / (4 + 4) = 2.00 (10)

As described above, the value of the feature value 1 of the representative feature amount can be set to “2” in FIG. Similarly, for the feature values 2 and 3 which are the remaining feature values, a new average can be obtained by excluding the feature amount that becomes noise, and the representative feature amount can be calculated.

The representative feature amount determination unit 109 determines that the reliability is low when the variation of the image feature amount used to calculate the representative feature amount is large for the same keyword, and the reliability is high when the variation is small. .

The representative feature quantity determination unit 109 may obtain the reliability C by another method in order to simplify the calculation. For example, the representative feature amount determination unit 109 sets the reliability C as the following equation (11), where n is the number of feature amounts excluding noise, a is the number of images before noise removal, and N is the number of images on the official site. Ask for. Similarly, in this case, the reliability is low when the variation in the feature amount is large, and the reliability is high when the variation in the feature amount is small.

The representative feature quantity determination unit 109 holds the performer name, the representative feature quantity F _t , and the reliability C thus obtained in the performer information holding unit 110 in an associated state. The representative feature amount determination unit 109 may store the role name in the performer information holding unit 110 when the role name is obtained in addition to the performer name as the program information. When an image search is performed via a network, many related images may be searched. Therefore, the representative feature quantity determining unit 109 may perform the processing until the representative feature quantity F _t is determined asynchronously step by step during program display. That is, while increasing the number of related images used for calculating the representative feature quantity F _t in stages, the representative feature quantity F _t is calculated for each stage, and the later calculated representative feature quantity F _t is used as a collation process described later. Used for. Thereby, the collation process can be started from the early stage of the program viewing start, and the reliability of the representative feature quantity F _t can be increased with the passage of time.

The representative feature amount determination unit 109 processes the next performer name when several related images are processed for the performer name, processes all performer names, and then again performs each performer name. Is processed with several other related images. By performing the processing step by step in this way, the collation processing can be started from the early stage of the program viewing start, and the reliability of the representative feature amount F _t can be increased with the passage of time. Become.

Next, processing up to the display of the performer name will be described using the flowchart of FIG. When the user wants to know the performer's person while viewing the program, the user performs a predetermined user instruction operation on the program information display device. The flow in FIG. 5 is executed with this user instruction as a trigger.

In step S20, the display image acquisition unit 102 acquires a display image in units of frames from the moving image acquired from the program acquisition unit 101, and holds the acquired display image.

In step S21, the display unit 103 displays the acquired display image as a moving image as it is.

In step S 22, the search unit 105 cuts out the face area of the person from the frame-by-frame display image acquired by the display image acquisition unit 102, outputs the cut out area as a display extracted image, and extracts the display extracted image ( (Hereinafter simply referred to as [position]). The feature amount calculation unit 111 calculates an extracted image feature amount from the display extraction image. When a plurality of persons are shown in the display image, a plurality of display extraction images are obtained. The feature amount calculation unit 111 has the same function as the feature amount calculation unit 108 that has already been described in detail. However, while the feature amount calculation unit 108 calculates the feature amount of the related extracted image, the feature amount calculation unit 111 calculates the feature amount of the display extracted image.

In step S23, the search unit 105 repeats the process between the start and end of the following loop for “all related images”.

In step S24, the search unit 105 acquires the representative feature amount from the performer information holding unit 110 via the performer information management unit 104 until there is no more representative feature amount that has been verified in the performer information holding unit 110.

In step S25, the search unit 105 collates the extracted image feature quantity with the representative feature quantity, and calculates the similarity. If the representative feature amount is F _t , the extracted image feature amount is F _e , and the maximum value that can be taken by the feature amount is F _max (however, the maximum value of each feature value is greater than 0), the similarity S is For example, it can obtain | require by following Formula (12).

Here, the search unit 105 may acquire the representative feature amount _Ft and the corresponding reliability C, and may not collate the representative feature amount having a reliability lower than a preset threshold value. In this case, the processing can be speeded up.

In step S26, the search unit 105 extracts the maximum similarity among the calculated similarities, and specifies a representative feature amount corresponding to the extracted similarity.

In step S27, the search unit 105 holds the performer name associated with the corresponding representative feature amount as a search result. The search unit 105 matches the extracted image feature amount calculated by the feature amount calculation unit 111 with the representative feature amount related to the related image obtained by the representative feature amount determination unit 109 in the flow of FIG. Specifically, the search unit searches the performer information holding unit 110 via the performer information management unit 104 and extracts the performer name associated with the representative feature amount specified in step S26.

In step S28, the display information generation unit 106 generates display information for displaying the search accuracy of the performer name simultaneously with the performer name from the performer name extracted by the search unit 105 and the display extracted image. Specifically, the display information generation unit 106 displays at least one of the display content determined from the performer name, the display position determined from the position of the display extracted image, and the reliability or similarity of the representative feature amount. From the matching result between the extracted image feature quantity and the representative feature quantity, the search accuracy of the performer name is calculated. Then, the display information generation unit 106 generates display information based on this search accuracy. At this time, the display information generation unit 106 basically generates display information for displaying the display contents with high search accuracy more prominently.

In step S29, the display unit 103 displays the search result by displaying the display information generated by the display information generation unit 106.

The program display device 100 ends the series of processes when the processes up to the display of the names of the performers described above are performed for “all related images”.

As described above, by using the program information display system, the user can know the information of the person displayed on the display screen when he / she wants to know.

Next, a specific example of display information generation processing by the display information generation unit 106 will be described.

The display information generation unit 106 determines the display content from the performer name extracted by the search unit 105. The display content is, for example, a description of a performer name, a role name, and a role.

Then, the display information generation unit 106 determines the display position of the display content from the determined display content and the position of the display extracted image acquired by the search unit 105. For example, the display information generation unit 106 confirms whether the display content can be arranged in the order of upper, right, lower, and left in the display extracted image area, and determines the position where the display content can be arranged as the display position of the display content. “Placeable” means that the display content does not exceed the range of the display screen, does not overlap with other display content whose display position has already been determined, and is close to other display content whose display position has already been determined. It means that the conditions such as not being met.

The display information generation unit 106 determines the display form according to the accuracy of the searched display content. The display form is a form in which display contents such as a display position, display contents, font type, font size, character color, background color, presence / absence of a border line, or border line color are displayed as a character string or the like. The accuracy of the display content corresponds to at least one of the reliability of the representative feature quantity used by the search unit 105 when collating and the similarity calculated when the extracted image feature quantity and the representative feature quantity are collated. It is a fixed index. When calculating using both the reliability C and the similarity S, the accuracy A used for determining the display form may be a value obtained by multiplying both values as shown in the following equation (13). Alternatively, the coefficients α and β may be used as a value obtained by summing the weighted values of the reliability C and the similarity S as shown in the following equation (14).

For example, the accuracy A can be reflected in the font size Fs of the performer name using the above formula (14). In this case, the font size Fs of the performer name can be obtained by the following equation (15), where Fm is the maximum standard font size.

When obtaining the accuracy A using the above equation (15), the degree of reflection of the reliability and similarity on the display information can be changed by changing the values of the coefficients α and β.

FIG. 6 is a table showing an example of display information generation in the above formula (15) when attention is paid only to the font size in the display form.

As shown in FIG. 6, when the calculation standard of the accuracy A is “based on reliability”, the coefficient α = 1 and the coefficient β = 0, and the font size Fs = C ^* Fm. When the calculation criterion is “based on similarity”, the coefficient α = 0 and the coefficient β = 1, and the font size Fs = S ^* Fm.

In addition, when the above equation (13) is used, it is possible to calculate the font size Fs in a similar manner by reflecting only the other value by setting one of the reliability C and the similarity S to a fixed value. it can.

In the above, the display information generation example in which the display information generation unit 106 generates display information using at least one of the reliability C and the similarity S of the representative feature amount has been described.

The display information generation unit 106 calculates the reliability or the similarity based on the performer name acquired by the search unit 105 and the display extracted image. Therefore, which region is cut out as a display extraction image has an influence when calculating reliability and similarity. When the region of the display extraction image is inappropriate (for example, when only a part of the face is set as the region), the reliability and similarity may not be calculated correctly.

FIG. 8 is a diagram for explaining a display area and a display image when the display unit 103 is a television display unit. In FIG. 8, a display area 140 is a television screen (display screen), and a display image 141 is a video of a program. The display image 141 is displayed in the display area 140. In the display image 141, a person and a performer name 142 are displayed. A region surrounding the human face in the display image 141 is a region cut out as the display extraction image 143. FIG. 8 shows a case where the display image (program video) 141 is not displayed in the full display area (television screen) 140. In this case, the display screen can take a form in which the name of the performer is displayed around the display image 141. An example of the case where the display image 141 is not displayed in the full display area 140 is a case where a broadcast video is reduced and displayed, such as a screen using a picture-in-picture function or a data broadcast display function in digital terrestrial broadcasting. .

The change in the display form of the performer name includes, for example, a change in transparency.

For example, when the accuracy of reliability and similarity is low, that is, when it is determined that there is a high possibility that the search result is not correct, the character name of the performer is increased and the performer name is increased. Make it difficult to see. On the other hand, when the accuracy is high, that is, when it is determined that there is a high possibility that the search result is correct, the character name transparency is lowered to make the performer name clearly visible.

Other changes in the display mode include increasing the character size when the accuracy is high and changing the color of the character according to the difference in accuracy. In the latter case, it is desirable to display a table that associates the color of the character with the accuracy that the color means together with the name of the performer. Further, as another change in the display form, the contrast of the character displaying the performer name with respect to the background color may be changed according to the accuracy.

Also, by determining the display position in order from the name of the performer with high accuracy, the display priority can be increased with respect to the name of the performer with high accuracy. If the accuracy is low, the display position of the performer name and the position of the image of the person to be displayed may be separated, and a lead line connecting the person image and the performer name may be displayed. In this case, if the accuracy is low, the lead line becomes long. Therefore, the difference in accuracy can be shown by the length of the lead line. Furthermore, when there are a plurality of search results, the difference in accuracy can be compared by a common display form. That is, when the accuracy changes due to the progress of the display of moving images, a plurality of display contents can be narrowed down to one display content.

As a display form, if a frame indicating the extracted human face area is added to the display, the visibility is further improved. Further, the visibility is further improved by matching the frame of the area with the display color of the display content. Further, as a display form, if a role name is registered in the performer information holding unit 110, the role name may be written as the display content.

FIG. 9 is a diagram showing an example of the display of the search result displayed by the display unit 103.

In FIG. 9, two persons are shown on the display screen 150 which is a television screen. The performer name 151 is also written in the face area 153 of the left person. Similarly, the performer name 152 is also written in the face area 154 of the right person. Here, the performer name of FIG. 3 is displayed with a larger character size as the reliability is higher, and an example in which the performer name 152 has a higher reliability than the performer name 151 is shown. According to the display example shown in FIG. 9, when the performer name 152 is displayed larger than the performer name 151, the user intuitively knows that the display content of the performer name 152 is correct. Can do. That is, the user can intuitively determine the correctness of the display content. Also, the method of displaying the performer name may be a change of the character or background color, a change of the transparency, a display position, etc. instead of or in combination with the change of the character size. In either case, the same effect can be obtained.

As described in detail above, in the program information display device 100, the program acquisition unit 101 acquires a program that is a moving image and program information including a performer name, and the related image acquisition unit 107 performs a performer name. Based on the network, a related image is acquired from the network, and the feature amount calculation unit 108 calculates a feature amount of a related extracted image obtained by cutting out a human face area from the related image. Then, the representative feature amount determination unit 109 determines the representative feature amount and its reliability based on the feature amount of the related extracted image and the information on the acquisition destination of the related image, and the performer information management unit 104 determines The representative feature amount and the reliability are managed as performer information in association with the performer name. On the other hand, the display image acquisition unit 102 acquires a display image from the frames constituting the moving image, and the feature amount calculation unit 111 calculates the feature amount of the display extracted image for the display image. Then, the search unit 105 calculates the similarity between the feature amount of the display extracted image calculated by the feature amount calculation unit 111 and the representative feature amount held in the performer information holding unit 110, and the similarity is the maximum. The name of the performer associated with the representative feature amount is acquired. Then, the display information generation unit 106 generates display information based on at least one of reliability or similarity, the performer name acquired by the search unit 105, and the region of the display extracted image, and the display unit 103 The name of the person is displayed in association with the face area.

As described above, the program information display apparatus 100 obtains performer information from the program information of the moving image, obtains the performer's face image from the network, and obtains the image feature amount. And the program information display apparatus 100 specifies a performer from the image feature-value obtained by extracting a face image from a moving image, and superimposes and displays a performer name on a moving image. Thereby, the program information display apparatus 100 can display a performer name in association with a performer's area without preparing a performer's face image database in advance. Moreover, the program information display apparatus 100 reflects the reliability based on the database searched dynamically, when specifying a performer from the image feature-value obtained by extracting a face image from a moving image. As a result, there is an effect of enabling a search with less noise and high accuracy. At this time, the program information display device 100 displays the correctness of the determination result of the performer according to the difference in display form. Thereby, the effect that a user can judge the determination result of a performer intuitively is acquired. For example, when the user does not agree with the name of the performer while watching the program, the user can respond to a request to know the name of the performer in association with the face on the screen. Moreover, it can be made to grasp | ascertain intuitively how reliable the determination result of a performer is.

In addition, since the representative feature amount determination unit 109 according to the present embodiment determines whether the related image is an image acquired from the official site and weights the related image, an effect of increasing the reliability of the feature amount can be obtained. Can do.

(Embodiment 2)
FIG. 10 is a diagram showing a configuration of a program information display system including a program information display device according to Embodiment 2 of the present invention. The same components as those in FIG. 1 are denoted by the same reference numerals, and description of overlapping portions is omitted.

10, the program information display device 300 includes a search information holding unit 301 in addition to the program information display device 100 of FIG.

The search information holding unit 301 holds search results obtained by the search unit 105. The search results by the search unit 105 are, as described above, the representative image feature amount, the reliability, the display extracted image region, the similarity, and the performer name.

FIG. 11 is a flowchart showing processing up to display of the performer name of the program information display apparatus 300. Steps that perform the same processing as the flow shown in FIG. 5 are denoted by the same step numbers, and description of overlapping portions is omitted.

In step S31, the search unit 105 closes the area of the newly extracted display extraction image (hereinafter referred to as “new display extraction image”) and has been extracted in the past (referred to as past display extraction image). It is confirmed whether or not this area is registered in the search information holding unit 301.

In step S 32, when the past display extracted image area close to the new display extracted image area is registered in the search information holding unit 301 in step S 32, the search unit 105 selects the new display extracted image area in step S 33. The extracted image feature quantity of the display extracted image is compared with the representative feature quantity of the corresponding past display extracted image, the similarity is calculated, and the process proceeds to step S26.

If the area of the past display extraction image adjacent to the area of the new display extraction image is not registered in the search information holding unit 301, the search unit 105 proceeds to the following in step S24 and performs the performer name in FIG. This is the same as the process up to step S27.

In step S27, the search unit 105 holds the performer name associated with the corresponding representative feature amount as a search result.

In step S34, after determining the performer name, the search information retaining unit 301 retains the representative image feature value, the reliability, the region of the display extracted image, the similarity, and the performer name, and proceeds to step S28.

Here, when the process proceeds to step S33, the search unit 105 calculates the maximum similarity and proceeds to step S26. This is because the past display extraction image adjacent to the area of the new display extraction image is almost always a face image of the same person. In this case, in step S 26, the search unit 105 compares the similarity to the past display extracted image and the similarity to the related image registered in the search information holding unit 301.

If the degree of similarity slightly falls within a predetermined threshold, it is considered an error in the feature amount comparison. Therefore, in this case, the search unit 105 holds the larger similarity. Thereby, the search part 105 can make the similarity extraction process act only in the direction in which the similarity increases while capturing the face area of the same person.

On the other hand, if the degree of similarity falls greatly exceeding a predetermined threshold, it can be determined that another performer is projected in the almost same area by switching scenes. Therefore, in this case, the search unit 105 discards the registration data related to the past past display extraction area, that is, the process of registering invalid data instead of the process of holding the search result (the process of step S34). It becomes processing.

As described above, in this embodiment, the search unit 105 includes the search information holding unit 301 that holds the search result. The search unit 105 searches the search result held in the search information holding unit 301 only for the display extracted image close to the search result. As a result, the search unit 105 does not need to compare the display extracted image with the similarity of the feature amounts of all registered representative images, and can speed up the processing.

(Embodiment 3)
FIG. 12 is a diagram showing a configuration of a program information display system including a program information display device according to Embodiment 3 of the present invention. In the present embodiment, the same components as those in FIG.

12, the program information display system includes a program information display device 400, a network 200, an image search device 500, an image server 220, a program information server 230, a program guide server 240, and a broadcast station 250. That is, in the program information display system according to the present embodiment, a program information display device 400 and an image search device 500 are arranged instead of the program information display device 100 and the image search device 210 of the first embodiment.

The program information display device 400 is a digital broadcast receiver that reproduces program information transmitted from the broadcast station 250. The program information display device 400 includes a program acquisition unit 101, a display image acquisition unit 102, a display unit 103, a search unit 105, a display information generation unit 106, and a feature amount calculation unit 111. That is, the program information display device 400 has a configuration including a functional unit related to image display among the functional units of the program information display device 100 of the first embodiment.

The image search device 500 is an image search site. The image search apparatus 500 includes a performer information management unit 104, a related image acquisition unit 107, a feature amount calculation unit 108, a representative feature amount determination unit 109, a performer information holding unit 110, an image information holding unit 501, and an image search unit 502. It is comprised with. That is, the image search device 500 has a configuration including a function unit related to related image search, an image information holding unit 501, and an image search unit 502 among the function units of the program information display device 100 of the first embodiment. is doing. The image information holding unit 501 and the image search unit 502 correspond to the functional units of the image search apparatus 210 according to the first embodiment.

That is, the program information display system according to the present embodiment has a configuration in which the function unit related to the related image search of the program information display device 400 of the first embodiment is moved to the image search device 210 of the first embodiment. ing.

The program information display device 400 and the image search device 500 may each be configured such that the CPU executes a program stored in a storage medium such as a ROM, similarly to the program information display device 100 of the first embodiment.

Instead of the program information display device 400, the program information display device 100 of the first embodiment or the program information display device 300 of the second embodiment may be combined with the image search device 500 of the present embodiment. .

In the above configuration, the related image acquisition unit 107 of the image search apparatus 500 acquires an image via the network 200 based on a URL link or the like. The related image acquisition unit 107 holds the acquired image, the keyword obtained from the description of the corresponding image or the content of the corresponding page, and the URL of the image in the image information holding unit 501.

When searching for an image, the image search unit 502 receives program information from the program information display device 400 via the network 200, and returns a corresponding URL as image information from the image information holding unit 501 based on the received program information. .

This configuration eliminates the processing from the program information display device 400 until the representative feature amount is determined. Further, the search unit 105 receives the representative feature amount and the reliability via the network 113 using the performer name as a keyword. These points are changes from the first embodiment.

As described above, in this embodiment, the processing until the representative feature amount is determined is performed by the image search device 500, so that the processing of the program information display device 400 can be expected to be reduced and speeded up due to the distribution of functions.

In the present embodiment, the example in which the image information holding unit 501 and the performer information holding unit 110 are configured separately has been described, but these may be integrated. Further, the example in which the image search device 500 receives program information from the program information display device 400 has been described. However, the program information may be received from the program information server 230, and the process of determining the representative feature amount may be executed in advance to receive the performer name and the program name from the program information display device 400.

The above description is an illustration of a preferred embodiment of the present invention, and the scope of the present invention is not limited to this. In each of the above embodiments, a broadcast program has been described. However, any apparatus and method using program information may be used, and the present invention is not limited to a broadcast program.

In each embodiment, the names of the program information display device and the program information display method are used. However, this is for convenience of explanation, and the program information display device may be an image search device, a program playback device, and the program information display method may be a program information search method or the like.

Furthermore, each part constituting the program information display apparatus and method, for example, the type, number and connection method of the program acquisition part and performer information holding part are not limited.

In addition, the detailed program information constituting the program guide is acquired from the program guide server via the network. However, when the detailed program information is superimposed on the broadcast wave, it may be acquired from the broadcast wave.

When a program is held on a medium such as a DVD or a hard disk, information in which the image data of the performer and the name of the performer are associated with each other is often held separately from the program. . In this case, the related image may be acquired from the media instead of via the network. In this case, the broadcast station and the program guide server may not be included in the overall configuration of the program information display system.

Also, the program information device is not limited to application to a television receiver, and can be applied to other various devices that display images on which performers are displayed. For example, the program information display device can be applied to a device that displays or reproduces a moving image, such as a BD (Blue-ray Disc) player, a DVD player, or a hard disk recorder. Furthermore, the program information display device is also applicable to portable terminals such as mobile phones / PHS (Personal Handy-Phone System), portable information terminals (hereinafter referred to as PDA (Personal Digital Assistants)), personal computers, and portable game machines. can do.

The program information display device and the program information display method described above are also realized by a program for causing the program information display method to function. This program is stored in a computer-readable recording medium.

The disclosure of the description, drawings and abstract contained in the Japanese application of Japanese Patent Application No. 2008-076310 filed on Mar. 24, 2008 is incorporated herein by reference.

A program information display apparatus and a program information display method according to the present invention extract a person's face area from a moving image and display a performer name in association with the person's face area, a mobile phone, a DVD player, It can be applied to video playback terminals such as personal computers and game machines.

Claims

A program acquisition unit for acquiring a program that is a moving image and program information including a performer name;
Based on the performer name, a related image acquisition unit that acquires a related image from the network;
A first feature amount calculation unit that calculates a feature amount of a first extracted image obtained by cutting out a person's region from the related image;
A representative feature amount determining unit that determines a representative feature amount and its reliability based on the feature amount of the first extracted image and the information on the source of the related image;
Performer information management unit for managing the determined representative feature amount and reliability as performer information in association with the performer name;
A display image acquisition unit for acquiring a display image from a frame constituting the moving image;
A second feature amount calculating unit that calculates a feature amount of a second extracted image obtained by cutting out a person's region from the display image;
The similarity between the feature amount of the second extracted image calculated by the second feature amount calculation unit and the representative feature amount held in the performer information management unit is calculated, and the similarity is maximum. A search unit that obtains performer names associated with representative feature quantities,
A display information generating unit that generates display information based on at least one of the reliability or the similarity, the performer name acquired by the search unit, and the region of the second extracted image;
A display unit for displaying the display image and the display information;
A program information display device comprising:
The related image acquisition unit acquires a related image of a program performer from a site on the network,
The representative feature amount determination unit determines the reliability by weighting the related image according to the site of the acquisition destination.
The program information display device according to claim 1.
The display information generation unit generates display information for changing at least one of transparency, display color, character size, or display position based on the reliability or the similarity.
The program information display device according to claim 1.
A search information holding unit for holding a search result by the search unit;
The search unit searches the search result held in the search information holding unit based on the performer information.
The program information display device according to claim 1.
Obtaining program information including a program that is a moving image and a performer name;
Obtaining a related image from the network based on the performer name;
A first feature amount calculating step of calculating a feature amount of a first extracted image obtained by cutting out a person's region from the related image;
Determining a representative feature amount and its reliability based on the feature amount of the first extracted image and the information on the source of the related image;
Managing the determined representative feature quantity and reliability as performer information in association with the performer name;
Obtaining a display image from a frame constituting the moving image;
A second feature amount calculating step of calculating a feature amount of a second extracted image obtained by cutting out a person region from the display image;
The similarity between the feature quantity of the second extracted image calculated by the second feature quantity calculation step and the retained representative feature quantity is calculated, and is associated with the representative feature quantity having the maximum similarity. Obtaining a given performer name,
Generating display information based on at least one of the reliability or the similarity, the performer name acquired by the search unit, and the region of the second extracted image;
A program information display method comprising: displaying the display image and the display information.