JP2009021732A - Storage type broadcasting receiver and digest video reproduction program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a storage type broadcasting receiver capable of reproducing digest video of broadcast programs. <P>SOLUTION: The storage type broadcasting receiver 1 is provided with a feature quantity generating means 20 for generating image feature quantity for each sampled frame image, from video data of a broadcast program, a storing means 30 for storing the image feature quantity, and a digest video specifying means 60 for specifying a video section to be digest video of video data selected by a viewer on the basis of similarity between image feature quantity of the video data selected by the viewer and image feature quantity of a video section for digest video time of video data other than the video data selected by the viewer. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a storage-type broadcast receiving apparatus and a summary video reproduction program for searching and reproducing a summary video of a broadcast program desired by a viewer from video data of a plurality of stored broadcast programs.

In recent years, a storage type equipped with a large-capacity storage device such as a hard disk, accumulates (stores) broadcast program video data, and searches and reproduces video data desired by the viewer from the stored video data. Broadcast receivers are becoming popular.
In such a storage-type broadcast receiving apparatus, when searching for video data, the viewer wants to use not only text data that simply describes the program name and its contents, but also the video as a criterion for selection. There is a request. However, if the video data of the broadcast program is reproduced as it is, there is a problem that it takes time to search the video data desired by the viewer from a large amount of video data.

  Therefore, a digest video (summary video) of the broadcast program is created separately from the broadcast program, and the digest video of the broadcast program is reproduced when searching for the broadcast program by associating the broadcast program with the digest video. Thus, techniques for assisting in searching for broadcast programs are disclosed (see Patent Documents 1 and 2).

  According to the technique disclosed in Patent Document 1, a transmission apparatus multiplexes a digest video created in advance into a broadcast signal together with program guide information in which the digest video and a broadcast program are associated with each other. In the receiving device, the digest video corresponding to the broadcast program is reproduced with reference to the program guide information, thereby enabling the search of the broadcast program by the digest video.

  Further, according to the technique disclosed in Patent Document 2, the digest video created in advance is stored in the content server, and the location of the digest video for the broadcast program is notified to the receiving device by URL (Uniform Resource Locator). To do. Then, in the receiving device, the digest video corresponding to the broadcast program is obtained from the content server and reproduced, thereby enabling the search for the broadcast program by the digest video.

JP 2002-344858 A JP 2004-304430 A

According to the conventional technology described above, the viewer can search for a desired broadcast program by actually viewing the video by playing back the digest video.
However, according to the conventional technology, a broadcast program provider (broadcaster or the like) has to create a digest video that is used only for searching for a broadcast program in advance, and it takes time and effort to create it. There's a problem.

Further, according to the conventional technique, it is necessary to create program guide information and URL in which a broadcast program and a digest video are associated in advance by a broadcast program provider. Since it is necessary to associate the broadcast program with the digest video manually, there is a problem that it takes much time to associate the digest video with a large number of broadcast programs.
Further, according to the conventional technology, since it is necessary to transmit the digest video to the receiving device via the broadcast wave or the communication line, there is a problem that the transmission band of the broadcast wave or the communication line is compressed.

  The present invention has been made in order to solve the above-described problems. The broadcast program provider associates a digest video used only for searching for a broadcast program, or associates a broadcast program with a digest video. It is an object of the present invention to provide a storage-type broadcast receiving apparatus and a summary video reproduction program capable of reproducing a digest video (summary video) of a broadcast program in a receiving apparatus without creating program guide information and the like.

  The present invention was devised to achieve the above object. First, the storage-type broadcast receiving apparatus according to claim 1 is a program information including video data of a broadcast program and time information of the broadcast program. In a storage-type broadcast receiver that stores data in video data storage means and reproduces the broadcast program, sampling means, feature value calculation means, feature value storage means, list generation means, and program video feature value extraction And a summary video candidate feature amount extraction unit, a similarity calculation unit, a summary video segment identification unit, and a summary video playback unit.

  In such a configuration, the storage-type broadcast receiving apparatus samples the video data at a predetermined frame interval by the sampling means. Then, the storage-type broadcast receiving apparatus calculates an image feature amount for each sampled frame image by the feature amount calculation unit, and stores it in the feature amount storage unit in association with the video data. As a result, the storage-type broadcast receiving apparatus can extract a feature quantity in which video data changes with time, and the feature quantity can be used as information for specifying video data.

The storage-type broadcast receiving apparatus generates a list of video data stored in the video data storage unit by referring to the program information data by the list generation unit. Thus, the viewer can select a broadcast program (video data) from the list.
Then, the storage-type broadcast receiving apparatus reads the image feature quantity of the video data selected by the viewer in the list from the feature quantity storage means by the program video feature quantity extraction means.

  Further, the storage-type broadcast receiving apparatus uses the summary video candidate feature amount extraction unit to calculate the image feature amount of the video section for a predetermined summary video time among video data other than the video data selected by the viewer in the list. And sequentially read out from the feature amount storage means. As a result, the image feature quantities of the video section that are candidates for the summary video are sequentially read out from the feature quantity storage means.

  Then, the storage-type broadcast receiving apparatus uses the similarity calculation unit to calculate the similarity between the image feature amount read by the program video feature amount extraction unit and the image feature amount sequentially read by the summary video candidate feature amount extraction unit. Calculate the degree. Thereby, the storage-type broadcast receiving apparatus can determine whether or not the video data selected by the viewer is similar to the summary video candidate.

Furthermore, the storage-type broadcast receiving device specifies the video segment for the summary video time most similar to the video data selected by the viewer based on the similarity calculated by the similarity calculation unit by the summary video segment specifying unit. To do. As a result, the summary video corresponding to the video data selected by the viewer is specified.
Then, the storage-type broadcast receiving apparatus reads and reproduces the video data of the video section specified by the summary video section specifying means from the video data storage means by the summary video playback means. As a result, the summary video corresponding to the video data selected by the viewer is reproduced.

  The storage type broadcast receiving apparatus according to claim 2 is the storage type broadcast receiving apparatus according to claim 1, wherein the video data of the broadcast program is a substance of the broadcast program to which the program information data is added. Video data and second video data to which no program information data is added, and the summary video candidate feature amount extraction unit extracts a video segment corresponding to the summary video time from the image feature amount corresponding to the second video data. An image feature amount is extracted.

  In such a configuration, the storage-type broadcast receiving device searches the summary video in the second video data (non-broadcast program) to which the program information data is not added, and summarizes the spot video broadcast for the promotion of the broadcast program. Can be specified as

  Further, the storage type broadcast receiving apparatus according to claim 3 is the storage type broadcast receiving apparatus according to claim 1, wherein the video data of the broadcast program is a substance of the broadcast program to which the program information data is added. The video data and the second video data to which no program information data is added, and the summary video candidate feature quantity extraction unit is specified by the program information data in the image feature quantity corresponding to the first video data The image feature amount of the video section for the summary video time is extracted from the time that is a predetermined time before the time.

  In such a configuration, the storable broadcast receiving apparatus searches the first video data (broadcast program) to which the program information data is added for a summary video from the video for the last predetermined time of the broadcast program, and notifies the broadcast program in advance. As a summary video, it is possible to specify the spot video broadcasted as

  According to a fourth aspect of the present invention, there is provided a summary video reproduction program that stores video data of a broadcast program and program information data including time information of the broadcast program in a video data storage means, and reproduces the broadcast program. In the broadcast receiving apparatus, in order to reproduce the summary video of the broadcast program selected by the viewer, the computer includes a sampling unit, a feature amount calculation unit, a list generation unit, a program video feature amount extraction unit, and a summary video candidate feature amount extraction unit. , A similarity calculating means, a summary video section specifying means, and a summary video reproducing means.

In such a configuration, the summary video reproduction program samples the video data at a predetermined frame interval by the sampling means. Then, the summary video reproduction program calculates an image feature amount for each sampled frame image by the feature amount calculation unit, and stores it in the feature amount storage unit in association with the video data.
The summary video reproduction program generates a list of video data stored in the video data storage unit by referring to the program information data by the list generation unit.

Then, the summary video reproduction program reads the image feature quantity of the video data selected by the viewer in the list from the feature quantity storage means by the program video feature quantity extraction means.
In addition, the summary video reproduction program uses the summary video candidate feature amount extraction means to extract the image feature amount of a video section for a predetermined summary video time among video data other than the video data selected by the viewer in the list, Read sequentially from the feature quantity storage means.
Then, the summary video reproduction program stores the similarity between the image feature amount read by the program video feature amount extraction unit and the image feature amount sequentially read by the summary video candidate feature amount extraction unit by the similarity degree calculation unit. Is calculated.

Further, the summary video reproduction program specifies the video section for the summary video time most similar to the video data selected by the viewer based on the similarity calculated by the similarity calculation means by the summary video section specifying means. .
Then, the summary video playback program reads the video data of the video section specified by the summary video section specifying means from the video data storage means and plays back by the summary video playback means.

  According to the first and fourth aspects of the present invention, the storage-type broadcast receiving device can search by looking at the summary video when searching for a desired broadcast program from the stored broadcast programs. The search can be performed at a higher speed than the case where the search is performed by sequentially watching the video of the program. In addition, according to the present invention, since a summary video is extracted from stored video data in a storage-type broadcast receiver, a summary video (digest used only by a broadcast program provider for searching for a broadcast program). Video), program guide information in which a broadcast program and a summary video are associated with each other, and the summary video can be reproduced without human intervention on the provider side. Furthermore, according to the present invention, since it is not necessary to distribute the summary video used only for the search through the broadcast wave or the communication line, the transmission band of the broadcast wave or the communication line is not compressed.

  According to the second aspect of the present invention, the storage-type broadcast receiver searches for a summary video in a non-broadcast program to which no program information data is added, and summarizes a spot video broadcast for the promotion of the broadcast program. It can be specified as a video. Thus, since the present invention searches for a summary video in a non-broadcast program, it can search for a summary video at a higher speed than when searching for a summary video from the entire broadcast program.

  According to the third aspect of the present invention, the storage-type broadcast receiving device searches the summary video in the last predetermined time of the broadcast program to which the program information data is added, and uses it for the broadcast program notice. The broadcast spot video can be specified as a summary video. As described above, according to the present invention, since a summary video is searched in a video section of a predetermined time in a broadcast program, it is possible to search for a summary video at a higher speed than when searching for a summary video from the entire broadcast program.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[Outline of the present invention]
First, the outline of the present invention will be described with reference to FIG. FIG. 1 is a schematic diagram for explaining the contents of program organization.
FIG. 1A shows a time table in which a broadcast program (main broadcast A) is broadcast at 11:00 on the day of the month of 2007, and the broadcast program ends at 11:54.
Further, FIG. 1B shows a time table in which a broadcast program (main broadcast B) is broadcast at 16:00 on the day of the month of 2007, and the broadcast program ends at 16:24. .
The start time, end time, etc. of such a broadcast program are usually distributed as program information data (EPG [electronic program guide], etc.).

On the other hand, non-broadcast programs such as commercials that are not notified as program information data are also broadcast during the broadcast program (first video data). For example, a non-broadcast program (second video data) is broadcast from 11:54 to 12:00 after the main broadcast A and from 16:24 to 16:30 after the main broadcast B. .
This non-broadcast program may include a spot video summarizing the contents of the broadcast program for the purpose of promoting the program of the broadcast program. For example, the spot video X included after the main broadcast A may be a summary video of the main broadcast B. In addition, a broadcast program may include a preview video at the end of the main broadcast (not shown).

Such a spot video is composed of a part of video data of a broadcast program, and therefore includes a frame image whose image feature is similar to that of the video data of the broadcast program.
Therefore, the storage-type broadcast receiving apparatus according to the present invention stores broadcast programs (including non-broadcast programs), and when a viewer searches for a broadcast program, a spot video similar to the main broadcast of the broadcast program is Play as a summary video of a broadcast program. As a result, the viewer can search for the broadcast program that he / she wants to view using the summary video.

[Configuration of storable broadcasting receiver]
Next, with reference to FIG. 2, the structure of the storage type broadcast receiving apparatus according to the embodiment of the present invention will be described. FIG. 2 is a block diagram showing the configuration of the storage-type broadcast receiving apparatus according to the embodiment of the present invention.

The storage-type broadcast receiving apparatus 1 is a storage-type broadcast receiving apparatus that stores video data (including audio) of a broadcast program and reproduces it when a viewer wants to view the broadcast program. Here, it is assumed that the storage-type broadcast receiving apparatus 1 is operated by a remote control device 3 with a display device 2 for displaying reproduced video data connected to the outside.
In addition, the storage-type broadcast receiving apparatus 1 has a function of assisting viewer search by playing back a summary video that is a summary of video data when a viewer searches for a broadcast program that the viewer wants to watch. .
Hereinafter, each configuration of the storage-type broadcast receiving apparatus 1 will be described individually.

  As shown in FIG. 2, the storage-type broadcast receiving apparatus 1 includes a receiving unit 10, a feature amount generating unit 20, a storage unit 30, a summary video specifying unit 40, a search list presenting unit 50, and a video reproducing unit 60. And.

  The receiving means 10 receives video data of broadcast programs (including non-broadcast programs) distributed via broadcast waves and program information data such as an electronic program guide (EPG). The receiving means 10 writes the received video data and program information data in the storage means 30 as file data (video file data) for each broadcast program. As a result, the video data is accumulated in the storage means 30 and can be reproduced at any time. Further, when reproducing video data in real time, the receiving means 10 outputs the video data to the video reproducing means 60. The receiving unit 10 includes a recording unit (not shown), and writes the received video data in the storage unit 30 only when recording is instructed by the remote control device 3. Alternatively, the video data is written in the storage means 30 when the recording reservation time is reached.

In addition, the receiving unit 10 writes the video data or the like in the storage unit 30 and then notifies the feature amount generating unit 20 that the writing is completed.
Here, the receiving means 10 receives the video data of the broadcast program via the broadcast wave. However, for example, the reception means 10 may receive the video data distributed via the communication line. .

  The feature amount generation unit 20 generates a feature amount of the video data stored in the storage unit 30. Here, the feature quantity generation means 20 includes a sampling means 21 and a feature quantity calculation means 22.

  The sampling means 21 samples the video data stored in the storage means 30 at a predetermined frame interval. For example, the sampling means 21 extracts frame images of video data at 10 frame intervals. The frame image sampled by the sampling unit 21 is output to the feature amount calculating unit 22 together with the sampling time.

  The feature amount calculating unit 22 calculates an image feature amount for each frame image sampled by the sampling unit 21. The image feature amount may be anything that is generally used in the field of image processing. For example, the RGB or HSV average value vector of all pixels of the frame image, the luminance histogram of the entire frame image, and the edge of the frame image The edge direction histogram or the fractal dimension indicating the complexity of the frame image can be used as the image feature amount.

Here, the feature amount calculation unit 22 calculates a plurality of image feature amounts and expresses the image feature amount of the frame image as an image feature vector. That is, the feature amount calculation means 22 calculates m image feature amounts (v ₁ , v ₂ ,..., V _m ) from the frame image, and an m-dimensional image feature vector V = [v ₁ , v ₂ , .., V _m ] are generated.
The feature amount calculating unit 22 writes the calculated image feature amount (image feature vector V) in the storage unit 30 together with an identifier for specifying video data and a sampling time.

The storage means (video data storage means, feature amount storage means) 30 includes the video data received by the reception means 10, the feature amounts of the video data generated by the feature amount generation means 20 (image feature amounts for each frame image), and Is a general storage device such as a hard disk.
The video data and the image feature quantity stored in the storage unit 30 are referred to by the summary video specifying unit 40, the search list presentation unit 50, and the video reproduction unit 60.

Here, referring to FIG. 3 (refer to FIG. 2 as appropriate), data contents stored in the storage means 30 will be specifically described. FIG. 3 is a structural diagram schematically showing the data structure of data stored in the storage means of the storage type broadcast receiving apparatus according to the embodiment of the present invention.
As shown in FIG. 3, the storage means 30 is divided into a video file data storage area 31 and a frame image feature quantity structure storage area 32 as storage areas.

  The video file data storage area (video data storage means) 31 is an area for storing video file data 31a including a video file identifier ID, video data D, and program information data P. The video file data 31a is written by the receiving means 10.

The video file identifier ID is unique identification information for identifying the video file data 31a, and is, for example, a unique numerical value, a file name, or the like.
The video data D is an entity of a broadcast program and is video digital data. For example, the video data D is digitally encoded data encoded by MPEG-2 or the like.

  The program information data P is data indicating a video section (start time, end time, etc.) of a broadcast program such as an electronic program guide (EPG) distributed from a broadcast station or the like. This program information data P is data added only to the video section of the main broadcast of the broadcast program, and is not added to non-program video sections such as commercial videos. In the present embodiment, the spot video inserted in the non-program video section is used as a summary video.

  The frame image feature quantity structure storage area (feature quantity storage means) 32 stores a frame image feature quantity structure 32a having an image feature quantity V, a video file identifier ID, and a sampling time T as elements of the structure. It is an area. The frame image feature quantity structure 32 a is written by the feature quantity generation unit 20.

The image feature amount V is a feature amount of a frame image obtained by sampling the video data D of the video file data 31a stored in the video file data storage area 31 at a predetermined frame interval. Here, an m-dimensional image feature vector V = [v ₁ , v ₂ ,..., V _m ] is stored as an image feature amount for each frame image.

  The video file identifier ID is identification information indicating which video file data 31a is the feature quantity of the frame image feature quantity structure 32a, and is the same identifier as the video file identifier ID of the corresponding video file data 31a. Indicates.

  The sampling time T is time information indicating which sampling time of the video file data 31a the frame image corresponding to the frame image feature quantity structure 32a is. This sampling time T may be the time code (hour: minute: second: frame) of the video data D, or may be a frame number numbered from the first frame image of the video data D.

Here, the video file data 31a and the frame image feature quantity structure 32a are stored in different storage areas of the storage means 30, but the video file data 31a and the frame image feature quantity structure are stored. The body 32a may be stored in separate storage means (video data storage means, feature amount storage means: not shown).
Returning to FIG. 2, the description of the configuration of the storage-type broadcast receiving apparatus 1 will be continued.

  The summary video specifying means 40 receives identification information (video file identifier) for specifying the video data of the broadcast program selected by the viewer from the search list presentation means 50 described later, and stores the summary video for the video data. To identify the video section (summary video section). Here, the summary video specifying unit 40 includes a feature amount extracting unit 41, a similarity calculating unit 42, and a summary video section specifying unit 43.

  The feature quantity extraction means 41 is an image feature quantity of the video data of the broadcast program selected by the viewer and a video section of a video section corresponding to a predetermined summary video of video data (non-program video data) other than the video data. Are extracted. Here, the feature quantity extraction means 41 includes a program video feature quantity extraction means 41a and a non-program video feature quantity extraction means 41b.

The program video feature amount extraction unit 41 a extracts an image feature amount corresponding to video data of a broadcast program selected by the viewer from the image feature amount stored in the storage unit 30. Here, the program video feature quantity extraction unit 41a refers to the program information data P of the video file data 31a described with reference to FIG. 3, and the frame image feature quantity (frame image feature quantity structure 32a) corresponding to the video section. To extract. Then, the program image feature quantity extraction unit 41a arranges the extracted frame image feature quantity structures 32a in time series according to the sampling time T, thereby corresponding to the image data of the broadcast program selected by the viewer. (Program image feature structure series).
The image feature amount (program image feature structure series) extracted by the program video feature amount extraction unit 41 a is output to the similarity calculation unit 42.

  The non-program video feature amount extraction means (summary video candidate feature amount extraction means) 41b is an image feature amount of a video section for a predetermined summary video time of non-program video data (second video data) that is a candidate for summary video. Is extracted. Here, the non-program video feature amount extraction means 41b is a frame image feature amount corresponding to a non-program video section (where program information data P is not added to the video file data 31a described in FIG. 3) as non-program video data. From the frame image feature amount structure 32a), image feature amounts corresponding to the summary video time (for example, 15 seconds that are general as spot images) are sequentially extracted. Then, the non-program video feature quantity extraction unit 41b arranges the extracted frame image feature quantity structures 32a in time series according to the sampling time T, so that the image feature quantities (summary video candidates) of the video section for the summary video time. Characteristic structure series).

  The image feature quantity (summary video candidate feature structure series) extracted by the non-program video feature quantity extraction unit 41b is output to the similarity calculation unit 42 together with the video file identifier ID and the sampling time T.

  The similarity calculation unit 42 sequentially includes the image feature amount (program image feature structure series) of the video data of the broadcast program selected by the viewer extracted by the program video feature amount extraction unit 41a of the feature amount extraction unit 41, The similarity with the image feature quantity (summary video candidate feature structure series) of the video section for the summary video time output from the non-program video feature quantity extraction unit 41b is calculated.

  Here, the similarity calculation means 42 is a program video feature vector that is a sequence of image feature amounts (image feature vectors) for each frame image of the program image feature structure sequence and each frame image of the summary video candidate feature structure sequence. Similarity with a summary video candidate feature vector, which is a series of image feature amounts (image feature vectors), is calculated.

  For example, the similarity calculation means 42 averages each vector of the program video feature vector and the summary video candidate feature vector in time series, then performs an inner product operation of the vectors, and sets the calculation result as the similarity. Instead of the inner product calculation of the vectors, the distance between the vectors is obtained, and the distance is set to “0” to “1” so that the closer the distance is, the value is “0”, and the farther the value is “1”. It is good also as calculating | requiring a similarity by normalizing by ".

Further, for example, the similarity calculation means 42 obtains the maximum similarity between the element vector (image feature vector) of the program video feature vector for each element vector (image feature vector) of the summary video candidate feature vector. The overall similarity may be obtained by averaging the number of elements of the summary video candidate feature vector. For the similarity between the element vectors, vector inner product, vector distance, and the like can be used.
The similarity calculated by the similarity calculating means 42 is output to the summary video section specifying means 43 together with the video file identifier and the sampling time.

The summary video section specifying unit 43 specifies a video section that becomes a summary video of the broadcast program selected by the viewer based on the similarity calculated by the similarity calculation unit 42.
Here, the summary video section specifying means 43 starts the summary video section with a frame image corresponding to the first sampling time of the non-program image feature structure sequence having the maximum similarity calculated by the similarity calculation means 42. Frame. Then, the summary video section specifying unit 43 specifies the summary video section by setting a frame image that has passed a predetermined number of frames (15 seconds; 450 frames) from the start frame as the end frame.

The section information that specifies the summary video section includes a video file identifier that specifies video data of the broadcast program, a start frame time, and an end frame time. Note that the time interval or the number of frames for the summary video may be used instead of the time of the end frame.
The section information of the summary video section specified by the summary video section specifying means 43 is output to the search list presenting means 50.

  The search list presenting means 50 generates a list indicating which broadcast programs are stored (stored) in the storable broadcast receiving apparatus 1 and outputs the list to the display apparatus 2. Here, the search list presentation unit 50 includes a list generation unit 51, a list selection unit 52, and a summary video reproduction unit 53.

  When the viewer is instructed to display the list via the remote control device 3, the list generation unit 51 stores the broadcast program stored based on the program information data of the video file data stored in the storage unit 30. A list including a program name, a video section (start time, end time), a genre, and the like is generated. The list generated by the list generating means 51 is output to the display device 2 and displayed on the display screen of the display device 2 as a list screen.

The list selection unit 52 receives a selection from the viewer in the list displayed on the display device 2. Here, the list selection means 52 identifies broadcast programs in the list by analyzing the pressing signals of the operation buttons such as the up / down / left / right buttons and the decision button output from the remote control device 3.
Here, the list selection unit 52 is instructed to reproduce the selected broadcast program by the operation button, and thereby the identification information (video file identifier) for identifying the video data of the selected broadcast program is displayed as the video reproduction unit 60. Output to.

  In addition, the list selection means 52 is instructed to reproduce the summary video of the selected broadcast program by the operation button, and thereby the identification information (video file identifier) for specifying the video data of the selected broadcast program is summarized video. Output to the reproduction means 53. The summary video playback instruction is given by, for example, displaying a summary video playback button on the list screen and selecting it by the remote control device 3.

The summary video reproduction means 53 reproduces the summary video of the broadcast program selected by the list selection means 52 and instructed to be reproduced.
Here, the summary video reproduction means 53 outputs the identification information (video file identifier) input from the list selection means 52 to the summary video specification means 40. Then, the summary video reproduction means 53 acquires the summary video section information corresponding to the video file identifier from the summary video specifying means 40, reads out the video data corresponding to the summary video section from the storage means 30, and reproduces it.

The video reproduction means 60 reproduces the video of the broadcast program selected by the list selection means 52 and instructed to be reproduced. Here, the video reproduction means 60 reads out and reproduces video data corresponding to the identification information (video file identifier) input from the list selection means 52 from the storage means 30.
By configuring the storable broadcast receiving apparatus 1 in this way, the storable broadcast receiving apparatus 1 can reproduce a summary video that is a summary of the broadcast program when a viewer searches for a broadcast program that the viewer wants to view. .

The configuration of the storable broadcast receiving apparatus 1 has been described above, but the storable broadcast receiving apparatus 1 can be operated by a summary video presentation program that causes a general computer to function as each of the above-described means.
Here, in the storage-type broadcast receiving apparatus 1, the summary video that is the summary of the broadcast program is extracted from the video other than the video section of the main broadcast of the broadcast program. It is good also as extracting from. In this case, for example, as shown in FIG. 4, the feature type extracting unit 41 is configured as a feature amount extracting unit 41B to which a program video partial feature amount extracting unit 41c is added, so that the storage type broadcast receiving apparatus 1B is configured.

The program video partial feature amount extraction means 41c is a predetermined time from the end time specified by the program information data in the image feature amount corresponding to the video data (first video data) other than the video data of the broadcast program selected by the viewer. The image feature amount of the video section for the summary video is extracted from the retroactive time.
The program video partial feature amount extraction unit 41c is, for example, an image feature amount for a summary video (for example, 15 seconds) in video data from 5 minutes before the end time to the end time of the video data (first video data). Are extracted sequentially. The method of extracting the feature amounts sequentially is the same as the non-program video feature amount extraction unit 41b described with reference to FIG.

  In this case, the similarity calculation means 42 extracts the image feature quantity extracted by the program video feature quantity extraction means 41a, and the image feature quantity and program video partial feature quantity extracted by the non-program video feature quantity extraction means 41b. The similarity with the image feature amount extracted by the means 41c is sequentially calculated.

  By configuring the storable broadcast receiving apparatus 1B as described above, the storable broadcast receiving apparatus 1B allows the summary video of the broadcast program, the video data to which no program information data is added (non-program video), and the program information data. Can be generated from video data that is a predetermined time later than the end time of the video data (program video) to which is added.

[Operation of storable broadcast receiver]
Next, the operation of the storage type broadcast receiving apparatus according to the embodiment of the present invention will be described. Here, a storage operation for storing a broadcast program and a summary video reproduction operation for reproducing a summary video in the storage-type broadcast receiving apparatus according to the embodiment of the present invention will be described separately.

<Accumulation operation>
First, with reference to FIG. 5 (refer to FIG. 2 as appropriate), the accumulation operation of the accumulation-type broadcast receiving apparatus according to the embodiment of the present invention will be described. FIG. 5 is a flowchart showing an accumulation operation of the accumulation type broadcast receiving apparatus according to the embodiment of the present invention.
First, the storage-type broadcast receiving apparatus 1 receives video data and program information data of a broadcast program distributed via broadcast waves by the receiving means 10 (step S1), and writes it in the storage means 30 (step S2).
Thereafter, the storage-type broadcast receiving apparatus 1 samples the video data at a predetermined frame interval by the sampling unit 21 of the feature amount generation unit 20 (step S3).

Then, the storage-type broadcast receiving apparatus 1 calculates an image feature amount for each frame image sampled in step S3 by the feature amount calculating means 22 (step S4), and an identifier (video file identifier) for specifying video data, A frame image feature quantity structure is generated together with the sampling time (step S5). Then, the feature quantity calculation unit 22 writes the frame image feature quantity structure in the storage unit 30 (step S6).
Through the above operation, the video data and program information data of the broadcast program and the feature amount of the video data are accumulated in the storage means 30.

<Summary video playback operation>
Next, referring to FIG. 6 (refer to FIG. 2 as appropriate), the summary video playback operation of the storage-type broadcast receiving apparatus according to the embodiment of the present invention will be described. FIG. 6 is a flowchart showing the summary video playback operation of the storage-type broadcast receiving apparatus according to the embodiment of the present invention.

First, the storage-type broadcast receiving apparatus 1 receives the video stored in the storage means 30 when the list generation means 51 of the search list presentation means 50 instructs the list display from the viewer via the remote control device 3. Based on the program information data of the file data, a list made up of program names of broadcast programs is generated and displayed on the display device 2 (step S11).
Then, the storage-type broadcast receiving apparatus 1 specifies the broadcast program selected by the viewer via the remote control device 3 by the list selection unit 52 (step S12).

  Further, the storage-type broadcast receiving apparatus 1 adds program information data by referring to the program information data of the broadcast program specified in step S12 by the program video feature value extracting means 41a of the feature value extracting means 41. Image feature quantities are extracted from a frame image feature quantity structure (program video feature structure) corresponding to a video section of video data, and a program video feature structure series arranged in time series according to the sampling time is generated (step S13). .

Here, the video data of the broadcast program selected by the viewer is composed of N _ref frame images, and the image feature amounts (image feature vectors) of the respective frame images are V (1) and V (2). ,..., V (N _ref ), program video feature vector V _ref = [V (1), V (2),. N _ref )].
Then, the storage-type broadcast receiving apparatus 1 uses the time (sampling time) t as a variable and initializes it (“0”) (step S14).

  Then, the storage-type broadcast receiving apparatus 1 uses the non-program video feature quantity extraction unit 41b of the feature quantity extraction unit 41 to generate a frame image feature quantity structure (non-program) corresponding to a video section of video data to which no program information data is added. From the program video feature structure), the image feature quantity for the video section that is a candidate for the summary video is extracted from time t, and a summary video candidate feature structure sequence arranged in time series according to the sampling time is generated ( Step S15).

Here, the video data of the non-broadcast program is composed of N _npg frame images, and the image feature amounts (image feature vectors) of the respective frame images are V ′ (1), V ′ (2) _,. , V ′ (N _npg ), the non-program video feature structure sequence is a non-program video feature vector V _npg = [V ′ (1), V ′ (2) _,. , V ′ (N _npg )].

In this case, in step S15, the summary video candidate feature structure series for the summary video starting from time t generated by the non-program video feature amount extraction unit 41b is the summary video candidate feature vector V _spt = [V ′ ( t), V ′ (t + 1),..., V ′ (t + L−1)]. Note that L is the number of summary video candidate feature structures extracted from the non-program video feature structure. For example, the sampling interval is 30 frames and the number of frames corresponding to the summary video time of 15 seconds is 450 frames. Then, L = (450/30) = 15.

  Then, the storage-type broadcast receiver 1 uses the similarity calculation means 42 to extract the image feature amount (program video feature structure series) of the broadcast program extracted in step S13 and the image for the summary video extracted in step S15. A similarity S (t) with the feature amount (summary video candidate feature structure series) is calculated (step S16). The similarity calculation operation will be described later in detail.

Then, the storage-type broadcast receiving apparatus 1 adds “1” to the time (sampling time) t (step S17). Here, when the extraction of the summary video candidate feature structure series has reached the end of the non-program video feature structure, that is, “t + L−1” is the number of frame images of video data of the non-broadcast program (N _npg If the number exceeds the number of sheets (Yes in step S18), the storable broadcast receiver 1 proceeds to step S19.

On the other hand, when the extraction of the summary video candidate feature structure series has not reached the end of the non-program video feature structure, that is, “t + L−1” is the number of frame images of video data of the non-broadcast program (N _npg Sheet) In the following cases (No in Step S18), the storage-type broadcast receiving apparatus 1 returns to Step S15 and continues the operation.

  Then, the storage-type broadcast receiving apparatus 1 uses the summary video section specifying unit 43 to display the frame image corresponding to the sampling time T at which the similarity S (t) calculated in step S16 is the maximum as the start frame of the summary video section. A summary video section is specified by setting a frame image that has passed a predetermined number of frames (15 seconds; 450 frames) from the start frame as an end frame (step S19).

Then, the storage-type broadcast receiving apparatus 1 reads and reproduces the video data corresponding to the summary video section specified in step S19 by the summary video reproduction means 53 from the storage means 30 (step S20).
With the above operation, the storage-type broadcast receiving apparatus 1 can reproduce a summary video that is a summary of the selected broadcast program when the viewer selects the broadcast program from the list.

[Similarity calculation operation of storable broadcasting receiver]
Next, with reference to FIG. 7 and FIG. 8, the similarity calculation operation performed by the similarity calculation means 42 of the storable broadcast receiving apparatus 1 will be described. FIG. 7 is a flowchart showing an example of the similarity calculation operation performed by the similarity calculation unit of the storage-type broadcast receiving apparatus according to the embodiment of the present invention. FIG. 8 is a flowchart showing another example of the similarity calculation operation performed by the similarity calculation unit of the storage-type broadcast receiving apparatus according to the embodiment of the present invention. The operations described in FIGS. 7 and 8 correspond to the operation in step S16 described in FIG.

<Similarity Calculation Operation: First Example>
First, a first example of the similarity calculation operation performed by the similarity calculation means 42 will be described with reference to FIG. 7 (refer to FIG. 2 as appropriate).
First, the similarity calculation means 42 has program video feature vectors V _ref = [V (1), V (2),..., V (N _ref ) in a program video feature structure sequence composed of N _ref frame images. )], An average vector V _{ref_ave} that is an average of the vectors is calculated (step S31).

In addition, the similarity calculation means 42 summarizes the video candidate feature vector V _spt (t) = [V ′ (t), V ′ in the video summary feature sequence series composed of L frame images at a certain time t. (T + 1),..., V ′ (t + L−1)], an average vector V _{spt_ave} (t) that is the average of the vectors is calculated (step S32).
Then, the similarity calculation means 42 _obtains the inner product of the vectors as the similarity S (t) in the average vector V _{ref_ave} calculated in step S31 and the average vector V _{spt_ave} (t) calculated in step S32 (step S33). ).

With the above operation, the similarity calculation means 42 calculates the similarity from the average vector of the program video feature vector and the average vector of the summary video candidate feature vector.
Here, the similarity is calculated by performing the inner product calculation of the vectors. However, the method is not limited to the vector inner product as long as it is a method for obtaining the similarity of the vectors. For example, in step S33, the similarity calculating unit 42 obtains a distance between vectors, and sets the distance to “0” so that the closer the distance is, the value is “0”, and the farther the value is “1”. It is good also as calculating | requiring a similarity by normalizing by ~ "1".

<Similarity Calculation Operation: Second Example>
Next, a second example of the similarity calculation operation performed by the similarity calculation unit 42 will be described with reference to FIG. 8 (refer to FIG. 2 as appropriate).
First, the similarity calculation means 42 initializes (“0”) a variable i indicating the number of loops (step S41).

Then, the similarity calculation means 42 includes element vectors V ′ (t), V of the summary video candidate feature vector V _spt (t) in the summary video candidate feature structure sequence composed of L frame images at a certain time t. ′ (T + 1),..., V ′ (t + L−1) i-th element vector V ′ (t + i) and a program video feature vector V _ref in a program video feature structure sequence composed of N _ref frame images. the element vector V (1), V (2 ), ..., V (N ref) each vector inner product of the s (i, 1), s (i, 2), ..., s a _{(i, N ref)} The maximum value s _max (i) is calculated (step S43).

Then, the similarity calculation means 42 adds “1” to the loop count i (step S44). Here, when the number of loops reaches the number (L) of element vectors of the summary video candidate feature vector (Yes in step S45), the similarity calculating unit 42 proceeds to step S46.
On the other hand, when the number of loops has not reached the number (L) of element vectors of the summary video candidate feature vector (No in step S45), the similarity calculation means 42 returns to step S42 and continues the operation.
Then, the similarity calculation means 42 calculates the average of the maximum values s _max (0), s _max (1),..., S _max (L−1) of the vector inner products sequentially calculated in step S43 as the similarity S (t ) (Step S46).

By the above operation, the similarity calculation means 42 obtains the most similar similarity between the element vector of the program video feature vector and the element vector of the summary video candidate feature vector, and averages them to obtain the entire vector. Calculate similarity.
Here, the similarity is calculated by performing the inner product calculation of the vectors. However, the method is not limited to the vector inner product as long as it is a method for obtaining the similarity of the vectors. For example, the similarity calculation means 42 obtains the distance between the vectors in step S42, as in the first example, so that the closer the distance is, the value is “0”, and the farther the value is “1”. The similarity may be obtained by normalizing the distance from “0” to “1”.

It is a schematic diagram for demonstrating the content of program organization. It is a block diagram which shows the structure of the accumulation | storage type | mold broadcast receiver which concerns on embodiment of this invention. It is a structure figure which shows typically the data structure of the data memorize | stored in the memory | storage means of the storage type broadcast receiver which concerns on embodiment of this invention. It is a block diagram which shows the structure of the accumulation | storage type | mold broadcast receiver which concerns on other embodiment of this invention. It is a flowchart which shows the accumulation | storage operation | movement of the accumulation | storage type | mold broadcast receiver which concerns on embodiment of this invention. It is a flowchart which shows the summary video reproduction | regeneration operation | movement of the storage-type broadcast receiver which concerns on embodiment of this invention. It is a flowchart which shows an example of the similarity calculation operation | movement which the similarity calculation means of the storage type | mold broadcast receiver which concerns on embodiment of this invention performs. It is a flowchart which shows the other example of the similarity calculation operation | movement which the similarity calculation means of the storage type broadcast receiver which concerns on embodiment of this invention performs.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1,1B Storage type broadcast receiver 10 Receiving means 20 Feature quantity generating means 21 Sampling means 22 Feature quantity calculating means 30 Storage means (video data storage means, feature quantity storage means)
40 summary video specifying means 41 feature quantity extracting means 41a program video feature quantity extracting means 41b non-program video feature quantity extracting means (summary video candidate feature quantity extracting means)
41c Program video partial feature amount extraction means (summary video candidate feature amount extraction means)
42 similarity calculation means 43 summary video section identification means 50 search list presentation means 51 list generation means 52 list selection means 53 summary video playback means 60 video playback means

Claims (4)

In a storage-type broadcast receiving apparatus for storing video data of a broadcast program and program information data including time information of the broadcast program in video data storage means and reproducing the broadcast program,
Sampling means for sampling the video data at predetermined frame intervals;
Feature quantity calculating means for calculating an image feature quantity for each frame image sampled by the sampling means;
Feature quantity storage means for storing the image feature quantity calculated by the feature quantity calculation means in association with the video data;
List generating means for generating a list of video data stored in the video data storage means based on the program information data;
A program video feature quantity extracting means for reading out an image feature quantity of video data selected by a viewer in the list from the feature quantity storage means;
Summary video candidate feature quantity extraction means for sequentially reading out image feature quantities of video sections for a predetermined summary video time from video data other than video data selected by the viewer in the list from the feature quantity storage means; ,
Similarity calculation means for calculating the similarity between the image feature quantity read by the program video feature quantity extraction means and the image feature quantity sequentially read by the summary video candidate feature quantity extraction means;
Based on the similarity calculated by the similarity calculation means, summary video section specifying means for specifying a video section for the summary video time most similar to the video data selected by the viewer;
Summary video playback means for reading out video data of the video section specified by the summary video section specifying means from the video data storage means and playing back as summary video;
A storage-type broadcast receiving apparatus comprising: The video data of the broadcast program includes first video data that is an entity of the broadcast program to which the program information data is added and second video data to which the program information data is not added.
The accumulation according to claim 1, wherein the summary video candidate feature amount extraction unit extracts an image feature amount of a video section corresponding to the summary video time from an image feature amount corresponding to the second video data. Type broadcast receiver. The video data of the broadcast program includes first video data that is an entity of the broadcast program to which the program information data is added and second video data to which the program information data is not added.
The summary video candidate feature amount extraction unit is configured to extract a video section corresponding to the summary video time from a time that is a predetermined time later than an end time specified by the program information data in the image feature amount corresponding to the first video data. The storage type broadcast receiving apparatus according to claim 1, wherein an image feature amount is extracted. Summary of a broadcast program selected by a viewer in a storage-type broadcast receiving apparatus that stores video data of a broadcast program and program information data including time information of the broadcast program in a video data storage unit and reproduces the broadcast program. In order to play the video,
Sampling means for sampling the video data at predetermined frame intervals;
Calculating an image feature amount for each frame image sampled by the sampling unit, and storing it in the feature amount storage unit in association with the video data;
List generating means for generating a list of video data stored in the video data storage means based on the program information data;
A program video feature quantity extraction unit for reading out an image feature quantity of video data selected by the viewer in the list from the feature quantity storage unit;
Among video data other than video data selected by the viewer in the list, summary video candidate feature quantity extraction means for sequentially reading out image feature quantities of video sections for a predetermined summary video time from the feature quantity storage means,
Similarity calculation means for calculating the similarity between the image feature quantity read by the program video feature quantity extraction means and the image feature quantity sequentially read by the summary video candidate feature quantity extraction means;
Summarized video section specifying means for specifying a video section for the summary video time most similar to the video data selected by the viewer based on the similarity calculated by the similarity calculating means;
Summary video playback means for reading out video data of the video section specified by the summary video section specifying means from the video data storage means and playing back as summary video,
A summary video playback program characterized by functioning as

Images