JP2007288608A - Method for preparing thumbnail and moving picture data reproducing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a thumbnail preparing method and a moving picture data reproducing apparatus where a highly probable image which is a feature picture of moving picture data can be surely extracted as a candidate of a thumbnail. <P>SOLUTION: The moving picture data reproducing apparatus is provided with: a picture group extractor 12 for extracting a picture group from a video stream including a plurality of I pictures encoded in frames, a plurality of P pictures predicted in a forward direction, and a plurality of B pictures predicted in bidirections; a first calculator 14 for calculating the data volume of at least one of the P picture and the B picture in each picture group; and a thumbnail candidate extractor/storage part 16 for selectively storing the I picture in the picture group, in which a feature value related to the data volume and corresponding to the movement of an image between frames reproduced in the picture group is minimum as a thumbnail candidate. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a thumbnail creation method and a moving image data reproduction device, and more particularly to a thumbnail creation method and a moving image data reproduction device capable of creating a thumbnail from moving image data compressed and encoded in a video stream format.

  Conventionally, various techniques have been proposed for extracting still images as thumbnails from moving image data.

  For example, Patent Document 1 discloses that one or a plurality of still images are extracted between a predetermined time before and after a CM (commercial message) portion of a moving image. Patent Document 2 discloses that a reduced image of a video signal is generated in synchronization with a timing signal periodically generated in a non-CM period. Patent Document 3 discloses that an attribute of a video signal (for example, a broadcasting station, a recording start time, etc.) is detected and a thumbnail image is generated based on the detected attribute.

A technique for detecting an image including characters has also been proposed. Patent Document 4 discloses detecting a character appearance frame from a plurality of frames constituting a moving image. Patent Document 5 discloses detecting a character area included in a moving image.
JP 2003-274361 A JP 2000-175133 A Japanese Patent Laid-Open No. 2004-15611 JP-A-4-316177 Japanese Patent Laid-Open No. 10-49682

  In the techniques disclosed in Patent Documents 1 to 3, thumbnails are extracted based on time, such as how many seconds after a certain reference point. However, the time at which a characteristic image appears varies depending on the program. For this reason, in the methods as described in Patent Documents 1 to 3, even if the techniques of Patent Documents 4 and 5 are used, it is not always possible to extract an optimal thumbnail.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to reliably extract an image that is likely to be a feature screen of moving image data as a thumbnail candidate. To provide a thumbnail creation method and a moving image data reproduction apparatus.

  A thumbnail creation method according to an aspect of the present invention includes a plurality of intra-frame-encoded pictures, a plurality of first-predicted pictures that are forward-predicted, and a plurality of second-predicted pictures that are bi-predicted. A method of creating a thumbnail from a video stream including: an extraction step of extracting a picture group from the video stream; and at least one of the first predicted picture and the second predicted picture in units of picture groups A first calculation step for calculating a data amount; and an intra-frame coded picture in a picture group that is related to the data amount and has a minimum feature value corresponding to the motion of an image between frames reproduced in the picture group And a first storage step for selectively storing as thumbnail candidates That.

Preferably, the feature value is a data amount.
Alternatively, the first calculation step includes a step of calculating a predicted picture number that is at least one of the first predicted picture and the second predicted picture in units of picture groups, and divides the data amount by the predicted picture number. It is preferable to further include a second calculation step, and the value calculated in the second calculation step is used as the feature value.

  Alternatively, the first calculation step includes a step of calculating a predicted picture number that is at least one of the first predicted picture and the second predicted picture in units of picture groups, and the data amount is divided by the predicted picture number. It is preferable that the method further includes a second calculation step of dividing the value by the number of pictures included in the picture group, and the value calculated by the second calculation step is used as the feature value.

  Preferably, in the first storing step, an intra-frame encoded picture in a picture group having a minimum feature value is stored as a thumbnail candidate.

  Alternatively, it is preferable that the extraction step extracts a picture group along the time axis, and the first storage step stores a minimum intra-frame encoded picture having a minimum feature value as a thumbnail candidate.

  Preferably, the first storage step includes a predetermined number of stored thumbnail candidates, and the feature value of the minimal intra-frame encoded picture is the largest of the feature values associated with the stored thumbnail candidates. When the feature value is equal to or larger than the feature value, a step of stopping the storing process of the minimum intra-frame coded picture is included.

  Preferably, the method further includes a display step of displaying a plurality of stored thumbnail candidates, and a second storage step of storing thumbnail candidates selected by the user among the plurality of thumbnail candidates as thumbnails.

  Preferably, the display step includes a step of displaying a plurality of thumbnail candidates in ascending order of feature values.

  Preferably, the extracting step includes a first intra-frame coded picture and a first intra-frame coded picture to a second intra-frame coded picture next to the first intra-frame coded picture. The thumbnail creation method according to claim 1, wherein the first predicted picture and the second predicted picture are extracted as a picture group.

  The moving image data reproducing device according to another aspect of the present invention includes a plurality of intra-frame encoded pictures, a plurality of first predicted pictures that are forward-predicted, and a plurality of second-predicted second pictures. Extraction means for extracting a picture group from a video stream including a prediction picture, and first calculation means for calculating the data amount of at least one of the first prediction picture and the second prediction picture in units of picture groups Intra-frame encoded pictures in a picture group that has a minimum feature value corresponding to the amount of data and corresponding to the motion of an image between frames reproduced in the picture group are selectively stored as thumbnail candidates. First storage means.

Preferably, the feature value is a data amount.
Alternatively, the first calculation means further calculates, for each picture group, a predicted picture number that is at least one of the first predicted picture and the second predicted picture, and divides the data amount by the predicted picture number. It is preferable that a second calculation unit is further provided, and the value calculated by the second calculation unit is a feature value.

  Alternatively, the first calculation means further calculates a predicted picture number that is at least one of the first predicted picture and the second predicted picture for each picture group, and calculates a value obtained by dividing the data amount by the predicted picture number. It is preferable that the image processing apparatus further includes second calculation means for dividing by the number of pictures included in the picture group, and the value calculated by the second calculation means is used as the feature value.

  Preferably, the first storage means stores an intra-frame coded picture in a picture group having a minimum feature value as a thumbnail candidate.

  Alternatively, it is preferable that the extraction unit extracts a picture group along the time axis, and the first storage unit stores a minimum intra-frame encoded picture having a minimum feature value as a thumbnail candidate.

  Preferably, the first storage means has a predetermined number of stored thumbnail candidates, and the feature value of the minimum intra-frame encoded picture is the largest of the feature values associated with the stored thumbnail candidates. If it is greater than or equal to the feature value, the storage process of the minimal intra-frame coded picture is stopped.

  Preferably, generating means for generating a signal for displaying a plurality of stored thumbnail candidates, and second storage means for storing thumbnail candidates selected by the user among the plurality of thumbnail candidates as thumbnails Is further provided.

  Preferably, the generation unit generates a signal for displaying a plurality of thumbnail candidates in ascending order of feature values.

  Preferably, the extracting means includes a first intra-frame encoded picture and a first intra-frame encoded picture to a second intra-frame encoded picture next to the first intra-frame encoded picture. One predicted picture and second predicted picture are extracted as a picture group.

  According to the present invention, it is possible to extract an intra-frame coded picture in a picture group with little image motion between reproduced frames. Thereby, a picture that is a feature of the moving image data can be extracted as a thumbnail candidate.

  Further, according to the present invention, it is not necessary to decode a picture when extracting a picture as a thumbnail candidate, so that the processing speed can be increased.

  Embodiments of the present invention will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals and description thereof will not be repeated.

  FIG. 1 is a block diagram showing a hardware configuration of moving image data reproducing apparatus 100 according to the embodiment of the present invention. The moving image data reproducing device 100 may be any device capable of reproducing moving image data compressed and encoded in a format such as MPEG (Moving Picture Experts Group), for example, a DVD (Digital Versatile Disc) recorder, an HDD (Hard disk drive) recorder. A personal computer (application), a mobile communication terminal such as a mobile phone, and the like are assumed.

  The video stream includes a plurality of intra-frame encoded pictures, a plurality of first-predicted pictures that are forward-predicted, and a plurality of second-predicted pictures that are bi-directionally predicted. including. In the present embodiment, it is assumed that the video stream is in the MPEG format. Therefore, hereinafter, an intra-frame coded picture will be described as an I picture, a first predicted picture as a P picture, and a second predicted picture as a B picture. The “picture” may also be referred to as a “frame”.

  Referring to FIG. 1, moving image data reproducing apparatus 100 includes a control unit 10 for performing overall control of moving image data reproducing apparatus 100, a memory 11 for storing programs and various data, and recorded moving images. Recorded data recording unit 20 for recording data (hereinafter also referred to as “content”), TV broadcast receiving unit 30 for receiving a television broadcast via an antenna 61, and picture data storage for storing picture data Unit 41, thumbnail candidate storage unit 45 for storing thumbnail candidates to be described later, display unit 32 for displaying thumbnail candidate images, etc., input unit 34 for receiving input of instructions from the user, And a drive device 36 for reading and writing data from the recording medium 62.

  The recorded data recording unit 20 includes a thumbnail data storage unit 21 that stores information related to thumbnails for each content. The data structure of the thumbnail data storage unit 21 will be described later.

  The display part 32 is comprised by LCD (Liquid Crystal Display), for example. The input unit 34 includes a plurality of keys. Note that the input unit 34 may be configured to receive a command / instruction by an infrared signal from a remote controller (not shown), convert the data into data, and provide the data to the control unit 10.

  FIG. 2 is a functional block diagram of the moving image data reproducing apparatus 100 according to the embodiment of the present invention.

  The control unit 10 includes a picture group extraction unit 12 for extracting a picture group from a video stream, and a data amount (total amount of data) and a number of B pictures representing difference data for each picture group. Calculated by the first calculation unit 14, the second calculation unit 15 for calculating the degree of change of the B picture for each picture group, and the second calculation unit 15 based on the calculation result by the first calculation unit 14. The thumbnail candidate extraction / storage unit 16 for extracting and storing thumbnail candidates based on the degree of change, and the sort unit 17 for rearranging the data stored in the thumbnail candidate storage unit 45 are included. Further, the control unit 10 decodes the thumbnail candidates from the one or more thumbnail candidates stored in the thumbnail candidate storage unit 45 and performs thumbnail registration processing, and displays the thumbnail candidates on the display unit 32. And a decoding processing unit 19 that generates a signal to be transmitted.

  The picture group extraction unit 12 determines the type of each picture (I picture, B picture, or P picture) by searching for the headers of the pictures constituting the video stream. When the I picture is detected, the I picture and the B picture and the P picture existing between the I picture and the next I picture in time are extracted as a picture group. The picture group may be a GOP (Group of Pictures) representing a group of one or more I pictures and a plurality of P / B pictures in an MPEG video stream.

  The picture group extraction unit 12 has a counter 2 for counting the number of the extracted picture group. Specific processing of the picture group extraction unit 12 will be described in detail later.

  In the present embodiment, the picture group extraction unit 12 will be described as extracting a picture group from recorded content, but the present invention is not limited to recorded content. For example, it may be extracted from moving image data being recorded. Alternatively, the content recorded on the recording medium 62 may be read by the drive device 36 and a picture group may be extracted from the read content.

  The second calculation unit 15 divides the data amount of the B picture calculated by the first calculation unit 14 by the number of B pictures calculated (counted) by the first calculation unit 14. The degree of change of the B picture is calculated for each picture group. Note that “degree of change” is an index that represents a change in image between frames reproduced in a picture group. In the present embodiment, for example, it represents an average of the data amount of B pictures.

  The thumbnail candidate extraction / storage unit 16 selectively extracts I pictures in a picture group in which the degree of change calculated by the second calculation unit 15 is minimal as thumbnail candidates. In the present embodiment, the following processing is performed to extract thumbnail candidates. That is, the thumbnail candidate extraction / storage unit 16 temporarily stores the value of the current degree of change and the current picture data in the picture data storage unit 41 when the current degree of change is smaller than the previous degree of change. Perform processing. Then, when it is detected that the degree of change corresponding to the temporarily stored picture data is a minimum value and a predetermined condition (as will be described later) is satisfied, it is stored in the picture data storage unit 41. The picture data is stored in the thumbnail candidate storage unit 45 as thumbnail candidates. Further, the thumbnail candidate extraction / storage unit 16 stores the value of the degree of change in the thumbnail candidate storage unit 45 in association with each thumbnail candidate.

  The sorting unit 17 performs a process of rearranging the thumbnail candidates stored in the thumbnail candidate storage unit 45 in ascending order of change value at a predetermined timing.

  The thumbnail registration unit 18 outputs the thumbnail candidates stored in the thumbnail candidate storage unit 45 to the decode processing unit 19. The decode processing unit 19 decodes the input thumbnail candidates and outputs them to the display unit 32. The thumbnail registration unit 18 registers information regarding the picture selected by the user via the input unit 34 among the thumbnail candidates displayed on the display unit in the thumbnail data storage unit 21 as content thumbnail information.

  The operation of each functional block shown in FIG. 2 may be realized by executing software stored in the memory 11, or at least one may be realized by hardware.

FIG. 3 is a diagram illustrating an example of the data structure of the thumbnail data storage unit 21.
The thumbnail data storage unit 21 stores management data A1, A2,..., Ak,. The management data Ak includes content program name (content name) data D1, recording date and time data D2, channel data D3, content data start address data D4, and thumbnail data D5.

  In the present embodiment, the thumbnail data D5 stores information on the thumbnails of the content to be managed, for example, one piece of reduced image data (thumbnail). Therefore, the thumbnail registration unit 18 reduces the picture selected by the user, and stores the reduced image data as thumbnail data D5. Two or more thumbnails may be stored. Here, although thumbnails are stored, picture data (not reduced) may be stored, or position information (for example, address data, time data) indicating the position of the thumbnail is stored. May be. In this example, the management data Ak for managing the content includes the thumbnail data D5. However, it is only necessary that each content is associated with the thumbnail information.

  FIG. 4 is a diagram showing an example of the data structure of the thumbnail candidate storage unit 45 in the embodiment of the present invention. This figure shows an example in which the number of thumbnail candidates that can be stored in the thumbnail candidate storage unit 45 (hereinafter referred to as “the number that can be stored”) is 30. Note that the storable number may be determined in advance at the time of shipment or may be determined by the user. Further, the memory amount necessary for storing the storable number is 8 bytes, the memory amount necessary for storing the degree of change (represented as “Dch_cd”) is 4 bytes each, and the address of the picture data (represented as “pPic_cd”). It is assumed that the amount of memory necessary for storage of 4 bytes is stored for each area, and the area reserved for picture data is each 0x10000 bytes.

  As shown in FIG. 4, the storable number is stored in area 451, and the picture data extracted as thumbnail candidates is stored in area 453. In the area 452, for each picture (thumbnail candidate) stored in the area 453, the change value and the address in the area 453 are stored in association with each other. The area 454 stores data that cannot be written to the area 453 when the amount of picture data exceeds 0x10000. Since processing when picture data cannot be written in the area 453 is known, detailed description thereof is omitted here.

  5 and 6 are flowcharts showing the flow of the main routine of the thumbnail candidate determination process in the embodiment of the present invention. The processing shown in the flowcharts of FIGS. 5 and 6 is stored in advance in the memory 11 as a program, and the control unit 10 reads out and executes this program, thereby realizing the thumbnail candidate determination processing function. It should be noted that the thumbnail candidate determination process described below may be executed in accordance with an instruction from the user, or may be automatically executed after the recording is completed (for example, after a predetermined time has elapsed).

The meanings of symbols used in this flowchart are as follows.
I (i): i-th I picture,
N: the number of the picture in the picture group (I picture is 0th) (= number of B pictures and P pictures included in the picture group),
Nb: the number of the B picture in the picture group (= the number of B pictures included in the picture group),
D: amount of picture data,
DB: the total amount of data of B pictures in the picture group,
Dch [i]: degree of change of the B picture in the picture group,
Cd_last: Number of storable (number of thumbnail candidates that can be stored in the thumbnail candidate storage unit 45).

  Referring to FIG. 5, first, picture group extraction unit 12 sets count value i of counter 2 to 0 (S2). Next, the control unit 10 executes initialization processing of the thumbnail candidate storage unit 45 (S4). A subroutine for initialization processing of the thumbnail candidate storage unit 45 is shown in FIG.

FIG. 7 is a flowchart showing the initialization process of the thumbnail candidate storage unit 45.
Referring to FIG. 7, control unit 10 sets counter variable c to 1 (S102). Next, it is determined whether or not the counter variable c is less than or equal to the storable number Cd_last, that is, 30 or less (S104). If it is determined that the counter variable c is equal to or less than the storable number Cd_last (YES in S104), the process proceeds to S106.

  In S106, the value of the change degree Dch_cd [c] in the region 452 is set to zero. Further, the value of the address pPic_cd [c] in the area 452 is set to “0x10000 × c”.

  When the process of S106 is completed, the counter variable c is incremented by 1, and the process returns to S102.

  If it is determined in S104 that the counter variable c has exceeded the storable number Cd_last (NO in S104), the process returns to the main routine.

  Referring to FIG. 5 again, when the initialization process of thumbnail candidate storage unit 45 is completed, control unit 10 determines whether or not the i-th I picture is the last I picture (S6). The determination as to whether or not it is the last I picture may be made by determining whether or not it is the last I picture of the content. For example, an I picture that is a predetermined time (for example, 10 minutes) after the start is determined as the last I picture. The I picture may be determined by determining whether or not the I picture is a predetermined time before the start. In many cases, the optimal image for the thumbnail appears at the beginning of the program. Therefore, thumbnail candidates can be extracted efficiently by staying within a predetermined time from the start.

  If it is determined in S6 that the i-th I picture is not the last I picture (NO in S6), the process proceeds to S8. On the other hand, when it is determined that the i-th I picture is the last I picture (YES in S6), the thumbnail candidate determination process is terminated.

  In S8, the picture group extraction unit 12 substitutes the value of the head address of the i-th I picture into the variable apsc. At the same time, the picture number n, the B picture number nb, and the B picture data amount DB are set to 0.

  Next, the picture group extraction unit 12 assigns a value (1: I picture, 2: P picture, 3: B picture) according to the type of the nth picture after the ith I picture to the variable pct. (S10). In addition, the picture group extraction unit 12 substitutes the value of the top address of the (n + 1) th picture after the i-th I picture into the variable apsc_next. Further, in S10, the data amount D of the nth picture is calculated by the difference between the value of the variable apsc and the value of the variable apsc_next.

  Next, the picture group extraction unit 12 determines whether or not the value of the variable pct is 3 (S12). That is, it is determined whether or not the nth picture is a B picture. If it is determined in S12 that the value of variable pct is 3 (YES in S12), the process proceeds to S18. If it is determined that the value of variable pct is not 3 (NO in S12), the process proceeds to S14.

  In S <b> 14, the picture group extraction unit 12 determines whether or not the value of the variable pct is 2. That is, it is determined whether or not the nth picture is a P picture. If it is determined that the value of variable pct is 2 (YES in S14), the process proceeds to S20. If it is determined that the value of variable pct is not 2 (NO in S14), the process proceeds to S16.

  In S16, the picture group extraction unit 12 determines whether or not the variable pct is 1 and the picture number n is not 0. That is, it is determined whether or not the nth picture is an I picture and the picture number n is not 0. If it is determined that this condition is not satisfied, that is, if it is determined that it is the 0th I picture (NO in S16), the process proceeds to S20. On the other hand, if it is determined that this condition is satisfied, that is, if it is determined that the I picture is the first or subsequent I picture (the I picture next to the i th I picture) (YES in S16), the process proceeds to S22. .

  In S18, the first calculation unit 14 increments the B picture number nb in the i-th picture group by one. Further, the first calculation unit 14 adds the data amount D of the nth picture to the data amount DB of the B picture in the i-th picture group. When the process of S18 ends, the process proceeds to S20.

  In S20, the picture group extraction unit 12 assigns the value of the variable apsc_next to the variable apsc, and increments the picture number n in the i-th picture group by 1. When the process of S20 ends, the process returns to S10.

  Thus, the first calculation unit 14 calculates the number of B pictures and the data amount in the i-th picture group.

  Referring to FIG. 6, in S22, second calculation unit 15 calculates a B picture change degree Dch [i]. Specifically, it is calculated by dividing the data amount DB of the B picture in the i-th picture group by the number nb of the B picture in the i-th picture group.

  Subsequently, the thumbnail candidate extraction / storage unit 16 determines that the count value i of the counter 2 is 0, or the count value i is 1 or more and the (previous) degree of change Dch [i−1] is (current time). It is determined whether the degree of change is greater than Dch [i] (S24). If it is determined that this condition is satisfied (YES in S24), the process proceeds to S26. On the other hand, when it is determined that this condition is not satisfied (NO in S24), the process proceeds to S28.

  In S26, the thumbnail candidate extraction / storage unit 16 sets the flag dec for identifying that the value of the degree of change is monotonously decreasing to 1, and temporarily stores the picture data. More specifically, the value Dch [i] and the data of the i-th I picture are overwritten and copied in the picture data storage unit 41. When the process of S26 ends, the process proceeds to S38.

  In S28, the thumbnail candidate extraction / storage unit 16 determines whether or not the degree of change Dch [i-1] is smaller than the degree of change Dch [i] and the flag dec is 1. That is, it is determined whether or not the change value has changed from monotonic decrease to monotonic increase. If it is determined that this condition is satisfied, that is, if it is determined that the value of the degree of change is a minimum value (YES in S28), the process proceeds to S30. On the other hand, if it is determined that this condition is not satisfied, that is, if it is determined that the value of the degree of change is not a minimum value (NO in S28), the process proceeds to S38.

  In S30, the thumbnail candidate extraction / storage unit 16 sets the flag dec to 0. Then, the thumbnail candidate extraction / storage unit 16 refers to the thumbnail candidate storage unit 45 and the value of the degree of change Dch_cd [Cd_last (= 30)] in the area 452 is 0 or the degree of change Dch [i −1] or not is determined (S32). If it is determined that this condition is satisfied (YES in S32), the process proceeds to S34. On the other hand, when it is determined that this condition is not satisfied (NO in S32), the process proceeds to S38.

  In S34, the thumbnail candidate extraction / storage unit 16 substitutes the value of the degree of change (Dch [i-1]) stored in the picture data storage unit 41 into the degree of change Dch_cd [Cd_last] of the thumbnail candidate storage unit 45. To do. At the same time, the data temporarily stored in the picture data storage unit 41 is fully stored in the thumbnail candidate storage unit 45. That is, the picture data stored in the picture data storage unit 41 is overwritten and copied to the picture data indicated by the address pPic_cd [Cd_last] in the thumbnail candidate storage unit 45.

  When the processing of S34 is completed, thumbnail candidate rearrangement processing is executed (S36). FIG. 8 shows a subroutine of thumbnail candidate rearrangement processing.

FIG. 8 is a flowchart showing thumbnail candidate rearrangement processing.
Referring to FIG. 8, the sorting unit 17 substitutes a value obtained by subtracting 1 from the storable number Cd_last for the counter variable c (S202). Next, the sorting unit 17 refers to the thumbnail candidate storage unit 45 and determines whether the counter variable c is greater than 0 and the value of the change degree Dch_cd [c] is 0 or greater than the change degree Dch_cd [c + 1]. It is determined whether or not (S204). If it is determined that this condition is satisfied (YES in S204), the process proceeds to S206.

  In S206, data replacement processing is performed. That is, the value of the degree of change Dc_cd [c] and the value of the degree of change Dc_cd [c + 1] are interchanged, and at the same time, the value of the address pPic_cd [c] and the value of pPic_cd [c + 1] are interchanged.

  When the processing of S206 ends, the sorting unit 17 decrements the counter variable c by 1 (S208). When the process of S208 ends, the process returns to S204.

  If it is determined in S204 that the above condition is not satisfied (NO in S204), the process returns to the main routine.

  By performing such rearrangement processing, the thumbnail candidates (addresses) in the thumbnail candidate storage unit 45 are in the descending order of change (Dch_cd [1] ≦ Dch_cd [2] ≦... Dch_cd [Cd_last]). It is rearranged to become.

  Referring to FIG. 6 again, when the thumbnail candidate rearrangement process ends, the process proceeds to S38.

  In S38, the picture group extraction unit 12 increments the count value i in the counter 2 by 1. When the process of S38 ends, the process returns to S6.

  By performing the thumbnail candidate determination process as described above, with compressed data such as MPEG using a picture group, in the case of a variable bit rate, an image between reproduced frames can be obtained without completely decoding the compressed data. It is possible to extract pictures in a picture group with little movement (change). This will be specifically described with reference to FIG. FIG. 9 shows an example in which the number of pictures included in the picture group is 15, and the period in which the I picture or P picture appears is 3. In this figure, the i-th I picture is represented by “Ii”, the P-pictures appearing in the i-th picture group (picture group including Ii) are “Pi1, Pi2, Pi3, Pi4” and the i-th picture in order. The B pictures appearing in the group are expressed as “Bi01, Bi02, Bi11, Bi12,... Bi42” in order.

  FIG. 9A shows the arrangement of frames along the time axis, and FIG. 9B shows the arrangement of data corresponding to FIG. 9A. Since a B picture represents a difference value between an I picture and a P picture, as described above, changes in all 15 frames in the i-th picture group can be detected by using the B picture.

  In the case of a video stream, the number of B pictures (and P pictures) may not be a fixed number. In the present embodiment, the degree of change of B pictures for each picture group, that is, data of B pictures for each picture group. Thumbnail candidates are extracted based on the average value. Therefore, thumbnail candidates can be extracted with high accuracy.

  As described above, according to the present embodiment, it is possible to automatically extract an I picture in a picture group with little image change between reproduced frames. An image with little change over a certain period of time is an indication of the intention of the producer that he / she wants to show to the viewer over a certain period of time, and thus is likely to be a feature screen of a program such as a program title or an opening screen. Therefore, according to the present embodiment, a picture suitable for a thumbnail is extracted.

  In the present embodiment, if the degree of change continues to decrease, it cannot become a thumbnail candidate until the degree of change is minimized (S28 in FIG. 6). Thereby, it is possible to prevent a similar picture from being extracted as a thumbnail candidate.

  In this embodiment, thumbnail candidates are extracted using B pictures. However, in order to simplify and / or speed up the processing, P pictures may be used instead of B pictures. Even if only the P picture is used, in the example shown in FIG. 9, it is possible to detect a change in the period of 12 frames out of 15 frames, and it is possible to detect a substantial change.

  Alternatively, considering that an irregular picture group (for example, a picture group including one I picture and two B pictures) appears, it is more preferable to use both B and P pictures. Therefore, the first calculation unit 14 calculates the number of B pictures and P pictures and the data amount of the B pictures and P pictures for each picture group, and the second calculation unit 15 is based on these values. Thus, the degree of change of the B picture and the P picture may be calculated. That is, the second calculation unit 15 divides the total data amount of the B picture and the P picture in the i th picture group by the number of the B picture and the P picture in the i th picture group, thereby obtaining the i th picture group. The degree of change in the picture group (the average value of the data amount of the B picture and the data amount of the P picture) may be calculated. The sum of the data amounts of the B picture and the P picture may be added by weighting as “a × (sum of the data amount of B picture) + b × (sum of the data amount of P picture)”.

  Alternatively, in consideration of the appearance of an irregular picture group, the degree of change may be calculated using another parameter (for example, the total number of pictures in the picture group). That is, the second calculation unit 15 divides the total amount of B picture data in the i th picture group by the number of B pictures in the i th picture group, and the number of pictures included in the i th picture group The degree of change in the i-th picture group may be calculated by dividing by. In this case, the calculation formula of the degree of change Dch [i] in S22 of FIG. 6 is “(DB / nb) / n”. Thus, by performing weighting according to the number of pictures included in a picture group, the accuracy of thumbnail candidate extraction can be further improved.

  Alternatively, in the present embodiment, thumbnail candidates are extracted based on the degree of change (for example, the average value of the data amount of B pictures), but at least corresponds to the motion of the image between frames reproduced in the picture group. It is only necessary to extract thumbnail candidates based on the value (feature value), and the degree of change is not limited. For example, for simplification of processing, thumbnail candidates may be extracted using only the data amount of B pictures and / or P pictures. In this case, the first calculation unit 14 calculates only the total data amount of the B picture and / or P picture in the i-th picture group. Then, the thumbnail candidate extraction / storage unit 16 selectively stores the i-th I picture as a thumbnail candidate in the thumbnail candidate storage unit 45 when the calculated total amount of data is minimized. In this case, the second calculation unit 15 is not necessary.

  As described above, in the present embodiment, the storable number is determined in advance. Therefore, it is possible to prevent an enormous number of thumbnail candidates from being registered. This will be specifically described with reference to FIG. This figure shows a graph in which the vertical axis represents the degree of change and the horizontal axis represents the number of picture groups. Here, for the sake of simplicity of explanation, it is assumed that there are three storable numbers.

  Referring to FIG. 10, in the case where the storable number is not provided, at the time point E, I pictures corresponding to all the minimum values A, B, C, D, and E are registered as thumbnail candidates. On the other hand, when a storable number is provided, I pictures corresponding to the minimum values B, C, and D are stored as thumbnail candidates at the time D, and correspond to the minimum values B, D, and E at the time E. The I picture is stored as a thumbnail candidate. In this way, at each time point, three thumbnail candidates are stored in ascending order of the minimum value. Depending on the scene change timing, the degree of change may increase suddenly as in Z, but the above method can be applied even when the graph does not draw a smooth curve.

  In the present embodiment, when the number of storable thumbnail candidates is already stored in the thumbnail candidate storage unit 45, even if the degree of change is minimal, the thumbnail candidate storage process is not necessarily performed. That is, the thumbnail candidate extraction / storage unit 16 stops the storage process to the thumbnail candidate storage unit 45 when the value of the degree of change is equal to or greater than the maximum value of the degree of change of the stored thumbnail candidates (thumbnail thumbnail display). It is not stored in the candidate storage unit 45). As a result, it is possible to prevent later inappropriate pictures from being stored as thumbnail candidates.

  In the above flowchart, even when the number of thumbnail candidates stored in the thumbnail candidate storage unit 45 is less than the storable number, the degree of change is equal to or greater than the maximum value of the degrees of change of the stored thumbnail candidates. (NO in S32 in FIG. 6), the storage process to the thumbnail candidate storage unit 45 is stopped. However, if the number of thumbnail candidates stored in the thumbnail candidate storage unit 45 is less than the storable number, the main saving of the picture data (S34 in FIG. 6) is performed without performing the determination in S32 of FIG. It is good.

  FIG. 11 is a flowchart showing thumbnail registration processing according to the embodiment of the present invention. The process shown in the flowchart of FIG. 11 is also stored in advance in the memory 11 as a program, and the function of the thumbnail registration process is realized by the controller 10 reading and executing this program.

  Referring to FIG. 11, thumbnail registration unit 18 reads out thumbnail candidates of pages to be displayed on display unit 32 from thumbnail candidate storage unit 45 (S302). More specifically, the picture data stored in area 453 is read in the order of the addresses stored in area 452. Thereby, thumbnail candidates are read out in ascending order of change. Assuming that the number of pictures that can be displayed on one page (screen) in the display unit 32 is six, for example, the thumbnail registration unit 18 reads picture data in units of six. The read picture data is output to the decoding processing unit 19.

  Next, the decoding processing unit 19 decodes the thumbnail candidate picture data and displays the decoded image on the display unit 32 (S304).

  As described above, the thumbnail registration unit 18 and the decoding processing unit 19 generate signals for displaying the thumbnail candidates on the display unit 32 in ascending order of change.

  An example of the thumbnail candidate list screen displayed in S304 is shown in FIG. FIG. 12 is a diagram illustrating an example of a thumbnail candidate list screen. Referring to FIG. 12, one or more thumbnail candidates are displayed in a predetermined area 51 of display unit 32. At the same time, a cursor key 52 for selecting one picture (image) from the displayed thumbnail candidates is displayed in a predetermined area. Further, a page switching key 53 for switching pages and a determination key 54 for determining a picture to be a thumbnail are displayed in predetermined areas, respectively.

  Referring to FIG. 11 again, the thumbnail registration unit 18 waits for an input from the user (S306). If there is an input from the user and the input data from the user is “input P”, it is determined whether or not the input P is one of the cursor keys 52 (S308). If it is determined that the input P is one of the cursor keys 52 (YES in S308), the process proceeds to S310. On the other hand, when it is determined that the input P is not one of the cursor keys 52 (NO in S308), the process proceeds to S312.

  In S310, the thumbnail registration unit 18 moves the displayed cursor as much as possible, and returns to S306 again.

  In S312, it is determined whether or not the input P is the enter key 54. When it is determined that the input P is the enter key 54 (YES in S312), the process proceeds to S320. On the other hand, when it is determined that the input P is not the enter key 54 (NO in S312), the process proceeds to S314.

  In S314, it is determined whether or not the input P is an input for interrupting this flow. When it is determined that the input P is not an input for interrupting this flow (NO in S314), the process proceeds to S316.

  In S316, it is determined whether or not the input P is the page switching key 53 and page switching is possible. When it is determined that this condition is satisfied (YES in S316), thumbnail registration unit 18 switches pages (S318). When the process of S318 ends, the process returns to S302.

  If it is determined in S316 that the input P is the page switching key 53 and that page switching is not possible (NO in S316), the process returns to S306.

  If it is determined in S314 that the input P is an input for interrupting this flow (YES in S314), the thumbnail registration process is terminated.

  As described above, the moving image data playback apparatus 100 according to the embodiment of the present invention automatically extracts and stores thumbnail candidates, but entrusts the registration of actual thumbnails to the user. Thereby, it is possible to prevent a frame that is not intended by the user from being registered as a thumbnail.

  Further, according to the present embodiment, the thumbnail candidates are rearranged in ascending order of change by the sorting unit 17 in the thumbnail candidate determination process described above. Thumbnail candidates can be displayed in ascending order. This makes it easier for the user to select a thumbnail. When the thumbnail candidates are presented to the user, the thumbnail registration unit 18 and the decoding processing unit 19 may generate a signal for displaying the thumbnail candidates in the order of the small degree of change. You may decide not to rearrange. In this case, in the thumbnail registration process, the thumbnail registration unit 18 may rearrange the thumbnail candidates in ascending order of change, and the decode processing unit 19 may decode the thumbnail candidates in the rearranged order.

  Alternatively, in the thumbnail candidate determination process, all I pictures in a picture group having a minimum change degree may be stored. In this case, in the thumbnail candidate determination process, the top 30 thumbnail candidates may be presented to the user in ascending order of change.

  As described above, in the present embodiment, the picture group is extracted along the time axis. However, the picture group is not necessarily extracted along the time axis. For example, a plurality of (for example, all) picture groups may be extracted at once from the recorded content, and for example, 30 thumbnail candidates may be registered in ascending order of the minimum value. Alternatively, only one I picture in the picture group having the smallest change degree may be registered as a thumbnail candidate.

  In the present embodiment, the degree of change is calculated every time a picture group is extracted, but the present invention is not necessarily limited to such an example. For example, every other degree of change may be calculated, or the degree of change may be calculated for each of a plurality of picture groups.

  In addition, the thumbnail candidate determination method and the thumbnail registration method performed by the moving image data reproduction apparatus 100 of the present invention can be provided as a program. Such a program can be recorded on an optical medium such as a CD-ROM (Compact Disc-ROM) or a computer-readable recording medium 62 such as a memory card and provided as a program product. A program can also be provided by downloading via a network.

  The provided program product is installed in a program storage unit such as a hard disk and executed. The program product includes the program itself and a recording medium on which the program is recorded.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

It is a block diagram which shows the hardware constitutions of the moving image data reproduction apparatus in embodiment of this invention. It is a functional block diagram of the moving image data reproducing device in the embodiment of the present invention. It is a figure which shows an example of the data structure of the thumbnail data storage part in embodiment of this invention. It is a figure which shows an example of the data structure of the thumbnail candidate memory | storage part in embodiment of this invention. It is a flowchart which shows the flow of the main routine of the thumbnail candidate determination process in embodiment of this invention. It is a flowchart which shows the flow of the main routine of the thumbnail candidate determination process in embodiment of this invention. It is a flowchart which shows the initialization process of the thumbnail candidate memory | storage part in embodiment of this invention. It is a flowchart which shows the thumbnail candidate rearrangement process in embodiment of this invention. (A) is a figure which shows the arrangement | sequence of the frame along a time-axis, (B) is a figure which shows the arrangement | sequence of the data corresponding to (A). It is a figure for demonstrating the memory | storage process of the thumbnail candidate in embodiment of this invention. It is a flowchart which shows the thumbnail registration process in embodiment of this invention. It is a figure which shows an example of a thumbnail candidate list screen.

Explanation of symbols

  2 counter, 10 control unit, 11 memory, 12 picture group extraction unit, 14 first calculation unit, 15 second calculation unit, 16 thumbnail candidate extraction / storage unit, 17 sort unit, 18 thumbnail registration unit, 19 decoding process , 20 Recording data recording unit, 21 Thumbnail data storage unit, 30 TV broadcast receiving unit, 32 Display unit, 34 input unit, 36 Drive device, 41 Picture data storage unit, 45 Thumbnail candidate storage unit, 61 Antenna, 62 Recording medium , 100 video data playback device.

Claims (20)

To create a thumbnail from a video stream including a plurality of intra-frame encoded pictures, a plurality of forward-predicted first predicted pictures, and a plurality of bi-predicted second predicted pictures A thumbnail creation method for
An extraction step of extracting a picture group from the video stream;
A first calculation step of calculating a data amount of at least one of the first predicted picture and the second predicted picture in units of the picture group;
The intra-frame coded picture in the picture group that has a minimum feature value corresponding to the amount of data and corresponding to the motion of an image between frames reproduced in the picture group is selectively selected as a thumbnail candidate. And a first storage step for storing the thumbnail creation method.   The thumbnail creation method according to claim 1, wherein the feature value is the data amount. The first calculating step includes a step of calculating a predicted picture number that is at least one of the first predicted picture and the second predicted picture in the picture group unit;
A second calculation step of dividing the amount of data by the predicted number of pictures;
The thumbnail creation method according to claim 1, wherein the value calculated in the second calculation step is used as the feature value. The first calculating step includes a step of calculating a predicted picture number that is at least one of the first predicted picture and the second predicted picture in the picture group unit;
A second calculation step of dividing the value obtained by dividing the amount of data by the number of predicted pictures by the number of pictures included in the picture group;
The thumbnail creation method according to claim 1, wherein the value calculated in the second calculation step is used as the feature value.   5. The thumbnail creation method according to claim 2, wherein the first storing step stores an intra-frame encoded picture in the picture group having the minimum feature value as the thumbnail candidate. 6. The extracting step extracts the picture group along a time axis;
5. The thumbnail creation method according to claim 2, wherein the first storing step stores a minimal intra-frame coded picture having a minimum feature value as the thumbnail candidate. 6.   In the first storing step, a predetermined number of stored thumbnail candidates exist, and the feature value of the minimal intra-frame encoded picture is the largest of the feature values associated with the stored thumbnail candidates. The thumbnail creation method according to claim 6, further comprising a step of stopping the storing process of the minimal intra-frame encoded picture when the feature value is equal to or greater than the feature value. A display step of displaying a plurality of stored thumbnail candidates;
The thumbnail creation method according to claim 6, further comprising: a second storage step of storing a thumbnail candidate selected by a user among the plurality of thumbnail candidates as the thumbnail.   The thumbnail creation method according to claim 8, wherein the displaying step includes a step of displaying the plurality of thumbnail candidates in ascending order of the feature values.   The extraction step includes a first intra-frame coded picture and a first intra-frame coded picture to a second intra-frame coded picture next to the first intra-frame coded picture. The thumbnail creation method according to claim 1, wherein one predicted picture and a second predicted picture are extracted as the picture group. A picture group is extracted from a video stream including a plurality of intra-frame encoded pictures, a plurality of forward-predicted first predicted pictures, and a plurality of bi-predicted second predicted pictures. Extraction means for,
First calculation means for calculating a data amount of at least one of the first predicted picture and the second predicted picture in units of the picture group;
The intra-frame coded picture in the picture group that has a minimum feature value corresponding to the amount of data and corresponding to the motion of an image between frames reproduced in the picture group is selectively selected as a thumbnail candidate. A moving image data reproducing apparatus comprising: first storage means for storing.   The moving image data reproducing apparatus according to claim 11, wherein the feature value is the data amount. The first calculation means further calculates a predicted picture number that is at least one of the first predicted picture and the second predicted picture in units of the picture group,
A second calculating means for dividing the amount of data by the predicted number of pictures;
12. The moving image data reproducing device according to claim 11, wherein the value calculated by the second calculating unit is used as the feature value. The first calculation means further calculates a predicted picture number that is at least one of the first predicted picture and the second predicted picture in units of the picture group,
A second calculation means for dividing a value obtained by dividing the amount of data by the number of predicted pictures by the number of pictures included in the picture group;
12. The moving image data reproducing device according to claim 11, wherein the value calculated by the second calculating unit is used as the feature value.   The moving image data reproduction device according to any one of claims 12 to 14, wherein the first storage unit stores an intra-frame encoded picture in the picture group having the minimum feature value as the thumbnail candidate. . The extraction means extracts the picture group along a time axis,
The moving image data reproduction device according to any one of claims 12 to 14, wherein the first storage unit stores a minimum intra-frame encoded picture having the minimum feature value as the thumbnail candidate.   The first storage means includes a predetermined number of stored thumbnail candidates, and a feature value of the minimal intra-frame encoded picture is a maximum of feature values associated with the stored thumbnail candidates. The moving image data reproduction device according to claim 16, wherein when the feature value is equal to or larger than the feature value, the storage processing of the encoded picture within the minimum frame is stopped. Generating means for generating a signal for displaying the plurality of stored thumbnail candidates;
The moving image data reproduction device according to claim 16, further comprising: a second storage unit for storing a thumbnail candidate selected by a user among the plurality of thumbnail candidates as the thumbnail.   The moving image data reproduction device according to claim 18, wherein the generation unit generates a signal for displaying the plurality of thumbnail candidates in ascending order of the feature values.   The extracting means includes a first intra-frame coded picture and a first intra-frame coded picture to a second intra-frame coded picture next to the first intra-frame coded picture. The moving image data reproduction device according to claim 11, wherein one predicted picture and a second predicted picture are extracted as the picture group.

Images