JP2007201643A - Thumbnail generating method, and moving picture data reproducing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a thumbnail generating method and a moving picture data reproducing apparatus capable of surely extracting an image being a characteristic image of moving picture data with high possibility as a candidate of a thumbnail. <P>SOLUTION: The thumbnail generating method for generating a thumbnail from moving picture data including a plurality of frames includes: an extraction step of extracting a plurality of frame groups from the moving picture data; a step (S108) of comparing the frames in the unit of pixels by each extracted frame group and counting the number of coincident pixels; and a first storage step (S132) for selectively storing an optional first frame as the thumbnail candidate from a frame group wherein the count value of the number of coincident pixels is at least a maximum. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a thumbnail creation method and a moving image data reproduction device, and more particularly to a thumbnail creation method and a moving image data reproduction device that can create a thumbnail from moving image data.

  Conventionally, various techniques have been proposed for extracting still images as thumbnails from moving image data.

  For example, Patent Document 1 discloses that one or a plurality of still images are extracted between a predetermined time before and after a CM (commercial message) portion of a moving image. Patent Document 2 discloses that a reduced image of a video signal is generated in synchronization with a timing signal periodically generated in a non-CM period. Patent Document 3 discloses that an attribute of a video signal (for example, a broadcasting station, a recording start time, etc.) is detected and a thumbnail image is generated based on the detected attribute.

A technique for detecting an image including characters has also been proposed. Patent Document 4 discloses detecting a character appearance frame from a plurality of frames constituting a moving image. Patent Document 5 discloses detecting a character area included in a moving image.
JP 2003-274361 A JP 2000-175133 A Japanese Patent Laid-Open No. 2004-15611 JP-A-4-316177 Japanese Patent Laid-Open No. 10-49682

  In the techniques disclosed in Patent Documents 1 to 3, thumbnails are extracted based on time, such as how many seconds after a certain reference point. However, the time at which a characteristic image appears varies depending on the program. For this reason, in the methods as described in Patent Documents 1 to 3, even if the techniques of Patent Documents 4 and 5 are used, it is not always possible to extract an optimal thumbnail.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to reliably extract an image that is likely to be a feature screen of moving image data as a thumbnail candidate. To provide a thumbnail creation method and a moving image data reproduction apparatus.

  A thumbnail creation method according to an aspect of the present invention is a thumbnail creation method for creating a thumbnail from moving image data including a plurality of frames, an extraction step of extracting a plurality of frame groups from the moving image data, and an extracted step For each frame group, a frame is compared in units of pixels, the number of matching pixels is counted, and an arbitrary first frame is selectively selected as a thumbnail candidate from the frame group in which the count value of the matching pixel count is at least maximum. And a first storage step for storing.

Thereby, an image with little pixel change is extracted as a thumbnail candidate.
Preferably, each frame group consists of frames having a constant interval.

The extracting step includes a step of sequentially extracting a group of frames along the time axis.
Preferably, the first storing step includes a step of storing the first frame as a thumbnail candidate when the count value is maximum and exceeds the determination reference value, and the count value is the maximum value. In the case, an update step of updating the determination reference value with the maximum value is further provided.

  Preferably, the updating step includes a step of excluding updating of the determination reference value when the count value exceeds a predetermined threshold value.

  Preferably, the first storing step further includes a step of storing an arbitrary second frame as a thumbnail candidate from a frame group in which the count value exceeds the determination reference value.

  Preferably, the method further includes a step of displaying a plurality of stored thumbnail candidates, and a second storage step of storing information on an image selected by the user from among the plurality of thumbnail candidates as thumbnail information of the moving image data. .

  Preferably, the first storing step includes a step of storing the thumbnail candidates in association with the count value every time the count value becomes maximum.

  Preferably, the plurality of stored thumbnail candidates are rearranged in descending order of the count value associated with each thumbnail candidate, the plurality of rearranged thumbnail candidates are displayed, and the user among the plurality of thumbnail candidates And a second storage step for storing the information related to the image selected in step (b) as thumbnail information of the moving image data.

Preferably, each frame group includes three frames.
Preferably, the step of counting includes a step of incrementing the number of matching pixels when the color components are the same among the pixels.

  Preferably, the step of counting includes a step of incrementing the number of matching pixels when the difference value of the color component between the pixels is less than a predetermined threshold value.

  According to another aspect of the present invention, there is provided a moving image data reproducing device that extracts a plurality of frame groups from moving image data including a plurality of frames, and compares the frames in units of pixels for each of the extracted frame groups. Counting means for counting the number of matching pixels, and first storage means for selectively storing an arbitrary first frame as a thumbnail candidate from a frame group in which the count value by the counting means is at least maximum With.

Thereby, an image with little pixel change is extracted as a thumbnail candidate.
Preferably, each frame group consists of frames having a constant interval.

Preferably, the extraction unit sequentially extracts the frame group along the time axis.
Preferably, the first storage means stores the first frame as a thumbnail candidate when the count value is maximal and exceeds the determination reference value, and when the count value is the maximum value, Update means for updating the determination reference value with the maximum value is further provided.

  Preferably, the updating unit includes an excluding unit for excluding updating of the determination reference value when the count value exceeds a predetermined threshold value.

Preferably, the first storage unit further stores an arbitrary second frame as a thumbnail candidate from a frame group in which the count value exceeds the determination reference value.

  Preferably, display means for displaying a plurality of thumbnail candidates stored in the first storage means, and information related to an image selected by the user among the plurality of thumbnail candidates is stored as thumbnail information of the moving image data. And a second storage means.

  Preferably, the first storage unit stores the thumbnail candidates in association with the count value every time the count value reaches a maximum.

  Preferably, a plurality of thumbnail candidates stored in the first storage unit and a rearrangement unit for rearranging the thumbnail candidates in descending order of count values associated with the thumbnail candidates, and the plurality of rearranged thumbnail candidates are displayed. Display means, and second storage means for storing information on an image selected by the user from among the plurality of thumbnail candidates as thumbnail information of the moving image data.

Preferably, each frame group includes three frames.
Preferably, the counting means increments the number of matching pixels when the color components are the same among the pixels.

  Preferably, the counting unit increments the number of matching pixels when the difference value of the color component between the pixels is less than a predetermined threshold value.

  According to the present invention, an image with little pixel change is detected for a certain period of time. As a result, an image that is a feature of the moving image data can be extracted as a thumbnail candidate.

  Embodiments of the present invention will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals and description thereof will not be repeated.

[Embodiment 1]
FIG. 1 is a block diagram showing a hardware configuration of moving image data reproducing apparatus 100 according to Embodiment 1 of the present invention.

  Referring to FIG. 1, moving image data reproducing apparatus 100 includes a control unit 10 for performing overall control of moving image data reproducing apparatus 100, a memory 11 for storing programs and various data, and recorded moving image data. (Hereinafter referred to as “content”), a recorded data recording unit 20, a TV broadcast receiving unit 30 for receiving a television broadcast via an antenna 61, and a first frame data storage for storing frame data Unit 41, second frame data storage unit 42, third frame data storage unit 43, fourth frame data storage unit 44, thumbnail candidate storage unit 45 for storing thumbnail candidates to be described later, and thumbnail candidates for display. Display unit 32, input unit 34 for accepting input of instructions from the user, and reading of data from the recording medium 62 And a drive unit 36 for performing fine writing.

  The recorded data recording unit 20 includes a thumbnail data storage unit 21 that stores information related to thumbnails for each content. The data structure of the thumbnail data storage unit 21 will be described later.

The display part 32 is comprised by LCD (Liquid Crystal Display), for example. The input unit 34 includes a plurality of keys. The input unit 34 may be configured to receive a command / instruction by an infrared signal from a remote controller (not shown), convert the data into data, and supply the data to the control unit 10.

  The playback device 100 may be any device that can play back moving image data, and is assumed to be a portable communication terminal such as a DVD (Digital Versatile Disc) recorder, HDD (Hard disk drive) recorder, personal computer (application), or mobile phone. Is done.

  FIG. 2 is a functional block diagram of the moving image data reproducing apparatus 100 according to Embodiment 1 of the present invention.

  A frame extraction unit 12 for extracting a plurality of frame groups from a content including a plurality of frames, and image detection for detecting an image with little pixel change by comparing frame data for each extracted frame group in units of pixels. And a thumbnail registration unit 18 for performing a thumbnail registration process from among one or more thumbnail candidates.

  The frame extraction unit 12 extracts a plurality of frame groups from the content recorded in the recorded data recording unit 20. It is preferable that the frame extraction unit 12 extracts a frame group including three frame data having a fixed interval from each other. For this purpose, the frame extraction unit 12 has a frame counter 2 for counting the interval of frame data. The frame extraction unit 12 stores the extracted three frame data in the first frame data storage unit 41, the second frame data storage unit 42, and the third frame data storage unit 43, respectively.

  In the present embodiment, the frame extraction unit 12 will be described as extracting a plurality of frame groups from the recorded content. However, a plurality of frame groups may be extracted from the content being recorded. Alternatively, the content recorded on the recording medium 62 may be read by the drive device 36, and a plurality of frame groups may be extracted from the read content.

  The image detection unit 14 compares the frame data for each frame group extracted by the frame extraction unit 12 and counts the number of matching pixels, and the count value by the count unit 4 The thumbnail candidate extraction / storage unit 6 for extracting / storing thumbnail candidates and the update unit 8 for updating the determination reference value are provided.

  More specifically, the counting unit 4 uses the three frame data stored in the first frame data storage unit 41, the second frame data storage unit 42, and the third frame data storage unit 43 as pixels. The unit is compared and the number of matching pixels is counted.

  The thumbnail candidate extraction / storage unit 6 selectively extracts frame data as thumbnail candidates from a frame group in which the count value of the number of matching pixels counted by the counting unit 4 is at least maximum. In the present embodiment, the following processing is specifically performed in order to extract thumbnail candidates. That is, the thumbnail candidate extraction / storage unit 6 first stores the frame data stored in the first frame data storage unit 41 in the fourth frame data storage unit 44 from the frame group whose count value exceeds a determination reference value described later. A temporary storage process is performed. Then, when it is detected that the count value corresponding to the temporarily stored frame data is the maximum value, the thumbnail data is obtained by reducing the image data obtained by reducing the frame data stored in the fourth frame data storage unit 44 as a thumbnail candidate. The process which memorize | stores in the candidate memory | storage part 45 is performed.

In the present embodiment, the reduced image data of the frame data having the maximum count value will be described as being stored in the thumbnail candidate storage unit 45, but the image data (frame data) before the reduction is stored. Also good.

  The update unit 8 updates the determination reference value with the maximum value when the count value is equal to or less than the predetermined threshold value, and excludes the update of the determination reference value when the count value exceeds the predetermined threshold value.

  The thumbnail registration unit 18 performs processing for displaying the thumbnail candidates stored in the thumbnail candidate storage unit 45 on the display unit 32. The image selected by the user via the input unit 34 is registered in the thumbnail data storage unit 21.

  The operation of each functional block may be realized by executing software stored in the memory 11, or at least a part thereof may be realized by hardware.

FIG. 3 is a diagram illustrating an example of the data structure of the thumbnail data storage unit 21.
In the thumbnail data storage unit 21, management data A1, A2,..., Ak,. The management data Ak includes content program name (content name) data D1, recording date and time data D2, channel data D3, content data start address data D4, and thumbnail data D5.

  In the present embodiment, one thumbnail image data (hereinafter referred to as “thumbnail data”) is stored in the thumbnail data D5. Two or more thumbnail data may be stored. Here, image data is stored, but position information (for example, address data, time data) representing the image data may be stored. In this example, the thumbnail data D5 is included in the management data Ak for managing the content. However, it is only necessary that each content and the thumbnail data are associated with each other.

  FIG. 4A is a diagram showing the coordinates of each pixel of the frame data according to the first embodiment of the present invention. When the frame data is 800 × 600 image data, the X coordinate is composed of 0 to 799, and the Y coordinate is composed of 0 to 599. In this embodiment, when the coordinates of each pixel are represented by “(x, y)” and “n = x + 800 × y”, (x, y) is referred to as the nth pixel.

  FIG. 4B is a diagram illustrating an example of the data structure of the first to third frame data storage units 41 to 43. In this figure, an example in which one pixel is a 32-bit image is shown.

  The color component of each pixel is represented as “color component (coordinates)”. In the present embodiment, it is assumed that the data of one pixel is composed of “R component”, “G component”, “B component”, and “empty”.

FIG. 5 is a diagram showing the concept of frame extraction processing in Embodiment 1 of the present invention.
With reference to FIG. 5, the frame extraction unit 12 extracts a plurality of frame groups along the time axis. In this embodiment, a frame group consisting of three frame data having an interval of 15 frames is extracted. Specifically, a frame group consisting of frame data whose count numbers in the frame counter 2 are 0, 15, and 30 is extracted and a first comparison is performed. Next, the frame extraction unit 12 extracts a frame group including frame data whose count numbers in the frame counter 2 are 1, 16, and 31, and performs a second comparison. Similarly, a frame group consisting of frame data whose count numbers in the frame counter 2 are 2, 17, and 32 is extracted, and a third comparison is performed.

As described above, in this embodiment, pixel values are compared not between consecutive frames but between frames having a fixed interval such as an interval of 15 frames. Assuming that the time difference between consecutive frames is 1/30 second, by setting the interval to 15 frames, it is possible to detect pixels that are expected to have the same pixel value for a certain period (1/2 second) (continuous). If the comparison is performed between frames, the number of pixels having the same pixel value increases).

  In the present embodiment, comparison is made between three frame data, not between two frame data. Thereby, since it is detected whether it is the same pixel value for 1 second, the detection precision can be improved.

  In the present embodiment, the comparison is made at intervals of 15 frames, but is not limited to such intervals. Alternatively, in this embodiment, the comparison is performed using three frames, but the comparison may be performed using two frames or four or more frames. Alternatively, in this embodiment, the second comparison is performed after one frame after the first comparison, but the second comparison is performed after two or more frames are advanced after the first comparison. You may do it. In this case, the same applies to the third and subsequent comparisons.

  In the present embodiment, a plurality of frame groups are extracted along the time axis. However, they may not necessarily be extracted along the time axis. For example, a plurality of frame groups may be extracted at once from the recorded content, and the following processing may be performed.

  FIG. 6 is a flowchart showing a flow of a main routine of processing (hereinafter referred to as “thumbnail candidate determination processing”) executed in the control unit 10 of the moving image data reproducing apparatus 100 according to the first embodiment of the present invention. The process shown in the flowchart of FIG. 6 is stored in advance in the memory 11 as a program, and the thumbnail candidate determination process is realized by the control unit 10 reading and executing this program. It should be noted that the thumbnail candidate determination process described below may be executed in accordance with an instruction from the user, or may be automatically executed after the recording is completed (for example, after a predetermined time has elapsed).

  Referring to FIG. 6, first, frame extraction unit 12 performs an initialization process (step S (hereinafter abbreviated as “S”) 2). In the following description, it is assumed that the number of frame intervals is “i”, the count number in the frame counter 2 is “fc”, and the frame represented by the count number fc is “F [fc]”.

  In S2, the frame extraction unit 12 substitutes 2 * i for the count number fc (constant i = 15). Also, the lower limit value of the number of matched pixels used for the thumbnail candidate is substituted for the determination reference value “max”. The lower limit value may be determined in advance at the time of shipment or may be determined by the user.

  If the threshold value (lower limit value) at which the determination reference value max is not updated is “max_max”, the value of the threshold value max_max is substituted into the maximum value storage variable “over_max” that is equal to or greater than the threshold value max_max. Here, the threshold max_max may be determined in advance at the time of shipment or may be determined by the user.

  In addition, a flag “max_lock” indicating whether or not to update the determination reference value max is set to 0. When the flag max_lock is 0, it indicates that the determination reference value max can be updated, and when the flag max_lock is 1, it indicates that the determination reference value max cannot be updated.

  Further, the number of matching pixels “sk [−1]” one frame before is set to zero. The “number of matching pixels one frame before” represents the number of matching pixels of the three frame data in the immediately preceding frame group.

  When the initialization process of S2 is completed, the frame extraction unit 12 writes the frame data of F [fc-2 * i] into the first frame data storage unit 41 (S4). Subsequently, the frame extraction unit 12 writes the frame data of F [fc-i] into the second frame data storage unit 42 (S6). Furthermore, the frame extraction unit 12 writes the frame data of F [fc] into the third frame data storage unit 43 (S8).

  When the frame extraction process of S4 to S8 is completed, a thumbnail candidate extraction / storage process is executed (S10). The thumbnail candidate extraction / storage process of S10 will be described later with a subroutine.

  When the process of S10 ends, the frame extraction unit 12 determines whether or not the frame with the count number fc is the last frame (S12). The determination as to whether or not it is the final frame may be made by determining whether or not it is the final frame of the content. For example, a frame that is a predetermined time (for example, 10 minutes) ahead of the start is used as the final frame, The determination may be made by determining whether the frame is ahead of time. In many cases, the optimal image for the thumbnail appears at the beginning of the program. Therefore, thumbnail candidates can be extracted efficiently by staying within a predetermined time from the start.

  If it is determined in S12 that the frame having the count number fc is not the last frame (NO in S12), the count number fc of the frame counter 2 is incremented by 1 (S14). When the process of S14 ends, the process returns to S4. On the other hand, when it is determined that the frame with count number fc is the final frame (YES in S12), this series of processes is terminated.

  FIG. 7 is a flowchart showing thumbnail candidate extraction / storage processing according to Embodiment 1 of the present invention.

  First, the image detection unit 14 performs an initialization process (S102). Specifically, the address reference counter j is set to zero. Also, the matching pixel count counting variable sk [0] is set to zero. In the following description, a threshold (constant) regarded as the same value is represented as “eq”, and a byte size variable for storing color components is represented as “rx”, “gx”, “bx” (x∈1, 2, 3). .

  Next, 4 * j, 4 * j + 1, 4 * j + 2nd byte data of the first frame data storage unit 41 are assigned to variables r1, g1, and b1, respectively. Similarly, each data from the second frame data storage unit 42 is substituted for the variables r2, g2, and b2, and each data from the third frame data storage unit 43 is substituted for r3, g3, and b3 (S104).

  Next, the count unit 4 determines the degree of coincidence of pixels (S106). More specifically, "| r1-r2 | <eq and | r2-r3 | <eq and | g1-g2 | <eq and | g2-g3 | <eq and | b1-b2 | <eq and | b2- It is determined whether or not b3 | <eq ". If it is determined that this condition is satisfied (YES in S106), the counting variable sk [0] is incremented by 1 (S108). When the process of S108 ends, the process proceeds to S110. On the other hand, when it is determined in S106 that the condition is not satisfied (NO in S106), the process proceeds to S110.

  In S110, the image detection unit 14 increments the address reference counter j by 1 (S110).

As shown in S106 above, in the present embodiment, even if the pixel values do not completely match, it is determined that they are the same pixel. Even when the same object is shot, it is not always possible to broadcast and record with the same pixel value due to fluctuations, but in this way, fluctuations can be allowed. Accordingly, the accuracy of extracting thumbnail candidates can be improved.

  Next, the image detection unit 14 determines whether the address reference counter j is less than 480000 (S112). If it is determined that the address reference counter j is less than 480000 (YES in S112), the process returns to S104. On the other hand, when it is determined that the address reference counter j has reached 480000 (NO in S112), the process proceeds to S114.

  In S114, the thumbnail candidate extraction / storage unit 6 determines whether or not the matching pixel number sk [-1] of the previous frame is larger than the current matching pixel number sk [0]. When it is determined that the matching pixel number sk [−1] one frame before is equal to or less than the current matching pixel number sk [0] (NO in S114), the process proceeds to S116. On the other hand, if it is determined that the number of matching pixels sk [−1] one frame before is larger than the current number of matching pixels sk [0] (YES in S114), the process proceeds to S120.

  In S116, the thumbnail candidate extraction / storage unit 6 determines that the current matching pixel count sk [0] exceeds the variable over_max, or the current matching pixel count sk [0] is greater than the determination reference value max. It is determined whether or not the threshold is max_max or less. If it is determined that the condition is satisfied (YES in S116), temporary storage is performed (S118). More specifically, the data of the first frame data storage unit 41 is overwritten on the fourth frame data storage unit 44. At the same time, the flag max_lock is set to 0. On the other hand, when it is determined that the condition is not satisfied (NO in S116), the process proceeds to S134.

  In S120, the thumbnail candidate extraction / storage unit 6 determines whether or not the number of matching pixels sk [−1] one frame before is larger than the threshold max_max. If it is determined that the condition is satisfied (YES in S120), the process proceeds to S126. On the other hand, when it is determined that the condition is not satisfied (NO in S120), the process proceeds to S122.

  In S122, it is determined whether or not the number of matching pixels sk [−1] one frame before is larger than the determination reference value max and the flag max_lock is zero. If it is determined that the condition is satisfied (YES in S122), the process proceeds to S124. On the other hand, when it is determined that the condition is not satisfied (NO in S122), the process proceeds to S134.

  In S124, the updating unit 8 updates the determination reference value max to a value (corresponding to the maximum value) of the matching pixel number sk [−1] one frame before. When the process of S124 is completed, the process proceeds to S132.

  In S126, the thumbnail candidate extraction / storage unit 6 sets the flag max_lock to 1. Next, the thumbnail candidate extraction / storage unit 6 determines whether or not the number of matching pixels sk [−1] one frame before is larger than the variable over_max (S128). If it is determined that the condition is satisfied (YES in S128), the variable over_max is set to the value of the number of matching pixels sk [−1] one frame before (S130). When the process of S130 ends, the process proceeds to S132.

  In S132, the thumbnail candidate extraction / storage unit 6 performs the main storage. More specifically, data in the fourth frame data storage unit 44 is reduced and added to the thumbnail candidate storage unit 45. When the process of S132 ends, the process proceeds to S134.

In S134, the current number of matching pixels sk is added to the number of matching pixels sk [-1] one frame before.
Substitute the value of [0].

When the process of S134 is completed, the process is returned to the main routine.
According to the above process, it is possible to automatically extract an image with little pixel change in the content. An image with little pixel change is an expression of the intention of the producer that he / she wants to show the viewer over a certain period of time, so that an image suitable as a thumbnail is extracted.

  In the first embodiment of the present invention, thumbnail candidates are extracted and stored (mainly stored) when the count value is maximal and exceeds the determination reference value max. Thereby, when the number of matching pixels continuously increases, it is possible to prevent the same image from being extracted as a thumbnail candidate.

  In the present embodiment, the determination reference value max is updated when the count value is maximal, exceeds the determination reference value max, and is within the threshold max_max. That is, when extracted as a thumbnail candidate, the determination reference value max is basically updated, but when the count value exceeds the threshold max_max, the determination reference value max is not updated. As a result, when a frame with an extremely large number of matching pixels (for example, a frame in which the company name of the provider appears) appears, a frame suitable for a thumbnail appearing later is prevented from being excluded from the candidates. can do.

  FIG. 8A shows a frame extracted as a thumbnail candidate in the first embodiment of the present invention. This graph shows a graph in which the vertical axis represents the matching pixel ratio (%) and the horizontal axis represents the frame count. Referring to FIG. 8A, when the above processing is performed, frames at positions A1, B1,... G1 are extracted as thumbnail candidates.

  In the present embodiment, the frame having the maximum count value is selectively extracted as a thumbnail candidate. However, as shown in FIG. 8B, even if the count value is not maximum, the determination reference value Frames exceeding max (corresponding to the past maximum value) may also be extracted as thumbnail candidates.

  Alternatively, in the present embodiment, when the count value exceeds the threshold max_max, the thumbnail value is extracted only when the count value is equal to or greater than the maximum value storage variable over_max. However, it is not limited to such an extraction method. For example, when the count value exceeds the threshold max_max, it may be extracted as a thumbnail candidate every time the maximum value comes. Alternatively, when the count value exceeds the threshold max_max, only the maximal frame may be extracted as a thumbnail candidate first. Alternatively, when the threshold value max_max is exceeded, even if the count value is a maximum value, it may not be extracted as a thumbnail candidate.

  Alternatively, the threshold max_max may not be provided. That is, a frame whose count value exceeds the determination reference value max (in this case, always the maximum value immediately before) and is maximal may be sequentially set as thumbnail candidates.

  In the present embodiment, the degree of coincidence between the RGB color components is determined. However, it is only necessary to determine the degree of coincidence between the values in the color space, and the degree of coincidence between the components other than RGB is determined. Also good.

FIG. 9 is a flowchart showing thumbnail registration processing according to Embodiment 1 of the present invention. The process shown in the flowchart of FIG. 9 is also stored in advance in the memory 11 as a program, and the thumbnail registration process is realized by the control unit 10 reading and executing this program.

  Referring to FIG. 9, thumbnail registration unit 18 reads one or more thumbnail candidates stored in thumbnail candidate storage unit 45 (S302). Next, the thumbnail registration unit 18 displays the read thumbnail candidates on the display unit 32 (S304). An example of the thumbnail candidate list screen displayed in S304 is shown in FIG.

  FIG. 10 is a diagram illustrating an example of a thumbnail candidate list screen. Referring to FIG. 10, one or more thumbnail candidates are displayed in a predetermined area 51 of display unit 32. At the same time, a cursor key 52 for selecting one image from the displayed thumbnail candidates is displayed in a predetermined area.

  Referring to FIG. 9 again, the thumbnail registration unit 18 determines whether or not a determination key (not shown) has been pressed (S306). If pressing of the determination key (not shown) is not detected (NO in S306), the process proceeds to S308. In S308, the thumbnail registration unit 18 determines whether or not the cursor key 52 is operated. If it is determined that the cursor key 52 is being operated (YES in S308), the process returns to S306.

  In S306, when pressing of the determination key (not shown) is detected (YES in S306), the thumbnail registration unit 18 stores the thumbnail candidate determined by the user in the thumbnail data storage unit 21 in association with the content (S310). ). More specifically, the thumbnail candidate selected by the user is stored as the thumbnail data D5 of the corresponding management data Ak in the thumbnail data storage unit 21.

  If it is determined in S308 that the cursor key 52 has not been operated (NO in S308), other processing is executed (S312).

This is the end of the thumbnail registration process according to Embodiment 1 of the present invention.
As described above, the moving image data playback apparatus 100 according to Embodiment 1 of the present invention automatically extracts and stores thumbnail candidates, but entrusts the actual thumbnail registration to the user. Thereby, it is possible to prevent an image unintended by the user from being registered as a thumbnail.

[Embodiment 2]
Next, a second embodiment of the present invention will be described.

  In Embodiment 1 described above, thumbnail candidates are extracted when the count value of the number of matching pixels is maximal and exceeds the determination reference value. In this case, even if the count value is maximal, if it does not exceed the determination reference value, it is not adopted as a thumbnail candidate. Therefore, there is a possibility that the optimal thumbnail candidate is not necessarily extracted. Therefore, in Embodiment 2 of the present invention, every time the count value reaches a maximum, thumbnail candidates are extracted to prevent such inconvenience.

  Note that the hardware configuration of the moving image data reproducing apparatus according to the second embodiment of the present invention is the same as that of the moving image data reproducing apparatus 100 according to the first embodiment. Therefore, the description will be given also using the reference numerals shown in FIG. Further, the functional configuration of the moving image data reproducing device in the second embodiment is the same as that in the first embodiment except for the updating unit 8 in the functional block diagram shown in FIG. Therefore, the description will be given also using the reference numerals shown in FIG.

Hereinafter, differences from the first embodiment will be described.
FIG. 11 is a flowchart showing thumbnail candidate extraction / storage processing according to Embodiment 2 of the present invention.

  First, the image detection unit 14 performs an initialization process (S102 ′). Specifically, the address reference counter j is set to zero. Also, the matching pixel count counting variable sk [0] is set to zero. Also, the number of matching pixels sk [−1] one frame before and the number of matching pixels “sk [−2]” two frames before are both set to zero.

  When the processing of S102 ′ is completed, the processing of S104 to S112 is executed as in the first embodiment.

  If it is determined in S112 that the address reference counter j has reached 480000 (NO in S112), the current matching pixel count sk [0] is greater than the previous matching pixel count sk [-1]. Whether or not (S202). If it is determined that the condition is satisfied (YES in S202), the process proceeds to S204. On the other hand, when it is determined that the condition is not satisfied (NO in S202), the process proceeds to S206.

  In S204, the thumbnail candidate extraction / storage unit 6 performs temporary storage. More specifically, the data of the first frame data storage unit 41 is overwritten on the fourth frame data storage unit 44. When the processing of S204 ends, the process proceeds to S210.

  In S206, the thumbnail candidate extraction / storage unit 6 determines whether or not the number of matching pixels sk [−1] one frame before is larger than the number of matching pixels sk [−2] two frames before. If it is determined that the condition is satisfied (YES in S206), the process proceeds to S208. On the other hand, when it is determined that the condition is not satisfied (NO in S206), the process proceeds to S210.

  In S208, the thumbnail candidate extraction / storage unit 6 performs the main storage. More specifically, the value of the matching pixel count sk [−1] one frame before and the reduced image data of the frame data in the fourth frame data storage unit 44 are additionally recorded in the thumbnail candidate storage unit 45 in association with each other. When the processing of S208 ends, the process proceeds to S210.

  In S210, the thumbnail candidate extraction / storage unit 6 substitutes the value of the current matching pixel count sk [0] for the matching pixel count sk [-1] of the previous frame. At the same time, the value of the number of matching pixels sk [-1] one frame before is substituted for the number of matching pixels sk [-2] two frames before.

When the process of S210 is completed, the process is returned to the main routine.
FIG. 12 is a diagram illustrating an example of a data structure of the thumbnail candidate storage unit 45 according to the second embodiment of the present invention.

  Referring to FIG. 12A, the thumbnail candidate storage unit 45 includes a thumbnail candidate management information storage area 451 and a thumbnail candidate image storage area 452.

  An example of the structure of the thumbnail candidate management information storage area 451 is shown in FIG. An example of the structure of the thumbnail candidate image storage area 452 is shown in FIG.

  Referring to FIG. 12B, the thumbnail candidate management information storage area 451 includes count value data D11 that is the number of matching pixels of the corresponding thumbnail candidates, and address data D12 that stores thumbnail candidate image data. Are stored in a one-to-one relationship. These data D11 and D12 may be stored in the order of extracting thumbnail candidates.

  Referring to FIG. 12C, thumbnail candidate image storage area 452 stores image data as thumbnail candidates for each address.

  FIG. 13 is a flowchart showing thumbnail registration processing according to Embodiment 2 of the present invention. The processing shown in the flowchart of FIG. 13 is also stored in advance in the memory 11 as a program, and the thumbnail registration processing is realized by the control unit 10 reading and executing this program. In addition, the same number is attached | subjected about the process similar to the process shown in FIG. Therefore, the description thereof will not be repeated here.

  Referring to FIG. 13, in the thumbnail registration process according to the second embodiment of the present invention, the process shown in S303 is executed between S302 and S304 described above. That is, when the processing of S302 ends, the thumbnail registration unit 18 selects thumbnail candidates in the descending order of the number of matching pixels, specifically, in descending order of the count value data D11 stored in the thumbnail candidate management information storage area 451. Rearrange (S303).

  When the process of S303 ends, the processes of S304 to S312 are executed as in the first embodiment, and the thumbnail registration process ends.

  Thus, according to Embodiment 2 of the present invention, thumbnail candidates are extracted each time the count value reaches a maximum. Thereby, it is possible to improve the possibility that optimal image data is extracted as a thumbnail.

  In the second embodiment, when the user selects a thumbnail, the thumbnail candidates are rearranged and displayed in descending order of the count value. Thereby, even when a huge number of thumbnail candidates have been extracted, the user can easily select a thumbnail.

  In the present embodiment, the thumbnail candidates are once read in S302, and the thumbnail candidates are rearranged in descending order of the number of matching pixels in S303. However, the present invention is not limited to this method. For example, thumbnail candidates may be read out in descending order of the number of matching pixels.

  In addition, the thumbnail candidate determination method and the thumbnail registration method performed by the moving image data reproduction apparatus 100 of the present invention can be provided as a program. Such a program can be recorded on an optical medium such as a CD-ROM (Compact Disc-ROM) or a computer-readable recording medium 62 such as a memory card and provided as a program product. A program can also be provided by downloading via a network.

  The provided program product is installed in a program storage unit such as a hard disk and executed. The program product includes the program itself and a recording medium on which the program is recorded.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

It is a block diagram which shows the hardware constitutions of the moving image data reproduction apparatus in Embodiment 1, 2 of this invention. It is a functional block diagram of the moving image data reproducing apparatus in the first and second embodiments of the present invention. It is a figure which shows an example of the data structure of a thumbnail data storage part. (A) is a figure which shows the coordinate of each pixel of the frame data in Embodiment 1, 2 of this invention, (b) is a figure which shows an example of the data structure of the 1st-3rd frame data storage part. It is. It is a figure which shows the concept of the frame extraction process in Embodiment 1, 2 of this invention. It is a flowchart which shows the flow of the main routine of the thumbnail determination process performed in the control part of the moving image data reproduction apparatus of Embodiment 1, 2 of this invention. It is a flowchart which shows the thumbnail candidate extraction and memory | storage process in Embodiment 1 of this invention. (A) is a figure showing the frame extracted as a thumbnail candidate in Embodiment 1 of this invention, (b) is extracted as a thumbnail candidate in the other example of Embodiment 1 of this invention. It is a figure showing a frame. It is a flowchart which shows the thumbnail registration process in Embodiment 1 of this invention. It is a figure which shows an example of a thumbnail candidate list screen. It is a flowchart which shows the thumbnail candidate extraction and memory | storage process in Embodiment 2 of this invention. (A)-(c) is a figure which shows an example of the data structure of the thumbnail candidate memory | storage part 45 in Embodiment 2 of this invention. It is a flowchart which shows the thumbnail registration process in Embodiment 2 of this invention.

Explanation of symbols

41 to 44 Frame data storage unit, 2 frame counter, 4 count unit, 6 thumbnail candidate extraction / storage unit, 8 update unit, 10 control unit, 11 memory, 12 frame extraction unit, 14 image detection unit, 18 thumbnail registration unit, 20 recording data recording unit, 21 thumbnail data storage unit, 30 TV broadcast receiving unit, 32 display unit, 34 input unit, 36 drive device, 45 thumbnail candidate storage unit, 61 antenna, 62 recording medium, 100
Movie data playback device, 451 thumbnail candidate management information storage area, 452 thumbnail candidate image storage area.

Claims (24)

A thumbnail creation method for creating a thumbnail from video data including a plurality of frames,
An extraction step of extracting a plurality of frame groups from the moving image data;
For each of the extracted frame groups, comparing the frames in pixel units and counting the number of matching pixels;
And a first storage step of selectively storing an arbitrary first frame as a thumbnail candidate from the frame group in which the count value of the number of matching pixels is at least maximum.   The thumbnail creation method according to claim 1, wherein each of the frame groups includes the frames having a constant interval.   The thumbnail creation method according to claim 1, wherein the extracting step includes a step of sequentially extracting the frame group along a time axis. The first storing step includes a step of storing the first frame as the thumbnail candidate when the count value is maximal and exceeds a determination reference value;
The thumbnail creation method according to claim 1, further comprising an updating step of updating the determination reference value with the maximum value when the count value is a maximum value.   The thumbnail creation method according to claim 4, wherein the updating step includes a step of excluding updating of the determination reference value when the count value exceeds a predetermined threshold value.   5. The thumbnail creation according to claim 4, wherein the first storing step further includes a step of storing an arbitrary second frame from the frame group in which the count value exceeds the determination reference value as the thumbnail candidate. Method. Displaying a plurality of stored thumbnail candidates;
7. The method according to claim 1, further comprising: a second storage step of storing information related to an image selected by a user from among the plurality of thumbnail candidates as the thumbnail information of the moving image data. To create thumbnails.   The thumbnail creation according to any one of claims 1 to 3, wherein the first storing step includes a step of storing the thumbnail candidates in association with the count value every time the count value reaches a maximum. Method. Rearranging the plurality of stored thumbnail candidates in descending order of the count value associated with each of the thumbnail candidates;
Displaying the sorted thumbnail candidates; and
9. The thumbnail creation method according to claim 8, further comprising: a second storage step of storing information on an image selected by a user among the plurality of thumbnail candidates as information on the thumbnail of the moving image data.   The thumbnail creation method according to claim 1, wherein each frame group includes three frames. The thumbnail creation method according to claim 1, wherein the counting step includes a step of incrementing the number of matching pixels when the color components are the same among the pixels.   The thumbnail creation according to any one of claims 1 to 3, wherein the counting step includes a step of incrementing the number of matching pixels when a difference value of a color component between pixels is less than a predetermined threshold value. Method. Extraction means for extracting a plurality of frame groups from moving image data including a plurality of frames;
For each of the extracted frame groups, the frame is compared in units of pixels, and counting means for counting the number of matching pixels;
A moving image data reproducing apparatus comprising: a first storage unit for selectively storing an arbitrary first frame as a thumbnail candidate from the frame group in which the count value by the counting unit is at least maximum.   The moving image data reproducing apparatus according to claim 13, wherein each frame group includes the frames having a constant interval.   15. The moving image data reproduction device according to claim 13, wherein the extraction unit sequentially extracts the frame groups along a time axis. The first storage means stores the first frame as the thumbnail candidate when the count value is maximal and exceeds a determination reference value;
The moving image data reproduction device according to claim 13, further comprising an updating unit configured to update the determination reference value with the maximum value when the count value is a maximum value.   The moving image data reproducing device according to claim 16, wherein the updating unit includes an excluding unit for excluding an update of the determination reference value when the count value exceeds a predetermined threshold value.   The moving image data reproduction device according to claim 16, wherein the first storage unit further stores an arbitrary second frame from the frame group in which the count value exceeds the determination reference value as the thumbnail candidate. Display means for displaying the plurality of thumbnail candidates stored in the first storage means;
19. The storage device according to claim 13, further comprising: a second storage unit configured to store information related to an image selected by a user from among the plurality of thumbnail candidates as thumbnail information of the moving image data. The moving image data reproducing apparatus described.   The moving image data reproduction device according to any one of claims 13 to 15, wherein the first storage unit stores the thumbnail candidates in association with the count value every time the count value reaches a maximum. Reordering means for reordering the plurality of thumbnail candidates stored in the first storage means in descending order of count value associated with each of the thumbnail candidates;
Display means for displaying the plurality of thumbnail candidates that have been rearranged;
21. The moving image data reproduction device according to claim 20, further comprising: second storage means for storing information on an image selected by a user from among the plurality of thumbnail candidates as information on the thumbnail of the moving image data. .   16. The moving image data reproducing device according to claim 13, wherein each of the frame groups includes three frames.   The moving image data reproduction device according to any one of claims 13 to 15, wherein the counting unit increments the number of matching pixels when the color components are the same among the pixels.   The moving image data reproduction device according to any one of claims 13 to 15, wherein the counting unit increments the number of matching pixels when a difference value of a color component between pixels is less than a predetermined threshold value.

Images