JP2005318618A - Moving image management apparatus and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve operability of various kinds of motion picture data operations using a representative image by having information related to the selection of the representative image as an attribute of that representative image. <P>SOLUTION: An apparatus that manages a moving image recorded on a storage medium in association with one or more representative images stores the representative image in accordance with the moving image, accesses a storage section storing selection reference information of the representative image in accordance with the representative image and selects a representative image corresponding to a purpose of processing out of representative images stored in accordance with a desired moving image based on the selection reference information. If one of selected representative images is designated, processing corresponding to the purpose of processing is applied to a moving image of a section indicated by section information corresponding to the designated representative image. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a moving image data management apparatus and method for managing moving image data stored on a storage medium using one or more representative images.

  Conventionally, a method for managing a moving image recorded on a storage medium using a representative image has been proposed. Japanese Patent No. 03081304 (Patent Document 1) discloses a method of using a representative image in a general moving image or still image playback device, and is particularly representative of a simple display when selecting a still image or a moving image. The use of images is described. In JP-A-11-176137 (Patent Document 2) and JP-A-2000-322875 (Patent Document 3), a representative image is assigned to a section of a moving image, and a target of a reproduction instruction is used using the representative image. A method for selecting a certain section is disclosed.

As the use forms of such representative images become common, the use forms have also diversified. For example, it is conceivable that image management using representative images is applied not only to the playback device as described above but also to an editing device.
Japanese Patent No. 03081304 JP 11-176137 A JP 2000-322875 A

  However, for example, even for representative images for the same moving image section, a representative image suitable for a user interface at the time of reproduction is different from a representative image suitable for a user interface at the time of editing. Therefore, when the representative image is used in various models including the editing apparatus, the following problems occur.

  For example, in FIG. 1, 11 indicates a series of moving images. In the moving image 11, it is assumed that a series of actions in which a batter holds and hits is shot in the section (a). The representative image 12 and the representative image 13 are representative images relating to the same section (a). The representative image 12 is created from the top frame of the section (a) and is an image when the batter is set up. Further, the representative image 13 is created from an intermediate frame, is an image when the batter hits, and can be said to be the most suitable representative image for grasping the contents of the section (a).

  In general, one representative image is assigned to one section and registered. Therefore, in practice, either the representative image 12 or 13 is registered as the representative image of the section (a). The representative image registered in this way can be used as an index image for a playback device, for example. However, when the representative image 13 is registered, the playback device starts playback from a temporally previous state when the representative image is selected in the playback device (because playback starts from the beginning of the section (a)), and the user feels uncomfortable. I remember it. On the other hand, when the representative image 12 is registered, the content of the section (a) cannot be accurately grasped only by the image when it is used in the editing apparatus, which is inconvenient. In the example of FIG. 1, this is because it cannot be determined from the representative image 12 whether the scene is a hit by the batter or a scene that has been missed. Thus, the appropriate representative image for the same section in the moving image differs depending on the purpose of the device used.

  Conventionally, it has been necessary to assign and manage such representative images for each use and usage of moving image data, such as for reproduction and editing. Therefore, it is necessary to have representative image information redundantly for each purpose of use, such as editing or playback, for each moving image or still image, or for each device or method. There is a problem in that it is necessary to separately hold program codes and methods for the processes to be used. That is, when moving image data using representative images is used in various moving image data processing, information such as representative images is individually managed for each device or program to be used. For this reason, although there are only a plurality of devices and programs for management, there is a waste that representative image data and the like are individually stored. Thus, if the operability of the moving image data using the representative image is to be maintained, there is a problem that management of the representative image becomes complicated.

  The present invention has been made in view of the above-described problems, and improves the operability of various moving image data operations using representative images by providing information on selection of representative images as attributes of the representative images. The purpose is to do.

In order to achieve the above object, a moving image management apparatus according to the present invention comprises the following arrangement. That is,
An apparatus for managing one or a plurality of representative images in association with a moving image recorded on a storage medium,
The representative image is stored in association with the moving image, and the storage unit storing the selection reference information of the representative image in association with the representative image is accessed, and the representative image stored in correspondence with the desired moving image is stored. Selecting means for selecting a representative image according to the processing purpose from among the selection criteria information;
When one of the representative images selected by the selection unit is designated, an application unit that applies a process according to the processing purpose to a moving image in a section indicated by the section information corresponding to the designated representative image; Prepare.
In this specification, the representative image is a still image (for example, a thumbnail) generated based on one of the frames in the section. It is needless to say that one representative image does not have to be one in one section, and a plurality of representative images may be set in the same section.

In addition, a moving image management method according to another aspect of the present invention that achieves the above object is
A method for managing one or a plurality of representative images in association with a moving image recorded on a storage medium,
The representative image is stored in association with the moving image, and the storage unit storing the selection reference information of the representative image in association with the representative image is accessed, and the representative image stored in correspondence with the desired moving image is stored. A selection step of selecting a representative image according to the processing purpose from among the selection criteria information;
When one of the representative images selected in the selection step is designated, an application step of applying a process according to the processing purpose to a moving image in a section indicated by the section information corresponding to the designated representative image; Prepare.

  According to the present invention, since information related to selection of a representative image is provided as an attribute of the representative image, operability using the representative image is improved in executing various moving image data processing.

  Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings.

  In the present embodiment, in the method of using a representative image, in order to be able to manage / use information related to the representative image collectively, the information can be shared, and the information can be shared depending on the usage form. It is characterized by having information at the time of creating a representative image so that information related to the representative image can be distinguished or shared. Here, the information at the time of creating the representative image indicates which part of the image in the corresponding video segment is generated based on the representative image, or what type of image (motion) in the corresponding video segment. It is shown whether it was produced | generated based on the intense part etc.). Here, the information at the time of creation of the representative image includes information (hereinafter referred to as a selection criterion) relating to selection of a frame below the representative image.

  According to the present embodiment, the characteristic of providing information at the time of creating a representative image allows a plurality of representative images to be recorded and stored in a unified manner, thereby eliminating waste (for example, corresponding to a plurality of types of usage purposes). A plurality of representative images can be managed as one file). In addition, since a representative image suitable for realizing a unique function in each apparatus or program can be selected by referring to a selection criterion for the representative image, the operability of the function can be improved. Similarly, at the time of storing information, the arrangement method and the like can be varied according to the use form and method of the information related to the representative image. In other words, it is possible to arrange the locations of important thumbnails (headers), index regions, and thumbnail regions (footers) on the storage medium so that they can be easily accessed.

  Hereinafter, the moving image processing apparatus of this embodiment will be described in detail.

  FIG. 2 is a block diagram illustrating a control configuration of the moving image processing apparatus according to the present embodiment. In FIG. 2, 201 is a CPU that executes various controls in the moving image processing apparatus of the present embodiment. Reference numeral 202 denotes a ROM which stores a boot program executed when the apparatus is started up and various data. A RAM 203 stores a control program to be processed by the CPU 201 and provides a work area when the CPU 201 executes various controls. A keyboard 204 and a mouse 205 provide various input operation environments for the user.

  Reference numeral 206 denotes an external storage device, which includes a hard disk, floppy (registered trademark) disk, optical disk, magnetic disk, magneto-optical disk, magnetic tape, and the like. Reference numeral 207 denotes a display, which is composed of a display or the like, and displays the result and the like to the user. A network interface 208 enables communication with each device on the network. Reference numeral 209 denotes a video interface, which enables capturing of moving images from the video camera 210 and the VTR 212. Reference numeral 211 denotes a bus connecting the above-described components.

  In the above configuration, the video camera 210, VTR 212, and external storage device 206 may be replaced with those arranged on the network.

  FIG. 3 is a block diagram illustrating a functional configuration of the moving image processing apparatus according to the present embodiment. Reference numeral 300 denotes a user operation unit that acquires user instructions and inputs via the keyboard 204 and the mouse 205. A moving image input unit 301 reads out moving images from the video camera 210 and the VTR 212 via the video I / F 209 and inputs them to the moving image processing apparatus of this embodiment. A moving image file storage unit 302 includes an external storage device 206, and stores moving images input by the moving image input unit 301 in a file format.

  A thumbnail creation unit 303 presents the moving images stored in the moving image file storage unit 302 on the display 207, causes the user to specify a desired section and a frame image in the section, and displays the specified frame image. Originally, a reduced image (hereinafter referred to as a thumbnail image) as a representative image of the section is created and stored in the moving image file storage unit 302 as a moving image file together with information on the specified section. That is, the thumbnail creation unit 303 allows the user to specify a section in the moving image and a frame representing the section through the user operation unit 300 with reference to the moving image presented on the display device 207. Then, the thumbnail image based on the designated frame image is stored in the moving image file storage unit 302 together with information such as the designated section. Note that the entire moving image can be designated as the designated section.

  A browse unit 304 reads a thumbnail from the moving image file storage unit 302, displays the thumbnail on the display unit 207, and reproduces a moving image corresponding to the thumbnail selected by the user. In the reproduction of the moving image in this case, for example, only the section represented by the selected thumbnail is reproduced, or the section represented by the selected thumbnail and the subsequent moving images are reproduced. An editing unit 305 edits the moving image in the moving image file storage unit 302. Further, the progress and result of the editing are displayed via the display unit 207.

  Note that the function of each unit described above is realized by the CPU 201 executing a control program stored in the ROM 202. Alternatively, the control program may be appropriately loaded from the external storage device 206 into the RAM 203 and executed by the CPU 201.

<1. Thumbnail storage method>
FIG. 4 is a schematic diagram showing an example of a thumbnail storage form in the moving image file in the present embodiment.

  Reference numeral 400 denotes a moving image file recorded on the storage medium by the moving image file storage unit 302. Note that the unit for making one moving image file is not particularly limited. For example, a shot from when the recording button of the imaging apparatus is pressed until it is stopped may be used as a unit, or a program in TV broadcasting may be used as one unit. It doesn't matter. The moving image file 400 includes three parts: a header 401 that records management information and the like, a moving image 402 that records moving image data itself, and a footer 403 that mainly records thumbnails.

  The header 401 records moving image file management information 404 in which information such as the data size of moving image files, the format of moving image data, and the date and time of creation are recorded, and one thumbnail 405. The footer 403 includes a thumbnail area 407 in which a plurality of thumbnails can be recorded, and an index area 406 in which index information of each thumbnail is recorded.

  FIG. 5 shows an example of the data structure for one index in the index area 406 in the footer 403 of FIG.

  Start Point and End Point are the start point and end point of the section in the moving image represented by the thumbnail, respectively. The form of the start point and the end point is not particularly limited as long as the section of the moving image can be specified. However, it is preferable that the start point and the end point are described by, for example, a time code or a frame ID. When the entire moving image is designated as a section, Start Point indicates the first frame of the moving image, and End Point indicates the last frame of the moving image.

  The Thumbnail Type ID is defined for each image selection criterion when creating a thumbnail, and is defined as follows, for example.

Thumbnail Type ID = 1: First frame of the section
Thumbnail Type ID = 2: Frame in the middle of the section
Thumbnail Type ID = 3: Frame at the end of the section
Thumbnail Type ID = 4
Thumbnail Type ID = 5: A frame with gentle movement in the section
Thumbnail Type ID = 6: Frame in which the object appears in the section
Thumbnail Type ID = 7: Frame in which the object disappeared in the section
Thumbnail Type ID = 8: Frame with good image quality in the section
Thumbnail Type ID = 9: Image not in the section The above example defines the ID for each type of frame from which the thumbnail was created. These IDs represent selection criteria for frames in the moving image that is the basis of the thumbnail.

  Further, Thumbnail Offset shown in FIG. 5 describes an offset to the thumbnail indicated by the index (the position of the thumbnail corresponding to the index in the thumbnail area). In the case of an image that is not in the section of Thumbnail Type ID = 9, it is unused, so that nothing is described or a special value such as a maximum value may be described.

  FIG. 6 is a diagram showing an example of the data structure of one thumbnail in the thumbnail 405 in the header 401 and the thumbnail area 407 in the footer 403 in FIG. In FIG. 6A, Thumbnail Header describes thumbnail image management information and the like. In addition, Thumbnail Image describes thumbnail image data itself. FIG. 6B shows an example of the data structure of the Thumbnail Header. In Thumbnail Image Offset, an offset value to the storage position of the thumbnail image (Thumbnail Image) is described. Thumbnail Image Size describes the size of the thumbnail image. In Thumbnail Key Relative Point, the position of the frame in the moving image that is the basis of the thumbnail is described by the relative position from the Start Point registered in the index.

<Thumbnail registration process>
FIG. 7 is a flowchart showing an example of thumbnail registration operation according to this embodiment.

  First, in step S100, the moving image input unit 301 inputs a moving image and records it in the moving image file storage unit 302 in the moving image file format shown in FIG. At this time, an area for storing thumbnails in the header 401 and an area for the footer 403 are reserved as reserved areas. Next, in step S101, it is determined whether a user has designated a thumbnail. The designation of the thumbnail here means designation of an arbitrary frame in the video that is the basis of the thumbnail, designation of a section in the moving image that the thumbnail represents, and selection of the video frame that is the basis of the thumbnail. To specify the selection criteria, and to specify whether or not it is an important thumbnail. Here, the selection criterion corresponds to the above-described Thumbnail Type ID. An important thumbnail represents a moving image file and is registered as a thumbnail 405.

  The user designates the thumbnail by using the function of the apparatus. For example, when the user performs manual setting, (1) this apparatus is set to a predetermined mode for specifying a thumbnail, (2) a desired section in the moving image to be set is specified, and (3 (4) A method of generating a thumbnail based on the image at that time by playing this section and (4) inputting a predetermined instruction to the device when a video to be made into a thumbnail is displayed during the playback. It is done. In this case, the Start Point and End Point are determined by the section in the designated moving image. Also, the designation of a desired section and the designation of an image to be used as a thumbnail are performed using a keyboard and a mouse.

  Alternatively, thumbnails may be generated by automatically selecting from a section designated by the user based on a selection criterion designated by the user, such as a rapidly changing image or an image having little change. .

  Furthermore, as the selection criterion, any one of the beginning, middle, and end of the section can be specified. When “Start” is specified, the head frame of the section is specified, and when “End” is specified, the section of the section is specified. When “intermediate” is designated as the last frame, the frame designated in the procedures (1) to (4) may be adopted as the base image of the thumbnail.

  As described above, various methods such as those designated by the user and those automatically designated by the device can be applied to the method of designating the video frame that is the basis of the thumbnail. Either method is acceptable. However, in the flowchart of FIG. 7, the following description will be given on the assumption that the procedure using the top, middle, and end is adopted as the selection criterion.

  Next, in step S102, it is determined whether there is a designation that the thumbnail designated in step S101 is an important thumbnail. In the present embodiment, the important thumbnail means a thumbnail for representing the entire moving image file. If the user designates that the thumbnail is important, the process proceeds to step S103, where the position of the frame designated to be adopted for the thumbnail is corrected to the relative position from the top of the moving image file, and the Thumbnail Key Relative Point is obtained. . Next, in step S104, the header is a thumbnail with the Thumbnail Key Relative Point calculated in step S103, the thumbnail image size (Thumbnail Image Size), and the information on the relative distance to the thumbnail image storage position (Thumbnail Image Offset) as a header. Record in 401. At this time, the header 401 may be stored at a position where access on the storage medium is relatively easy and at a high speed, for example, in the case of an optical disc, in the inner periphery. By doing so, it is possible to realize high-speed index display for selecting a moving image file. And it returns to determination of step S101. Note that registration of important thumbnails is necessary when selecting a moving image file in a later operation, so that this processing may not be completed until registration of important thumbnails is completed. Of course, if an important thumbnail is not stored, it will be apparent to those skilled in the art that one of the thumbnails in the footer may be substituted, for example, the first stored thumbnail may be substituted. Let's go. In this case, it is needless to say that it is not necessary to check the completion of registration of an important thumbnail at the end of the above process.

  On the other hand, if it is not designated as an important thumbnail in step S102, the process proceeds to step S105. In step S105, a new index is secured in the unused area of the index area 406 of the footer 403 in the moving image file 400. In step S106, a new thumbnail is secured in an unused area of the thumbnail area 407. Next, in step S107, an offset value to the thumbnail secured in step S106 is calculated.

  Next, in step S108, the type of the designated thumbnail is determined. Here, a process of classification according to selection criteria designated by the user (in this example, whether the position of the image that is the basis of the thumbnail is the beginning, middle, or end of the section represented) is shown.

  The process branches to any of steps S109 to S111 according to the type determined in step S108, and a corresponding Thumbnail Type ID is set. This can be set by referring to the above-mentioned Thumbnail Type ID as a table. In this embodiment, Thumbnail Type ID = 1 (S109) if the selection criterion is the first frame of the section, Thumbnail Type ID = 2 (S110) if the frame is in the middle of the section, and the end frame of the section. If so, Thumbnail Type ID = 3 (S111) is set. Here, only three types are shown, but this is an example and of course not limited to these.

  Next, in Step S112, the offset calculated in Step S107 (Thumbnail Offset), the Thumbnail Type ID set in Steps S108 to S111, and the start point (Start Point) and end point (End Point) indicating the section specified by the user. Is generated and stored in the footer 403. At this time, the index area 406 of the footer 403 may be stored at a position where access on the storage medium is relatively easy and at a high speed, for example, in the case of an optical disc, in the inner periphery. By doing so, it is possible to speed up the selection of thumbnails in the footer.

  Next, in step S113, the position of the frame adopted for the thumbnail is corrected to the relative position from the beginning of the section, and a Thumbnail Key Relative Point is obtained. In step S114, the Thumbnail Key Relative Point calculated in step S113 and the thumbnail image size (Thumbnail Image Size) are stored in the footer 403 together with the thumbnail image obtained by reducing the specified frame. Information on the relative distance to the storage position of the thumbnail image is stored in Thumbnail Image Offset. At this time, the thumbnail area 407 of the footer 403 may be stored in a position where access on the storage medium is relatively easy and can be performed at high speed, for example, in the case of an optical disc, in the inner periphery. By doing so, it is possible to increase the speed of displaying thumbnails in the footer. Next, returning to step S101, if the thumbnail is designated again, step S102 to step S114 are repeated.

  In step S101, if a thumbnail is not designated, the process proceeds to step S115, and if the end of the registration process is instructed here, the process ends. If the end of the registration process is not instructed, the process returns to step S101 and the above process is repeated.

  Through the above processing, any one of the important thumbnails, indexes, and thumbnail images can be stored in a location that can be easily accessed on the storage medium, and thumbnails with different purposes of use can be stored according to their selection criteria, namely Thumbnail Type ID. Since they can be distinguished, these thumbnails can be stored in the same file, and management becomes easy.

<Thumbnail utilization process>
Next, processing when using the thumbnail registered by the above-described thumbnail registration processing will be described.

[Utilization by browser]
First, a case where it is used in a browser is described. FIG. 8 is a diagram illustrating an example of a user interface of a browser. FIG. 8A shows a display state after a moving image file to be browsed is selected by a predetermined selection operation. Therefore, each thumbnail shown in FIG. 8A is a thumbnail registered in the footer of the selected file. Here, the predetermined selection operation is to display a list of thumbnails registered in the header of each video file as important thumbnails as shown in FIG. 8A, and select a video file by specifying a desired thumbnail. To do. In FIG. 8A, reference numeral 801 denotes the entire display area of the browser. Reference numeral 802 denotes an area where thumbnails are displayed side by side. The thumbnails are displayed in order from the left to the right, starting from the top row in order. Reference numeral 803 denotes a scroll bar. By scrolling the scroll bar, thumbnails that cannot be displayed at a time in the thumbnail display area 802 can be displayed in order. Reference numeral 804 denotes a pointer, and the user can specify a desired thumbnail in the thumbnail display area 802 by moving the pointer 804 by operating the user operation unit 300. When a thumbnail is selected with the pointer 804, the screen is switched to a video playback screen 805 as shown in FIG. 8B, and playback of the video starts.

  Processing for realizing the use of thumbnails in the above browser will be described below. 9A and 9B are flowcharts showing processing during browser operation of the moving image processing apparatus of the present embodiment. The processing shown in FIGS. 9A and 9B is executed when browser processing is activated in the moving image processing apparatus and a moving image file to be browsed is designated. When designating a moving image file to be browsed, thumbnails representative of the entire moving image file (thumbnail stored in the header of each moving image file) are displayed side by side in the thumbnail display area of FIG. A desired moving image file can be selected by designating a desired thumbnail with the pointer 804. When a desired moving image file is selected, the processing shown in FIGS. 9A and 9B is started.

  First, in step S <b> 201, the browsing unit 304 reads the header 401 in the moving image file 400 selected by the above operation from among the moving image files accumulated by the moving image file accumulation unit 302. Then, various basic management information such as a file size is obtained from the moving image file management information 404 and stored in the memory (RAM 203). In step S202, the index area 406 in the footer 403 is read and held in the memory (RAM 203). In step S203, the indexes included in the index area 406 held in step S202 are sorted in ascending order by the start position (Start Point shown in FIG. 5). This is for arranging the thumbnail display described later in chronological order.

  In step S204, the display start number N of thumbnails (indicating that the Nth thumbnail is at the left end of the top row) is obtained from the current position of the scroll bar 803. For example, if the scroll bar 803 is at the upper end (the state shown in FIG. 8), the display start number N = 1. Therefore, the display is started from the first thumbnail.

  Next, the following thumbnails for browsing are displayed in the thumbnail display area 802 by executing the following steps S205 to S212 in order from the top index.

  First, in step S205, the browsing unit 304 determines whether or not the thumbnail is suitable for browsing based on a selection criterion at the time of thumbnail registration. What kind of thumbnail is most suitable as an index in the browse operation may differ depending on the device even if the same browser is used. In the present embodiment, it is assumed that the browse unit 304 determines that the first frame of the section is suitable as an index thumbnail. Here, in order to check whether the frame is the first frame of the section, it is only necessary to check whether Thumbnail Type ID = 1. At the start of processing, the check starts from the first index after sorting.

  If it is determined in step S205 that the thumbnail is for the browser, the process proceeds to the next step S206. In step S206, the number C (the number of thumbnails for browsers) is counted. In step S207, it is checked whether the counted number C is less than the display start number N (C <N). This is a process for skipping without displaying thumbnails less than the Nth.

  If C is equal to or greater than the display start number N, a thumbnail image is read for thumbnail display in the next step S208. In step S209, the thumbnail image is displayed in the thumbnail display area 802, and the process proceeds to step S210. On the other hand, if it is determined in step S205 that the thumbnail is not a browsing thumbnail, or if it is determined in step S207 that the thumbnail is less than the Nth, the process proceeds directly to step S210.

  In step S210, the processing target is moved to the next index in order to process the next thumbnail. Here, if the immediately preceding process is for the final index and there is no next index, the process proceeds to step S213. If the next index exists, the process advances to step S212 to check whether the number of thumbnails that can be displayed in the thumbnail display area 802 has been exceeded. For example, in the example shown in FIG. 8A, the number of thumbnails that can be displayed is nine. Therefore, if C ≧ N + 9, it can be determined that displayable thumbnails have been displayed.

  If the displayable number is not exceeded, the process returns to step S205. That is, the processes in steps S205 to S210 are performed to check the indexes in order until there are no more indexes to be processed or the number of displayable sheets is exceeded.

  If there are no more indexes to be processed, or if the number of thumbnails displayed exceeds the number of thumbnails that can be displayed in the thumbnail display area 802, the process proceeds to step S213. In step S213, it is checked whether or not the scroll bar has been operated by the user. If the user has operated the thumbnail, the thumbnail corresponding to the position of the scroll bar after the operation is displayed by repeating the processing from step S204 to step S212. If not, the process proceeds to step S214.

  In step S214, it is determined whether one of the thumbnails displayed in the thumbnail display area 802 is selected by the user. If a thumbnail has been selected, the process proceeds to step S215, and the Start Point is read from the index corresponding to the selected thumbnail. Next, in step S216, the playback of the moving image is started from the Start Point, and the moving image is displayed on the moving image playback screen 805. Thereafter, the process proceeds to step S217. On the other hand, if no thumbnail is selected in step S214, the process proceeds to step S217.

  In step S217, it is determined whether or not an instruction to end browsing is given by the browsing unit 304, and if the end is not instructed, the process returns to step S213. If termination is instructed, this process is terminated.

  In the above example, when a thumbnail is selected, the Start Point is acquired from the corresponding index, playback is started from that position, and playback is continued until the end of the moving image data. Not limited to. The start point and end point may be acquired from the index corresponding to the selected thumbnail, and the corresponding section may be reproduced. It should be noted that a stop or pause button may be provided so that moving image playback can be stopped at an arbitrary position.

  In the above, for simplification of explanation, the example of browsing within one video file has been described. However, even when selecting thumbnails for a plurality of video files, each video file is similarly selected. This can be realized by repeating the process of displaying thumbnails.

  Through the above processing, the browser unit 304 can extract and use only thumbnails suitable for browsing from the thumbnail area 407.

[Utilization processing by video editing]
Next, a case where it is used in moving image editing will be described. FIG. 10 shows an example of a user interface when the editing unit 305 edits a moving image. Reference numeral 1000 denotes an editing work window, which is an area for performing editing work. Reference numeral 1001 denotes a playback window for playing back and confirming the selected moving image file or editing result. A movie file window 1002 displays a list of thumbnails (important thumbnails) in the header of each movie file. Reference numeral 1003 is used to operate a moving image to be played back in the playback window 1001, and “playback / pause / stop / rewind / fast-forward” operations can be performed from the left operation button.

  Reference numeral 1004 denotes a scene designation window, which is an area for arranging the selected moving image files and designating a section to be adopted by each moving image. In the figure, files represented by thumbnails of “sun” and “baseball player” are arranged. Note that thumbnail images are superimposed and displayed in the area of each moving image file so that the user can recognize them. Reference numeral 1005 denotes a scene designation bar, which can be operated to designate the start point and end point of the selected clip (shaded portion).

  Reference numeral 1006 denotes a pointer. The user can operate the user operation unit 300 to move the pointer 1006 to select various buttons and specify operations. As an operation procedure, a moving image file is selected from the moving image file window 1002 and dragged and dropped into the scene designation window 1004 in the order of reproduction. Then, the start point and end point are corrected using the scene designation bar 1005. At this time, the user can play back the moving image in the playback window 1001 by operating the operation unit 1003 as necessary, and can check the content as needed.

  FIG. 11 is a flowchart showing the thumbnail display operation when the editing unit 305 edits a moving image. Note that the processing in FIG. 11 is started when a moving image editing process is executed in the moving image processing apparatus.

  First, in step S300, the editing unit 305 reads the headers 401 of all moving image files from the moving image file storage unit 302. Next, in step S301, the thumbnail 405 in the header 401 of each file is read. In step S302, a thumbnail (that is, an important thumbnail) in the header is displayed on the moving image file window 1002 as a representative image of each moving image file.

  In step S303, it is determined whether a moving image file has been selected. In this embodiment, it is determined whether a moving image file has been dragged and dropped from the moving image file window 1002 to the scene designation window 1004 by operating the pointer 1006. If a moving image file is selected, the process advances to step S304 to read the footer index area 406 in the moving image file selected by the user. On the other hand, if it is determined in step S303 that no moving image file has been selected, step S304 is skipped.

  Next, in step S305, the user checks whether or not the scene range is designated by the scene designation bar 1005 in the scene designation window 1004 for the moving image file. If it is designated, the thumbnails for editing existing within the range designated by the user are extracted and displayed in the scene designation window 1004 in steps S306 to S311.

  First, in step S306, it is determined whether or not the section indicated by the Start Point and End Point is within the user-specified range with reference to the Start Point and End Point in the index (at the beginning if processing is started). To do. If it is within the range, the process proceeds to step S307, and it is determined based on the selection criterion of the thumbnail whether the thumbnail corresponding to the index is a thumbnail suitable for editing. The kind of thumbnail that is suitable for editing may be different depending on the device even if the same editing mode is used. It is assumed that the editing unit 305 of this embodiment determines that the middle frame of the section is a thumbnail suitable for editing. To check whether the frame is in the middle of the section, it is only necessary to check whether Thumbnail Type ID = 2. If the thumbnail is suitable for editing, the process proceeds to step S310.

  On the other hand, if it is determined in step S306 that the corresponding section of the index is not within the user-specified range, or if it is determined in step S307 that it is not a thumbnail for editing, the process proceeds to step S308, and the next index is set. set to target. In step S309, it is determined whether or not the next index exists. If there is, the processes in steps S306 and S307 are repeated.

  When the condition of step S307 is satisfied, the process proceeds to step S310, and the thumbnail image corresponding to the index is read. In the next step S311, a thumbnail image is displayed. For example, as shown in FIG. 10, in the scene designation window 1004, thumbnails corresponding to the bars indicating the scenes are displayed in an overlapping manner. This makes it easier for the user to recognize what kind of scene the range has been designated. Thereafter, the process proceeds to step S308, and the above-described processing is repeated for the next index.

  Through the processing as described above, the thumbnails for editing in all sections included in the range instructed by the user are displayed so as to overlap the bar indicating the scene. In this case, the display range of the thumbnail is an area sandwiched between two scene designation bars 1005. If the size of the sandwiched area is insufficient with respect to the thumbnail display, a part of the thumbnail is overlapped. The display may be made possible, or a scroll bar may be displayed to enable scrolling.

  If no scene range instruction is given from the user in step S305, or if there are no more indexes to be processed in step S309, the process proceeds to step S312. In step S312, it is determined whether or not an instruction to end moving image editing work has been issued. If there is no instruction to end (if work is in progress), the process returns to step S303 and the above processing is repeated. On the other hand, if the end of the editing work is instructed, the process is finished.

  With the above processing, the editing unit 305 can use the important thumbnail stored in the header for file selection, and extracts only the thumbnail for moving image editing corresponding to the selected range from the thumbnail area 407. Can be used.

  In the above embodiment, two examples of thumbnail use are browse and movie editing. However, the present invention is not limited to these two. For example, a frame corresponding to the selected thumbnail is used. It can be used for various usage methods such as printing images.

  For example, when printing a frame image corresponding to the selected thumbnail, a thumbnail suitable for printing is extracted from thumbnails corresponding to the specified image data (thumbnail recorded in the footer) as print frame candidates. Display and print the frame on which the thumbnail selected from them is based. The standard of thumbnails suitable for printing may differ depending on the device even if the usage form is the same printing. For example, if the frame has a gentle motion in the section and the frame has a good image quality in the section, it may be confirmed whether Thumbnail Type ID = 5 and Thumbnail Type ID = 8. When a frame is designated, an image of the frame specified by the Thumbnail Key Relative Point shown in FIG. 6 of the designated thumbnail is acquired and printed. Therefore, it is possible to add other functions such as a printing unit to the moving image processing apparatus of FIG.

  In the above embodiment, the browse unit and the edit unit have two functions that utilize thumbnails. However, any one of these functions including the above-described print unit and the like may be used. May have only some functions, or may have some functions.

  In the above embodiment, the moving image processing apparatus shown in FIG. 3 includes all of the moving image input unit 301, the thumbnail creation unit 303, the browse unit 304, and the editing unit 305 in one device. In addition, other devices such as the above-described printing unit may be configured as separate devices for each function or for some combination of functions. At this time, a moving image file including thumbnails may be exchanged between the devices via a storage medium or via communication means such as IEEE1394.

  In the above-described embodiment, since there is no index for the header thumbnail, the Start Point, End Point, and Thumbnail Type ID are not held, but these may be held as management information.

  In the above embodiment, the thumbnail (thumbnail representing the file) recorded in the header is not recorded in the footer, but it may be recorded in the header and footer.

  If the same thumbnail is created from multiple creation criteria (for example, if the middle frame of the section and the frame that moves rapidly in the section are the same), assign the same thumbnail to a different Thumbnail Type ID. It may be registered redundantly, or a corresponding Thumbnail Type ID (for example, Thumbnail Type ID = 10: a frame that is in the middle of the section and has a strong motion in the section) may be prepared. In the latter case, it is not necessary to register the same thumbnail image repeatedly, so that the data use efficiency is improved. A plurality of selection criteria (Thumbnail Type ID) may be registered in one thumbnail.

  In the above embodiment, the thumbnail registration is determined by the user's instruction, but it is of course possible to automate this. For example, in order to automatically extract frames with intense motion in a section, a plurality of blocks are divided vertically and horizontally into the previous frame and the current frame as shown in FIG. An average value of RGB is calculated, and a sum of squares of differences between RGB channels of corresponding blocks of the previous frame and the current frame is obtained, and this is used as an interframe similarity distance. An example of the calculation formula is shown below.

However,
i: Indicates the block being processed
K: Number of divided blocks
P1 _iR : Average value of the R channel of the i-th block of the previous frame
P1 _iG : Average value of the G channel of the i-th block of the previous frame
P1 _iB : Average value of the B channel of the i-th block of the previous frame
P2 _iR : Average value of R channel of i-th block of current frame
P2 _iG : Average value of the G channel of the i-th block of the current frame
P2 _iB : Average value of the B channel of the i-th block of the current frame.

  The smaller the similarity distance between the frames, the more similar, and the larger the similarity distance, the less similar. As the motion becomes more intense, the frames before and after become less similar. Therefore, there is a method in which a frame having a distance between frames equal to or greater than a certain threshold is used as a frame with intense motion.

  As described above, according to the present embodiment, thumbnails having different usage purposes and selection criteria can be stored in one file, and management thereof is facilitated. In addition, by referring to the selection criteria (Thumbnail Type ID) given to the thumbnail when using it, it is possible to use only the thumbnail according to its purpose and the characteristics of the device that uses the thumbnail. Become.

  Another object of the present invention is to supply a storage medium storing software program codes for implementing the functions of the above-described embodiments to a system or apparatus, and the computer (or CPU or MPU) of the system or apparatus stores the storage medium. Needless to say, this can also be achieved by reading and executing the program code stored in.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention.

  As a storage medium for supplying the program code, for example, a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile raw memory card, a ROM, or the like can be used.

  Further, by executing the program read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) running on the computer is actually executed based on an instruction of the program code. It goes without saying that a case where the function of the above-described embodiment is realized by performing a part or all of the process and the process is included.

  Further, after the program code read from the storage medium is written to a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. It goes without saying that the CPU or the like provided in the board or the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

It is a figure for demonstrating the problem in utilizing a representative image. It is a block diagram which shows the control structure of the moving image processing apparatus in this embodiment. It is a block diagram which shows the function structure of the moving image processing apparatus in this embodiment. It is the schematic which shows an example of the thumbnail storage form in the moving image file in this embodiment. FIG. 5 is a diagram illustrating a data structure example for one index in an index area 406 in the footer 403 of FIG. 4. FIG. 5 is a diagram illustrating a data structure example of one thumbnail in a thumbnail 405 in a header 401 and a thumbnail area 407 in a footer 403 in FIG. 4. It is the flowchart which showed the operation example of the thumbnail registration by this embodiment. It is a figure which shows an example of the user interface at the time of browser execution. It is a flowchart which shows the process at the time of the browser operation | movement of this moving image processing apparatus. It is a flowchart which shows the process at the time of the browser operation | movement of this moving image processing apparatus. It is an example of the user interface at the time of the moving image edit by the edit part 305. FIG. 10 is a flowchart showing an operation of displaying a thumbnail during editing of a moving image by the editing unit 305. It is a figure which shows the example of the block division of a frame.

Claims (2)

An apparatus for managing one or a plurality of representative images in association with a moving image recorded on a storage medium,
The representative image is stored in association with the moving image, and the storage unit storing the selection reference information of the representative image in association with the representative image is accessed, and the representative image stored in correspondence with the desired moving image is stored. Selecting means for selecting a representative image according to the processing purpose from among the selection criteria information;
When one of the representative images selected by the selection unit is designated, an application unit that applies a process according to the processing purpose to a moving image in a section indicated by the section information corresponding to the designated representative image; A moving image management apparatus comprising: A method for managing one or a plurality of representative images in association with a moving image recorded on a storage medium,
The representative image is stored in association with the moving image, and the storage unit storing the selection reference information of the representative image in association with the representative image is accessed, and the representative image stored in correspondence with the desired moving image is stored. A selection step of selecting a representative image according to the processing purpose from among the selection criteria information;
When one of the representative images selected in the selection step is designated, an application step of applying a process according to the processing purpose to a moving image in a section indicated by the section information corresponding to the designated representative image; A moving image management method comprising:

Images