JP2011130007A - Moving picture processing apparatus and moving picture processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To efficiently search a scene image of a desired moment in a moving picture having a plurality of chapters. <P>SOLUTION: A moving picture processing apparatus 11 according to the invention includes a moving picture data memory 17, a large block dividing unit 31, a complexity analysis unit 33, a small block dividing unit 35, and a chapter creation unit 37. The moving picture data memory 17 stores moving picture data presenting a predetermined reproduction time. The large block dividing unit 31 divides the moving picture into a plurality of large blocks for each predetermined unit time. The complexity analysis unit 33 quantifies the complexity of the image change in each large block. The small block dividing unit 35 divides the reproduction time for each large block into a plurality of small blocks, respectively, by a numeric value determined by the complexity analysis unit 33 and related to the corresponding complexity. The chapter creation unit 37 partitions the plurality of small blocks one by one for eash predetermined number along with time series to create a chapter. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a moving image processing apparatus and a moving image processing program capable of efficiently retrieving a scene image at a desired moment from a moving image having a plurality of chapters.

  Due to the recent broadbandization of the Internet and the like, the distribution amount of digital contents is increasing. Attempts have been made to efficiently manage and process such digital contents by adding metadata to computers.

  As an example of such an attempt, a moving image is divided into a plurality of chapters and metadata is added to the division points (see, for example, Patent Document 1). In addition, as a method of dividing a moving image into a plurality of chapters, as shown in Patent Document 1, there are a method of equally dividing a moving image into a predetermined number of chapters, and a method of equally dividing a moving image into predetermined reproduction times. Are known.

  According to the chapter division technique according to Patent Document 1, it is easy to find and search a scene image, and it is possible to improve user convenience, such as cutting out and printing a desired scene image from a moving image.

  By the way, in general, in a moving image, a change in the movement of an image along a time series is gradual in some places but abrupt in some places and is constantly changing and not constant.

  However, the chapter division method described in Patent Document 1 does not take into account changes in image motion that a moving image generally has, in other words, regardless of whether the image motion changes slowly or suddenly. Are uniformly divided into a plurality of chapters. For this reason, there is a possibility that a large variation occurs in the data amount of the moving image belonging to each chapter.

  When searching for scene images efficiently across chapters with large variations in data volume, the search time is short for chapters with small data volumes, but the search time is long for chapters with large data volumes. For example, the time required to search for a scene image may be greatly different among a plurality of chapters. As a result, it takes a long time to search for a scene image at a desired moment from a moving image having a plurality of chapters, which may impair user convenience.

JP 2006-246446 A

  The problem to be solved by the invention is that, in the conventional chapter division method, it takes a long time to search for a scene image at a desired moment from a moving image having a plurality of chapters, which may impair user convenience. It is a point.

  A moving image processing apparatus according to the present invention includes a moving image data storage unit that stores moving image data, a large block dividing unit that divides the moving image into a plurality of large blocks every predetermined unit time, and each divided large block A complexity analysis unit for obtaining a numerical value related to the complexity of the image change, and a reproduction time for each of the large blocks are divided into a plurality of small blocks according to the numerical value for the corresponding complexity obtained by the complexity analysis unit. The main feature is that it includes a small block dividing unit that generates a chapter by sequentially dividing the plurality of small blocks by a predetermined number from the top in time series.

  A moving image processing program according to the present invention includes a large block dividing function for dividing a moving image having a predetermined reproduction time into a plurality of large blocks every predetermined unit time, and a complicated image change of each divided large block. A complexity analysis function for obtaining a numerical value relating to the size, and a small block dividing function for dividing the reproduction time relating to each large block into a plurality of small blocks according to the numerical value relating to the corresponding complexity obtained by the analysis function And a chapter creation function for creating chapters by sequentially dividing the plurality of small blocks in a predetermined number from the top in chronological order.

  In the moving image processing apparatus and the moving image processing program according to the present invention, a scene image at a desired moment can be efficiently searched from a moving image having a plurality of chapters.

It is a functional block diagram of the moving image processing apparatus according to the embodiment. It is a flowchart with which it uses for operation | movement description of the moving image processing apparatus which concerns on an Example. FIG. 3 is a diagram for explaining the operation of the moving image processing apparatus according to the embodiment. FIG. 3A is an explanatory diagram illustrating a process of equally dividing the moving image into large blocks, and FIG. It is explanatory drawing which shows the state which divided | segmented the small block sequentially for every predetermined number along the time series from the head, and was chapter-ized.

  For the purpose of enabling efficient retrieval of a scene image at a desired moment from a moving image having a plurality of chapters, (1) the moving image is divided into a plurality of large blocks every predetermined unit time, and (2) each large block The reproduction time is divided into a plurality of small blocks according to the complexity of the image change, and (3) the plurality of divided small blocks are sequentially divided in a predetermined number from the top in time series. Realized by creating chapters.

A moving image processing apparatus and a moving image processing program according to an embodiment of the present invention will be described in detail with reference to the drawings, taking as an example a digital multi-function peripheral in which the moving image processing program is installed.
[Block configuration of video processing device]
FIG. 1 is a functional block diagram of the moving image processing apparatus according to the embodiment.

  As shown in FIG. 1, the digital multifunction peripheral 11 corresponding to the moving image processing apparatus of the present invention includes a computer 13, an operation panel 15, a storage unit 17, and a printing unit 21.

  The computer 13 is a device that controls the operation of the digital multi-function peripheral 11 by executing a software program. The computer 13 has a CPU (Central Processing Unit) and a RAM (Random Access Memory). The CPU is an arithmetic processing unit that executes processing described in the software program. The RAM is a memory that temporarily stores the software program and data when the software program is executed and uses it as a work area.

  The computer 13 realizes various functions by executing an instructed predetermined program among various software programs stored in the storage unit 17 described later. In particular, when the predetermined program is an operation processing program, the computer 13 executes the moving image processing program to thereby execute a large block dividing unit 31, a complexity analyzing unit 33, a small block dividing unit 35, a chapter creating unit 37, and Each functional unit including the scene image extraction unit 39 is configured.

  The large block dividing unit 31 has a function of dividing a moving image having a predetermined reproduction time into a plurality of large blocks every predetermined unit time.

  In this embodiment, as the moving image data 45 to be processed, moving image data in MPEG (Motion Picture Experts Group) format that continuously reproduces a plurality of still image frames composed of image data serving as a reproduction reference and difference data thereof at a predetermined speed is used. The explanation will be given by way of example.

  The complexity analysis unit 33 has a function of quantifying the degree of complexity of the image change of each large block divided by the large block division unit 31. Here, the complexity of the image change is an index representing the intensity of the movement of the image. In this embodiment, the complexity of the image change is set to 6 levels, and for an image that is changing slowly, the numerical values of 1 to 2 are set for the image that is changing to the middle level. By assigning a numerical value of 3 to 4 according to the stage and assigning a numerical value of 5 to 6 to the image that changes drastically according to the stage, the complexity of the image change of each large block is quantified.

  Note that the complexity data of the image change may be acquired based on the color change of the pixel by paying attention to a certain pixel (including a group of a plurality of pixels) in the image frame, for example. Further, image change complexity data may be acquired based on known MPEG motion detection data (motion vector) as disclosed in, for example, Japanese Patent Laid-Open No. 2009-76972. By the way, a motion vector is one of the characteristic values for expressing moving image data, and it includes “reference frame (image corresponding to a certain moment of moving image)” and “motion from the reference frame”. It is expressed as vector data.

  The small block dividing unit 35 has a function of dividing the reproduction time of each large block into a plurality of small blocks by dividing the reproduction time by the numerical value related to the corresponding complexity obtained by the complexity analyzing unit 33.

  The chapter creation unit 37 has a function of creating chapters by sequentially dividing the plurality of small blocks by a predetermined number along a time series. The predetermined number is actually a value obtained by dividing the sum of numerical values related to the complexity of each large block by the number of divisions of the plurality of large blocks. Each chapter created by the chapter creation unit 37 will be described in detail later, but the time integration values of the complexity of each image change are equalized.

  Based on a scene image selected and operated by the user and a later-described similar image group table 47 stored in the storage unit 17, the scene image extraction unit 39 selects the sharpest scene from the similar image group of the one scene image. It has a function to extract an image.

  As illustrated in FIG. 1, the operation panel 15 includes a touch panel unit 41 and a function key unit 43. The operation panel 15 is used for the user to perform various operations, and gives operation commands and the like by the user to the computer 13.

  The touch panel unit 41 corresponds to the preview display unit of the present invention, and includes a touch panel unit that combines a touch panel and a color LCD (Liquid Crystal Display). The touch panel unit 41 displays operation buttons for inputting various operation commands when the user touches the corresponding part.

  The function key unit 55 corresponds to the scene image selection operation unit of the present invention. The function key unit 55 includes a plurality of function keys that are operated when items related to various functions relating to image formation and moving image processing are selected or set. When the user operates this function key portion 55, a scene image at a moment desired by the user is selected from the scene images displayed as previews on the touch panel portion 41.

  The storage unit 17 includes a primary storage device made up of a semiconductor storage device and a secondary storage device made up of a magnetic disk device or the like. The storage unit 17 has a function of storing various software programs and data related to image formation and moving image processing.

  Specifically, the storage unit 17 stores moving image data 45 to be processed and a similar image group table 47. The similar image group table 47 stores corresponding similar image groups in association with a plurality of scene images selected from a plurality of viewpoints.

The printing unit 21 corresponds to the scene image printing unit of the present invention, and is a print engine that transfers and outputs a scene image selected by the user onto a sheet.
[Operation of video processing device]
Next, the operation of the moving image processing apparatus according to the embodiment will be described with reference to FIGS.

  FIG. 2 is a flowchart for explaining the operation of the moving image processing apparatus according to the embodiment. FIG. 3 is a diagram for explaining the operation of the moving image processing apparatus. FIG. 3 (A) is an explanatory diagram showing a process of equally dividing moving image data in MPEG format into large blocks, and FIG. It is explanatory drawing which shows the state which divided | segmented every small block of this to a predetermined number along a time series from the head, and was chapter-ized.

  In step S <b> 11, as shown in FIG. 2, the computer 13 reads MPEG format moving image data 45 to be processed from the storage unit 17.

  In step S12, as shown in FIG. 3 (A), the large block dividing unit 31 converts the moving image data 45 into a plurality of large blocks X1, X2, X3,. , X10. Specifically, assuming that the reproduction time of the moving image data 45 is “30 minutes”, the moving image data 45 includes a plurality of “10” large blocks X1, X2, X3 every unit time “3 minutes”. ,..., X10 are equally divided.

  The graph shown in FIG. 3A is a bar graph obtained by sampling the complexity of the image change of a large block per unit time. The horizontal axis shows the playback time. The vertical axis indicates the complexity of the image change, and the longer the bar, the more complicated the image change per unit time. In the example of FIG. 3A, the total playback time of the moving image is T, and T is divided by a certain constant. In this embodiment, the total reproduction time T is divided by “10”.

  In step S13, as shown in FIG. 3A, the complexity analysis unit 33 determines the complexity of the image change for each of the large blocks X1, X2, X3,..., X10 (unit time “3 minutes”). Digitize the degree. Numerical values relating to the complexity of the image change thus obtained are described in association with each of the plurality of large blocks X1, X2, X3,..., X10 at the top of each graph in FIG. Yes.

  Specifically, in the present embodiment, the large block X1 (1; however, the numerical value relating to the complexity of the image change is shown in parentheses. The same applies hereinafter), the large block X2 (1), and the large block X3 (2). , Large block X4 (3), large block X5 (5), large block X6 (6), large block X7 (6), large block X8 (4), large block X9 (4), large block X10 (3) Become.

  Note that the complexity data of the image change is obtained based on a color change of the pixel by paying attention to a certain pixel (including a group of a plurality of pixels) in the image frame, or an MPEG motion Acquired based on detection data (motion vector).

  Next, in step S14, the small block dividing unit 35, as shown in FIG. 3B, reproduces each large block X1, X2, X3,..., X10 (the unit time is “3 minutes”). “)” Is divided by a numerical value related to the corresponding complexity obtained by the complexity analysis unit 33 to divide each into a plurality of small blocks.

  Specifically, for example, in the large block X3 (3), the reproduction time “3 minutes” is divided by the numerical value “3” relating to the corresponding complexity to obtain three small blocks (reproduction time of each block is "1 minute"). Also, in the large block X7 (6), the reproduction time “3 minutes” is divided by the corresponding complexity value “6” to obtain six small blocks (reproduction time of each block is “0.5 minutes each”). ”).

  In step S15, the chapter creation unit 37 creates chapters by sequentially dividing the plurality of small blocks by a predetermined number along a time series. The predetermined number is actually “35”, which is the sum of numerical values related to the complexity of the large blocks X1, X2, X3,..., X10, and the division number “10” of the plurality of large blocks. The divided value is “3.5”.

  As shown in FIG. 3B, the chapter creation process of the chapter creation unit 37 starts a chapter branch point CH0, which is the start point of the moving image data 45, and sets a predetermined number of the plurality of small blocks in time series. By dividing every 3.5 ", a chapter" A "partitioned by chapter branch points CH0 to CH1 is created. In FIG. 3B, the branch point of each chapter is indicated by an arrow.

  In the same manner as described above, the chapter “B” defined by the chapter branch points CH1 and CH2 is divided by a predetermined number “3.5” along the time series starting from the chapter branch point CH1. "Is created.

  In the same manner as described above, the chapter “C” defined by the chapter branch points CH3 to CH4 is divided by a predetermined number “3.5” along the time series starting from the chapter branch point CH3. "Is created.

  In the same manner, other chapters “D to I” are sequentially created.

  When FIG. 3B is analyzed, it can be seen that the playback time is shorter for chapters having a larger image change complexity (for example, chapters E and F shown in FIG. 3B).

  The important point in the present embodiment is that the chapters A, B, C,... Created in the above-described procedure are equalized in time integral values of the complexity of each image change. In other words, the amount of information for each chapter A, B, C,... Is almost equalized. This is generally based on the premise that there is a correlation between the amount of information contained in the video and the search effort, and the search effort among the chapters A, B, C,. Means you can.

  Therefore, according to the moving image processing method according to the present embodiment, a scene image at a desired moment can be efficiently searched from a moving image having a plurality of chapters.

  In step S16, the computer 13 determines whether a scene image search process has been executed by the user.

  As a result of the determination in step S16, when the scene image search process is not executed, the computer 13 terminates the entire process flow. On the other hand, when the scene image search process is executed, the computer 13 advances the process flow to step S17. Make it.

  In step S <b> 17, the computer 13 displays a preview of the scene image belonging to the chapter selected by the user on the display screen of the touch panel unit 41. By sequentially confirming the scene images thus previewed visually, a search operation for a scene image at a moment desired by the user is executed (step S18).

  As a preview display mode at the time of retrieval, a scene image belonging to a chapter having a large image change complexity is displayed as a preview by slow reproduction. Here, chapters with large image change complexity (for example, chapters E and F shown in FIG. 3B) generally mean chapters with a large amount of information included therein.

  For chapters with a large amount of information, a desired scene image can be searched while sequentially displaying a preview in the slow (frame advance) playback mode, so that the user can accurately find the scene image at a desired moment without overlooking it.

  On the other hand, a scene image belonging to a chapter with a small image change complexity is displayed as a preview by fast-forward playback. Here, chapters with small image change complexity (for example, chapters A and B shown in FIG. 3B) generally mean chapters with a small amount of information included therein.

  For chapters with a small amount of information, by searching for a desired scene image while sequentially displaying a preview in the fast-forward playback mode, it is possible to accurately find a scene image at a user's desired moment while shortening the search time.

  In step S19, the computer 13 determines whether or not a scene image selection operation has been performed by the user.

  As a result of the determination in step S19, when the scene image selection operation is not executed, the computer 13 terminates the entire process flow. On the other hand, when the scene image selection operation is executed, the computer 13 advances the process flow to step S20. Make it.

  In step S <b> 20, the scene image extraction unit 39 most sharply selects the similar image group of the one scene image based on the scene image selected and operated by the user and the similar image group table 47 stored in the storage unit 17. A simple scene image.

  In this scene image extraction process, it is only necessary to pay attention to the edge portion of each scene image belonging to the group of similar images, and to extract the scene image having a clear outline of the edge portion as the sharpest scene image.

  In step S21, the computer 13 causes the printing unit 21 to transfer the extracted scene image onto a sheet for output. As a result, the user can easily and automatically obtain an output product of a scene image at a moment desired by the user and having a high definition.

Then, after the printing process in step S21, the computer 13 ends the flow of a series of processes.
[Effect of Example]
In the moving image processing apparatus according to the present embodiment, the moving image is divided into a plurality of large blocks every predetermined unit time, the complexity of the image change of each divided large block is quantified, and the reproduction time associated with each large block Is divided into a plurality of small blocks by numerical values related to the corresponding complexity, and a configuration is adopted in which chapters are created by sequentially dividing the plurality of small blocks by a predetermined number along the time series from the top. For this reason, a scene image at a desired moment can be efficiently searched from a moving image having a plurality of chapters.

  Further, in the moving image processing apparatus according to the present embodiment, the predetermined number used when dividing a plurality of small blocks in the chapter creating unit 37 is the sum of numerical values related to the complexity of each large block, and the plurality of large blocks The chapters A, B, C,... Created by the chapter creation unit 37 are equalized in time integral values of the complexity of image changes. In other words, the amount of information for each of the chapters A, B, C,.

  This is generally based on the premise that there is a correlation between the amount of information contained in a moving image and the search effort, and the search effort among the chapters A, B, C,. Means that

  Therefore, according to the moving image processing apparatus according to the present embodiment, a scene image at a desired moment can be efficiently searched from a moving image having a plurality of chapters.

  Furthermore, the moving image processing apparatus according to the present embodiment includes a preview display unit (touch panel 41) for previewing scene images belonging to the chapters. In this preview display, scene images belonging to chapters having a large image change complexity. Adopted a configuration in which a preview is displayed by slow playback.

  Here, chapters with large image change complexity (for example, chapters E and F shown in FIG. 3B) generally mean chapters with a large amount of information included therein. For chapters with a large amount of information, a desired scene image can be searched while sequentially displaying a preview in the slow (frame advance) playback mode, so that the user can accurately find the scene image at a desired moment without overlooking it.

  On the other hand, a configuration is adopted in which a scene image belonging to a chapter with a small image change complexity is displayed as a preview by fast-forward playback. Here, chapters with small image change complexity (for example, chapters A and B shown in FIG. 3B) generally mean chapters with a small amount of information included therein. For chapters with a small amount of information, by searching for a desired scene image while sequentially displaying a preview in the fast-forward playback mode, it is possible to accurately find a scene image at a user's desired moment while shortening the search time.

  In addition, in the moving image processing apparatus according to the present embodiment, the scene image selection unit (operation key unit 43) that selects one scene image from the scene images that belong to the chapter previewed by the preview display, and the selected one A scene image extraction unit 39 that extracts the sharpest scene image from a group of similar images of the scene image, and a scene image printing unit (printing unit 21) that prints the extracted scene image. For this reason, the user can easily and automatically obtain an output product of a scene image at a moment desired by the user and having a high definition.

  In the moving image processing program according to the present embodiment, the computer is divided into a large block dividing function for dividing a moving image having a predetermined reproduction time into a plurality of large blocks every predetermined unit time, and the large block dividing function. In addition, a complexity analysis function for obtaining a numerical value related to the complexity of the image change of each large block, and a reproduction time relating to each of the large blocks are divided into a plurality of small blocks according to the calculated numerical value relating to the corresponding complexity, respectively. And a chapter creation function for creating chapters by sequentially dividing the plurality of small blocks in a predetermined number from the top in time series. For this reason, a scene image at a desired moment can be efficiently searched from a moving image having a plurality of chapters.

Finally, in the moving image processing apparatus or the moving image processing program according to the present embodiment, for example, when a common capacity limit is imposed on each chapter, the data amount of each chapter can be accurately suppressed within the limited capacity. Such secondary effects can be expected.
[Others]
The present invention is not limited to the above-described embodiments, and can be changed as appropriate within the scope of the invention or the technical idea that can be read from the claims and the entire specification. The apparatus and the moving image processing program are also included in the scope of the technical scope of the present invention.

  That is, in the embodiments of the present invention, the example in which the moving image processing apparatus is configured by installing the moving image processing program according to the present invention in the digital multifunction peripheral has been described, but the present invention is not limited to this example. The moving image processing apparatus according to the present invention may be configured, for example, by installing the moving image processing program according to the present invention in a personal computer (PC) capable of reproducing and processing a moving image.

  In the above embodiment, moving image data in the MPEG format is targeted for moving image processing, but moving image data in other formats may be targeted. For example, moving image processing may include moving image data in a slide show format including a plurality of individual still image data and still image reproduction instruction data including the order, time, and various display effects for displaying each still image data.

11 Digital multi-function peripheral (video processing device)
13 Computer 15 Operation Panel 17 Storage Unit 21 Printing Unit (Scene Image Printing Unit)
31 Large block division unit 33 Complexity analysis unit 35 Small block division unit 37 Chapter creation unit 39 Scene image extraction unit 41 Touch panel unit (preview display unit)
43 Operation key section (scene image selection operation section)
45 Movie data (video)
47 Similar Images Group Table

Claims (5)

A video data storage unit for storing video data;
A large block dividing unit that divides the video into a plurality of large blocks every predetermined unit time;
A complexity analysis unit for obtaining a numerical value related to the complexity of the image change of each divided large block;
A small block dividing unit that divides the reproduction time of each large block into a plurality of small blocks according to the numerical values related to the corresponding complexity obtained by the complexity analyzing unit;
A chapter creating unit that creates chapters by sequentially dividing the plurality of small blocks into a predetermined number along a time series; and
A moving image processing apparatus comprising: The moving image processing apparatus according to claim 1,
The predetermined number is a value obtained by dividing the sum of numerical values related to the complexity of each large block by the number of divisions of the plurality of large blocks,
A plurality of chapters created by the chapter creation unit, the time integration value of the complexity of each image change is equalized,
A moving picture processing apparatus. The moving image processing apparatus according to claim 1 or 2,
A preview display unit for previewing scene images belonging to the chapters;
In the preview display section, a scene image belonging to a chapter with a large image change complexity is preview-displayed by slow playback, and a scene image belonging to a chapter with a small image change complexity is preview-displayed by fast-forward playback,
A moving picture processing apparatus. The moving image processing apparatus according to any one of claims 1 to 3,
A scene image selection operation unit for selecting and operating one scene image from the scene images belonging to the chapter previewed in the preview display unit;
A similar image group storage unit that stores a similar image group of a corresponding scene image in association with each scene image;
A scene image extraction unit that extracts the sharpest scene image from the similar image group of the one scene image based on the selected one scene image and the stored content of the similar image group storage unit;
A scene image printing unit for printing the extracted scene image;
A moving image processing apparatus comprising: On the computer,
A large block division function for dividing a video having a predetermined playback time into a plurality of large blocks every predetermined unit time;
A complexity analysis function for obtaining a numerical value related to the complexity of the image change of each divided large block;
A small block dividing function that divides the reproduction time of each large block into a plurality of small blocks by numerical values related to the corresponding complexity obtained by the analysis function;
Realizing a chapter creation function for creating chapters by sequentially dividing the plurality of small blocks from the top in a time-series manner by a predetermined number;
A moving picture processing program characterized by the above.