JP2013198012A - Information processor, still-image frame sampling method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To sample a more suitable still-image frame representing the whole of a moving image.SOLUTION: The information processor 1 comprises: a vector calculation section 112 for calculating an amount of movement for each still-image frame by comparison of neighbouring still-image frames; an image selection section 113 for selecting as a marking frame a still-image frame whose calculated amount of movement exceeds a predetermined threshold; a sampling interval setting section 116 for setting, under the condition that the total number of selected marking frames exceeds the number of samples, a sampling interval on the basis of the total number and the number of samples; and a sampling section 117 for sampling still-image frames for the number of samples from the marking frames lined up in an order of appearance in a moving image, according to the set sampling intervals.

Description

  The present invention relates to an information processing apparatus, a still image frame extraction method, and a program for extracting a predetermined number of still image frames with large motion from a moving image including a plurality of continuous still image frames.

  In recent years, digital images are used to manage captured images with the development of digital technology. In recent years, imaging devices incorporating functions for processing captured images are known. For example, digital stills having a motion print (registered trademark) function are known. The camera is known. Note that the motion print function is a function for extracting one or a plurality of still image frames from a moving image captured at a predetermined frame rate and generating image data of one still image.

  As a method of extracting a plurality of still image frames from a moving image, for example, a method of extracting a plurality of still image frames constituting a moving image at equal intervals in time is known. However, in such a method, only a similar still image frame is extracted for a moving image with a small amount of movement and a long time zone.

Therefore, in recent years, a method of extracting a still image frame having a large motion (change) between adjacent still image frames is also known (Patent Documents 1 and 2).
Here, if a large number of still image frames with large motion are included in a plurality of still image frames constituting a moving image, more still image frames than expected are extracted. For this reason, Patent Documents 1 and 2 also disclose a function of adjusting the number of still image frames to be extracted. For example, Patent Document 1 discloses that similar still image frames among extracted still image frames are combined into one to reduce the number of extractions, and Patent Document 2 discloses a predetermined number (20 images in descending order of motion). ) Is extracted.

JP 09-107517 A JP 2009-141882 A

However, when similar still image frames are combined into one as in Patent Document 1, the number of extractions cannot be reduced unless similar still image frames are included in many extracted still image frames. Furthermore, in the method of Patent Document 1, it is necessary to perform image recognition such as specifying a similar still image frame, which requires complicated processing, leading to an increase in the cost of the apparatus and a reduction in reaction speed.
In addition, when extracting a predetermined number of images in descending order of movement as in Patent Document 2, a still picture that is extracted in a moving picture that is biased in a time zone in which movement is large, for example, a moving picture that is 60 minutes and has a large movement for the last 5 minutes. There is a possibility that the position of the image frame in the moving image is biased and the still image frame representing the entire moving image cannot be extracted properly.

  The present invention has been made in view of such a situation, and an information processing apparatus, a still image frame extraction method, and a method for extracting a more appropriate still image frame that represents the entire moving image without requiring complicated processing. The purpose is to provide a program.

  In order to achieve the above object, an information processing apparatus according to one embodiment of the present invention is an information processing apparatus that extracts a predetermined number of still image frames with large movement from a moving image including a plurality of continuous still image frames. A vector calculation unit that calculates a motion amount for each still image frame by comparing the image frames, and a still image frame in which the motion amount calculated by the vector calculation unit exceeds a predetermined threshold. An extraction interval is set based on the total number and the predetermined number on condition that the total number of still image frames selected by the image selection unit and the image selection unit selected as the frame exceeds the predetermined number. Extraction interval setting means for performing the extraction interval setting from the extraction candidate still image frames arranged according to the appearance order in the moving image. There accordance extraction interval set, characterized in that it comprises extraction means for extracting a still image frame of the predetermined number.

  According to the present invention, it is possible to extract a more appropriate still image frame that represents the entire moving image without requiring complicated processing.

It is a block diagram which shows the hardware constitutions of the information processing apparatus which concerns on one Embodiment of this invention. It is a functional block diagram which shows the functional structure for performing a motion print process among the functional structures of the information processing apparatus of FIG. It is a figure explaining the function of the image selection part of FIG. It is a figure explaining the function of the extraction space | interval setting part of FIG. 3 is a flowchart for explaining a flow of motion print processing executed by the information processing apparatus of FIG. 1 having the functional configuration of FIG. 2. 6 is a flowchart for explaining the continuation of the motion print process in FIG. 5. It is a figure which shows an example of the motion print image produced | generated by the motion print process of FIG.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a hardware configuration of an information processing apparatus 1 according to an embodiment of the present invention. The information processing apparatus 1 is configured as a digital still camera, for example.

  The information processing apparatus 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input / output interface 15, a display unit 16, and an input unit. 17, an imaging unit 18, a storage unit 19, a communication unit 20, and a drive 21.

  The CPU 11 executes various processes according to a program recorded in the ROM 12 or a program loaded from the storage unit 19 to the RAM 13.

  The RAM 13 appropriately stores data necessary for the CPU 11 to execute various processes.

  The CPU 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output interface 15 is also connected to the bus 14. A display unit 16, an input unit 17, an imaging unit 18, a storage unit 19, a communication unit 20, and a drive 21 are connected to the input / output interface 15.

The display unit 16 is configured by a display, and displays images, moving images, and the like captured by the imaging unit 18.
The input unit 17 inputs various types of information according to user instruction operations.

The imaging unit 18 is an optical camera that includes an imaging device, AFE (Analog Front End), and the like.
In the present embodiment, the imaging element is configured by a CMOS (Complementary Metal Oxide Semiconductor) sensor type photoelectric conversion element. The image sensor is provided with a color filter such as a Bayer array. The image sensor photoelectrically converts (captures) an optical signal of a subject image incident and accumulated during a certain time interval, and sequentially supplies an analog electric signal obtained as a result to the AFE. The AFE performs various signal processing such as A / D (Analog / Digital) conversion processing on the analog electric signal, and outputs the resulting digital signal as an output signal of the imaging unit 18.
Here, the imaging unit 18 in the present embodiment is configured to capture a moving image having a predetermined frame rate in addition to a still image. Hereinafter, a digital signal representing a still image is referred to as “image data”, and a digital signal representing a moving image is referred to as “moving image data”. Note that the moving image data includes still image data (hereinafter referred to as “still image frames”) in units of frames constituting the moving image. Image data and moving image data output from the imaging unit 18 are stored in the storage unit 19 automatically or in response to a user instruction, and are appropriately read out by the CPU 11 or the like.

The storage unit 19 is configured by a hard disk or a DRAM (Dynamic Random Access Memory) or the like, and stores various programs such as application programs in addition to various image data.
The communication unit 20 controls communication performed with other devices (not shown) via a network including the Internet.

  A removable medium 41 made of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately attached to the drive 21. The program read from the removable medium 41 by the drive 21 is installed in the storage unit 19 as necessary. The removable medium 41 can also store various data such as image data stored in the storage unit 19 in the same manner as the storage unit 19.

  FIG. 2 shows a motion print process of the functional configuration of the information processing apparatus 1, that is, one or a plurality of still image frames extracted from a moving image captured at a predetermined frame rate, and image data of one still image. It is a functional block diagram which shows the functional structure for performing the process which produces | generates. In the present embodiment, in the motion print process, nine still image frames are extracted from a moving image to generate image data of one still image. Hereinafter, the image data of one still image generated in the motion print process may be referred to as a “motion print image”.

  In the CPU 11, when the execution of the motion print process is controlled, the moving image acquisition unit 111, the vector calculation unit 112, the image selection unit 113, the threshold adjustment unit 114, the setting unit 115, and the extraction interval setting unit 116 The extraction unit 117 functions.

  The moving image acquisition unit 111 acquires moving image data from the storage unit 19 that stores moving image data or the removable medium 41. The moving image data acquired by the moving image acquisition unit 111 is supplied to the vector calculation unit 112 to the extraction unit 117 as a target of motion print processing.

The vector calculation unit 112 extracts a still image frame constituting the moving image data and compares adjacent still image frames to calculate a motion amount (hereinafter referred to as “vector”) for each still image frame. The adjacent still image frames are the n-th (n is an integer value of 1 or more) still image frames and the (n−1) (or n + 1) -th frame among the continuous still image frames constituting the moving image. The still image frame.
As a method for calculating a vector of still image frames, a publicly known method can be employed. For example, after dividing a still image frame to be processed (nth image) into a predetermined number of regions, This is done by calculating feature points and calculating how the feature points have moved in adjacent still image frames. That is, when a plurality of feature points are detected from the nth still image frame, the feature points correspond to the feature points of the n−1th still image frame among the detected feature points of the nth still image frame. The corresponding point is specified, and a vector from the feature point of the (n-1) th still image frame to the corresponding point of the nth still image frame is calculated.

The image selection unit 113 selects a still image frame in which the vector calculated by the vector calculation unit 112 exceeds a predetermined threshold as an extraction candidate still image frame (hereinafter referred to as a “marking frame”). That is, the image selection unit 113 selects a still image frame with a large motion as a marking frame from among the still image frames constituting the moving image data.
By the way, the motion of a still image frame in a moving image varies depending on the type of the moving image, and there is a moving image with a small movement and a moving image with a large movement. For example, a landscape video or the like by a fixed-point camera is a video with a small motion, and a video such as a sports broadcast is a video with a large motion. For this reason, it is preferable that the predetermined threshold value for determining the magnitude of the motion can be adjusted as appropriate in order to support various types of moving images.

The threshold adjustment unit 114 adjusts the predetermined threshold appropriately. As an example, the threshold adjustment unit 114 lowers the threshold when the number of marking frames selected by the image selection unit 113 is too small, and sets a predetermined threshold so that more still image frames are selected as marking frames. Make adjustments. On the other hand, the threshold adjustment unit 114 increases the threshold when the number of marking frames selected by the image selection unit 113 is too large, and adjusts the predetermined threshold to narrow down the selected marking frames. In other words, the threshold adjustment unit 114 decreases the threshold when the number of marking frames is less than the Min value (minimum value), and increases the threshold when the number of marking frames exceeds the Max value (maximum value).
When the threshold adjustment unit 114 adjusts the threshold, the image selection unit 113 selects the marking frame again based on the adjusted predetermined threshold.
By the way, in the configuration in which marking frames are selected based on a predetermined threshold, the number of marking frames to be selected is approximately proportional to the length of the moving image. For example, when comparing a 30-minute moving image and a 60-minute moving image capturing the same sport, marking frames that are approximately twice as many as the 30-minute moving image are selected from the 60-minute moving image. As a result, the adjustment of the threshold value and the selection of the marking frame are repeated as the moving image becomes longer, and the processing load increases.

Therefore, the setting unit 115 may set the permitted number (Min value and Max value) of still image frames that can be selected as marking frames in accordance with the total length of the moving image. Specifically, the setting unit 115 sets the Min value and the Max value low when the total length of the moving image is short, and sets the Min value and the Max value high when the total length of the moving image is long. The total length of the moving image may be the time length of the moving image or the total number of frames constituting the moving image. In the present embodiment, since nine still image frames are finally extracted, the Min value is preferably 9 or more.
As described above, by setting the Min value and the Max value according to the entire length of the moving image, it is possible to reduce repetition of adjustment of the threshold and selection of the marking frame.

  The extraction interval setting unit 116 sets extraction conditions for extracting nine still image frames that are finally extracted from the marking frame selected by the image selection unit 113. Here, in the present embodiment, an interval for extracting nine still image frames from the marking frame (hereinafter referred to as “skip number”) is set as the extraction condition. Details of the skip number setting by the extraction interval setting unit 116 will be described later with reference to FIG.

The extraction unit 117 extracts nine still image frames from the marking frame selected by the image selection unit 113 based on the number of skips set by the extraction interval setting unit 116. For example, when the number of skips is “2” and the number of marking frames is “18”, the extraction unit 117 extracts nine still image frames at intervals of two according to the appearance order of the 18 marking frames in the moving image. That is, nine still image frames are extracted by extracting marking frames appearing in even numbers such as second and fourth, without extracting marking frames appearing in odd numbers such as first and third. Of course, it is also possible to extract marking frames that appear at odd numbers and not to extract marking frames that appear at even numbers.
The nine still image frames extracted by the extraction unit 117 are stored in the storage unit 19 and appropriately displayed on the display unit 16 under the control of the CPU 11 or the like.

  Next, the execution function of the motion print process will be specifically described with reference to FIGS. FIG. 3 is a diagram for explaining the function of the image selection unit 113, and FIG. 4 is a diagram for explaining the function of the extraction interval setting unit 116.

A vector of a still image frame is supplied from the vector calculation unit 112 to the image selection unit 113 that selects a marking frame. As an example, the image selection unit 113 is supplied with a vector of feature points for each region of a still image frame as shown in FIG.
Referring to FIG. 3B, the image selection unit 113 determines the size of the feature point vector for each region, and specifies the number of regions for which a vector having a magnitude greater than or equal to a predetermined value is calculated.
Subsequently, with reference to FIG. 3C, the image selection unit 113 determines whether or not the number of specified regions exceeds the determination value. At this time, the image selection unit 113 selects the still image frame as a marking frame when the determination value is exceeded, and does not select the still image frame as a marking frame when the determination value is equal to or less than the determination value.

As described above, the image selection unit 113 selects a marking frame based on the number of areas whose vector magnitude is equal to or greater than a predetermined value. Therefore, the predetermined threshold value used as a reference for the image selection unit 113 to select the marking frame may be the vector size (predetermined value) for each region, or the number of regions (determination value). Alternatively, both may be used.
Note that the selection of the marking frame shown in FIG. 3 is merely an example and does not preclude selection by any other method. That is, for example, the vector of the entire still image frame may be calculated from the vector for each region, and the marking frame may be selected based on the size of the entire vector.

  Next, setting of the number of skips by the extraction interval setting unit 116 will be described with reference to FIG. In the present embodiment, the extraction interval setting unit 116 determines the number of skips at substantially equal intervals (FIG. 4A), the number of skips according to the degree of congestion in the moving image of the marking frame (FIG. 4B), The number of skips (FIG. 4C) corresponding to the appearance position of is set.

Referring to FIG. 4A, when setting the number of skips at substantially equal intervals, the extraction interval setting unit 116 finally extracts the number of marking frames selected by the image selection unit 113 (the number of still image frames ( Hereinafter, the value divided by “extraction number”) is set as the skip number. By extracting nine still image frames from the marking frame using the skip number set in this way, a still image frame with a large motion can be extracted from the entire moving image.
By the way, when the number of skips is an integer value as shown in FIGS. 4A and 4A, still image frames can be extracted at equal intervals, but depending on the number of marking frames, FIGS. In some cases, a fraction is generated in the skip number. In such a case, the extraction unit 117 uses the skip number m (m is an integer value and satisfies the relationship m <skip number <m + 1) and the skip number m + 1 as appropriate so that the intervals are substantially equal. Extract a picture frame.

Referring to FIG. 4B, when setting the number of skips according to the density of marking frames, as shown in FIGS. 4B and 4A, the extraction interval setting unit 116 first includes It is determined whether there is a dense part of the marking frame, and its position is specified. The dense portion can be specified by an arbitrary method. For example, the extraction interval setting unit 116 sets a time zone in which the number of marking frames is a predetermined ratio or more as a dense portion with respect to the total number of frames in an arbitrary time. Identify.
The extraction interval setting unit 116 sets the number of skips according to the degree of congestion as shown in FIGS. Here, in FIGS. 4B and 4B, the number of skips at substantially equal intervals is set only in the dense part. That is, since there are six marking frames other than the dense part, the number of skips is set to six so that three still image frames are extracted from the dense part. As a result, the extraction unit 117 extracts still image frames at intervals of one sheet from areas other than the dense part, and extracts still image frames at intervals of six sheets from the dense part. In this way, by setting the number of skips according to the degree of congestion, it is not necessary to extract a large number of still image frames from a series of scenes (scenes) in the movie. Can be extracted.
In FIGS. 4B and 4B, all marking frames other than the dense part are extracted. However, the present invention is not limited to this. It is sufficient to extract marking frames other than the dense part than the dense frames, and it is also possible to set the number of skips exceeding 1 for other parts than the dense part. That is, the number of skips corresponding to a dense part means that the number of skips with a larger interval is set for a dense part, not the number of skips only for a dense part.

Referring to FIG. 4C, when setting the number of skips according to the appearance position in the moving image, as shown in FIGS. 4C and 4A, the extraction interval setting unit 116 first selects the moving image. The marking frame is divided into a predetermined range, and the marking frame belonging to the predetermined range is specified based on the appearance position of the marking frame in the moving image. The division of the moving image can be arbitrarily set. For example, the moving image may be divided temporally as in the first and second half, or may be divided according to the scene (scene) of the moving image. In FIGS. 4C and 4A, the first half, the middle, and the second half of the moving image are divided into three in terms of time.
The extraction interval setting unit 116 sets the number of skips for each predetermined range according to the number of marking frames belonging to the marking frame, as shown in FIGS. 4C and 4B, when the marking frame belonging to the divided predetermined range is specified. When the moving image is divided into three predetermined ranges as shown in FIG. 4C, for example, the number of skips for extracting three still image frames from each predetermined range is set. In FIGS. 4C and 4B, 2 is set as the skip number for the first half part to which the 6 marking frames belong, and 3 is set as the skip number for the middle part to which the 9 marking frames belong. The number of skips is set to 1 in the latter half to which the marking frame belongs. Thus, by setting the number of skips according to the appearance position in the moving image, it is possible to appropriately extract a still image frame with a large movement from the entire moving image.
In FIGS. 4C and 4B, all the still image frames extracted from the predetermined range are the same (three frames each). However, the present invention is not limited to this, and extraction is performed according to the predetermined range into which the moving images are divided. The number of still image frames to be performed may be varied.

The function of the CPU 11 when executing the motion print process has been described above. Next, the motion print process executed by the information processing apparatus 1 (CPU 11) having the functional configuration of FIG. 2 will be described.
5 and 6 are flowcharts for explaining the flow of motion print processing executed by the information processing apparatus 1 of FIG. 1 having the functional configuration of FIG. When the motion printing process is executed by the information processing apparatus 1, the functional blocks shown in FIG. 2 function in the CPU 11, and the following process is performed. That is, the operation subject of the following steps corresponds to the CPU 11 in hardware. However, in order to facilitate the understanding of the present invention, the processing of the following steps will be described on the assumption that each functional block functioning in the CPU 11 is an operation subject.

The motion print process is started when a start instruction is received from the user via the input unit 17.
First, in step S <b> 1, the moving image acquisition unit 111 acquires moving image data from the storage unit 19. Subsequently, in step S2, the setting unit 115 sets the Min value and the Max value of the number of marking frames according to the total length of the acquired moving image data.

  Subsequently, in step S3, the vector calculation unit 112 reads the still image frame constituting the moving image data acquired in step S1 as a processing target. Subsequently, in step S4, the vector calculation unit 112 reads the still image of the previous frame. It is determined whether there is an image frame, that is, whether the read still image frame is the second or later frame. At this time, if there is no previous still image frame, NO is determined in step S4, the process returns to step S3, and the next still image frame is read. On the other hand, if there is a previous still image frame, the vector calculation unit 112 subsequently compares the still image frame read in step S3 with the previous still image frame to calculate a vector ( Step S5).

  Subsequently, in step S6, the image selection unit 113 determines whether or not the still image frame read in step S3 is a moving frame. That is, the image selection unit 113 determines whether or not the size and number of vectors calculated in step S5 exceed a predetermined threshold. At this time, the image selection unit 113 selects a still image frame exceeding a predetermined threshold as a marking frame, and adds 1 to the number of marking frames. On the other hand, for a still image frame that is equal to or less than the predetermined threshold, the image selection unit 113 moves the process to step S8 without selecting it as a marking frame.

  In step S8, the image selection unit 113 determines whether there is a next still image frame. That is, it is determined whether or not the processing in steps S3 to S7 has been performed for all the still image frames constituting the moving image data. At this time, if there is a next still image frame, the image selection unit 113 moves the process to step S3, and if there is no next still image frame, moves the process to step S9.

  In step S9, the threshold adjustment unit 114 determines whether or not the number of marking frames selected in step S7 is greater than or equal to the Min value set in step S2 and less than or equal to the Max value. At this time, when the number of marking frames is equal to or greater than the Min value and equal to or less than the Max value, the threshold adjustment unit 114 moves the process to Step S14 of FIG. The threshold adjustment unit 114 moves the process to step S10.

  In step S10, the threshold adjustment unit 114 determines whether or not the number of marking frames is less than the Min value. At this time, when the number of marking frames is less than the Min value, the threshold adjustment unit 114 moves the process to Step S11, and when the number of marking frames is not less than the Min value, that is, when the number of marking frames exceeds the Max value. The threshold adjuster 114 moves the process to step S12.

  When the number of marking frames is less than the Min value, in step S11, the threshold adjustment unit 114 adjusts the predetermined threshold low so that more still image frames are selected as marking frames. On the other hand, when the number of marking frames exceeds the Max value, in step S12, the threshold value adjustment unit 114 adjusts the predetermined threshold value high so as to narrow down still image frames selected as marking frames. Subsequent to step S11 or step S12, the threshold adjustment unit 114 clears the number of marking frames (step S13), and moves the process to step S3. Accordingly, the image selection unit 113 selects a marking frame again based on the adjusted threshold value. In addition, since the vector for every still image frame has already been calculated, when it transfers to step S3 from step S13, it is good also as omitting the process of step S4, S5 which calculates a vector.

  Subsequently, referring to FIG. 6, when the number of marking frames is not less than the Min value and not more than the Max value, in step S14, the extraction interval setting unit 116 sets the number of skips. Note that the setting of the number of skips by the extraction interval setting unit 116 is as described above with reference to FIG.

  Subsequently, in step S15, the extraction unit 117 reads the marking frame selected by the image selection unit 113 in step S7. Here, the extraction unit 117 reads the marking frames in the order of appearance in the moving image. Subsequently, in step S16, the extraction unit 117 determines whether or not the marking frame read in step S15 is a marking frame to be skipped. At this time, if it is a marking frame to be skipped, the extraction unit 117 moves the process to step S19 without extracting the marking frame.

On the other hand, if the marking frame is not a skip target marking frame, the extraction unit 117 extracts the marking frame as a still image frame used for the motion print image (step S17). Subsequently, in step S18, the extraction unit 117 attaches the extracted marking frame to the specified position of the motion print image, and the process proceeds to step S19.
Here, FIG. 7 is an example of a motion print image generated by the motion print process. As shown in FIG. 7, the motion print image is composed of nine still image frames. The still image frames extracted by the extraction unit 117 are pasted in order from the upper left of the motion print image. That is, the first extracted still image frame is pasted on the upper left, and the second extracted still image frame is pasted on the right side. Thereafter, when the fourth extracted still image frame is pasted on the upper right, the fifth extracted still image frame is pasted on the center portion. Then, the still image frames extracted from the sixth to the ninth are pasted in order from the lower left. Note that the still image frame attached to the center portion may be larger than other still image frames.

  Returning to FIG. 6, in step S19, the extraction unit 117 determines whether there is a next marking frame. That is, the extraction unit 117 determines whether or not the processing of steps S15 to S18 has been performed on all the marking frames selected by the image selection unit 113 in step S7. At this time, if there is a next marking frame, the extraction unit 117 moves the process to step S15, and if there is no next marking frame, ends the motion print process.

As described above, the information processing apparatus 1 according to the present embodiment selects a still image frame having a large motion as a marking frame, and the selected marking frames are more than nine necessary for generating a motion print image. In this case, unnecessary marking frames are skipped (not extracted). As a result, a moving still image frame can be extracted, and it can be prevented that only a similar still image frame is extracted.
At this time, in the information processing apparatus 1 of the present embodiment, the interval (the number of skips) for extracting still image frames is set based on the number of marking frames and the number of extractions. As a result, the still image frames necessary for generating the motion print image are extracted from the entire moving image at a predetermined interval. Therefore, the appearance position of the extracted still image frame in the moving image is not biased, and There is no need to perform complicated processing such as image recognition. As a result, according to the information processing apparatus 1 of the present embodiment, a more appropriate still image frame that represents the entire moving image can be extracted.

  Note that the intervals at which the still image frames are extracted may be approximately equal intervals, may be intervals according to the degree of density of the marking frame in the moving image, or may be intervals according to the appearance position in the moving image. . By extracting still image frames using such various intervals, appropriate still image frames can be extracted from a wide variety of moving images.

  At this time, the information processing apparatus 1 appropriately adjusts a predetermined threshold (that is, the magnitude of the motion of the still image frame) serving as a reference for selecting a marking frame in accordance with the number of selected marking frames. As a result, regardless of the type of moving image, that is, a moving image with a small movement such as a landscape moving image or a moving image with a large movement such as a sports broadcast, an appropriate still image frame representing the moving image is appropriately displayed. Can be extracted.

  Further, in the information processing apparatus 1, the number of marking frames that triggers adjustment of a predetermined threshold is set according to the total length of the moving image. That is, since the number of marking frames to be selected increases when the total length of the moving image is long, the number of marking frames for threshold adjustment is set to a large value, while it is selected when the total length of the moving image is short. Since the number of marking frames is reduced, the number of marking frames for threshold adjustment is set to be small. Thereby, it is possible to prevent the adjustment of the threshold value and the selection of the marking frame from being repeated, and to reduce the processing load.

  In addition, this invention is not limited to the above-mentioned embodiment, The deformation | transformation in the range which can achieve the objective of this invention, improvement, etc. are included in this invention.

  In the above-described embodiment, the information processing apparatus 1 to which the present invention is applied has been described using a digital still camera as an example, but is not particularly limited thereto. For example, the present invention can be applied to general electronic devices having a function of processing moving image data. Specifically, for example, the present invention can be applied to a notebook personal computer, a digital video camera, a television receiver, a portable navigation device, a mobile phone, a portable game machine, and the like.

The series of processes described above can be executed by hardware or can be executed by software.
In other words, the functional configuration of FIG. 2 is merely an example and is not particularly limited. That is, it is sufficient that the information processing apparatus 1 has a function capable of executing the above-described series of processing as a whole, and what functional block is used to realize this function is not particularly limited to the example of FIG. .
In addition, one functional block may be constituted by hardware alone, software alone, or a combination thereof.

When a series of processing is executed by software, a program constituting the software is installed on a computer or the like from a network or a recording medium.
The computer may be a computer incorporated in dedicated hardware. The computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose personal computer.

  The recording medium including such a program is not only configured by the removable medium 41 of FIG. 1 distributed separately from the apparatus main body in order to provide the program to the user, but also in a state of being incorporated in the apparatus main body in advance. It is comprised with the recording medium etc. which are provided in. The removable medium 41 is composed of, for example, a magnetic disk (including a floppy disk), an optical disk, a magneto-optical disk, or the like. The optical disk is composed of, for example, a CD-ROM (Compact Disk-Read Only Memory), a DVD (Digital Versatile Disk), or the like. The magneto-optical disk is configured by an MD (Mini-Disk) or the like. In addition, the recording medium provided to the user in a state of being preliminarily incorporated in the apparatus main body includes, for example, the ROM 12 in FIG. 1 in which a program is recorded, the hard disk included in the storage unit 19 in FIG.

  In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in time series along the order, but is not necessarily performed in time series, either in parallel or individually. The process to be executed is also included.

  As mentioned above, although several embodiment of this invention was described, these embodiment is only an illustration and does not limit the technical scope of this invention. The present invention can take other various embodiments, and various modifications such as omission and replacement can be made without departing from the gist of the present invention. These embodiments and modifications thereof are included in the scope and gist of the invention described in this specification and the like, and are included in the invention described in the claims and the equivalent scope thereof.

The invention described in the scope of claims at the beginning of the filing of the present application will be appended.
[Appendix 1]
In an information processing apparatus for extracting a predetermined number of still image frames with a large movement from a moving image including a plurality of continuous still image frames,
A vector calculation means for calculating the amount of movement for each still image frame by comparing adjacent still image frames;
Image selection means for selecting still image frames in which the amount of motion calculated by the vector calculation means exceeds a predetermined threshold as extraction candidate still image frames;
Extraction interval setting means for setting an extraction interval based on the total number and the predetermined number of sheets, on the condition that the total number of still image frames of the extraction candidates selected by the image selection unit exceeds the predetermined number;
Extraction means for extracting the predetermined number of still image frames in accordance with the extraction interval set by the extraction interval setting means from the extraction candidate still image frames arranged according to the appearance order in the moving image;
An information processing apparatus comprising:
[Appendix 2]
The extraction interval setting means sets a value obtained by dividing the total number by the predetermined number as the extraction interval.
The information processing apparatus according to appendix 1, wherein
[Appendix 3]
The extraction interval setting means sets different extraction intervals according to the density of appearance positions of the still image frames of the extraction candidates in the moving image;
The information processing apparatus according to appendix 1, wherein
[Appendix 4]
The extraction interval setting means sets the extraction interval for each predetermined range of appearance positions in the video,
The extraction means extracts still image frames from the extraction candidate still image frames belonging to the predetermined range at an extraction interval corresponding to the range.
The information processing apparatus according to appendix 1, wherein
[Appendix 5]
Threshold adjustment means for adjusting the predetermined threshold according to the total number of still image frames of the extraction candidates selected by the image selection means;
The image selection unit reselects a still image frame as an extraction candidate on condition that the predetermined threshold is adjusted by the threshold adjustment unit.
The information processing apparatus according to any one of appendices 1 to 4, characterized in that:
[Appendix 6]
A setting means for setting a permitted number of still image frames that can be selected as the still image frame of the extraction candidate according to the total length of the moving image;
The threshold adjustment means adjusts the predetermined threshold on the condition that the total number of still image frames of the extraction candidates does not satisfy the permitted number;
The information processing apparatus according to appendix 5, wherein the information processing apparatus is provided.
[Appendix 7]
The setting means sets a maximum value of still image frames that can be selected as the extraction candidate still image frame as the permitted number,
The threshold adjustment means adjusts the predetermined threshold to be high on the condition that the total number of still image frames of the extraction candidates exceeds the maximum value.
The information processing apparatus according to appendix 6, wherein:
[Appendix 8]
The setting means sets a minimum value of still image frames that can be selected as the extraction candidate still image frame as the permitted number,
The threshold adjustment means adjusts the predetermined threshold low on condition that the total number of still image frames of the extraction candidates is less than the minimum value;
The information processing apparatus according to appendix 6 or 7, characterized by the above.
[Appendix 9]
In the still image frame extraction method, the information processing apparatus extracts a predetermined number of still image frames with a large movement from a moving image including a plurality of continuous still image frames.
A vector calculation step of calculating the amount of movement for each still image frame by comparing adjacent still image frames;
An image selection step of selecting, as an extraction candidate still image frame, a still image frame in which the amount of motion calculated in the vector calculation step exceeds a predetermined threshold;
An extraction interval setting step for setting an extraction interval based on the total number and the predetermined number on the condition that the total number of still image frames of the extraction candidates selected in the image selection step exceeds the predetermined number;
An extraction step of extracting the predetermined number of still image frames from the extraction candidate still image frames arranged according to the appearance order in the moving image, according to the extraction interval set in the extraction interval setting step;
A still image frame extracting method.
[Appendix 10]
Computer
A vector calculating means for calculating the amount of motion for each still image frame by comparing adjacent still image frames of a plurality of continuous still image frames constituting a moving image;
Image selecting means for selecting, as a candidate still image frame, a still image frame in which the amount of motion calculated by the vector calculating means exceeds a predetermined threshold;
An extraction interval setting means for setting an extraction interval based on the total number and the predetermined number on the condition that the total number of still image frames of the extraction candidates selected by the image selection unit exceeds a predetermined number;
Extraction means for extracting the predetermined number of still image frames according to the extraction interval set by the extraction interval setting means from the extraction candidate still image frames arranged according to the appearance order in the moving image;
A program characterized by functioning as

  DESCRIPTION OF SYMBOLS 1 ... Information processing apparatus, 11 ... CPU, 12 ... ROM, 13 ... RAM, 14 ... Bus, 15 ... Input / output interface, 16 ... Display part, 17 ... Input unit, 18 ... imaging unit, 19 ... storage unit, 20 ... communication unit, 21 ... drive, 41 ... removable media, 111 ... video acquisition unit, 112 ... Vector calculation unit, 113 ... image selection unit, 114 ... threshold adjustment unit, 115 ... setting unit, 116 ... extraction interval setting unit, 117 ... extraction unit

Claims (10)

In an information processing apparatus for extracting a predetermined number of still image frames with a large movement from a moving image including a plurality of continuous still image frames,
A vector calculation means for calculating the amount of movement for each still image frame by comparing adjacent still image frames;
Image selection means for selecting still image frames in which the amount of motion calculated by the vector calculation means exceeds a predetermined threshold as extraction candidate still image frames;
Extraction interval setting means for setting an extraction interval based on the total number and the predetermined number of sheets, on the condition that the total number of still image frames of the extraction candidates selected by the image selection unit exceeds the predetermined number;
Extraction means for extracting the predetermined number of still image frames in accordance with the extraction interval set by the extraction interval setting means from the extraction candidate still image frames arranged according to the appearance order in the moving image;
An information processing apparatus comprising: The extraction interval setting means sets a value obtained by dividing the total number by the predetermined number as the extraction interval.
The information processing apparatus according to claim 1. The extraction interval setting means sets different extraction intervals according to the density of appearance positions of the still image frames of the extraction candidates in the moving image;
The information processing apparatus according to claim 1. The extraction interval setting means sets the extraction interval for each predetermined range of appearance positions in the video,
The extraction means extracts still image frames from the extraction candidate still image frames belonging to the predetermined range at an extraction interval corresponding to the range.
The information processing apparatus according to claim 1. Threshold adjustment means for adjusting the predetermined threshold according to the total number of still image frames of the extraction candidates selected by the image selection means;
The image selection unit reselects a still image frame as an extraction candidate on condition that the predetermined threshold is adjusted by the threshold adjustment unit.
The information processing apparatus according to claim 1, wherein the information processing apparatus is an information processing apparatus. A setting means for setting a permitted number of still image frames that can be selected as the still image frame of the extraction candidate according to the total length of the moving image;
The threshold adjustment means adjusts the predetermined threshold on the condition that the total number of still image frames of the extraction candidates does not satisfy the permitted number;
The information processing apparatus according to claim 5. The setting means sets a maximum value of still image frames that can be selected as the extraction candidate still image frame as the permitted number,
The threshold adjustment means adjusts the predetermined threshold to be high on the condition that the total number of still image frames of the extraction candidates exceeds the maximum value.
The information processing apparatus according to claim 6. The setting means sets a minimum value of still image frames that can be selected as the extraction candidate still image frame as the permitted number,
The threshold adjustment means adjusts the predetermined threshold low on condition that the total number of still image frames of the extraction candidates is less than the minimum value;
The information processing apparatus according to claim 6, wherein the information processing apparatus is an information processing apparatus. In the still image frame extraction method, the information processing apparatus extracts a predetermined number of still image frames with a large movement from a moving image including a plurality of continuous still image frames.
A vector calculation step of calculating the amount of movement for each still image frame by comparing adjacent still image frames;
An image selection step of selecting, as an extraction candidate still image frame, a still image frame in which the amount of motion calculated in the vector calculation step exceeds a predetermined threshold;
An extraction interval setting step for setting an extraction interval based on the total number and the predetermined number on the condition that the total number of still image frames of the extraction candidates selected in the image selection step exceeds the predetermined number;
An extraction step of extracting the predetermined number of still image frames from the extraction candidate still image frames arranged according to the appearance order in the moving image, according to the extraction interval set in the extraction interval setting step;
A still image frame extracting method. Computer
A vector calculating means for calculating the amount of motion for each still image frame by comparing adjacent still image frames of a plurality of continuous still image frames constituting a moving image;
Image selecting means for selecting, as a candidate still image frame, a still image frame in which the amount of motion calculated by the vector calculating means exceeds a predetermined threshold;
An extraction interval setting means for setting an extraction interval based on the total number and the predetermined number on the condition that the total number of still image frames of the extraction candidates selected by the image selection unit exceeds a predetermined number;
Extraction means for extracting the predetermined number of still image frames according to the extraction interval set by the extraction interval setting means from the extraction candidate still image frames arranged according to the appearance order in the moving image;
A program characterized by functioning as

Images