JP2011250284A - Output apparatus and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To output an image having no space even when a total number of frame images that are extractable from a video is less than the number of frame images to be arranged.SOLUTION: Nine frame images are extracted from a video represented by video data, and a frame-arranged image formed by time-sequentially arranging the nine frame images in three rows by three columns is generated (c). When the total number of frame images that are extractable from the video is less than nine, a change amount of between time-sequential frame images among all the frame images that are extractable from the video is calculated. An insertion location for inserting interpolating frame images for an insufficient number of frame images are determined so that at least an interval between two frame images having the least change amount is used as the insertion location (a). By reproducing one of two frame images adjacent to the determined insertion location to form interpolating frame images, and inserting the formed frame images, a frame-arranged image is generated (b).

Description

  The present invention relates to an output device and a program.

  Conventionally, a printing apparatus that prints an image represented by image data is known. Image data representing still images such as photographs (still image data) is generally printed by such a printing apparatus, but in recent years image data representing moving images (moving image data) is also targeted for printing. Printing apparatuses that can do this have been proposed.

  For example, the printing apparatus described in Patent Document 1 extracts a plurality of frame images from a moving image represented by moving image data, and prints an image in which they are arranged on the same page. Here, when the total number of frame images that can be extracted from the moving image is smaller than the number to be arranged in the image for printing, a lacking portion of the frame image becomes a blank space.

JP 2005-130254 A

  However, if there is a lack of frame images that can be extracted from a moving image, there is a problem that if the portion where the frame image is supposed to be placed is left blank, the appearance of the frame image is deteriorated and the appearance is deteriorated. It was.

  The present invention has been made in view of these problems, and an output device and program capable of outputting an image without a margin even when the total number of frame images that can be extracted from a moving image is smaller than the number to be arranged. The purpose is to provide.

  In order to achieve the above object, an output device of the present invention extracts a plurality of frame images from a moving image represented by moving image data, and a frame arrangement image in which these are arranged so as to have chronological regularity. Arrangement image generation means for generating and output means for outputting the frame arrangement image. Then, the layout image generation means, when the total number of frame images that can be extracted from the moving image is less than the number of layout frames that should be positioned in the frame layout image, for all the frame images that can be extracted from the moving image, in chronological order. Based on the amount of change calculated by the amount of change calculated by the amount of change calculated by the amount of change calculated by the amount of change between the successive frame images and the amount of change calculated by the change amount calculator, And an interpolation position generating means for generating a frame arrangement image in which a complementary frame image based on a frame image around the insertion position is inserted at the insertion position determined by the insertion position determining means. Specifically, the insertion position determination unit determines, as an insertion position, a frame image having the smallest change amount among the change amounts calculated by the change amount calculation unit.

  According to such an output device, even when the total number of frame images that can be extracted from a moving image is less than the number of arranged frames, it is possible to output a frame arranged image having no blank space (which looks better). In particular, among the amount of change between frame images that are consecutive in time series, at least the frame image having the smallest change amount is set as the insertion position of the complementary frame image, so that the complementary frame image can be inserted in an inconspicuous manner.

  Here, the complementary frame image can be, for example, an image obtained by duplicating a frame image around the insertion position. That is, the complementing means inserts a duplicate image obtained by duplicating one of the two frame images before and after the insertion position as a complementation frame image at the insertion position. According to such an output device, the complement frame image can be made inconspicuous. In addition, it is possible to simplify the process of generating the complementary frame image.

  In this case, the frame image to be copied may be selected based on the arrangement of the complementary frame image in the frame arrangement image. That is, the complementing unit selects one of the two frame images before and after the insertion position based on the arrangement of the insertion position in the frame layout image, and a duplicate image of the selected frame image is used as the complementary frame image at the insertion position. insert. According to such an output device, it is possible to select a more appropriate frame image as compared to the case of randomly selecting one of the two frame images before and after the insertion position.

  For example, the complementing unit may select one of the two frame images before and after the insertion position that is farther from the insertion position in the frame layout image. According to such an output device, it is possible to make it difficult to produce a sense of incongruity due to the complementary frame image and the copy source frame image being arranged close to each other.

  Specifically, if the frame arrangement image is a plurality of frame images extracted from a moving image arranged in a plurality of columns in time series, the complementing unit arranges the two frame images before and after the insertion position in different columns. In such a case, it is possible to select the one arranged in a different column from the complement frame image. According to such an output device, it is possible to easily arrange the complementary frame image and the copy source frame image in different columns.

  On the other hand, a program of the present invention made to achieve the above object extracts a plurality of frame images from a moving image represented by moving image data, and arranges them so as to have time-series regularity. The computer is caused to function as an output device including arrangement image generation means for generating the image and output means for outputting the frame arrangement image. Then, the layout image generation means, when the total number of frame images that can be extracted from the moving image is less than the number of layout frames that should be positioned in the frame layout image, for all the frame images that can be extracted from the moving image, in chronological order. Based on the amount of change calculated by the amount of change calculated by the amount of change calculated by the amount of change calculated by the amount of change between the successive frame images and the amount of change calculated by the change amount calculator, And an interpolation position generating means for generating a frame arrangement image in which a complementary frame image based on a frame image around the insertion position is inserted at the insertion position determined by the insertion position determining means. Specifically, the insertion position determination unit determines, as an insertion position, a frame image having the smallest change amount among the change amounts calculated by the change amount calculation unit.

  According to such a program, it is possible to cause the computer to function as the above-described output device, thereby obtaining the above-described effects.

1 is a block diagram illustrating a schematic configuration of a multifunction machine according to an embodiment. It is explanatory drawing of the insertion position of a complement frame image. It is explanatory drawing of an input image information storage area. It is explanatory drawing of a moving image information storage area. It is explanatory drawing of a frame image information storage area. It is explanatory drawing of a frame difference information storage area. It is a flowchart of a media image printing process. It is a flowchart of an output image generation process. It is a flowchart of a frame difference calculation process.

Embodiments to which the present invention is applied will be described below with reference to the drawings.
[1. overall structure]
FIG. 1 is a block diagram illustrating a schematic configuration of a multifunction machine 10 as an output device according to the embodiment.

  The multifunction machine 10 is a multi-function device having a scanner function, a copy function, and the like in addition to a printer function, and includes a CPU 11, a ROM 12, an internal memory (RAM) 13, a scanner reading unit 14, a print control unit 15, and a liquid crystal display unit 16. The operation input unit 17 and the media card slot 18 are connected to each other via a signal line.

The CPU 11 is a device for performing all calculations in the multifunction machine 10.
The ROM 12 is a device in which a program for causing the CPU 11 to execute processes (FIGS. 7 to 9) described later is stored in advance.

The internal memory 13 is a device for temporarily storing calculation results, input data, and the like by the CPU 11. Details of the storage area of the internal memory 13 will be described later.
The print control unit 15 is a device for printing image data for which a print command has been issued, and is capable of printing a color image with CMYK color materials (toner, ink, etc.).

The liquid crystal display unit 16 is a device for displaying an image (including an image representing a character string such as a message) on a small color liquid crystal display.
The operation input unit 17 is a device for arranging various operation keys pressed by the user and inputting information based on the operation. Specifically, an up key, a down key, a left key, and a right key for performing up / down / left / right operations and an OK key for performing a determination operation are provided.

  The media card slot 18 stores an SD card in which an image file (image data representing a still image or a moving image) generated by a photographing device such as a digital still camera or a digital video camera capable of taking a still image and a moving image is stored. And a media card (nonvolatile portable storage medium) such as a CF card can be inserted. The multifunction device 10 has a function of directly reading an image file from a media card inserted in the media card slot 18 and printing an image representing the content of the read image file (so-called direct print function). .

  Here, as an image file, not only a still image file representing a still image but also a moving image file representing a moving image is a target. Specifically, when the image file to be printed is a still image file, only the still image represented by the still image file is printed with a layout arranged in one page of the output image. On the other hand, when the image file to be printed is a moving image file, a predetermined number of arrangement frames (9 in this embodiment) among all frame images constituting the moving image represented by the moving image file. A frame image is extracted, and a frame arrangement image having a layout in which the extracted frame image is arranged in one page of the output image is generated and printed.

  Here, it is conceivable that the total number of frame images that can be extracted from a moving image is less than the number of arranged frames. In this case, even if all the frame images are arranged in the frame arrangement image, a margin (empty portion where the frame image can be arranged) is generated. Therefore, the multifunction machine 10 according to the present embodiment inserts a shortage of complementary frame images so that no blank space is generated in the frame layout image.

  For example, as shown in FIG. 2A, when 8 frame images can be extracted from a moving image, one complementary frame image is inserted. Specifically, for all frame images that can be extracted from a moving image, a difference between frame images that are consecutive in time series (interframe difference) is calculated, and a complementary frame image is obtained between two frame images having the smallest interframe difference. Is the insertion position. FIG. 2A shows an example in which the inter-frame difference between the third frame image and the fourth frame image from the left among the eight frame images is the smallest, and a complementary frame between these two frame images. This is the image insertion position.

  Then, as shown in FIG. 2B, a frame layout image without a margin is generated from the nine frame images including the complementary frame image. In this frame arrangement image, the frame images are arranged in chronological order from the upper left to the horizontal direction. The maximum arrangement number of one column in this frame arrangement image is set to three, and the portion exceeding this is arranged in chronological order from the left of the next (lower) column, 3 × 3 columns (9 in total) Frame images) are arranged.

  In this embodiment, a duplicate image obtained by duplicating (copying) one of the two frame images before and after the insertion position is used as a complementary frame image. Specifically, when two frame images before and after the insertion position are arranged in different columns, the frame image arranged in a column different from the complementary frame image is set as a replication target.

  For example, as shown in FIG. 2B, when the frame image before the insertion position is arranged at the right end (end of the column) in the frame arrangement image, the complementary frame image is arranged at the beginning of the next column. Therefore, a duplicate image obtained by duplicating (copying) the frame image is set as a complementary frame image.

  On the other hand, for example, as shown in FIG. 2C, when the frame image behind the insertion position is arranged at the left end (the top of the column) in the frame arrangement image, the complementary frame image is arranged at the end of the previous column. Therefore, a duplicate image obtained by duplicating (copying) the frame image is set as a complement frame image.

  Note that the multifunction device 10 displays on the liquid crystal display unit 16 a thumbnail image that can grasp the contents of the image file so that the user can select an image file to be printed from among a plurality of image files stored in the media card. An image represented by an image file corresponding to the thumbnail image selected by the user is printed.

[2. Internal memory storage area]
Next, the storage area of the internal memory 13 will be described.
As shown in FIG. 1, in the internal memory 13, as storage areas for storing various information, an input image information storage area 31, a moving image information storage area 32, an output image data storage area 33, and enlarged / reduced image data are stored. Storage area 34, frame image information storage area 35, first frame image data storage area 36, second frame image data storage area 37, frame difference information storage area 38, complementary frame image data storage area 39, print data storage area 40, A temporary variable storage area 41 and a thumbnail image data storage area 42 are prepared.

  The input image information storage area 31 is an area for storing image file information stored in the media card. Specifically, as shown in FIG. 3, the input image information storage area 31 includes an input image ID storage area 51, an input image file name storage area 52, and an input image file size storage area 53.

  The input image ID storage area 51 is an area for storing image file IDs (hereinafter referred to as “input image IDs”) sorted in order from 0 according to the number of image files stored in the media card. Specifically, input image IDs are assigned to the image files read from the media card in the order of reading.

  The input image file name storage area 52 is an area for storing the file name of the image file stored in the media card. Specifically, for example, 256-character (1 character / byte) character data (including information without characters) is stored in a 256-byte area.

  The input image file size storage area 53 is an area for storing the file size (data size) of the image file stored in the media card as a numerical value (in this example, a numerical value in kilobytes).

  Returning to FIG. 1, the moving image information storage area 32 is an area for temporarily storing information of a moving image file to be processed among image files stored in the media card. Specifically, as shown in FIG. 4, the moving image information storage area 32 includes a format type storage area 61, a codec type storage area 62, a horizontal size storage area 63, a vertical size storage area 64, and a total frame number storage. A region 65 is provided.

  The format type storage area 61 is an area for storing the file format (data format) type of the moving image file to be processed. In the present embodiment, numerical values associated with the three types of file formats (for example, “0” for the AVI format, “1” for the MOV format, “2” for the MPEG format) are stored. .

  The codec type storage area 62 is an area for storing the codec type of the moving image file to be processed. In the present embodiment, numerical values associated with three types of codecs (for example, “0” for the Motion JPEG codec, “1” for the MPEG1 codec, “2” for the DivX codec) are stored.

The horizontal size storage area 63 is an area for storing the pixel size in the horizontal direction of the frame image data constituting the moving image file to be processed as numerical data.
The vertical size storage area 64 is an area for storing, as numerical data, the vertical pixel size of the frame image data constituting the moving image file to be processed.

The total frame number storage area 65 is an area for storing the total number of frame image data (total number of frames) constituting the moving image file to be processed as numerical data.
Returning to FIG. 1, the output image data storage area 33 is an area for temporarily storing output image data (image data representing an image for printing) to be transferred to the print control unit 15. Since the still image file is composed of one image data, this one image data is arranged in the entire output image data storage area 33. On the other hand, since the moving image file arranges and outputs a plurality of frame images, each frame image data is arranged at a corresponding position in each arrangement area equally divided by the number of arrangement frames to be arranged.

The enlarged / reduced image data storage area 34 is an area for storing image data obtained by converting (enlarging or reducing) the image data into a predetermined thumbnail image size.
The frame image information storage area 35 is an area for storing information relating to the frame image to be arranged in the frame arrangement image. Specifically, as shown in FIG. 5, the frame image information storage area 35 includes a frame arrangement position number storage area 71, a moving picture file name storage area 72, an extracted frame number storage area 73, an offset size information storage area 74, and a frame. A size information storage area 75 is provided.

  The frame arrangement position number storage area 71 is an area for storing numbers corresponding to the arrangement positions of the frame images in the frame arrangement image (numbers 0 to 8, hereinafter referred to as “frame arrangement position numbers”). Specifically, the arrangement position in the frame arrangement image is “0”, “1”, “2” from the left in the top row, and “3”, “4”, “from the left in the second row. “5”, numbers “6”, “7”, “8” from the left in the third column are associated with each other.

The moving image file name storage area 72 is an area for storing the file name of a moving image file from which frame image data is extracted.
The extracted frame number storage area 73 is an area for storing an extracted frame number that is a numerical value (frame number) indicating which frame of the original moving image file the frame image corresponds to. In the present embodiment, the frame number of the first frame image is “0”.

  The offset size information storage area 74 is an area for storing information for specifying the position of the frame image in the moving image file. Specifically, in the offset size information storage area 74, an offset size indicating how many bytes the head of the frame image data corresponds to from the head of the original moving image file is a numerical value (in this example, a numerical value in units of bytes). It is remembered. That is, the file structure of a moving image file is a structure in which frame image data is sequentially arranged between the header information at the beginning and the index information at the end. The offset size here is the moving image file's file structure. This is the data size from the beginning (that is, the beginning of the header information) to the beginning of the frame image data to be extracted. The reason why the offset size is stored in bytes instead of kilobytes is to accurately specify the position of the frame image data.

  The frame size information storage area 75 is an area for storing the data size of frame image data (data size in a compressed state (for example, JPEG format) before decompression (decoding) processing), and the data size is a numerical value (in this example) Is stored as a numerical value in bytes.

  Returning to FIG. 1, the first frame image data storage area 36 is an area for temporarily storing the frame image data extracted from the moving image file. The frame image data stored in this area is image data in a compressed state before decompression processing.

  The second frame image data storage area 37 is frame image data extracted from the moving image file, and is used for calculating a difference (interframe difference) from the frame image data stored in the first frame image data storage area 36. This is an area for temporarily storing the frame image data. The frame image data stored in this area is also image data in a compressed state before decompression processing.

  The frame difference information storage area 38 is an area for storing the number of frame image data from which the inter-frame difference is calculated and the inter-frame difference. Specifically, as illustrated in FIG. 6, the frame difference information storage area 38 includes a difference information number storage area 81, a first difference acquisition frame number storage area 82, a second difference acquisition frame number storage area 83, and a determination value storage. A region 84 is provided.

The difference information number storage area 81 is an area for storing a difference information number, which is a serial number for managing information on interframe differences.
The first difference acquisition frame number storage area 82 is an area for storing the frame number of the first frame image data (first difference acquisition frame) for acquiring the inter-frame difference.

  The second difference acquisition frame number storage area 83 is an area for storing the frame number of the second frame image data (second difference acquisition frame) for acquiring the inter-frame difference.

  The determination value storage area 84 is an area for storing a determination value for determining the insertion position of the complementary frame image. In the present embodiment, the inter-frame difference (absolute value) of the frame image data corresponding to the frame numbers stored in the first difference acquisition frame number storage area 82 and the second difference acquisition frame number storage area 83 is stored as a determination value. .

  Returning to FIG. 1, the complementary frame image data storage area 39 is a complementary frame image generated when the total number of frames constituting the moving image represented by the moving image file is smaller than the number of arranged frames (9 in the present embodiment). This is an area for storing data (image data in an expanded state (RGB format)). Specifically, the complementary frame image data storage area 39 is divided into “number of arrangement frames−1” areas (eight areas from the first to the eighth).

The print data storage area 40 is an area for temporarily storing image data (such as CMYK binary image data) for actually printing the image represented by the output image data.
The temporary variable storage area 41 is an area for storing temporary information such as variables and counters.

  The thumbnail image data storage area 42 is an area for temporarily storing thumbnail image data representing a thumbnail image to be displayed on the liquid crystal display unit 16 so that the user can select an image file to be printed.

[3. Processing executed by CPU]
Next, when the media card storing the image file is inserted into the media card slot 18, a user's operation for selecting the “media image printing” mode is performed on the operation input unit 17, so that the multifunction device The media image printing process executed by the CPU 11 will be described with reference to the flowchart of FIG.

  When starting the media image printing process, the CPU 11 first reads out the file name and file size for each image file stored in the media card in S101, and reads the input image file name storage area 52 in the input image information storage area 31 and Media information extraction processing for sequentially storing in the input image file size storage area 53 is performed. In this media information extraction process, input image IDs are assigned in order from 0 to the information read from the media card and stored in the input image ID storage area 51.

  Subsequently, in S102, a thumbnail image capable of grasping the contents of the image file stored in the media card is displayed on the liquid crystal display unit 16, and a print image selection process for allowing the user to select an image file to be printed is performed. Specifically, one thumbnail image is displayed on the liquid crystal display unit 16, and the thumbnail image displayed on the liquid crystal display unit 16 in response to the operation of the upper key, the lower key, the left key, or the right key of the operation input unit 17. Is changed to that of another image file, and the image file corresponding to the thumbnail image displayed when the OK key is pressed is selected as the print target. In the present embodiment, for a moving image file, the top frame image is displayed as a thumbnail image, and an icon indicating moving image data is displayed at a predetermined position (for example, the upper left position in the thumbnail image), and a still image is displayed. For data, a still image is displayed as a thumbnail image (no icon is displayed). Here, the image data representing the thumbnail image is generated by reducing the original image (the still image or the first frame image of the moving image) and stored in the thumbnail image data storage area 42, and the thumbnail image representing the image data is displayed. Is displayed on the liquid crystal display unit 16.

Subsequently, in S103, output image generation processing for generating output image data representing an output image based on the image file selected as the print target in S102 is executed.
Here, the details of the output image generation processing will be described with reference to the flowchart of FIG.

When the output image generation processing is started, the CPU 11 first reads out the input image ID (value stored in the input image ID storage area 51) of the image file to be printed in S201.
Subsequently, in S202, it is determined with reference to the header information or the like whether the type of the image file of the input image ID read in S201 is classified as a still image file or a moving image file. . More specifically, for example, the JPEG format still image file has a feature that the top data starts with FFD8 in hexadecimal, and the AVI format video file starts with “RIFF” with the top data converted to a character string. Therefore, the type of the image file can be determined by analyzing the first few bytes of the image file. In the present embodiment, even if the format is a moving image file, if there is only one frame image, it is processed as a still image file.

  If it is determined in S202 that the type of the image file is a moving image file, the process proceeds to S203, and various information is acquired from the moving image file. Specifically, the format type information, codec type information, horizontal size information, vertical size information, and total frame number information of the moving image file are acquired. Each information acquired in this way is stored in the format type storage area 61, codec type storage area 62, horizontal size storage area 63, vertical size storage area 64, and total frame number storage area 65 in the moving image information storage area 32, respectively. Remembered. These parameters are generally described in the header of a moving image file.

  In step S204, extracted frame image information (offset size and data size before decompression processing) is extracted for each frame image data to be extracted. In the present embodiment, the frame image data to be extracted is the first and last (final) frame images of all the frame images constituting the moving image, and each equally-divided position obtained by dividing them into eight ( The data is a total of nine frame images consisting of seven frame images at approximate positions if they cannot be accurately divided equally. In other words, nine frame images that are equally spaced in time series in the moving image are extracted. Note that the number of frame image data to be extracted is normally nine. However, when the total number of frames is less than nine, which is the number of arranged frames, the number is less than nine.

  Subsequently, in S205, the extracted frame image information (offset size and data size before decompression processing) extracted in S204 is a number corresponding to each frame image data in the frame image information storage area 35 (frame arrangement position number storage area 71). The number is stored in the area.

Subsequently, in S206, it is determined whether or not the total number of frames constituting the moving image represented by the moving image file is less than the number of arranged frames.
If it is determined in S206 that the total number of frames is less than the number of arranged frames, processing (S207 to S210) for inserting a complementary frame image is executed. That is, first, in S207, a frame difference calculation process for calculating a difference (interframe difference) between time-sequential frame images is executed for all frame images constituting the moving image.

Here, details of the frame difference calculation processing will be described with reference to the flowchart of FIG.
When starting the frame difference calculation process, the CPU 11 first generates a frame counter in the temporary variable storage area 41 and initializes it to 0 in S301.

  Subsequently, in S302, frame image data having a number corresponding to the value of the frame counter (the number of the frame arrangement position number storage area 71) is extracted from the moving image file based on the offset size and the data size.

Subsequently, in S303, the frame image data extracted in S302 is stored in the first frame image data storage area.
Subsequently, in S304, frame image data having a number corresponding to “frame counter value + 1” is extracted from the moving image file in the same manner as in S302.

Subsequently, in S305, the frame image data extracted in S304 is stored in the second frame image data storage area 37.
Subsequently, in S306, the difference between the frame image data stored in each of the first frame image data storage area 36 and the second frame image data storage area 37 is calculated. Specifically, the two frame image data are expanded, and the absolute value of the difference between the pixel values is calculated for each coordinate position unit. This is calculated for all coordinates for each of RGB, and this total value is used as the interframe difference. Note that the decompression data storage area is secured by preparing variables for storing pixel values for one pixel in the temporary variable storage area 41. Since dot sequential work is possible, there is no need to provide a dedicated area for storing images.

  Subsequently, in S307, the value of the frame counter is stored in the first difference acquisition frame number storage area 82 of the number corresponding to the value of the frame counter in the frame difference information storage area 38 (the number of the difference information number storage area 81). “Frame counter value + 1” is stored in the difference acquisition frame number storage area 83, and the inter-frame difference calculated in S 306 is stored in the determination value storage area 84.

Subsequently, in S308, 1 is added to the value of the frame counter.
Subsequently, in S309, it is determined whether or not the value of the frame counter has reached “total number of frames−1” (difference between frames is calculated for all frame images).

  If it is determined in S309 that the value of the frame counter has not reached “total number of frames−1”, the process returns to S302, and the above-described processing is performed based on the added value of the frame counter.

On the other hand, if it is determined in S309 that the value of the frame counter has reached “total number of frames−1”, the frame difference calculation process is terminated.
Returning to FIG. 8, in S <b> 208, the number obtained by subtracting the total number of frames from the number of arranged frames (number of deficiencies) is determined as the number for generating the complement frame image (number of complement frames).

  Subsequently, in S209, the insertion position of the complementary frame image and the content of the image (the frame image to be copied) are determined. Specifically, processing for updating the information in the frame difference information storage area 38 and the frame image information storage area 35 in the following procedures (1) to (4) is performed.

  (1) The minimum value of the judgment value stored in the judgment value storage area 84 of the frame difference information storage area 38 is searched, the judgment value is rewritten to a double value, and the judgment value corresponds to the judgment value (difference information The frame number of the first difference acquisition frame (the number is the same) (the value stored in the first difference acquisition frame number storage area 82) is read. That is, the frame image having the smallest determination value is determined as the insertion position of the complementary frame image, and the frame number of the frame image before the insertion position is read out from the two frame images before and after the insertion position. The reason why the determination value is rewritten to a double value is to make it difficult to continuously select the same determination value (insertion position) in subsequent processing. In other words, when a large number of complementary frame images are inserted between the same frame images, the movement of the portions seem to stop and the flow becomes uncomfortable, so that the insertion position is not concentrated.

  (2) Extracted frame numbers (extracted frame numbers) of the values read in (1) in order from the smallest frame placement position number stored in the frame placement position number storage area 71 of the frame image information storage area 35 The value stored in the storage area 73) is searched, and the frame arrangement position number (value stored in the frame arrangement position number storage area 71) corresponding to the extracted extracted frame number is read. That is, the number read here is the arrangement position in the frame arrangement image of the frame image before the insertion position.

  (3) In the frame image information storage area 35, 1 is added to the frame arrangement position number larger than the value of the frame arrangement position number read in (2) above, and the row of the frame arrangement position number read out in (2) above Is newly established. That is, in order to secure the insertion position of the complementary frame image, the arrangement position of the frame image behind the insertion position is shifted one by one.

  (4) Copy one piece of information of the frame images before and after the insertion position to the line newly established in (3) above. Specifically, when the arrangement of the complementary frame image in the frame arrangement image is at the right end (in other words, when the frame arrangement position number of the newly established row is 2 or 5), information on the frame image behind the insertion position (Information on the next line after the newly created line) is copied. On the other hand, in other cases, the information of the frame image before the insertion position (information of the line immediately before the newly established line) is copied.

  Subsequently, in S210, it is determined whether or not the insertion position and the content of the image have been determined in S209 for the number of complementary frame images determined in S208. Specifically, a generated frame number counter whose initial value (value at the start of output image generation processing) is 0 is prepared in advance in the temporary variable storage area 41, and the generated frame number counter is set every time the processing of S209 is completed. Add 1 to the value. When this value reaches the number of complementary frames, it is determined that complementary frame image data for the number of complementary frames has been generated.

  If it is determined in S210 that the insertion position and the content of the image have not been determined for the complementary frame images for the number of complementary frames, the process returns to S209. That is, the processing of S209 is repeated until the insertion position and the content of the image are determined for the complementary frame images for the number of complementary frames.

  On the other hand, when it is determined in S210 that the insertion position and the content of the image have been determined for the complementary frame images for the number of complementary frames, or in S206, the total number of frames is not less than the number of arranged frames (is more than the number of arranged frames). If it is determined, the process proceeds to processing (S211 to S215) in which frame image data representing a frame image (which may include a complementary frame image) to be arranged in the frame arrangement image is arranged in the output image data storage area 33. Specifically, the frame image data to be arranged is sequentially selected as a processing target, and the processing of S211 to S214 is performed on the processing target frame image data. This is repeated for the number of arrangement frames (9 times). Specifically, a processing frame image number counter whose initial value (value at the start of output image generation processing) is 0 is prepared in advance in the temporary variable storage area 41, and a frame corresponding to the value of this processing frame image number counter is prepared. Image data is a processing target. Note that the value of the processing frame image number counter is added every time processing for the processing target frame image data is completed, as will be described later.

  In S211, processing for reading out the frame image data to be processed from the moving image file based on the offset size stored in the offset size information storage area 74 and the data size before decompression stored in the frame size information storage area 75 is performed. Do.

Subsequently, in S212, the frame image data (image data before decompression processing) read in S211 is stored in the first frame image data storage area 36.
Subsequently, in S213, the frame image data stored in the first frame image data storage area 36 is subjected to decompression processing so that the frame image data can be subjected to pixel calculation (RGB values of 0 to 255 for each pixel, respectively). Image data represented by a value).

Subsequently, in S214, the processing target frame image data (decompressed pixel data) is stored at a position corresponding to each frame image in the output image data storage area 33.
Subsequently, in S215, it is determined whether or not all the frame image data has been arranged in the output image data storage area 33. Specifically, every time processing is completed for one frame image data, 1 is added to the value of the processing frame image number counter, and whether or not the value of the processing frame image number counter has reached the number of arranged frames is determined. By determining, it is determined whether or not all the frame image data are arranged.

  If it is determined in S215 that all the frame image data has not been arranged, the process returns to S211 and the above-described processing is performed on unprocessed frame image data.

  On the other hand, if it is determined in S215 that all the frame image data has been arranged, the output image generation process is terminated. Note that the image data stored in the output image data storage area 33 at this time is the output image data of the moving image file.

  If it is determined in S202 that the type of the image file is a still image file, the process proceeds to S216, the still image file is expanded, and the data is converted into a format capable of pixel calculation. .

  Subsequently, in S217, the image data expanded in S216 is stored in the output image data storage area 33, and then the output image generation process is terminated. That is, in the case of a still image file, unlike a moving image file, only one still image is arranged in the output image data storage area 33. Note that the image data stored in the output image data storage area 33 at this time is the output image data of the still image file.

  Returning to FIG. 7, in S <b> 104, an image printing process for printing an image represented by the output image data stored in the output image data storage area 33 is executed. In this image printing process, the output image data stored in the output image data storage area 33 is copied to the print data storage area 40 line by line, and color conversion processing (RGB → CMYK) and binarization processing are performed. After that, the data is output to the print control unit 15. As a result, the printing control unit 15 performs printing based on the binary data. Thereafter, the media image printing process ends.

[4. effect]
As described above, according to the multifunction machine 10 of the present embodiment, even when the total number of frame images that can be extracted from a moving image is less than the number of arranged frames, there is no blank space (which looks better). Can be printed.

  In particular, in this multi-function device 10, a frame image having a small inter-frame difference is set as an insertion position of a complementary frame image, and a duplicate image obtained by duplicating one of two frame images before and after the insertion position is inserted as a complementary frame image. Insert in position. For this reason, it is possible to insert the complement frame image in an inconspicuous form (in other words, so that the original originality of the moving image is not easily lost), and to simplify the process of generating the complement frame image. it can. That is, when a duplicate image C in which one of these is duplicated is inserted between the frame images A and B having a large inter-frame difference, the movement from the frame image A to the duplicate image C and the movement from the duplicate image C to the frame image A The gap between and becomes larger, so it is easy to feel uncomfortable. Such a sense of discomfort is less likely to occur as the inter-frame difference between the frame images A and B is smaller.

  In addition, in the multi-function device 10, when two frame images before and after the insertion position are arranged in different columns, the one arranged in a column different from the complementary frame image is set as a duplication target. For this reason, the complement frame image and the copy source frame image can be easily arranged in different columns, and the discomfort caused by arranging the complement frame image and the copy source frame image side by side is less likely to occur. be able to.

[5. Correspondence with Claims]
In the MFP 10 of the present embodiment, the CPU 11 that executes the process of S103 corresponds to a layout image generation unit, and the CPU 11 that executes the process of S104 corresponds to an output unit. Further, the CPU 11 that executes the process of S207 corresponds to a change amount calculating unit, the CPU 11 that executes the processes of S208 to S210 corresponds to an insertion position determining unit, and the CPU 11 that executes the processes of S211 to S215 corresponds to a complementing unit. To do.

[6. Other forms]
As mentioned above, although embodiment of this invention was described, it cannot be overemphasized that this invention can take a various form, without being limited to the said embodiment.

  For example, when the total number of frames constituting the moving image is extremely small (when it is equal to or less than a predetermined number), the complementary frame image may not be inserted. In this way, it is possible to prevent generation of a frame layout image in which a large number (for example, a number larger than the number of frame images that can be extracted from a moving image) is inserted.

  In the above embodiment, the layout of the frame arrangement image is fixed regardless of the total number of frames constituting the moving image, but the present invention is not limited to this. For example, when the total number of frames is 6 or less, a layout of 3 × 2 columns is used, and when the number of frames is 4 or less, a layout of 2 × 2 columns is used. The layout may be changed step by step. In this way, the number of complementary frame images can be reduced.

  In the above embodiment, the insertion position and the copy target are determined for each complementary frame image (S209), and this is repeated as many times as the number of complementary frame images (S210). is not. For example, the insertion position for a plurality of complementary frame images may be determined first, and then the copy target of each complementary frame image may be determined. In this way, the positional relationship between the complementary frame image in the frame layout image and the copy-source frame image can be more reliably defined.

  Further, in the above embodiment, when the arrangement of the complementary frame image in the frame arrangement image is the right end, the frame image after the insertion position is to be duplicated. In other cases, the frame image before the insertion position is to be duplicated. However, it is not limited to this. For example, when the arrangement of the complementary frame image in the frame arrangement image is at the left end, the frame image before the insertion position may be the duplication target, and in other cases, the frame image after the insertion position may be the duplication target. Further, if a frame image whose arrangement in the frame arrangement image is far from the complementary frame image is selected as a duplication target, the same effect can be obtained even in a layout other than the layout as in the above embodiment. As described above, the replication target can be selected. On the other hand, it is possible to select a copy target regardless of the arrangement of the complementary frame image in the frame arrangement image.

  In the above-described embodiment, the same determination value (insertion position) is not easily selected continuously by rewriting the determination value between frame images determined as the insertion position to a double value (S209). For example, the determination value may not be rewritten.

  In the above embodiment, the first frame image of a moving image file is displayed as a thumbnail image. However, the present invention is not limited to this. For example, an image that is actually printed (frame arrangement image) ) May be displayed as a thumbnail image.

  In the above embodiment, a duplicate image obtained by duplicating one of the two frame images before and after the insertion position is used as a complementary frame image. However, the present invention is not limited to this. For example, two frame images are synthesized. It is also possible to use a composite image as a complementary frame image.

  In the above embodiment, the difference (specifically, the total value of the absolute values of the pixel value differences in each coordinate position unit) is exemplified as the amount of change between the frame images. However, the present invention is not limited to this. Instead, any value that reflects the amount of change in the frame image may be used. For example, a “motion vector magnitude” between frame images calculated by a known optical flow method or the like may be used for difference calculation.

  In the MFP 10 of the above embodiment, the number of frame images extracted from the moving image file is nine. However, the number of frame images is not limited to this. For example, the number of frame images increases, for example, 30 or 40. The present invention has significance. Further, the number need not be fixed, and may be specified by the user, for example.

  In the above-described embodiment, the configuration in which the present invention is applied to the multifunction device 10 is illustrated. However, the present invention is not limited to this. For example, a printing device other than the multifunction device (for example, a printing device that does not have a scanner function) It can also be applied to. Further, the present invention is not limited to a printing apparatus, and any apparatus that outputs an image representing the contents of an image file may be used. That is, image output includes, for example, image display in addition to image printing. Therefore, the present invention can be applied to an information processing apparatus such as a personal computer and a program executed by the information processing apparatus.

  DESCRIPTION OF SYMBOLS 10 ... MFP, 11 ... CPU, 12 ... ROM, 13 ... Internal memory, 15 ... Print control part, 16 ... Liquid crystal display part, 17 ... Operation input part, 18 ... Media card slot, 31 ... Input image information storage area, 32 ... Moving image information storage area, 33 ... Output image data storage area, 34 ... Enlarged / reduced image data storage area, 35 ... Frame image information storage area, 36 ... First frame image data storage area, 37 ... Second frame image Data storage area, 38 ... frame difference information storage area, 39 ... complementary frame image data storage area, 40 ... print data storage area, 41 ... temporary variable storage area, 42 ... thumbnail image data storage area

Claims (6)

Arrangement image generation means for extracting a plurality of frame images from a moving image represented by the moving image data and generating a frame arrangement image in which these are arranged so as to have chronological regularity;
Output means for outputting the frame arrangement image;
An output device comprising:
The arrangement image generation means includes:
When the total number of frame images that can be extracted from the moving image is less than the number of arrangement frames to be arranged in the frame arrangement image, all frame images that can be extracted from the moving image are consecutive between frame images in time series. Change amount calculating means for calculating the change amount of
An insertion position determining means for determining an insertion position for inserting a complementary frame image corresponding to the total number of deficiencies with respect to the number of arranged frames based on the amount of change calculated by the amount of change calculating means;
Complementing means for generating a frame arrangement image in which a complementary frame image based on a frame image around the insertion position is inserted at the insertion position determined by the insertion position determining means;
And the insertion position determination means determines, as the insertion position, a frame image having the smallest change amount among the change amounts calculated by the change amount calculation means. The output device according to claim 1, wherein the complementing unit inserts a duplicated image obtained by duplicating one of two frame images before and after the insertion position as the complementary frame image at the insertion position. The complementing unit selects one of the two frame images before and after the insertion position based on the layout of the insertion position in the frame layout image, and inserts a duplicate image of the selected frame image as the complementary frame image. The output device according to claim 2, wherein the output device is inserted into a position. The output device according to claim 3, wherein the complementing unit selects one of the two frame images before and after the insertion position that is farther from the insertion position in the frame layout image. The frame arrangement image is obtained by arranging a plurality of frame images extracted from the moving image in a plurality of columns in time series.
The complementing means, when two frame images before and after the insertion position are arranged in different columns, selects the one arranged in a column different from the complementary frame image. The output device according to claim 4. Arrangement image generation means for extracting a plurality of frame images from a moving image represented by the moving image data and generating a frame arrangement image in which these are arranged so as to have chronological regularity;
Output means for outputting the frame arrangement image;
A program for causing a computer to function as an output device comprising:
The arrangement image generation means includes:
When the total number of frame images that can be extracted from the moving image is less than the number of arrangement frames to be arranged in the frame arrangement image, all frame images that can be extracted from the moving image are consecutive between frame images in time series. Change amount calculating means for calculating the change amount of
An insertion position determining means for determining an insertion position for inserting a complementary frame image corresponding to the total number of deficiencies with respect to the number of arranged frames based on the amount of change calculated by the amount of change calculating means;
Complementing means for generating a frame arrangement image in which a complementary frame image based on a frame image around the insertion position is inserted at the insertion position determined by the insertion position determining means;
And the insertion position determining means determines, as the insertion position, a frame image having the smallest change amount among the change amounts calculated by the change amount calculating means.