JP4569659B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus.

  In recent years, since a moving image and a still image can be photographed using a digital camera, there are scenes in which a moving image and a still image are separately photographed according to the photographing scene. For example, a moving scene such as an athletic meet is shot with a moving image, and a non-moving scene such as a group photo is shot as a still picture.

There has also been proposed an apparatus that prints out an image list in which a frame image obtained by extracting a moving image shot by a digital camera and a still image are mixed and displays the image list on a display. For example, the following Patent Document 1 describes a print system that displays a frame image and a still image on a display 7 in a distinguishable manner.
JP 2005-130254 A (paragraph “0036”, FIG. 9)

  However, in the print system described in Patent Document 1 described above, the image list including the frame images displayed on the display 7 and the still images is visually lacking in uniformity. Specifically, the frame image constituting the moving image of the digital camera has a smaller image size than the still image which is the main function of the digital camera. In addition, since moving images continue for a long time, flash shooting for improving shooting conditions at the time of shooting and adjustment methods such as ISO sensitivity, F value, and white balance are different from those for still images. For example, if the shooting conditions change while shooting, such as moving from indoors to outdoors while shooting, the video will cast color during the period until white balance is applied. There was a tendency to cast color. As a result, in the image list in which the frame image and the still image are mixed, there is a problem that the frame image and the still image lack visual uniformity and the image list is difficult to see.

  The present invention has been made to solve the above-described problems, and is an image list in which frame images extracted from a moving image and still images are mixed, and is visually displayed between the frame images and the still images. An object of the present invention is to provide an image processing apparatus capable of outputting an image list in which a sense of unity is ensured.

In order to achieve this object, the image processing apparatus according to claim 1 is configured to acquire one or more still image files and one or more moving image files, and a moving image file acquired by the acquiring unit. 1 and extracting means for extracting the above frame images, among the pre SL obtained by the obtaining unit still image files and moving image files, a feature amount representing an image feature of the still image picture of quiescent image file from the an image feature quantity calculating means for calculating for each one or more still image files acquired by the acquisition means, and one or more feature quantity or al criteria corresponding to one or more still image calculated by the image feature quantity calculating means Each of the one or more still images, the reference feature amount calculating means for calculating the reference feature amount, the feature amount of each of the one or more frame images, An image correcting unit that corrects an image so that a feature amount of a still image matches the reference feature amount calculated by the reference feature amount calculating unit, the frame image corrected by the image correcting unit, and the still image Output control means for outputting an image on the same output surface.

Information claim 2 image processing apparatus, wherein in the image processing apparatus according to claim 1, wherein the image feature quantity calculating means, as the feature quantity, related to the size of the color information and the image showing the information about the colors of the image At least one of the size information indicating the reference feature amount, the reference feature amount calculating unit calculates a reference feature amount serving as a reference from the feature amount obtained by the image feature amount calculating unit, and the image correcting unit The feature amount of each frame image of the one or more frame images and the feature amount of each still image of the one or more still images are matched with the reference feature amount calculated by the reference feature amount calculating means. Correct the image as follows.

The image processing apparatus according to claim 3, wherein, in the image processing apparatus according to claim 2, wherein the color information, luminance information indicating luminance, saturation information indicating the saturation, of the color information indicating a color, either One or more.

The image processing device according to claim 4 is the image processing device according to claim 2 , wherein the size information indicates width information indicating the width of the image, height information indicating the height of the image, and an aspect ratio of the image. Aspect ratio information.

The image processing apparatus according to claim 5 is the image processing apparatus according to any one of claims 1 to 4 , further comprising recording means for recording an image on a recording sheet, wherein the output control means is corrected by the image correction means. The recorded frame image and the still image are recorded on the same paper surface of the recording paper by the recording means.
The image processing apparatus according to claim 6 is the image processing apparatus according to any one of claims 1 to 5, wherein the acquisition unit acquires a plurality of still image files and one or more moving image files. The image feature amount calculating unit calculates feature amounts representing the image features of still images of all the still image files acquired by the acquiring unit, and the reference feature amount calculating unit is calculated by the image feature amount calculating unit. One reference feature value is calculated from a plurality of feature values corresponding to all still images.
The image processing device according to claim 7 is the image processing device according to claim 6, wherein the reference feature value calculating unit represents an image feature of each still image calculated by the image feature value calculating unit. The image processing apparatus according to claim 6, wherein an average value obtained by the step is calculated as a reference feature amount.
The image processing device according to claim 8 is the image processing device according to any one of claims 1 to 7, wherein the image feature amount calculation unit is configured to determine whether the file acquired by the acquisition unit is a still image file. A feature amount that represents an image feature of a still image in a still image file is calculated, and a feature amount that represents an image feature of a frame image extracted from the moving image file when the file acquired by the acquisition unit is a moving image file. It is characterized by not calculating.

According to the image processing apparatus of the first aspect, each frame image and each still image are output on the same output surface in a state in which the feature amount matches the reference feature amount. Therefore, there is an effect that it is possible to output an image list in which a frame image and a still image are mixed and an image list in which visual unification is ensured between the frame image and the still image.

The reference feature amount calculating means calculates at least one reference feature quantity from the feature quantity corresponding to one or more still images, the image correcting means, a feature amount of each frame image of the one or more frame images The image is corrected so as to match the reference feature amount. Therefore, although frame images tend to be smaller in size or have a color cast compared to still images, matching the feature values of the frame image with the feature values of the still image can reduce the size mismatch or color cast. It will be resolved. Therefore, there is an effect that it is possible to output an image list in which visual unification is ensured with high quality between the frame image and the still image.

According to the image processing apparatus according to claim 2, wherein, in addition to the effects of the image processing apparatus according to claim 1, of the size information indicating the information on the size of the color information and the image showing the information about the colors of the image At least one is calculated as a feature amount of each frame image and a feature amount of each still image, and is corrected so that at least one of the feature amounts coincides with each other. Therefore, there is an effect that it is possible to output an image list in which visual unification is ensured with respect to at least one of color and size between the frame image and the still image.

According to the image processing apparatus according to claim 3, wherein, in addition to the effects of the image processing apparatus according to claim 2, color information, luminance information indicating luminance, saturation information indicating the saturation, hue information indicating a hue Output a list of images that ensure a visual sense of uniformity in terms of one or more of luminance, saturation, and hue between the frame image and the still image. There is an effect that can be made.

According to the image processing device of claim 4 , in addition to the effect produced by the image processing device of claim 2 , the size information includes width information indicating the width of the image, height information indicating the height of the image, Since the aspect ratio information indicates the aspect ratio of the image, it is possible to output a list of images in which a visual unification feeling is ensured with higher accuracy between the frame image and the still image. There is an effect.

According to the image processing device of the fifth aspect , in addition to the effect produced by the image processing device according to any one of the first to fourth aspects, the output control means includes a frame image corrected by the image correction means, and a still image. Are recorded on the same surface of the recording paper by the recording means, so that a recording paper on which an image list in which visual unification is ensured between the frame image and the still image is recorded can be output. There is an effect that can be done.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. FIG. 1 is a perspective view showing an external configuration of a multifunction peripheral device (hereinafter, referred to as “MFP (Multi Function Peripheral)”) 1 as an embodiment of an image processing apparatus of the present invention. In particular, the MFP 1 outputs an image list in which a frame image extracted from a moving image and a still image are mixed, and an image list in which visual unification is ensured between the frame image and the still image is output. It is a device that can.

  The MFP 1 is a multifunction peripheral device having various functions such as a photo capture function, a copy function, a scanner function, and a media capture function. In the media capture function, a process for reading out an image file from a media card mounted in a slot unit 6 to be described later and printing it out, a process for displaying and outputting an image corresponding to the image file read out from the media card on the LCD 5, and a scan function The stored image data can be stored in a media card.

  A scanner 2 for reading a document is arranged on the top of the MFP 1. In addition, a printer 3 composed of a so-called inkjet printer is built in the housing as a device for printing an image on recording paper.

  A slot 6 is provided on the front surface of the MFP 1. The slot portion 6 is provided with a card slot into which various media cards such as a compact flash (registered trademark) and an SD memory card can be directly inserted.

  An operation panel is provided in front of the document cover body, and includes an operation unit 4 including a plurality of keys and a touch panel, and an LCD 5. The MFP 1 displays the operation procedure and the status of the process being executed on the LCD 5 and also displays information corresponding to the operation of the operation unit 4.

  The electrical configuration of the MFP 1 will be described with reference to FIG. FIG. 2 is a block diagram showing an electrical configuration of the MFP 1. The MFP 1 mainly includes a CPU 11, a ROM 12, a RAM 13, an EEPROM 14, a scanner 2, a printer 3, an operation unit 4, an LCD 5, and a slot unit 6.

  Among these, the CPU 11, ROM 12, RAM 13, and EEPROM 14 are connected to each other via a bus line 26. The scanner 2, the printer 3, the operation unit 4, the LCD 5, the slot unit 6, and the bus line 26 are connected to each other via an input / output port 27.

  The CPU 11 controls each function of the MFP 1 and controls each unit connected to the input / output port 27 according to fixed values and programs stored in the ROM 12, RAM 13, and EEPROM 14.

  The ROM 12 is a non-volatile memory that stores the control program 12a executed by the CPU 11 and can retain the contents even after the power is turned off. The control program 12a includes a program for executing processing shown in flowcharts in FIGS.

  The ROM 12 further includes a horizontal image arrangement number storage area 12b and a one-page image arrangement number storage area 12c.

  The horizontal image arrangement number storage area 12b is an area for storing “the number of horizontal arrangements”. “Number of sheets arranged in the horizontal direction” is a value indicating the number of index images that can be arranged in a horizontal row in the image list. Note that the MFP 1 according to the present embodiment includes a plurality of types of “horizontal arrangement number” according to the print paper size (L, A4, Letter, etc.) and the print mode (high definition, normal, high speed, etc.) in advance. In the arrangement printing process (see FIGS. 10 and 11), which will be described later, an appropriate value is read out and used.

  The one-page image arrangement number storage area 12c is an area for storing “the number of image arrangements in one page”. “Number of images per page” is a value indicating the maximum number of index images that can be arranged in an image list formed on one page of recording paper. In this embodiment, a plurality of types of “number of images per page” are stored in advance according to the print paper size and print mode, and in the layout print processing (see FIGS. 10 and 11) described later. In the following description, a suitable value is read out and used.

  The RAM 13 is a memory for temporarily storing various data when each operation of the MFP 1 is executed. The extracted frame number storage area 13a, the processed file number storage area 13b, the processed image total number storage area 13c, and the generated page number storage. An area 13d, a group information storage area 13g, a file information storage area 13h, an arrangement information storage area 13i, a frame image information storage area 13j, a decoded data storage area 13k, a counter storage area 13m, and an identification image storage area 13n are provided.

  Here, the meaning of terms used in the present embodiment will be described. In the present embodiment, “moving image” means an image reproduced by switching and displaying a large number of frame images. The “moving image file” is a file for reproducing a moving image, and includes, for example, an AVI moving image file, a quick time moving image file, and the like. However, any information file can be used as long as it can reproduce a moving image. In the present embodiment, “frame image information” means information corresponding to one frame image constituting a moving image.

  The extracted frame number storage area 13a is an area for storing “the number of frame images extracted”. The “number of frame image extractions” is a value indicating how many frame images corresponding to one moving image file are to be extracted. For example, when “5” is stored as the “number of extracted frame images”, the MFP 1 corresponds to five frame images per moving image file for each moving image file stored in the media card. Frame image information is extracted, and five index images per moving image file are arranged in the image list based on the extracted frame image information.

  The MFP 1 according to the present embodiment will be described assuming that an arbitrary value input by the user from the operation unit 4 is stored as the “frame image extraction number” in the extracted frame number storage area 13a. The “frame image extraction number” is automatically determined in accordance with a predetermined relationship so that the “frame image extraction number” increases as the size of the moving image file is stored in the ROM 12 or the like. May be.

  The processing file number storage area 13b is an area for storing the “total number of processing files”. The “total number of processed files” is a value indicating the total of still image files and moving image files stored in the media card.

  The processed image total number storage area 13c is an area for storing “total number of processed images”. The “total number of processed images” is a value indicating the total number of index images formed in the image list. The generated page number storage area 13d is an area for storing “number of generated pages”. The “number of generated pages” is a value indicating the total number of pages of the image list output by executing the layout printing process (FIGS. 10 and 11) described later.

  The group information storage area 13g, the file information storage area 13h, and the arrangement information storage area 13i will be described later with reference to FIGS.

  The frame image information storage area 13j is an area for storing frame image information extracted from the moving image file. The decoded data storage area 13k is an area for storing a result of decoding the frame image information or a result of decoding the still image file. The counter storage area 13m is an area for storing various counters.

  The reference feature quantity storage area 13n is an area for storing a reference feature quantity calculated as a reference from one or more still picture feature quantities (image size, image color). Since each index image output as an image list is corrected based on the reference feature amount, each index image included in the image list is visually unified.

  FIG. 3A illustrates an example of the image list 20 that the MFP 1 prints out on one page of recording paper. When the still image file and the moving image file are mixed on the media card, the MFP 1 converts the index image 21a, 21b, 21c corresponding to the frame image information and the still image file as shown in FIG. An image list 20 in which corresponding index images 22 are mixed is output. Therefore, the user can visually recognize the index images 21a, 21b, and 21c indicating the contents of the moving image file and the index image 22 indicating the contents of the still image file at a time using the one-page image list 20. Convenience is high.

  On the other hand, FIG. 3B is a diagram showing an image list 200 printed out on one page of recording paper by an apparatus different from the present invention. In both cases, the index images 21a, 21b, and 21c extracted from the same moving image file and the index image 22 extracted from the same still image file are printed out on one page of recording paper.

  Therefore, in the image list 200 shown in FIG. 3B, as in the image list 20 shown in FIG. 3A, the three index images 21a extracted from the first moving image file and the second moving image are displayed. Two index images 21b extracted from the image file, three index images 21c extracted from the third moving image file, and three index images 22 extracted from the three still image files are included. .

  However, in the image list 200 shown in FIG. 3B, the size (size) of each index image 21a, 21b, 21c extracted from the moving image file and the index image 22 extracted from the still image file. In addition, the colors of the index images are not unified to the same color (the hatching pattern and the dot pattern in the index images 21a, 21b, and 21c shown in FIG. 3B are the index images 21a and 21b). , 21c are in different color casts.) This is because a moving image file is used for a moving image file because the adjustment methods such as flash photography, ISO sensitivity, F value, and white balance are different. This is because the index images 21a, 21b, and 21c extracted from the image tend to have a color cast as compared with still images.

  As a result, as shown in the image list 200 in FIG. 3B, when the index images 21a, 21b, and 21c extracted from the moving image file and the index image 22 extracted from the still image file coexist. The image list 200 is difficult to see due to lack of visual unity between the two.

  On the other hand, in the image list 20 shown in FIG. 3A, the index images 21a, 21b, and 21c extracted from the moving image file and the index image 22 extracted from the still image file are all the same size ( Size), and the colors of the index images are unified to the same color.

  That is, when the MFP 1 prints out the image list 20 including the index images 21a, 21b, and 21c extracted from the moving image file and the index image 22 extracted from the still image file, The index images 21a, 21b, 21c, and 22 can be corrected, and the image list 20 in which a visual sense of uniformity is secured between the index images 21a, 21b, 21c, and 22 can be output.

  Further, the MFP 1 determines the arrangement order for each of the index images 21a, 21b, 21c, and 22 so that the image group including the index images 21a, 21b, 21c, and 22 is arranged in the order according to the shooting date and time. The image list 20 in which the index images 21a, 21b, 21c, and 22 are arranged in the determined arrangement order is output.

  The image list 20 shown in FIG. 3A is an example. For example, the index images are arranged in such an order that the shooting date and time becomes newer from the top to the bottom of the image list 20 and from the left to the right. The arrangement order is determined so that 21a, 21b, 21c, and 22 are arranged.

  Although detailed description will be given later, the MFP 1 sorts only on the shooting date and time, regardless of whether the index image 22 is based on a still image file or the index images 21a, 21b, and 21c based on a moving image file. The arrangement order is determined, and an image list 20 in which the index images 21a, 21b, 21c, and 22 are arranged in the arrangement order is output. Therefore, for example, even if the user does not remember whether the movie was shot or shot as a still image, the shooting date / time such as whether the shooting date / time is quite old or recent This makes it possible to quickly find the target image.

  As will be described in detail later, the MFP 1 according to the present embodiment is configured to classify the index images 21a, 21b, 21c, and 22 into groups for each shooting date and output an image list 20 for each group. Therefore, for example, the shooting date 23 of the group may be printed in the header portion of the image list 20.

  FIG. 4 is a diagram schematically showing the configuration of the file information storage area 13h. As shown in FIG. 4, the file information storage area 13h includes a file ID storage area 201 for storing the file ID, a file name storage area 202 for storing the file name, a file type storage area 203 for storing the file type, and an extraction frame. Extracted frame number storage area 204 for storing the number of frames, total frame number storage area 205 for storing the total number of frames, fps information storage area 206 for storing fps information, file date information storage area 207 for storing file date information, file time A file time information storage area 208 and a file identification information storage area 209 for storing information are provided, and file information 200 including a file name, file type, number of extracted frames, fps information, file date information, file time information, and file identification information. The file information 200 It is stored in association with the file ID to be. As a preparatory process for determining the arrangement order of the index images 21a, 21b, 21c, and 22 in the image list 20 (see FIG. 3), the MFP 1 performs each of the still image file or the moving image file stored in the media card. File information 200 is created and stored in the file information storage area 13h.

  The file ID storage area 201 is an area for storing a file ID. The file name storage area 202 is an area for storing a file name of a still image file or a moving image file. As shown in FIG. 4, an extension is added to each file name. For example, the extension “jpg” is added when the image file is a JPEG file, the extension “avi” is added when it is an AVI movie file, and the extension “mov” is added when it is a quick time movie file. Added.

  The file type storage area 203 is an area for storing a value indicating the type (format) of each image file. The type of the image file can be determined based on, for example, an extension added to the file name. In the present embodiment, when the image file format is a JPEG file (extension jpg), a value “0” indicating that it is a JPEG file is stored in the file type storage area 203, and the image file format is an AVI video file. In the case of (extension avi), the value “1” is stored in the file type storage area 203, and in the case where the format of the image file is a quick time video file (extension mov), the value “2” is stored in the file type storage area. 203.

  Note that the type of the image file can also be determined based on information stored in several bytes from the top of each image file. For example, if the first two bytes are “0xFFD8”, it can be determined that the file is a JPEG file. If the first two bytes are “0x424D”, it can be determined that the data is bitmap data. The type of image file may be determined based on such information.

  The extracted frame number storage area 204 is an area for storing a value indicating how many frame image information corresponding to the number of frame images is to be extracted from the image file. In the case of a still image file, “1” is stored in the extracted frame number storage area 204. On the other hand, in the case of a moving image file, the extracted frame number storage area 204 stores a value of “2” or more.

  The total frame number storage area 205 is an area for storing the total number of frames of each image file. In the case of a moving image file, the total number of frames included in the moving image file is stored, and in the case of a still image, 1 is always stored.

  The fps information storage area 206 is an area for storing fps (Frame Per Second) information of each image file. If it is a still image file, “0” is stored. On the other hand, in the case of a moving image file, for example, “60” indicating that there are 60 frame images per second, and “30” indicating that there are 30 frame images per second include the fps information. Stored in the storage area 206.

  The file date information storage area 207 is an area for storing file date information indicating the shooting date of each image file. For example, the shooting date “January 15, 2008” is represented as file date information “20080115”.

  The file time information storage area 208 is an area for storing file time information indicating the shooting time of each image file. For example, the shooting time “12:50:45” is represented as file time information “125045”. When the image file is a moving image file, information indicating the shooting start time is stored as file time information. Note that the file date information and the file time information correspond to the shooting date information described in the claims.

  In the following description, the total number of frames, fps information, file date information, and file time information are acquired by reading the information described in the header of each image file. You may acquire the information.

  The file additional information storage area 209 is an area for storing other unique information of each image file.

  The MFP 1 according to the present embodiment determines the arrangement order of the index images so that the index images are arranged in the order according to the shooting date / time based on the unique information of each image file stored in the file information storage area 13h. . Information indicating the determined arrangement order is stored in the arrangement information storage area 13i shown in FIG.

  FIG. 5 is a diagram schematically showing the configuration of the arrangement information storage area 13i. As shown in FIG. 5, the arrangement information storage area 13 i has an arrangement information 300 including a file ID, a group number, an extracted frame number, image date information, image time information, and image identification information, and an arrangement corresponding to the arrangement information 300. Remember the order.

  As shown in FIG. 5, the arrangement information storage area 13i includes an arrangement order storage area 301, a file ID storage area 302, a group number storage area 303, an extracted frame number storage area 304, an image date information storage area 305, and image time information. A storage area 306 and an image identification information storage area 307 are provided.

  The arrangement information storage area 301 is an area for storing the arrangement order. In the layout printing process (see FIGS. 10 and 11) described later, the layout information 300 is read in the order according to the layout order, and the frame image or still image index image specified by each layout information 300 is placed in the layout order. The images are arranged in the image list 20 in the order according to the order.

  The file ID storage area 302 is an area for storing a file ID. As described with reference to FIG. 4, since the file ID and the file name are stored in a one-to-one correspondence in the file information storage area 13h, one image file is specified from the file ID. be able to.

  The group number storage area 303 is an area for storing the group number. The group number will be described later with reference to FIG.

  The extracted frame number storage area 304 is an area for storing the extracted frame number. As described above, a plurality of pieces of frame image information are extracted for one moving image file. Therefore, information indicating the number of frame image information counted from the head is stored in the extracted frame number storage area 304 as “extracted frame number”.

Here, the “extracted frame number” includes the total number of frames of the moving image file stored in the “total frame number storage area 205”, the number of extracted frames stored in the “extracted frame number storage area 13a”, and the moving image. For example, it can be calculated as follows using a frame image extraction number counter value indicating the number of frames to be extracted.
(Extracted frame number) = [(frame image extraction number counter value−1) × (total number of frames) ÷ (number of extracted frames)]
Here, [] represents a Gaussian symbol, and [a] represents a maximum integer not exceeding a.

  The image date information storage area 305 is an area in which image date information indicating the shooting date of the image file specified by the file ID stored in the file ID storage area 302 is stored. The image time information storage area 306 is an area in which image time information representing the shooting time is stored. For still image files, the file time information stored in the corresponding area of the file information storage area 13h is copied to the image time information storage area 306.

  On the other hand, for a moving image file, since a plurality of pieces of frame image information are extracted from the moving image file, image time information representing the shooting time is acquired and stored in the image time information storage area 306 for each frame image information. . A specific calculation example for calculating the image time information for each frame image information will be described later with reference to the flowchart shown in FIG.

  The image feature storage area 307 is an area for storing still image feature quantities. In the present embodiment, the size of each still image and the color of the image are calculated and stored as the feature amount of the still image. The above-described reference feature value is calculated from all the feature values of the still image stored in the image feature storage area 307.

  In the layout printing process (see FIGS. 10 and 11) described later, the index images 21a, 21b, and 21c extracted from the moving image file based on the layout information 300 stored in the layout information storage area 13i, The index image 22 extracted from the image file is all printed with the same size (size), and the image list 20 in which the color of each index image is unified is the same color, and the index image is taken. The image list 20 arranged in the order according to the date and time is printed out.

  Further, the MFP 1 of the present embodiment classifies the index image into groups for each shooting date, and prints out the image list 20 for each group. For example, if an image file taken on “January 15, 2008” and an image file taken on “January 20, 2008” are stored on the media card, “January 15, 2008” The image list 20 composed of index images of the image files photographed in “and the image list 20 composed of index images of the image files photographed on“ January 20, 2008 ”are printed out. .

  FIG. 6 is a diagram showing an example of the configuration of the group information storage area 13g. As shown in FIG. 6, the group information storage area 13g is provided with a group number storage area 401, a group unit information storage area 402, and a group No storage area 403.

  The group number storage area 401 is an area for storing the number of groups generated for classifying the index image. In the example illustrated in FIG. 6, a group “10” is generated to classify the index image.

  The group unit information storage area 402 is an area for storing information for each group. The group unit information storage area 402 is provided with a storage area 403 for storing a group number and a group image number storage area 404 for storing the number of index images classified into the group specified by the group number. In the example illustrated in FIG. 6, “10” index images are classified into the group having the group number “1”.

  According to the MFP 1 of the present embodiment, by outputting the image list 20 for each group, the user can visually recognize the image list 20 for each group (for each shooting date), so that the user is classified into groups for each shooting date. It is possible to obtain a list of images with good browsing properties.

  Next, processing for outputting the image list 20 as shown in FIG. 3A will be described with reference to FIGS. If this process is briefly described, first, an arrangement order determination process for determining the arrangement order of index images is executed (S100). Next, a reference feature value calculation process (S150) for calculating a reference feature value for correcting each index image is executed. Then, according to the determined arrangement order, an arrangement printing process (S200) for outputting an image list 20 including visually unified index images is executed. Hereinafter, each process will be described in detail.

  FIG. 7 is a flowchart showing the arrangement order determination process (S100) executed by the MFP 1. The arrangement order determination process (S100) shown in FIG. 7 is executed when a media card is inserted into the slot unit 6 and an execution instruction for print output of the image list 20 is input by the user.

  In this arrangement order determination process (S100), for each of the frame image information extracted from the moving image file and the still image file read from the media card, image date information and image time information representing the shooting date and time of each image information are obtained. This is a process of determining the arrangement order of index images on the basis of the acquired image date information and image time information.

  First, the value input from the operation unit 4 by the user is stored in the extracted frame number storage area 13a as “frame image extraction number” (S101). Next, the “total number of processed files” indicating the total of still image files and moving image files stored in the media card is calculated and stored in the processed file number storage area 13b (S102).

  Next, a processing file number counter, a processing image number counter, a frame image extraction number counter, and a moving image file number counter are each initialized to 0 (S103). These counters are provided in the counter storage area 13m (see FIG. 2).

  Next, of the image files stored on the media card, one image file is selected as a processing target, and the file name of the image file and the type of the image file are acquired. Then, the acquired file name and value indicating the type of image file are stored in the file name storage area 202 and the file type storage area 203 (see FIG. 4) corresponding to the file ID of the processing file number counter value, respectively (S104). ).

  Then, it is determined whether the selected image file is a moving image file or a still image file (S105). If it is determined that the file is a moving image file (S105: moving image), file date information, file time information, and fps information representing fps are acquired from the header of the moving image file (S106).

  Then, the file date information is stored in the file date information storage area 207 corresponding to the arrangement order of the processed image counter value, the file time information is stored in the file time information storage area 208, and the fps information is stored in the fps information storage area 206. Store (S107).

  Next, the current processing file number counter value is stored in the file ID storage area 302 of the arrangement information storage area 13i (see FIG. 5) corresponding to the arrangement order of the processed image number counter value. Further, the extracted frame number is calculated from the value of the frame image extraction number counter at the present time, and stored in the extracted frame number storage area 304 corresponding to the arrangement order of the processed image number counter value (S108).

  Next, time information of the frame image information is calculated, and the calculated time information is stored as image time information in the image time information storage area 306 corresponding to the arrangement order (S109). Specifically, first, the file time information indicating the shooting start time of the moving image file is VideoTime, the fps information is FPS, the image time information indicating the shooting time of the frame image information is FrameTime, and the frame image information to be processed is specified. Let N be the extracted frame number.

  Then, the file time information VideoTime is decomposed into moving image time information VT_Hour, moving image portion information VT_Minute, and moving image second information VT_Second as follows.

VT_Hour = [VideoTime / 10000]
VT_Minute = [(VideoTime−VT_Hour × 10000) / 100]
VT_Second = VideoTime− (VT_Hour × 10000 + VT_Minute × 100)
Next, the frame time information Frame_Hour, the frame portion information Frame_Minute, and the frame second information Frame_Second are calculated as, for example, Equation 1 below.
(Formula 1)
Here, Cal_Second, Cal_Minute, and Cal_Hour are temporary variables for obtaining the above information.
A mod B represents a remainder when A is divided by B. Cal_Second = VT_Second + [N / FPS]
Cal_Minute = VT_Minute + [Cal_Second / 60]
Cal_Hour = VT_Hour + [Cal_Minute / 60]
Frame_second = Cal_Second mod 60
Frame_Minute = Cal_Minute mod 60
Frame_Hour = Cal_Hour mod 24
The above is combined using, for example, Equation 2 below, and image time information FrameTime representing the shooting time of the frame image information is calculated.
(Formula 2)
FrameTime = Frame_Hour × 10000 + Frame_Minute × 100 + Frame_Second
Similarly, when the file date information of the moving image file is VideoDate and the image date information of the frame image information to be extracted is FrameDate, first, the VideoDate is changed to moving image year information VD_Year, moving image month information VD_Month, moving image date information VD_Day, For example, it decomposes | disassembles using the following Numerical formula 3.
(Formula 3)
VD_Year = [VideoDate / 10000]
VD_Month = [(VideoDate−VD_Year × 10000) / 100]
VD_Day = VideoDate− (VD_Year × 10000 + VD_Month × 100)
Next, an image date calculation process is executed (S110). The image date calculation process is a process of calculating frame year information Frame_Year, frame month information Frame_Month, and frame date information Frame_Day.

  FIG. 8 is a flowchart showing the image date calculation process (S110). Note that a variable M shown in the flowchart of FIG. 8 is a variable that means that M days are advanced from the current date.

  First, moving image year information VD_Year, moving image month information VD_Month, and moving image date information VD_Day are assigned to frame year information Frame_Year, frame month information Frame_Month, and frame date information Frame_Day, respectively (S1201). Next, [Cal_Hour / 24] is substituted into the variable M (S1202). Next, it is determined whether M is 0 (S1203). If the determination is affirmative (S1203: Yes), this process is terminated.

  On the other hand, if negative (S1203: No), the frame date information Frame_Day is determined (S1204). If it is determined that “1 to 27” (S1204: 1 to 27), “1” is added to Frame_Day (S1205).

  On the other hand, if “28 or 29” is determined in the determination in S1204 (S1204: 28 or 29), it is determined whether or not Frame_Month is “2” (S1206). When negative (S1206; No), it transfers to the process of S1205.

  On the other hand, if affirmative (S1206: Yes), it is next determined whether Frame_Day is "28" (S1208). If negative (S1208: No), “1” is substituted into Frame_Day, and “1” is added to Frame_Month (S1210).

  On the other hand, if the determination in S1208 is positive (S1208: Yes), it is determined whether or not the year indicated by Frame_Year is a leap year (S1209). If the determination is affirmative (S1209: Yes), the process proceeds to S1205. When negative (S1209: No), it progresses to the process of S1210.

  Next, in the determination of S1204, if “30” is determined (S1204: 30), it is next determined whether Frame_Month is 4, 6, 9, or 11 (S1211). If the determination is affirmative (S1211: Yes), the process proceeds to S1210.

  On the other hand, if negative (S1211: No), “1” is added to Frame_Day (S1216). Next, if “31” is determined in the determination of S1204 (S1204: 31), it is next determined whether Frame_Year is 12 (S1215). When negative (S1215: No), it progresses to the process of S1216.

  On the other hand, if the result is affirmative (S1215: Yes), “1” is substituted into Frame_Day, “1” is substituted into Frame_Month, and “1” is added to Frame_Year (S1217).

  After the processing of S1205, S1210, S1216, and S1217, “1” is subtracted from the variable M (S1212), and the process returns to S1203. If the determination in S1203 is affirmed (S1203: Yes), this process ends.

Returning to FIG. The values calculated in the process shown in FIG. 8 are combined using, for example, the following Equation 5 to calculate image date information FrameDate representing the shooting date of the frame image information. The calculated image date information is stored in the image date information storage area 305 (S111).
(Formula 5)
FrameDate = Frame_Year × 10000 + Frame_Month × 100 + Frame_Day
Next, “1” is added to the processed image number counter and the frame image extraction number counter, respectively (S113).

  Next, it is determined whether or not the value of the frame image extraction number counter is equal to or greater than the number of frame image extractions, that is, the number of frame image information extracted from the moving image file (S114). If the determination in S114 is negative (S114: No), the process returns to S108 and is repeated.

  On the other hand, if the determination in S114 is affirmative (S114: Yes), then the frame image extraction number counter is initialized to 0 (S115), and 1 is added to the processing file number counter (S122). Then, it is determined whether or not the value of the processing file number counter is equal to or greater than the “total number of processing files” indicating the total of still image files and moving image files stored in the media card (S123). If the determination in S123 is negative (S123: No), the processing returns to S104, the next image file is selected as a processing target, and the processing is repeated.

  Next, a case where it is determined that the image file to be processed is a still image file (S105: still image) will be described. In the case of a still image file, the current processing file number counter value is stored in the file ID storage area 302 corresponding to the arrangement order of the processing image number counter value. Further, “0” is stored in the extracted frame number storage area 304 corresponding to the arrangement order of the processed image counter value (S115a).

  Next, file date information and file time information of the still image file to be processed are acquired. In the file information storage area 13h, the file date information is stored in the file date information storage area 207 (see FIG. 4) corresponding to the file ID of the processing file counter value, and the file time information corresponding to the file ID is stored. The file time information is stored in the storage area 208 (S117). Note that “0” is stored in the fps information storage area 206.

  Then, the file date information of the still image file to be processed is stored in the image date information storage area 305 corresponding to the arrangement order of the processed image number counter values in the arrangement information storage area 13i (see FIG. 5). Further, the file time information of the still image file to be processed is stored in the image time information storage area 306 corresponding to the same arrangement order (S119). Then, 1 is added to the processed image number counter (S121), and the process proceeds to S123.

  As described above, according to the processing from S101 to S123, the file ID, the extracted frame number, the image date information, the image time information, and the file identification information are stored in the arrangement information storage area 13i. The arrangement order of the index images is not yet determined. In the processing after S124 described later with reference to FIG. 9, the arrangement order according to the shooting date and time is determined by sorting the arrangement information on the condition of the image date information and the image time information.

  FIG. 9 is a flowchart showing a continuation of the arrangement order determination process (S100) shown in FIG. 7, and in particular, a process for classifying index images arranged in the image list 20 based on image date information, and arrangement information A part of processing for sorting the arrangement information (see FIG. 5) stored in the storage area 13i in the arrangement order according to the shooting date and time is shown.

  As shown in FIG. 9, first, the value of the processed image number counter at the present time is stored as the “total number of processed images” in the processed image total number storage area 13c (see FIG. 2) (S124). Then, the processing image counter is initialized with “0” (S125). Here, the processed image number counter is a counter provided in the counter storage area 13m (see FIG. 2).

  Next, a group number is determined according to the image date information stored in the image date information storage area 305 of the arrangement information storage area 13i (see FIG. 5) (S126). Specifically, when the value of the image date information storage area 305 corresponding to the arrangement order of the value of the processed image counter is the same as the value of the already existing group, the corresponding group number is set as the image information. If not, a new group number is determined as the group number of this image information. However, the initial value of No is 1, and the new group No is the largest group No + 1 so far.

  In this way, the index images corresponding to the frame image information or still image files whose shooting dates are the same day are classified into the same group, while the index images corresponding to the frame image information or still image files whose shooting dates are different. Are classified into different groups.

  Then, the determined group number is stored in the group number storage area 303 (see FIG. 5) corresponding to the arrangement order of the values of the processed image number counter (S127). Next, “1” is added to the processed image number counter (S128), and it is determined whether the value of the processed image number counter is equal to or greater than the value stored in the total number of processed images (S129). If the determination in S129 is negative (S129: No), the process is repeated from S126.

  On the other hand, if the determination in S129 is affirmative (S129: Yes), the maximum group number is stored in the group number storage area 401 (see FIG. 6) (S130). Then, the arrangement information is sorted in ascending order of group number (S131). In the process of S131, it is sufficient if the arrangement information can be sorted in ascending order of the group numbers. Therefore, the sorting algorithm is not particularly limited, and for example, a quick sort method or a bubble sort method can be used. .

  Note that the arrangement information storage area 13i illustrated in FIG. 5 schematically illustrates a state in which arrangement information stored in the arrangement information storage area 13i is sorted in ascending order of group numbers.

  Returning to FIG. Next, the number of index images that are included in the same group number is calculated (S132), and stored in the group image number storage area 404 in association with the group number (S133). Next, the group counter is initialized with 1 (S134). This group counter is a counter provided in the counter storage area 13m (see FIG. 2).

  Next, the arrangement information is sorted in ascending order of image time information (S135). This sorting is done in groups. Thereby, the corresponding arrangement order is determined for each index image specified by the arrangement information.

  That is, the arrangement order according to the image time information (photographing time information) is determined in the index images classified into the same group. In the layout printing process (S200) to be described later, the image list 20 is printed out for each group. Since the index images in the same group are arranged in ascending order of the image time information, a large number of indexes are included in one group. Even in the case where images are included, the user can obtain a list of images with good browsing properties arranged in order of photographing time.

  If the image time information is the same, it is preferable not to change the original arrangement order. Therefore, a simple sort algorithm such as bubble sort is preferably used in the process of S135.

  Next, 1 is added to the group counter (S136), and it is determined whether the value of the group counter is larger than the value of the group number storage area 401 (see FIG. 6) (S137). If the determination in S137 is negative (S137: No), the process returns to S135 and is repeated. On the other hand, when the processing for all the groups is finished and the determination in S137 is affirmed (S137: Yes), the arrangement order determination processing (S100) is finished.

  FIG. 10 is a flowchart showing the reference feature amount calculation process (S150) executed by the MFP 1. The reference feature quantity calculation process (S150) shown in FIG. 10 is a process executed after the above-described arrangement order determination process (S100), and the reference feature quantity serving as a reference for correcting each index image is set to each stationary image. This is a process of calculating from the feature amount of the image.

  First, the arrangement order counter is initialized with “0” (S151), and then the file ID and extracted frame number corresponding to the arrangement order counter are read from the arrangement information storage area 109 (S152). Then, it is determined whether the image file corresponding to the read file ID is a moving image file or a still image file (S153). Since this determination has already been made once in S105, the determination information at that time is read from the file type determination area 203 and determined.

  If it is determined in S153 that the file is a still image file (S153: still image), the still image file is decoded (S154).

  Then, the image feature amount SSp of the size of the still image file is calculated, and the calculated image feature amount SSp is stored in the image feature amount storage area 307 corresponding to the arrangement order counter (S155). Here, the image feature amount SSp of the size includes an image width (Width), an image height (Height), and an aspect ratio (Aspect Ratio), and the image width (Width) is the horizontal direction of the image. The number of pixels of the image, the height of the image (Height) is calculated from the number of pixels in the vertical direction of the image, and the aspect ratio (Aspect Ratio) is calculated from the height of the image (Height) / the height of the image (Height). Stored in area 307.

  Next, the image feature amount CSp of the color of the still image file is calculated, and the image feature amount CSp of the color is stored in the image feature amount storage area 307 corresponding to the arrangement order counter (S156). Here, the color image feature CSp is the hue value (H_R, H_G, H_B), luminance value (S), and saturation value (V) of each still image, and the hue value (H_R, H_G, H_B) is The average RGB value, luminance value (S) of pixels of brightness of a predetermined value or more in the RGB space of each still image is the average luminance value in HSV space, and the saturation value (V) is the saturation of the image in the HSV space. And is stored in the image feature amount storage area 307.

  If it is determined in step S153 that the image is a moving image, 0 (zero) is stored in the corresponding image feature amount storage area 307 (S157), and the process proceeds to step S158.

  Thus, when the image feature amount SSp of the size and the image feature amount CSp of the color are calculated for the still image file and stored in the image feature amount storage area 307, “1” is converted into the arrangement order counter in S158 (S158). ), It is determined whether the processing of all the arranged files has been completed (S159). If not completed (S159: No), the processing from S152 is repeated, and if completed (S159: Yes), the processing of S160 is performed. Transition.

  In S160, the reference feature value BSSp is calculated from the image feature value SSp stored in the image storage means region 307 for the case where the value stored in the image feature value storage region 307 in S155 is not 0 (zero). The calculated reference feature value BSSp is stored in the reference feature value storage area 114 (S160).

  The reference feature amount BSSp is a feature amount that serves as a reference for the image feature amount SSp stored in the image feature amount storage area 307, and the reference feature amount BWidth relating to the width (Width) of the image has the largest value (Width). ), The reference feature amount BAspectRatio relating to the aspect ratio (Aspect Ratio) is the most frequently used aspect ratio (Aspect Ratio), and the reference feature amount BHeight relating to the height of the image (Height) is the reference feature amount relating to the width (Width) of the image. The reference feature amount BAspectRatio relating to BWidth / Aspect Ratio (Aspect Ratio) is used.

  Thus, the reference feature amount BSSp amount is set from the image feature amount SSp of each still image file. The reference feature amount BWidth and the reference feature amount BAspectRatio may be average values of the width (Width) and aspect ratio (Aspect Ratio) of all images.

  Then, a reference feature value BCSp is calculated from the image feature value CSp of the color stored in the image storage unit area 307 as an object whose value stored in the image feature value storage area 307 is not 0 (zero). Then, the calculated reference feature value BCSp is stored in the reference feature value storage area 114 (S161), and this process ends.

  The reference feature amount BCSp is a reference for the image feature amount CSp of the color stored in the image feature amount storage area 307. In the present embodiment, the reference feature amount BH_R related to the hue value (H_R) is the hue value of each image. The reference feature value BH_G related to the average value (H_R) and the hue value (H_G) is the average value of the hue value (H_G) of each image, and the reference feature value BH_B related to the hue value (H_B) is the hue value (H_B) of each image. ) And the reference feature value BS regarding the saturation value (S) are the average value of the saturation value (S) of each image, and the reference feature value BV regarding the luminance value (V) is the luminance value of the image of each image Calculated as an average value of (V) and stored in the reference feature amount storage area 114.

  Then, using the reference feature value BSSp calculated in S160 and the reference feature value BCSp calculated in S161, each index image printed out on one page of recording paper in the next layout printing process (S200). By correcting this, the size and color of each index image can be visually unified.

  FIG. 11 is a flowchart showing the layout printing process (S200). The layout printing process (S200) shown in FIG. 11 is a process executed subsequent to the layout order determination process (S100) described with reference to FIGS. 7 to 9, and prints out the image list 20 on a recording sheet. It is processing.

  First, “horizontal arrangement number” indicating the number of index images for one horizontal row arranged in the image list 20 is read from the horizontal image arrangement number storage area 12b (S201). Next, “the number of image arrangements in one page” indicating the number of index images that can be arranged in the image list 20 for one page is read from the one-page image arrangement number storage area 12c (S202).

  Next, the group counter is initialized with 1 (S203). In the subsequent processing, this group counter functions as a counter indicating the group number of the group to be processed. Next, the page counter is initialized with 1 (S204). In the subsequent processing, this page counter functions as a counter indicating which page of the image list 20 is the processing target in the processing target group. The page counter is a counter provided in the counter storage area 13m (see FIG. 2).

Next, the number of group images corresponding to the group number of the group counter value is read from the group image number storage area 404 (S205). That is, the number of index images classified into the group of the group number is acquired. Then, from the read “number of group images” and “number of images arranged in one page”, a “number of generated pages” indicating how many pages of the image list 20 are printed out is calculated for the group to be processed. Stored in the generated page number storage area 13d (see FIG. 2) (S206). In the present embodiment, the “number of generated pages” is calculated using, for example, Equation 6 below.
(Formula 6)
(Number of generated pages) = [(Number of group images) / (Number of images arranged in one page)] + 1
Next, the layout image counter is initialized to 0 (S207), and the number of images layout_no arranged on one page of recording paper is calculated (S208). In this embodiment, layout_no is calculated using, for example, Equation 7 below.
(Formula 7)
(Page counter value) <number of generated pages layout_no = (number of images arranged in one page)
When (page counter value) ≧ number of generated pages layout_no = (number of group images) − {(number of images arranged in one page) × (page counter value)}
Next, the file ID, the extracted frame number, and the file identification information corresponding to the arrangement order are read from the arrangement information storage area 13i (S209). Then, it is determined whether the image file corresponding to the read file ID is a moving image file or a still image file (S210).

  If it is determined in S210 that the file is a moving image file (S210: moving image), then frame image information for one frame image specified by the extracted frame number is extracted from the moving image file (S211).

  Next, the extracted frame image information is decoded and stored in the decoded data storage area 13k (S212). Then, in S160 and S161 shown in FIG. 10, the feature amount of the frame image information matches each reference feature amount using the reference feature amount BSSp and the reference feature amount BCSp stored in the reference feature amount storage area 114. (S212a). Specifically, this correction is performed in the order of enlargement / reduction processing, trimming processing, hue conversion processing, luminance conversion processing, and saturation conversion processing.

  First, enlargement / reduction processing is performed. In this process, the enlargement / reduction ratio is set to SizeRate, and the SizeRate is calculated by the following equation.

そして、算出された拡大縮小率ＳｉｚｅＲａｔｅにより処理対象となるフレーム画像情報の拡大縮小処理を行う。尚、拡大縮小はバイリニアやバイキュービックなどの画像補間により行うことができる。そして、この拡大縮小後の画像の幅をＷｉｄｔｈ’、拡大縮小後の画像の高さをＨｅｉｇｈｔ’とする。

Then, the frame image information to be processed is enlarged / reduced by the calculated enlargement / reduction ratio SizeRate. The enlargement / reduction can be performed by image interpolation such as bilinear or bicubic. The width of the image after enlargement / reduction is Width ′, and the height of the image after enlargement / reduction is Height ′.

  Next, trimming processing is performed. In this processing, the image width Width ′ after enlargement / reduction and the image height Height ′ after enlargement / reduction become the reference feature amount BWidth relating to the image width and the reference feature amount BHeight relating to the image height, respectively. Trim the image. If the image before trimming is O (x, y) and the image after trimming is P (x, y), the image after trimming can be obtained by the following equation. However, x and y represent coordinate values.

上述した拡大縮小処理、トリミング処理によって、フレーム画像情報を基準特徴量ＢＳＳｐと一致させることができるので、各フレーム画像情報に対応するインデックス画像の大きさを統一させることができる。

The frame image information can be matched with the reference feature amount BSSp by the above-described enlargement / reduction processing and trimming processing, so that the size of the index image corresponding to each frame image information can be unified.

Next, a hue conversion process is performed. In this processing, each pixel value in the RGB space of the correction target image O is set as O (x, y) R, O (x, y) G, O (x, y) B, and the pixel value O ′ ( x, y) R, O ′ (x, y) G, and O ′ (x, y) B are obtained by the following equations. Here, x and y are coordinate values of the image.
Further, the average value of the luminance values (V) of the entire image of the frame image information is assumed to be AveV.

O '(x, y) R = O (x, y) R + H_R × AveV ÷ BV
O '(x, y) G = O (x, y) G + H_G × AveV ÷ BV
O '(x, y) B = O (x, y) B + H_B × AveV ÷ BV

Next, luminance conversion processing is performed. In this process, the pixel values O ′ (x, y) R, O ′ (x, y) G, and O ′ (x, y) B after the hue correction obtained in the hue conversion process are converted into the HSV space. Let O ′ (x, y) H, O ′ (x, y) S, and O ′ (x, y) V.

  Then, the pixel values O ″ (x, y) H, O ″ after luminance correction are obtained from the converted O ′ (x, y) H, O ′ (x, y) S, O ′ (x, y). (X, y) S, O ″ (x, y) V is obtained by the following equation.

O '' (x, y) H = O '(x, y) H
O '' (x, y) S = O '(x, y) S
O '' (x, y) V = O '(x, y) V × BV ÷ AveV

Finally, saturation conversion processing is performed. In this process, pixels after saturation conversion are obtained from pixel values O ′ ″ (x, y) H, O ′ ″ (x, y) S, O ″ ′ (x, y) V after luminance correction. The values O ′ ″ (x, y) H, O ′ ″ (x, y) S, and O ′ ″ (x, y) V are obtained by the following equations.
Further, the average value of the saturation values (S) of the entire image of the frame image information is AveS.

O '''(x, y) H = O''(x, y) H
O '''(x, y) S = O''(x, y) S × BS ÷ AveS
O '''(x, y) V = O''(x, y) V
Since the feature of the frame image information can be matched with the reference feature amount BCSp by the above-described hue, luminance, and saturation conversion processing, the color of the index image corresponding to each frame image information can be unified.

  Then, the frame image information corrected as described above is arranged at a predetermined position of the arrangement buffer provided in the RAM 13 (see FIG. 2) (S213).

  On the other hand, if it is determined in S210 that the file is a still image file (S210: still image), the still image file specified by the file ID corresponding to the arrangement order is decoded and stored in the decoded data storage area 13k. (S214). Then, as described in S212a described above, the still image file is corrected using the reference feature amounts BSSp and BCSp stored in the reference feature amount storage area 114 (S214a). By the correction of S214a, the still image file can be matched with the reference feature value BSSp and the reference feature value BCSp, so that the size and color of the index image corresponding to each still image file can be unified. The size and color of the image can be unified between the frame image information and the still image file.

  Then, the still image file corrected as described above is arranged at a predetermined position of the arrangement buffer provided in the RAM 13 (see FIG. 2) (S215).

  After executing the process of arranging either the corrected frame image information or the corrected still image file in the arrangement buffer in this way, the next step is “horizontal” which is the upper limit of the number of images that can be arranged in the horizontal direction. It is determined whether or not the processing has been performed for the “number of directional arrangements” (S216). If the determination in S216 is negative (S216: No), the horizontal position indicating the arrangement position of the next data is updated (S220).

  Then, “1” is added to the layout image counter (S221), and it is determined whether or not the value of the layout image counter is equal to or larger than the number “layout_no” of index images to be arranged on the current recording paper to be processed (S221). S222). If the determination in S222 is negative (S222: No), the process returns to S209, and the arrangement information corresponding to the next arrangement order is processed.

  In this way, by repeating the process, frame image information corresponding to a frame image of the number of extracted frames (corresponding to the predetermined number described in the claims) is extracted from each of the moving image files stored in the media card. Extracted and placed in the placement buffer. Also, a still image file stored in the media card is extracted and placed in the placement buffer.

  If the arrangement processing for “horizontal arrangement number” times is executed while the processing is repeated in this way, the determination in S216 is affirmed (S216: Yes). The data is transferred to the printer 3 (see FIG. 2) (S217). Thereby, the data corresponding to the index image for one horizontal row arranged in the arrangement buffer is subjected to the printing process by the printer 3, and the image list 20 can be output.

  Then, the horizontal position indicating the horizontal position of the next index image to be processed is initialized (S218), and the vertical position indicating the vertical position is updated (S219). That is, the next arrangement position is determined so that the next index image is arranged from the left end of the next column in the image list 20. And the process of S221 and S222 mentioned above is performed.

  When the number of index images processed in this way reaches the number of index images layout_no to be arranged on the recording paper to be processed, the determination in S222 is affirmed (S222: Yes), so the next process shown in FIG. Migrate to

  FIG. 11 is a flowchart showing the continuation of the layout printing process (S200) shown in FIG. 10, and is executed when the determination in S222 is affirmed.

  First, it is determined whether there is data that has not yet undergone printing processing in the arrangement buffer (S223). If the determination in S223 is affirmative (S223: Yes), the remaining data is transferred to the printer 3 (S224), the printer 3 prints it out, and the process proceeds to S225. However, an area where no image is arranged in the arrangement buffer is output as a white background. On the other hand, if the determination in S223 is negative (S223: No), the determination in S224 is skipped and the process proceeds to S225.

  Then, a print completion command indicating completion of data output for one page is passed to the printer 3 (S225). Then, “1” is added to the page counter (S226). Next, the horizontal position and the vertical position indicating the arrangement position of the next data in the arrangement buffer are initialized (S227). Then, it is determined whether the value of the page counter is greater than the “number of generated pages” indicating how many pages of the image list 20 are to be printed out for the current processing target group (S228). If the determination in S228 is negative (S228: No), the process returns to the process in S207 shown in FIG. 10, and the process for the next page is started for the same group.

  On the other hand, if the determination in S228 is affirmative (S228: Yes), 1 is added to the group counter (S229). Then, it is determined whether or not the value of the group counter has become larger than the value stored in the group number storage area 401 (S230). When determination of S230 is denied (S230: No), it returns to S204 and repeats a process. That is, the process is repeated with the group specified by the next group number as the process target.

  On the other hand, if the determination in S228 is affirmative (S228: Yes), the layout printing process (S200) is terminated.

  As described above, according to the MFP 1, as described with reference to FIG. 3A, the index images 21a, 21b, and 21c extracted from the moving image file and the index images extracted from the still image file. 22 is printed out, the size of each index image 21a, 21b, 21c, 22 is corrected so as to match the reference feature value BSSp, and each index image 21a, 21b, 21c, 22 is corrected. Since the color of the image is corrected so as to match the reference feature value BCSp and printed out, the image list 20 in which visual unification is ensured among the index images 21a, 21b, 21c, and 22 is output. Can do. Further, it is possible to output the image list 20 in which the index images are arranged in the arrangement order according to the file time information (that is, the shooting time of the still image file or the frame image information).

  Although the present invention has been described based on the embodiments, the present invention is not limited to the above-described embodiments, and various improvements and modifications can be easily made without departing from the spirit of the present invention. Can be inferred.

  For example, in the present embodiment, the image list 20 is described as being printed out on recording paper. However, the present invention can also be applied to the case where the image list 20 is displayed on a display device such as the LCD 5.

  The present invention is also applied to, for example, a case where the present invention is applied to a personal computer and the image list 20 is displayed on a PC monitor, or the image list is printed out by controlling the printer from the personal computer. Can be applied.

  In the above embodiment, the image list 20 is output for all groups. However, for example, the image list 20 may be output only for the group selected by the user.

  In the above-described embodiment, the moving image file and the still image file are both assumed to be image files generated by shooting. However, the moving image file and the still image file are generated by a method other than shooting. Even if it is a thing, this invention is applicable. In that case, the arrangement order of the index images is determined based on other conditions such as file generation date / time information indicating file creation date / time and file update date / time information instead of the shooting date / time. Also good.

  In the above-described embodiment, the case where the still image file and the moving image file are mixed in the media card has been described. However, the present invention is also applicable to the case where only the moving image file is stored in the media card. Is applicable.

  In the above-described embodiment, the image information stored in the removable media card is the processing target. However, the present invention is also applicable to the case where the image information stored in the internal memory of the MFP 1 is the processing target. Applicable.

  Further, in the above-described embodiment, the size of each index image is matched with the reference feature amount BSSp in order to unify the size of each index image, and each index image is unified in order to unify the color of each index image. In the above description, the color is matched with the reference feature value BCSp. However, the color may be corrected so as to match only one of the reference feature value BSSp and the reference feature value BCSp.

  In the above-described embodiment, the case of converting all of hue, luminance, and saturation in the processing of S212a and S214a shown in FIG. 11 has been described. However, any one or more of them may be converted.

  In the above-described embodiment, the case where the image feature amounts SSp and CSp are calculated only for the still image corresponding to the still image file in S155 and S156 shown in FIG. 10 has been described. SSp and CSp may be calculated, and in S160 and S161, the respective reference feature values may be calculated from the image feature values SSp and CSp of both (still image and moving image).

1 is a perspective view showing an external configuration of an MFP that is an embodiment of an image processing apparatus of the present invention. 2 is a block diagram illustrating an electrical configuration of the MFP. FIG. 3 is a diagram illustrating an example of an image list printed out by an MFP. FIG. It is a figure which shows typically the structure of a file information storage area. It is a figure which shows typically the structure of an arrangement | positioning information storage area. It is a figure which shows an example of a structure of a group information storage area. It is a flowchart which shows an arrangement | positioning order determination process. It is a flowchart which shows an image date calculation process. FIG. 8 is a flowchart showing a continuation of the arrangement order determination process shown in FIG. 7. FIG. It is a flowchart which shows a reference | standard feature-value calculation process. It is a flowchart which shows arrangement | positioning printing processing. FIG. 11 is a flowchart showing a continuation of the layout printing process shown in FIG. 10. FIG.

Explanation of symbols

1 MFP (an example of an image processing apparatus)
3 Printer (part of recording means)
6 Slot part (part of acquisition means)
20 Image list 21a, 21b, 21c Index image (an example of a frame image )
22 Index image (an example of a still image)
S211 Part of extraction means S225 Part of output control means S155 Part of image feature quantity calculation means S160, S161 Part of reference feature quantity calculation means S212a, S214a Part of image correction means

Claims (8)

Obtaining means for obtaining one or more still image files and one or more moving image files;
Extraction means for extracting one or more frame images from the moving image file acquired by the acquisition means;
Among prior SL still image files and moving image files acquired by the acquiring unit, a feature amount representing an image feature of the still image picture of quiescent image file, has been one or more still image files respectively acquired by the acquisition unit an image feature quantity calculating means for calculating for,
And the reference feature amount calculating means for calculating a reference feature amount as the one or more feature quantities or al criteria corresponding to one or more still image calculated by the image feature quantity calculating means,
The feature amount of each frame image of the one or more frame images and the feature amount of each still image of the one or more still images are matched with the reference feature amount calculated by the reference feature amount calculating means. Image correcting means for correcting the image,
An image processing apparatus comprising: output control means for outputting the frame image corrected by the image correction means and the still image on the same output surface. The image feature amount calculating means calculates at least one of color information indicating information about the color of the image and size information indicating information about the size of the image as the feature amount,
The reference feature value calculating means calculates a reference feature value serving as a reference from the feature value obtained by the image feature value calculating means,
The image correction means uses the reference feature value calculated by the reference feature value calculation means to calculate the feature value of each frame image of the one or more frame images and the feature value of each still image of the one or more still images. the image processing apparatus according to claim 1, characterized by the image correction to match the characteristic amount. The image processing apparatus according to claim 2 , wherein the color information is at least one of luminance information indicating luminance, saturation information indicating saturation, and hue information indicating hue. The image processing apparatus according to claim 2 , wherein the size information is width information indicating an image width, height information indicating an image height, and aspect ratio information indicating an image aspect ratio. A recording means for recording an image on a recording paper;
It said output control means includes: the frame image corrected by the image correcting means, wherein the still image, of claims 1 to 4, characterized in that to record on the same paper surface of the recording sheet by said recording means The image processing apparatus according to any one of the above. The acquisition means acquires a plurality of still image files and one or more moving image files,
The image feature amount calculating means calculates feature amounts representing image features of still images of all still image files acquired by the acquiring means,
6. The reference feature amount calculating unit calculates one reference feature amount from a plurality of feature amounts corresponding to all still images calculated by the image feature amount calculating unit. An image processing apparatus according to any one of the above. 7. The reference feature value calculating unit calculates an average value obtained from a feature value representing an image feature of each still image calculated by the image feature value calculating unit as a reference feature value. The image processing apparatus according to 1. When the file acquired by the acquisition unit is a still image file, the image feature amount calculation unit calculates a feature amount representing an image feature of a still image in the still image file, and the file acquired by the acquisition unit is 8. The image processing apparatus according to claim 1, wherein in the case of a moving image file, a feature quantity representing an image feature of a frame image extracted from the moving image file is not calculated.

Images