JP2009246888A - Image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor that outputs an easy-to-view list of images. <P>SOLUTION: An image file stored in a media card is analyzed to determine whether an image formed based upon the image file meets an image extraction criterion stored in an image extraction criterion storage area 13n, and the image list 20 including index images comprising images determined to meet the image extraction criterion or reduced or enlarged images of the images is output. Consequently, the image list easy-to-view for a user is output. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus.

  In recent years, since a moving image and a still image can be shot using a digital camera, there are cases in which a moving image and a still image are shot separately according to a shooting scene. For example, a scene without motion such as a group photo can be shot as a still image, and a scene with motion such as a sports day scene can be shot as a moving image.

There has also been proposed an apparatus that prints out a moving image or a still image captured by such a digital camera or displays it on a display.
JP 2005-130254 A JP 9-322113 A

  However, in the above-described apparatus, the moving image and the still image are simply arranged, and the moving image and the still image cannot be arranged with good viewability.

  The present invention has been made to solve the above-described problems, and an object thereof is to provide an image processing apparatus capable of outputting a list of images having good browsing properties.

  The image processing apparatus according to claim 1, a selection condition storage means for storing a selection condition for selecting an image to be arranged in an image list, and image information stored in a storage medium are analyzed, and based on the image information An image determination unit that determines whether an image to be formed satisfies a selection condition stored in the selection condition storage unit; and the image determination among images formed based on image information stored in the storage medium And an output means for outputting an image list in which an image determined to satisfy the selection condition by the means or an index image composed of a reduced or enlarged image of the image is arranged.

  The image processing device according to claim 2 is the image processing device according to claim 1, wherein the image determination means stores moving image information for reproducing a moving image as the image information in the storage medium. Determining whether or not the selection condition is satisfied for each of a plurality of frame images extracted from the moving image corresponding to the moving image information, and the output means determines that the selection condition is satisfied by the image determining means. An image list in which the determined frame image or a reduced or enlarged image of the frame image is arranged as the index image is output.

  The image processing apparatus according to claim 3, wherein the image determination unit includes, as the image information, still image information for outputting a still image and the moving image information as the storage medium. Stored in the frame image, it is determined whether the selection condition is satisfied for the still image formed based on the still image information, and the output means In addition to a frame image determined to satisfy the selection condition or a reduced or enlarged image of the frame image, a still image determined to satisfy the selection condition or a reduced or enlarged image of the still image is used as the index image. Output as.

  According to a fourth aspect of the present invention, in the image processing apparatus according to the second or third aspect, the frame image information extracted from the moving image information is analyzed and included in a frame image corresponding to the frame image information. Person detection means for detecting a person is provided, and the image determination means determines whether the frame image satisfies the selection condition based on a detection result by the person detection means.

  The image processing device according to claim 5 is the image processing device according to claim 4, wherein when the selection condition storage means stores the person image including a person as the selection condition, the image determination means Determines a frame image in which a person is detected by the person detection means as an image satisfying the selection condition.

  The image processing device according to claim 6 is the image processing device according to claim 4, wherein when the selection condition storage means stores the person image including a person as the selection condition, the image determination means Determines, among the frame images from which the person is detected by the person detection means, an image included within a predetermined number from the top in the order of the number of detected persons as an image satisfying the selection condition.

  The image processing device according to claim 7, wherein in the image processing device according to claim 4, when it is stored in the selection condition storage unit as the selection condition that the person image includes a person, the image determination unit Determines, among the frame images from which the person is detected by the person detecting means, those images included within a predetermined number from the top in the order of the detected area of the person as images satisfying the selection condition.

  Here, the area of the detected person means the area of the range detected by the person detecting means, and the person detecting means detects the person by detecting a face, for example. The area of the face corresponds to the area of the person.

  The image processing device according to claim 8 is the image processing device according to claim 4, wherein when the selection condition storage means stores the scenery image including a landscape as the selection condition, the image determination means Determines a frame image in which no person is detected by the person detection means as an image satisfying the selection condition.

  The image processing device according to claim 9 is the image processing device according to claim 3, wherein the frame image information and the still image information extracted from the moving image information are analyzed, and the colors included in the frame image and the still image are analyzed. Color detecting means for detecting is provided, and the image determining means determines whether the frame image and the still image satisfy the selection condition based on a detection result by the color detecting means.

  An image processing apparatus according to a tenth aspect is the image processing apparatus according to any one of the second to ninth aspects, wherein the image determination unit is configured to generate a predetermined number of frames in a reproduction order of a series of frame images constituting a moving image. It is determined whether or not the selection condition is satisfied for the frame image.

  The image processing apparatus according to claim 11 is the image processing apparatus according to claim 3, wherein the image information stored in the storage medium is analyzed, and each image is determined based on a feature condition satisfied by an image corresponding to the image information. Classification means for classifying information into groups for each feature condition; and group determination means for deciding a group in which the number of image information classified by the classification means is maximum among the groups for each feature condition, The selection condition storage means stores the group characteristic condition determined by the group determination means as the selection condition.

  The image processing device according to claim 12 is the image processing device according to claim 11, wherein the classification means classifies moving image information stored in a storage medium into groups for each feature condition. And still image information classification means for classifying still image information stored in the storage medium into groups for each feature condition, wherein the group determination means includes the moving image information classification among the plurality of groups. A group having the maximum sum of the number of moving image information classified by the means and the number of still image information classified by the still image information classification means is determined.

  The image processing device according to claim 13 is the image processing device according to claim 12, wherein the moving image information classification unit includes a largest number of frame images among frame images constituting a moving image corresponding to the moving image information. The moving image information is classified into a group of feature conditions satisfying.

  15. The image processing device according to claim 14, wherein the classification means is a group based on a feature condition that the classification means is a person image including a person, and a landscape including a landscape. The image information is classified into one of groups based on the feature condition of being an image.

  Here, the moving image information classified into the person image group may include information indicating that all of the frame images constituting the moving image include a person, but is not limited thereto. For example, it may be configured such that moving image information in which a person is included in a predetermined number of frames or more in the frame images constituting the moving image is classified into a person image group. In addition, when there are more frame images satisfying the feature condition of being a person image among all frame images than frame images satisfying other feature conditions, the moving image information is classified into a person image group. May be. In other words, the classification criterion may be set so that the moving image information mainly corresponds to a moving image including a person and is classified into a person image group.

  Similarly, the moving image information classified into the landscape image group may satisfy the characteristic condition that all the frame images constituting the moving image are landscape images, but is not limited thereto. For example, it may be configured to classify moving image information in which a landscape is included in a predetermined number of frames or more in a frame image constituting the moving image into a landscape image group. In addition, when there are more frame images satisfying the feature condition of being landscape images among all frame images than frame images satisfying other feature conditions, the moving image information is classified into a landscape image group. May be. In other words, the classification criterion may be set so that the moving image information mainly corresponds to a moving image including a landscape so that the moving image information is classified into a landscape image group.

  According to the image processing apparatus of the first aspect, the image processing apparatus includes an image determined to satisfy the selection condition among images formed based on the image information stored in the storage medium, or a reduced or enlarged image of the image. A list of images in which index images are arranged is output. As a result, it is possible to output a list of images that are easy to browse and that allow the user to browse images that satisfy the selection condition.

  According to the image processing device according to claim 2, in addition to the effect achieved by the image processing device according to claim 1, a list of images with good browsing properties capable of browsing frame images determined to satisfy the selection condition is provided. Can be output. Since a moving image is often composed of a large number of frame images, if index images are formed for all frame images and arranged in an image list, it is difficult to find a target image from them. Therefore, by arranging the index images of the frame images determined to satisfy the selection conditions in the image list, the user can easily find the target frame image.

  According to the image processing device according to claim 3, in addition to the effect produced by the image processing device according to claim 2, a list of images with good browsing properties capable of browsing the frame image and the still image satisfying the selection condition. Can be output.

  According to the image processing device of claim 4, whether the frame image satisfies the selection condition based on the detection result by the person detection means in addition to the effect of the image processing device of claim 2 or 3. It can be determined whether or not.

  According to the image processing apparatus of the fifth aspect, in addition to the effect produced by the image processing apparatus of the fourth aspect, it is possible to output an image list in which index images of frame images in which a person is detected are arranged. Therefore, for example, it is possible to output a useful image list when it is desired to selectively browse frame images including a person.

  According to the image processing apparatus of the sixth aspect, in addition to the effect produced by the image processing apparatus of the fourth aspect, the index images of the images included within a predetermined number from the top in the order in which the number of detected persons is large are arranged. It is possible to output a list of processed images. Therefore, for example, it is possible to output a useful image list when it is desired to selectively browse frame images including a large number of people.

  According to the image processing apparatus of the seventh aspect, in addition to the effect produced by the image processing apparatus of the fourth aspect, the index images of the images included within a predetermined number from the top are arranged in the order in which the area of the detected person is large. It is possible to output a list of processed images. Therefore, for example, it is possible to provide a list of images that are useful when it is desired to selectively browse frame images that are close-up of a person.

  According to the image processing apparatus of the eighth aspect, in addition to the effect produced by the image processing apparatus of the fourth aspect, it is possible to output an image list in which index images of frame images in which a person is not detected by the person detection unit are arranged. it can. Therefore, for example, it is possible to provide a list of images that are useful when it is desired to selectively view a frame image of a landscape.

  According to the image processing device according to claim 9, in addition to the effect produced by the image processing device according to claim 3, it is determined whether or not the frame image satisfies the selection condition based on the detection result by the color detection means. Can do.

  According to the image processing device of the tenth aspect, in addition to the effect produced by the image processing device according to any one of the second to ninth aspects, in a reproduction order of a series of frame images constituting the moving image, every predetermined number of frames. Since it is determined whether or not the frame image satisfies the selection condition, the frame image to be determined can be extracted without bias from the series of frame images constituting one moving image. Furthermore, it is possible to prevent a large amount of frame images from being extracted unnecessarily.

  According to the image processing apparatus of the eleventh aspect, in addition to the effect produced by the image processing apparatus of the third aspect, it is possible to output an image list according to user preferences.

  According to the image processing device of claim 12, in addition to the effect produced by the image processing device of claim 11, a group is determined by equalizing the weight of the number of moving image information and the number of still image information, Since the selection condition is stored, an appropriate selection condition can be selected.

  For example, consider a case where 10 still image information corresponding to 10 landscape images and 1 moving image information corresponding to a moving image including 20 frame images including a person are stored in a storage medium. According to the configuration of the invention of claim 12, when this image information is classified into a person image group and a landscape image group, it can be stored as a selection condition that it is a landscape image. As a result, there is a high possibility that an image list that suits the preference of the user who has taken a large number of landscape images can be output. On the other hand, if the selection conditions are set as 10 landscape images and 20 person frame images, the person image is set as the selection condition, and the still image index image is an image. Inconvenience that it is not output at all to the list can occur.

  According to the image processing device of the thirteenth aspect, in addition to the effect produced by the image processing device according to the twelfth aspect, the moving image information can be classified into an appropriate group, and as a result, an appropriate selection condition is stored. be able to.

  According to the image processing device of claim 14, in addition to the effect of the image processing device according to any of claims 11 to 13, for example, when image information corresponding to a landscape image occupies a large number, When an image list of images is output and, conversely, when image information corresponding to a person image occupies a large number, an image list of person images is output, so an image list that matches the user's preference may be output. it can.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. FIG. 1 is a perspective view showing an external configuration of a multifunction peripheral device (hereinafter referred to as “MFP (Multi Function Peripheral)”) 1 which is an embodiment of an image processing apparatus of the present invention.

  The MFP 1 is a multifunction peripheral device having various functions such as a photo capture function, a copy function, a scanner function, and a media capture function. In the media capture function, a process for reading out an image file from a media card (an example of a storage medium in claims) loaded in a slot unit 6 to be described later and printing it out, an image corresponding to the image file read from the media card is displayed. A process of displaying and outputting on the LCD 5 and a process of storing image data read by the scanner function in a media card can be executed.

  In particular, the MFP 1 of the present embodiment is configured to output a list of images with good browsing properties so that the user can grasp the contents of the image file in the media card mounted in the slot unit 6. Will be described later.

  A scanner 2 for reading a document is arranged on the top of the MFP 1. In addition, a printer 3 composed of a so-called inkjet printer is built in the housing as a device for printing an image on recording paper.

  A slot 6 is provided on the front surface of the MFP 1. The slot portion 6 is provided with a card slot into which various media cards such as a compact flash (registered trademark) and an SD memory card can be directly inserted.

  An operation panel is provided in front of the document cover body, and includes an operation unit 4 including a plurality of keys and a touch panel, and an LCD 5. The MFP 1 displays the operation procedure and the status of the process being executed on the LCD 5 and also displays information corresponding to the operation of the operation unit 4.

  The electrical configuration of the MFP 1 will be described with reference to FIG. FIG. 2 is a block diagram showing an electrical configuration of the MFP 1. The MFP 1 mainly includes a CPU 11, a ROM 12, a RAM 13, an EEPROM 14, a scanner 2, a printer 3, an operation unit 4, an LCD 5, and a slot unit 6.

  Among these, the CPU 11, ROM 12, RAM 13, and EEPROM 14 are connected to each other via a bus line 26. The scanner 2, the printer 3, the operation unit 4, the LCD 5, the slot unit 6, and the bus line 26 are connected to each other via an input / output port 27.

  The CPU 11 controls each function of the MFP 1 and controls each unit connected to the input / output port 27 according to fixed values and programs stored in the ROM 12, RAM 13, and EEPROM 14.

  The ROM 12 is a non-volatile memory that stores the control program 12a executed by the CPU 11 and can retain the contents even after the power is turned off. The control program 12a includes a program for executing processing shown in flowcharts in FIGS.

  The ROM 12 further includes a horizontal image arrangement number storage area 12b and a one-page image arrangement number storage area 12c.

  The horizontal image arrangement number storage area 12b is an area for storing “the number of horizontal arrangements”. “Number of sheets arranged in the horizontal direction” is a value indicating the number of index images that can be arranged in a horizontal row in the image list. Note that the MFP 1 according to the present embodiment includes a plurality of types of “horizontal arrangement number” according to the print paper size (L, A4, Letter, etc.) and the print mode (high definition, normal, high speed, etc.) in advance. In the arrangement printing process (see FIGS. 14 and 15), which will be described later, an appropriate value is read out and used.

  The one-page image arrangement number storage area 12c is an area for storing “the number of image arrangements in one page”. “Number of images per page” is a value indicating the maximum number of index images that can be arranged in an image list formed on one page of recording paper. In this embodiment, a plurality of types of “number of images per page” are stored in advance according to the printing paper size and printing mode, and in the layout printing process (see FIGS. 14 and 15) described later. In the following description, a suitable value is read out and used.

  The RAM 13 is a memory for temporarily storing various data when each operation of the MFP 1 is executed. The extracted frame number storage area 13a, the processed file number storage area 13b, the processed image total number storage area 13c, and the generated page number storage. Area 13d, file information storage area 13h, arrangement information storage area 13i, frame image information storage area 13j, decode data storage area 13k, counter storage area 13m, image extraction reference storage area 13n, extracted frame information storage area 13p, number of investigation frames A storage area 13q and a survey frame image storage area 13r are provided.

  Here, the meaning of terms used in the present embodiment will be described. In the present embodiment, “moving image” means an image reproduced by switching and displaying a large number of frame images. The “moving image file (corresponding to the moving image information described in the claims)” is a file for reproducing a moving image, and includes, for example, an AVI moving image file, a quick time moving image file, and the like. However, any information file can be used as long as it can reproduce a moving image. In the present embodiment, “frame image information” means information corresponding to one frame image constituting a moving image.

  The extracted frame number storage area 13a is an area for storing “the number of frame images extracted”. The “number of frame image extractions” is a value indicating how many frame images corresponding to one moving image file are to be extracted. For example, when “5” is stored as the “number of extracted frame images”, the MFP 1 corresponds to five frame images per moving image file for each moving image file stored in the media card. Frame image information is extracted, and five index images per moving image file are arranged in the image list based on the extracted frame image information.

  The MFP 1 according to the present embodiment will be described assuming that an arbitrary value input by the user from the operation unit 4 is stored as the “frame image extraction number” in the extracted frame number storage area 13a. The “frame image extraction number” is automatically determined in accordance with a predetermined relationship so that the “frame image extraction number” increases as the size of the moving image file is stored in the ROM 12 or the like. May be.

  The processing file number storage area 13b is an area for storing the “total number of processing files”. The “total number of processed files” is a value indicating the total of still image files (an example of still image information described in claims) and moving image files stored in the media card.

  The processed image total number storage area 13c is an area for storing “total number of processed images”. The “total number of processed images” is a value indicating the total number of index images formed in the image list. The generated page number storage area 13d is an area for storing “number of generated pages”. The “number of generated pages” is a value indicating the total number of pages of the image list output by executing the layout printing process (FIGS. 14 and 15) described later.

  The file information storage area 13h and the arrangement information storage area 13i will be described later with reference to FIGS.

  The frame image information storage area 13j is an area for storing frame image information extracted from the moving image file. The decoded data storage area 13k is an area for storing a result of decoding the frame image information or a result of decoding the still image file. The counter storage area 13m is an area for storing various counters.

  In the image extraction reference storage area 13n, as the “image extraction reference”, either a “person center flag” indicating a person image including a person or a “landscape center flag” indicating a landscape image including a landscape is used. Remember. The MFP 1 arranges and outputs a frame image or a still image index image determined to satisfy the “image extraction criterion” in the image list 20 (see FIG. 3), and details of the processing will be described later. The “image extraction criterion” corresponds to an example of a selection condition for selecting an image described in the claims.

  The extracted frame information storage area 13p will be described later with reference to FIG. Information stored in the survey frame number storage area 13q and the survey frame image storage area 13r will be described later with reference to FIG. The EEPROM 14 is a rewritable nonvolatile memory.

  FIG. 3 is a diagram showing a set P of images corresponding to image files stored in the media card and an example of the image list 20 output by the MFP 1 based on the image extraction criteria. As shown in FIG. 3, the image set P includes a still image P1 corresponding to the still image file and a frame image P2 extracted from the moving image corresponding to the moving image file. explain.

  The MFP 1 analyzes the image file stored in the media card and determines whether the images P1 and P2 formed based on each image file satisfy the “image extraction criterion”. Then, among the images P1 and P2 formed based on the image file, an index image 21 obtained by reducing or enlarging the still image P1 or the frame image P2 determined to satisfy the “image extraction criterion” to a specified size is displayed in the image list. 20 and output.

  More specifically, when the “person center flag” is stored in the image extraction reference storage area 13n (see FIG. 2), the MFP 1 sets the images P1 and P2 including the person as images satisfying the image extraction reference. Judgment is made, and an image list 20 in which the index images 21 are arranged is output. Therefore, it is possible to provide a useful image list 20 when the user wants to selectively browse images including a person.

  Conversely, when the “landscape center flag” is stored in the image extraction reference storage area 13n, the MF1 outputs an image list 20 in which the index images 21 of the images P1 and P2 in which no person is detected are arranged. Thus, for example, a useful image list 20 can be provided when the user wants to selectively view landscape images.

  Since a moving image is composed of a large number of frame images, it is difficult to view if all the frame images are arranged in the image list 21. As described above, an index image of a frame image determined to satisfy the “image extraction criterion”. By arranging only 21 in the image list 20, the user can easily find the target image.

  In addition, the frame list and still images satisfying the “image extraction criterion” can be browsed together by the user using the image list 20, so that the browsing property is good for the user.

  Although detailed description will be given later, the MFP 1 determines the arrangement order sorted based on only the shooting date and time regardless of whether the index image is based on a still image file or an index image based on a moving image file. An image list 20 in which index images are arranged in the arrangement order is output.

  The image list 20 shown in FIG. 3 is an example. For example, as the image list 20 moves from the left side to the right side, the index images 21 for one horizontal row are arranged in the order in which the shooting date and time are updated, and the horizontal The arrangement order is determined for each of the index images 21 so that the next row of index images 21 with the latest shooting date and time are arranged below the row of rows.

  Therefore, for example, even if the user does not remember whether the movie was shot or shot as a still image, the shooting date / time such as whether the shooting date / time is quite old or recent This makes it possible to quickly find the target image.

  FIG. 4 is a diagram schematically showing the configuration of the file information storage area 13h. As shown in FIG. 4, the file information storage area 13h includes a file ID storage area 201 for storing the file ID, a file name storage area 202 for storing the file name, a file type storage area 203 for storing the file type, and an extraction frame. Extracted frame number storage area 204 for storing the number of frames, total frame number storage area 205 for storing the total number of frames, fps information storage area 206 for storing fps information, file date information storage area 207 for storing file date information, file time A file time information storage area 208 for storing information and a file additional information storage area 209 are provided, and file information 200 including a file name, a file type, the number of extracted frames, fps information, file date information, file time information, and file additional information. The file information 200 Storing information associated with the file ID to. The MFP 1 creates file information 200 for each of the still image file or the moving image file stored in the media card as a preparation process for determining the arrangement order of the index images 21 in the image list 20 (see FIG. 3). And stored in the file information storage area 13h.

  The file ID storage area 201 is an area for storing a file ID. The file name storage area 202 is an area for storing a file name of a still image file or a moving image file. As shown in FIG. 4, an extension is added to each file name. For example, if the image file is a JPEG file, the extension “jpg” is added, if it is an AVI movie file, the extension “avi” is added, and if it is a quick time movie file, the extension “mov” is added. Added.

  The file type storage area 203 is an area for storing a value indicating the type (format) of each image file. The type of the image file can be determined based on, for example, an extension added to the file name. In this embodiment, when the format of the image file is a JPEG file (extension jpg), a value “0” indicating that it is a JPEG file is stored in the file type storage area 203, and the format of the image file is an AVI video file. If it is (extension avi), the value “1” is stored in the file type storage area 203, and if the image file format is a quick time video file (extension mov), the value “2” is stored in the file type storage area. 203.

  Note that the type of the image file can also be determined based on information stored in several bytes from the top of each image file. For example, if the first two bytes are “0xFFD8”, it can be determined that the file is a JPEG file. If the first two bytes are “0x424D”, it can be determined that the data is bitmap data. The type of image file may be determined based on such information.

  The extracted frame number storage area 204 is an area for storing a value indicating how many frame image information corresponding to the number of frame images is to be extracted from the image file. In the case of a still image file, “1” is stored in the extracted frame number storage area 204.

  The total frame number storage area 205 is an area for storing the total number of frames of each image file. In the case of a moving image file, the total number of frames included in the moving image file is stored, and in the case of a still image, 1 is always stored.

  The fps information storage area 206 is an area for storing fps (Frame Per Second) information of each image file. If it is a still image file, “0” is stored. On the other hand, in the case of a moving image file, for example, “60” indicating that there are 60 frame images per second, and “30” indicating that there are 30 frame images per second include the fps information. Stored in the storage area 206.

  The file date information storage area 207 is an area for storing file date information indicating the shooting date of each image file. For example, the shooting date “January 15, 2008” is represented as file date information “20080115”.

  The file time information storage area 208 is an area for storing file time information indicating the shooting time of each image file. For example, the shooting time “12:50:45” is represented as file time information “125045”. When the image file is a moving image file, information indicating the shooting start time is stored as file time information.

  In the following description, the total number of frames, fps information, file date information, and file time information are acquired by reading the information described in the header of each image file. You may acquire the information.

  Information stored in the file additional information storage area 209 will be described later with reference to FIGS.

  FIG. 5 is a diagram schematically showing the configuration of the arrangement information storage area 13i. As shown in FIG. 5, the arrangement information storage area 13 i includes an arrangement information 300 including a file ID, an extracted frame position, an extracted frame number, image date information, and image time information, and an arrangement order corresponding to the arrangement information 300. Remember.

  As shown in FIG. 5, the arrangement information storage area 13i includes an arrangement order storage area 301, a file ID storage area 302, an extracted frame position storage area 303, an extracted frame number storage area 304, an image date information storage area 305, an image time. An information storage area 306 is provided.

  The arrangement information storage area 301 is an area for storing the arrangement order. In the layout printing process (see FIGS. 14 and 15) described later, the layout information 300 is read in the order according to the layout order, and the frame image or the still image index image 21 specified by each layout information 300 is The images are arranged in the image list 20 in the order according to the arrangement order.

  The file ID storage area 302 is an area for storing a file ID. As described with reference to FIG. 4, since the file ID and the file name are stored in a one-to-one correspondence in the file information storage area 13h, one image file is specified from the file ID. be able to.

  The extracted frame position storage area 303 is an area for storing an “extracted frame position” indicating what number of frame image information is counted from the beginning in one moving image file. The information stored here will be described later with reference to FIG.

  The extracted frame number storage area 304 stores an “extracted frame number” indicating the number of extracted frames corresponding to one moving image from the top.

  The image date information storage area 305 is an area in which image date information indicating the shooting date of the image file specified by the file ID stored in the file ID storage area 302 is stored. The image time information storage area 306 is an area in which image time information representing the shooting time is stored. For still image files, the file time information stored in the corresponding area of the file information storage area 13h is copied to the image time information storage area 306.

  On the other hand, for the moving image file, image time information representing the shooting time is acquired for each frame image information and stored in the image time information storage area 306. A specific calculation example for calculating the image time information for each frame image information will be described later with reference to the flowchart shown in FIG.

  In an arrangement printing process (see FIGS. 14 and 15), which will be described later, based on the arrangement information 300 stored in the arrangement information storage area 13i, the image list 20 in which the index images 21 are arranged in the order according to the shooting date and time (see FIG. 3) is printed out.

  FIG. 6 is a diagram schematically showing the extracted frame information storage area 13p. As shown in FIG. 6, the extracted frame information storage area 13p is provided with an extracted frame number storage area 401, an extracted frame position storage area 402, and a file ID storage area 403. The extracted frame position storage area 402 stores an extracted frame position for specifying a frame image that satisfies the image extraction criterion. The extracted frame number storage area 401 stores a management number for managing the extracted frame position stored in the extracted frame position storage area 402. A file ID corresponding to the value in the file ID storage area 201 is stored in the file ID storage area 403 and is associated with the file ID in the file ID storage area 201.

  FIG. 7 is a flowchart showing a layout image printing process executed by the MFP 1. This layout image printing process is executed when the user inputs a print output start instruction for the image list 20.

  First, an image information acquisition process for setting the file information 200 in the file information storage area 13h (see FIG. 4) is executed (S10). Details of the image information acquisition process (S10) will be described later with reference to FIG.

  Next, an extracted image determination process for determining an image satisfying the image extraction criterion is executed (S11). Details of the extracted image determination process (S11) will be described later with reference to FIG.

  Next, an arrangement information determination process for setting the arrangement information 300 in the arrangement information storage area 13i (see FIG. 5) is executed (S12). Details of the arrangement information determination processing will be described later with reference to FIG.

  Then, layout printing processing for printing out the image list 20 based on the set layout information 300 is executed (S13). Details of the layout printing process (S13) will be described later with reference to FIGS. By executing the layout image printing process, the image list 20 described with reference to FIG. 3 can be output.

  FIG. 8 is a flowchart showing the image information acquisition process (S10). The image information acquisition process (S10) is a process for creating the file information 200 (see FIG. 4).

  First, the total number of processed files stored in the media card is calculated and stored in the processed file number storage area 13b (S299). Then, a file number counter for counting the number of files is initialized with 0 (S301). This file number counter is provided in the counter storage area 13m (see FIG. 2).

  Next, among the image files stored on the media card, one image file is selected as a processing target, and the file name, file date information, and file time information of the image file are acquired (S302). Then, the acquired file name, file date information, and file time information are stored in an area corresponding to the file ID of the file number counter value (S303). The file name is stored in the file name storage area 202, the file date information is stored in the file date information storage area 207, and the file time information is stored in the file time information storage area 208 (see FIG. 5).

  Next, the type of the selected image file is analyzed (S304). In this process, for example, the type of the image file is determined by analyzing the extension added to the file name. The determined image file type is stored in the file type storage area 203 (see FIG. 5) corresponding to the file ID of the file number counter value (S305). Next, 1 is added to the processed image number counter (S310), and it is determined whether or not the value of the file number counter exceeds the total number of processed files (S311).

  If the determination in S311 is negative (S311: No), the process returns to S302 and is repeated. If the determination is affirmative (S311: Yes), the image information acquisition process (S10) is terminated. And it transfers to the process shown in FIG.

  FIG. 9 is a flowchart showing the extracted image determination process (S11). The extracted image determination process (S11) is a process for determining an image to be arranged in the image list 20.

  First, the value input by the user from the operation unit 4 is stored as “frame image extraction number” in the extracted frame number storage area 13a (S351). Next, each of the moving image file and the still image file stored in the media card is analyzed, and the moving image file and the still image file are classified into one of a person image including a person and a landscape image including a landscape. Image analysis processing (S352) is executed. Details of the image analysis processing (S352) will be described later with reference to FIG.

  Next, as a result of the image analysis process (S352), it is determined whether there are more still image files and moving image files classified as person images or still images and moving image files classified as landscape images (S353). ). When it is determined that there are many person images (S353: many person images), the person center flag is stored in the image extraction reference area 13n (see FIG. 2) (S354). On the other hand, if it is determined that there are many landscape images (S353: many landscape images), the landscape center flag is stored in the image extraction reference area 13n (S355). In other words, the image file is classified into either a person image or a landscape image by the image analysis process (S352). Then, of the person image and the landscape image, one of the classified still image file and the moving image file having a larger total number is determined, and the determined one image is used as an image extraction criterion. Remember.

  Next, the file number counter and the processed image number counter are initialized with 0 (S356). Then, the type of the image file specified by the file ID of the file number counter value is determined (S357). When it is determined that the file is a moving image file (S357: moving image), a frame extraction position determination process for determining the position of a frame image that satisfies the image extraction criterion is executed (S358). The frame extraction position determination process (S358) will be described later with reference to FIG.

  Next, the frame number counter is initialized to 0 (S359), the frame position information is read from the frame position information storage area 402 corresponding to the frame number counter value, and the extracted frame position storage area 303 corresponding to the processed image number counter value is read. (See FIG. 5) (S360).

  Then, 1 is added to the frame number counter and the processed image number counter (S361), it is determined whether or not the frame number counter value is equal to or greater than the number of extracted frame images (S362), and the determination of S362 is negative. (S362: No), the process is repeated from S360.

  If the determination in S362 is affirmed while the process is repeated (S362: Yes), the process proceeds to S366.

  On the other hand, if it is determined in S357 that the file is a still image file (S357: still image), then the value stored in the file additional information storage area 209 (see FIG. 4) and the image extraction reference storage area 13n. It is determined whether or not the value stored in (see FIG. 2) is equal (S363). As will be described later, the file additional information storage area 209 stores either a “person center flag” indicating that the still image is a person image or a “landscape center flag” indicating that the still image is a landscape image. Has been. Therefore, whether or not the still image satisfies the image extraction criterion can be determined based on the determination in S363.

  If the determination in S363 is affirmative (S363: Yes), the “print target flag” is stored in the file additional information storage area 209 corresponding to the file number counter value (S364). On the other hand, if the determination in S363 is negative (S363: No), a “non-print target flag” is stored in the file additional information storage area 209 (S365). After the process of S364 or S365, the process proceeds to S366.

  Next, 1 is added to the file number counter (S366), and it is determined whether or not the file number counter value is equal to or greater than the total number of processed files (S367), while the determination of S367 is denied (S367: No), the process is repeated from S357. If the determination in S367 is affirmed while the process is repeated (S367: Yes), the extracted image determination process (S11) is terminated, and the process proceeds to the arrangement information determination process (S12), but the arrangement information determination process (S12). ) Will be described later with reference to FIG.

  According to the extracted image determination process (S11), the larger one of the landscape image and the person image is selected, and the image of that type is stored as the image extraction criterion. Then, the MFP 1 outputs a list of images 20 in which index images 21 of images satisfying the image extraction criteria are arranged by executing processing described later.

  Therefore, for example, when an image file capturing a landscape occupies a large number in the media card, an image list 20 of landscape images is output, and conversely, an image file capturing a person occupies a large number in the media card. Since the image list 20 of human images is output, the image list 20 that matches the user's preference can be output.

  FIG. 10 is a flowchart showing the image analysis process (S352). The image analysis process (S352) is a process for determining whether each image file is a person image or a landscape image.

  First, the value input by the user from the operation unit 4 is stored as the “number of survey frames” in the survey frame number storage area 13q (see FIG. 2) (S401). Then, the file number counter is initialized with 0 (S402).

  Next, the type of the image file specified by the file ID of the file number counter value is determined (S403). When it is determined that the file is a moving image file (S403: moving image), the frame extraction interval is calculated by dividing the total number of frames of the moving image file by the number of extracted frame images (S404).

  Next, the investigation frame number counter and the face counter are initialized with 0 (S405). These counters are provided in the counter storage area 13m.

  Then, a value obtained by multiplying the frame extraction interval by the survey frame counter value is calculated, and the frame image information of the frame image corresponding to the value number is extracted from the head (S406), and the survey frame image storage area 13r (FIG. 2) is extracted. Reference) is stored (S407).

  Next, the frame image information stored in the survey frame image storage area 13r is analyzed, and a face recognition process for detecting a face included in the frame image corresponding to the frame image information is executed (S408). As a face recognition processing method, a known face recognition process using a Haar-like feature quantity and an AdaBoost classifier can be adopted, but other known face recognition processes other than this may be adopted. In the present embodiment, a person included in an image is detected based on detecting a face included in the image.

  Next, it is determined whether or not a face has been detected (S409). If the result is affirmative (S409: Yes), 1 is added to the face counter (S410). On the other hand, when negative (S409: No), the process of S410 is skipped.

  Next, 1 is added to the survey frame number counter (S411), and it is determined whether the survey frame number counter value is equal to or greater than the number of survey frames (S412). If the determination in S412 is negative (S412: No), the process returns to S406 and is repeated.

  On the other hand, if the determination in S412 is affirmative (S412: Yes), it is determined whether the value of the face counter is equal to or greater than the “frame image extraction number” stored in the extracted frame number storage area 13a (S413). If the determination in S413 is affirmative (S413: Yes), the “person center flag” is stored in the file additional information storage area 209 corresponding to the file number counter value (S414). On the other hand, if the determination in S413 is negative (S413: No), the “landscape center flag” is stored in the file additional information storage area 209 (S415).

  That is, based on whether or not a face is detected in a predetermined number or more of frame images, the frame image satisfies a characteristic condition that it is a person image including a person, or is a landscape image that includes a landscape Determine whether the condition is met. Then, based on the determination result, the person center flag or the scenery center flag is stored in the file additional information storage area 209, so that the moving image file is a group based on the feature condition of being a person image or a landscape image. It is classified into one of the groups based on the characteristic condition of being.

  In order to avoid redundant description, in this embodiment, the person image group is simply described as “person image”, and the landscape image group is simply described as “landscape image”. FIG. 4 illustrates a state where “1” as the “person center flag” or “0” as the “landscape center flag” is stored in the file additional information storage area 209.

  Then, 1 is added to the file number counter (S420), and it is determined whether the file number counter value is equal to or greater than the total number of processed files (S421). If the determination in S421 is negative (S421: No), the process returns to S403 and is repeated.

  On the other hand, if it is determined in S403 that the file is a still image file (S403: still image), the still image file specified by the file ID of the file number counter value is acquired and analyzed, and the still image file is supported. A face included in the still image is detected (S416). Since the details of the processing have been described in the processing of S408, description thereof is omitted here.

  Next, it is determined whether or not a face has been detected (S417). If the determination is affirmative (S417: Yes), the “person center flag” is stored in the file additional information storage area 209 corresponding to the file number counter value (S418). On the other hand, if negative (S417: No), the “landscape center flag” is stored in the file additional information storage area 209 corresponding to the file number counter value (S419). In this way, based on whether the still image corresponding to the still image file satisfies the feature condition that it is a person image or the feature condition that it is a landscape image, the person center flag or the landscape center flag is By storing the file in the file additional information storage area 209, the still image file is classified into either a person image or a landscape image. Then, the process proceeds to S420.

  If the determination in S421 is affirmed while the processing is repeated in this manner, the image analysis processing is terminated, and the processing returns to S353 described with reference to FIG.

  FIG. 11 is a flowchart showing the frame extraction position determination process (S358). This frame extraction position determination process (S358) is a process for determining the frame position for specifying the frame image information that meets the image extraction reference.

  First, the frame extraction interval is calculated by dividing the total number of frames of the moving image file to be processed by the number of extracted frame images (S501). Then, the investigation frame number counter and the extracted frame number counter are initialized with 0 (S502).

  Next, a value obtained by multiplying the frame extraction interval by the survey frame counter value is calculated, the frame image information of the frame image corresponding to the value number is extracted from the head (S503), and the survey frame image storage area 13r (FIG. 2) (S504).

  Next, the frame image information stored in the survey frame image storage area 13r is analyzed, and a face recognition process similar to the process of S408 is executed to detect a face included in the frame image corresponding to the frame image information (S505). ).

  Next, it is determined whether or not the frame image satisfies the image extraction criterion (S506). Specifically, a frame image for which it is determined that a face has been detected is an image that satisfies the image extraction criterion when the “person center flag” is stored in the image extraction criterion storage area 13n (see FIG. 2). On the other hand, if the “landscape center flag” is stored in the image extraction criterion storage area 13n, it is determined that the image does not satisfy the image extraction criterion.

  Further, an image for which it is determined that a face is not detected is determined to be an image that does not satisfy the image extraction criterion when the “person center flag” is stored in the image extraction criterion storage area 13n. When the “landscape center flag” is stored in the extraction criterion storage area 13n, it is determined that the image satisfies the image extraction criterion.

  When the determination in S506 is affirmative (S506: Yes), a value obtained by multiplying the frame extraction interval by the survey frame counter value is calculated as a frame position, and an extracted frame position storage area 402 (FIG. 5) corresponding to the extracted frame counter value is calculated. 6) (S507). Then, 1 is added to the extracted frame counter (S508). If the determination in S506 is negative (S506: No), the processes in S507 and S508 are skipped.

  Next, 1 is added to the survey frame number counter (S509), and it is determined whether the survey frame number counter value is equal to or greater than the number of survey frames (S510). While the determination in S510 is negative (S510: No), the processing is repeated from S503.

  By repeating the processing in this manner, it is possible to determine whether or not the image extraction criterion is satisfied for the frame images for every predetermined number of frames from the top in the reproduction order of the series of frame images constituting the moving image. Therefore, it is possible to extract and judge the determination target frame image from the series of frame images constituting one moving image without deviation.

  If the determination in S510 is affirmed while the process is repeated in this manner (S510: Yes), the frame extraction position determination process (S358) is terminated.

  FIG. 12 is a flowchart showing the arrangement information determination process (S12). First, the values of a file number counter for counting the number of processed files, a processed image number counter for counting the number of processed images, and a frame image extraction number counter for counting the number of extracted frame images are each initialized to zero. (S1001). These counters are provided in the counter storage area 13m (see FIG. 2).

  Next, the type of the image file corresponding to the file ID of the file number counter value is determined (S1002). Initially, since the value of the file number counter is “0”, the type of the image file with the file ID “0” is determined.

  When it is determined that the type of the image file is a moving image (S1002: moving image), file date information indicating the shooting date of the moving image file, file time information indicating the shooting start time of the moving image file, and fps of the moving image file. And the total frame number information indicating the total number of frames of the moving image file are read out from the header of the moving image file (S1003).

  Then, the file date information is stored in the file date information storage area 207 corresponding to the arrangement order of the processed image counter value, the file time information is stored in the file time information storage area 208, and the fps information is stored in the fps information storage area 206. The total frame number information is stored in the total frame number storage area 205 (S1004).

  Next, a process of creating arrangement information for the processing target image file and storing it in the arrangement information storage area 13i (see FIG. 6) is performed. Specifically, first, the current value of the file number counter is stored as a file ID in the file ID storage area 302 (see FIG. 5) corresponding to the arrangement order of the processed image number counter value. Further, the extracted frame number is calculated from the value of the frame image extraction number counter at the present time, and stored in the extracted frame number storage area 304 (see FIG. 6) (S1004a).

  Next, the image time information of the frame image information corresponding to the extracted frame number is calculated and stored in the image time information storage area 306 corresponding to the arrangement order (S1005). In the present embodiment, as an example, image time information is calculated as follows.

  Here, the video time information indicating the shooting start time of the moving image file is VideoTime, the fps information is FPS, the image time information indicating the shooting time of the frame image information is FrameTime, and the extraction frame position (extraction frame of the frame image information to be processed) The value stored in the position storage area 303 is N.

Then, the file time information VideoTime is decomposed into moving picture time information VT_Hour, moving picture minute information VT_Minute, and moving picture second information VT_Second as shown in Equation 6 below.
(Formula 6)
VT_Hour = [VideoTime / 10000]
VT_Minute = [(VideoTime-VT_Hour × 10000) / 100]
VT_Second = VideoTime-(VT_Hour × 10000 + VT_Minute × 100)
Next, frame time information Frame_Hour, frame portion information Frame_Minute, and frame second information Frame_Second are calculated as shown in Equation 7 below.
(Formula 7)
Here, Cal_Second, Cal_Minute, and Cal_Hour are temporary variables for obtaining the above information.
A mod B represents the remainder when A is divided by B
Cal_Second = VT_Second + [N / FPS]
Cal_Minute = VT_Minute + [Cal_Second / 60]
Cal_Hour = VT_Hour + [Cal_Minute / 60]
Frame_second = Cal_Second mod 60
Frame_Minute = Cal_Minute mod 60
Frame_Hour = Cal_Hour mod 24
The above is combined using, for example, Equation 8 below to calculate image time information FrameTime representing the shooting time of the frame image information.
(Formula 8)
FrameTime = Frame_Hour × 10000 + Frame_Minute × 100 + Frame_Second
Next, an image date calculation process is executed (S1006). The image date calculation process is a process for calculating frame year information Frame_Year, frame month information Frame_Month, and frame date information Frame_Day.

  FIG. 13 is a flowchart showing the image date calculation process (S1006). Note that a variable M shown in the flowchart of FIG. 13 is a variable that means that M days are advanced from the current date.

  First, moving image year information VD_Year, moving image month information VD_Month, and moving image day information VD_Day are substituted into frame year information Frame_Year, frame month information Frame_Month, and frame date information Frame_Day, respectively (S1201). Next, [Cal_Hour / 24] is substituted into the variable M (S1202). Next, it is determined whether M is 0 (S1203). If the determination is affirmative (S1203: Yes), this process is terminated.

  On the other hand, if negative (S1203: No), the frame date information Frame_Day is judged (S1204). If it is determined that “1 to 27” (S1204: 1 to 27), “1” is added to Frame_Day (S1205).

  On the other hand, if “28 or 29” is determined in the determination in S1204 (S1204: 28 or 29), it is determined whether or not Frame_Month is “2” (S1206). When negative (S1206; No), it transfers to the process of S1205.

  On the other hand, if the determination is affirmative (S1206: Yes), it is next determined whether or not Frame_Day is “28” (S1208). If negative (S1208: No), “1” is substituted into Frame_Day, and “1” is added to Frame_Month (S1210).

  On the other hand, if the determination in S1208 is positive (S1208: Yes), it is determined whether or not the year indicated by Frame_Year is a leap year (S1209). If the determination is affirmative (S1209: Yes), the process proceeds to S1205. When negative (S1209: No), it progresses to the process of S1210.

  Next, if “30” is determined in the determination of S1204 (S1204: 30), it is next determined whether Frame_Month is 4, 6, 9, or 11 (S1211). If the determination is affirmative (S1211: Yes), the process proceeds to S1210.

  On the other hand, if negative (S1211: No), “1” is added to Frame_Day (S1216). Next, if “31” is determined in the determination in S1204 (S1204: 31), it is then determined whether Frame_Year is 12 (S1215). When negative (S1215: No), it progresses to the process of S1216.

  On the other hand, if affirmative (S1215: Yes), “1” is substituted into Frame_Day, “1” is substituted into Frame_Month, and “1” is added to Frame_Year (S1217).

  After the processing of S1205, S1210, S1216, and S1217, “1” is subtracted from the variable M (S1212), and the process returns to S1203. If the determination in S1203 is affirmed (S1203: Yes), this process ends.

Returning to FIG. The values calculated in the process shown in FIG. 13 are combined using, for example, the following Equation 11 to calculate image date information FrameDate representing the shooting date of the frame image information. The calculated image date information is stored in the image date information storage area 305 (S1007).
(Formula 11)
FrameDate = Frame_Year × 10000 + Frame_Month × 100 + Frame_Day
In this way, when the arrangement information is created for one frame image information extracted from the moving image file and stored in the arrangement information storage area 13i, a processed image number counter for counting the number of processed images, Then, “1” is added to the frame image extraction number counter (S1008).

  Then, it is determined whether or not the value of the frame image extraction number counter is equal to or greater than the frame image extraction number determined for the moving image file (S1009). If the determination in S1009 is negative (S1009: No), the process returns to S1004a and the process is repeated.

  On the other hand, if the determination in S1009 is affirmative (S1009: Yes), then the frame image extraction number counter is initialized with 0 (S1010), and 1 is added to the file number counter (S1016). Then, it is determined whether or not the value of the file number counter is equal to or greater than the “total number of processed files” indicating the total of still image files and moving image files stored in the media card (S1017). If the determination in S1017 is negative (S1017: No), the process returns to S1002, and the next image file corresponding to the file ID of the file number counter value is selected as a processing target, and the process is repeated.

  Next, a case where it is determined that the image file to be processed is a still image file (S1002: still image) will be described. Similarly, in the case of a still image file, first, file information relating to the still image file is created and stored in the file information storage area 13h (see FIG. 4).

  Specifically, first, the current file number counter value is stored in the file ID storage area 302 (see FIG. 6) corresponding to the arrangement order of the processed image number counter value, and the extracted frame number storage area 304 (FIG. 6). “1” is stored in (refer to) (S1011a). Then, the file date information indicating the shooting date of the still image file to be processed and the file time information indicating the shooting time are read (S1011), and the file information storage area 13h is moved to the area corresponding to the file ID of the file number counter value. The acquired file date information and file time information are stored (S1012). Note that “0” is stored in the fps information storage area 206.

  Next, the file date information of the still image file to be processed corresponding to the file ID of the file number counter value is set to the image date corresponding to the arrangement order of the processed image number counter value in the arrangement information storage area 13i (see FIG. 6). The information is stored in the information storage area 305, and the file time information of the still image file to be processed is stored in the image time information storage area 306 corresponding to the same arrangement order (S1013). Then, 1 is added to the processed image number counter (S1015), and the process proceeds to S1016.

  If the determination in S1017 is affirmed while the processing is repeated in this way (S1017: Yes), then the arrangement information is sorted in ascending order of the image time information (S1018), and the arrangement information determination process (S12) is performed. finish. Through the arrangement information determination process (S12), the corresponding arrangement order is determined for each of the index images 21 specified by the arrangement information.

  FIG. 14 is a flowchart showing the layout printing process. The layout printing process (S13) shown in FIG. 14 is a process of printing out the image list 20 in which the index images 21 are arranged in the order determined by the layout information determination process (S12; see FIG. 12) on a recording sheet. .

  First, “horizontal arrangement number” indicating the number of index images 21 for one horizontal row arranged in the image list 20 is read from the horizontal image arrangement number storage area 12b (S1101). Next, “the number of image arrangements in one page” indicating the number of index images 21 that can be arranged in the image list 20 for one page is read from the image arrangement number storage area 12c in one page (S1102).

  Next, the page counter is initialized with 1 (S1103), and the “total number of processed images”, which is the sum of the total number of still image files and the total number of extracted frame images, is stored in the total processed image storage area 13c (FIG. 2) (S1104). The page counter is provided in the counter storage area 13m.

  Next, a “generated page number” indicating how many pages of the image list are to be printed out is calculated from the total number of processed images and the number of images arranged per page, and the generated page number storage area 13d (see FIG. 2). (S1105).

In the present embodiment, the number of generated pages page_no is calculated using the following formula 12. Here, the total number of processed images is TotalImageNo, and the number of images arranged in one page is LAYOUT_NUM.
(Formula 12)
page_no = [TotalImageNo / LAYOUT_NUM] + 1
Next, the layout image counter is initialized to 0 (S1106), and the number of layout_no images to be placed on one recording sheet is calculated (S1107). In the present embodiment, layout_no is calculated using Equation 13 below. The layout image counter is provided in the counter storage area 13m.
(Formula 13)
(Page counter value) <Number of generated pages
layout_no = (Number of images on one page)
(Page counter value) ≥ Number of generated pages
layout_no = (number of group images)-{(number of images in one page) x (page counter value)}

  Next, the file ID corresponding to the arrangement order and the extracted frame number are read from the arrangement information storage area 13i (S1108). Then, it is determined whether the image file corresponding to the read file ID is a moving image file or a still image file (S1109).

  If it is determined in S1109 that the file is a moving image file (S1109: moving image), then frame image information for one frame image corresponding to the extracted frame number is extracted from the moving image file (S1110).

  Next, the extracted frame image information is decoded (S1111). Then, the decoded frame image information is reduced or enlarged to the specified size of the index image 21, and is arranged at a predetermined position in the arrangement buffer (S1112).

  On the other hand, if it is determined in S1109 that the file is a still image file (S1109: still image), the file additional information storage area 209 corresponding to the still image file is referred to and whether or not a non-print image flag is stored. Is determined (S1109a).

  If the determination in S1109a is negative (S1109a: No), the still image file specified by the file ID corresponding to the arrangement order is extracted and decoded (S1113). Then, the decoded still image file is reduced or enlarged to the specified size of the index image 21, and is arranged at a predetermined position in the arrangement buffer (S1114).

  On the other hand, if the determination in S1109a is affirmative (S1109a: Yes), the process returns to S1108, and the file ID and the extracted frame number corresponding to the next arrangement order are read (S1108). That is, the still image file associated with the non-print image flag is not extracted, and the index image 21 of this still image file is not formed.

  In this way, after executing the placement processing for decoding either the frame image information or the still image file and placing it in the placement buffer, the “horizontal direction” is the upper limit of the number of images that can be placed in the horizontal direction. It is determined whether or not the arrangement processing for “number of arrangements” has been performed (S1115). If the determination in S1115 is negative (S1115: No), the horizontal position indicating the arrangement position of the next data in the arrangement buffer is updated (S1119).

  Then, “1” is added to the layout image counter (S1120), and it is determined whether or not the value of the layout image counter is equal to or larger than the number “layout_no” of the index images 21 to be arranged on the current recording paper to be processed. (S1121). If the determination in S1121 is negative (S1121: No), the process returns to S1108, and arrangement information corresponding to the next arrangement order is processed.

  If the arrangement processing for “the number of arrangements in the horizontal direction” is executed while the processing is repeated in this way, the determination in S1115 is affirmed (S1115: Yes). Next, the data in the arrangement buffer is stored. The data is transferred to the printer 3 (see FIG. 2) (S1116). As a result, the data in the placement buffer is subjected to print processing in the printer 3. However, print output is not performed for an area where no image is arranged in the arrangement buffer.

  Next, the horizontal position indicating the horizontal arrangement position of the next data in the arrangement buffer is initialized (S1117), and the vertical position indicating the vertical arrangement position of the next data in the arrangement buffer is updated (S1117). S1118). The arrangement position of the next data in the arrangement buffer is determined by the horizontal position and the vertical position so that the next index image 21 is arranged from the left end of the next row in the image list 20. And the process of S1120 and S1121 mentioned above is performed.

  By repeating the process in this manner, the frame image information corresponding to the frame image determined to satisfy the image extraction criterion is extracted from the moving image file to form the index image 21, and when the image extraction criterion is satisfied. The determined still image file is extracted, and the index image 21 is formed.

  In this way, when the printing process is completed for all of the index images 21 arranged on the recording paper for one page to be processed, the determination in S1121 is affirmed (S1121: Yes), so the next process shown in FIG. Migrate to

  FIG. 15 is a flowchart showing the continuation of the layout printing process shown in FIG. 14, and is executed when the determination in S1121 is affirmed.

  First, it is determined whether there is data that has not yet undergone printing processing in the arrangement buffer (S1122). If the determination in S1122 is affirmative (S1122: Yes), the remaining data is transferred to the printer 3 (S1123), the printer 3 prints out, and the process proceeds to S1124. On the other hand, if the determination in S1122 is negative (S1122: No), the determination in S1123 is skipped and the process proceeds to S1124.

  Then, a print completion command indicating that data output for one page has been completed is passed to the printer 3 (S1124), and “1” is added to the page counter (S1125). Next, the horizontal position and the vertical position indicating the arrangement position of the next data in the arrangement buffer are initialized (S1126). Then, it is determined whether the value of the page counter is larger than “number of generated pages” (S1127). If the determination in S1127 is negative (S1127: No), the process returns to S1106 shown in FIG. 14, and the process for the next page is started.

  On the other hand, if the determination in S1127 is affirmative (S1127: Yes), the layout printing process ends. As a result, the image list 20 described with reference to FIG. 3 is output.

  The present invention has been described above based on the embodiments. However, the present invention is not limited to the above-described embodiments, and various modifications and changes can be easily made without departing from the spirit of the present invention. Can be inferred.

  For example, in the present embodiment, the image list 20 is described as being printed out on recording paper. However, the present invention can also be applied to the case where the image list 20 is displayed on a display device such as the LCD 5.

  The present invention is also applied to, for example, a case where the present invention is applied to a personal computer and the image list 20 is displayed on a PC monitor, or the image list is printed out by controlling the printer from the personal computer. Can be applied.

  In the above-described embodiment, the moving image file and the still image file are both assumed to be image files generated by shooting. However, the moving image file and the still image file are generated by a method other than shooting. Even if it is a thing, this invention is applicable. In this case, the arrangement order of the index images 21 is determined based on other conditions such as file generation date / time information indicating file creation date / time and file update date / time information instead of the shooting date / time. May be.

  In the above-described embodiment, the case where a still image file and a moving image file are mixed in the media card has been described. Is applicable.

  In the above-described embodiment, the image information stored in the removable media card is the processing target. However, the present invention is also applicable to the case where the image information stored in the internal memory of the MFP 1 is the processing target. Applicable.

  Furthermore, in the above embodiment, the arrangement order is sorted based on the shooting date and time. However, for example, the dynamic range, the average luminance, whether the image includes a face, the number and size of face images, and the like. The arrangement order may be sorted under the above conditions.

  In the above embodiment, the person included in the image is detected based on the detection of the face included in the image. However, instead of this, the person may be detected based on detecting the eyes included in the person's face or the outline of the human body.

  In the above embodiment, the moving image file is classified as a human image based on the detection of faces in a predetermined number or more of frame images. For example, all the frames constituting the moving image are examined, The moving image information may be classified into groups of feature conditions that the most frame images satisfy.

  In the above-described embodiment, the image extraction criterion is one of two types, that is, a person image and a landscape image, but any one of three or more types of image extraction criterion is selected. You may comprise so that it may be selected.

  In the above embodiment, whether the still image corresponding to the still image file is a person image or a landscape image is determined based on detecting a face included in the still image. Alternatively, it may be determined whether each still image is a person image or a landscape image based on a shooting mode included as header information of the still image file.

  Further, the present invention is not limited to arranging only index images of images satisfying the image extraction criteria in the image list 20. For example, for each of the still image file and the moving image file, one index image 21 is arranged, and in addition to that, frame image information satisfying the image extraction criterion is further extracted and arranged in the remaining space of the image list 20. It may be configured.

  In this way, the contents of each image file can be browsed by the user, and the image list 20 with an emphasis on either the person center or the landscape center can be output.

  In the above embodiment, it is determined whether the image is a person image or a landscape image based on whether or not a face is detected. For example, the frame image from which a face is detected is included in the image. The frame images included within a predetermined number from the top in the order of the number of faces to be displayed may be determined as images satisfying the image extraction criterion.

  In this way, for example, it is possible to provide the user with an image list 20 that is useful when it is desired to selectively browse a frame image including a large number of faces from a moving image.

  In addition, among the frame images in which the face is detected, frame images included within a predetermined number from the top in the order in which the area of the face included in the image is large may be determined as an image satisfying the image extraction criterion.

  In this way, for example, it is possible to provide an image list 20 that is useful when it is desired to selectively browse a frame image with a close-up face from a moving image.

  In the above-described embodiment, whether or not an image satisfies the “image extraction criterion” is determined based on whether or not a face is included in the image. It may be determined whether the image satisfies the “image extraction criterion” based on the detection result.

  For example, if the color detection means detects the color of the majority of pixels constituting the image as blue or green, the image may be determined to be a landscape image. Further, if a skin color of a predetermined area or more is detected as the pixel color, the image may be determined to be a person image.

1 is a perspective view showing an external configuration of a multifunction peripheral device according to an embodiment of an image processing apparatus of the present invention. 2 is a block diagram illustrating an electrical configuration of the MFP. FIG. 6 is a diagram illustrating an example of an image list output by an MFP. FIG. It is a figure which shows typically the structure of a file information storage area. It is a figure which shows typically the structure of an arrangement | positioning information storage area. It is a figure which shows an extraction frame information storage area typically. 6 is a flowchart illustrating a layout image printing process executed by the MFP. It is a flowchart which shows an image information acquisition process. It is a flowchart which shows an extraction image determination process. It is a flowchart which shows an image analysis process. It is a flowchart which shows a frame extraction position determination process. It is a flowchart which shows arrangement | positioning information determination processing. It is a flowchart which shows an image date calculation process. It is a flowchart which shows arrangement | positioning printing processing. FIG. 15 is a flowchart showing a continuation of the layout printing process shown in FIG. 14. FIG.

Explanation of symbols

1 MFP (an example of an image processing apparatus)
13n image extraction reference storage area (an example of selection condition storage means)
20 Image list 21 Index image S13 Arrangement printing process (an example of output means)
S352 Image analysis processing (an example of classification means)
S353 Example of group determination means S358 Frame extraction position determination processing (example of image determination means)
S363 An example of image determination means S414, S415 Moving image information classification means S418, S419 Still image information classification means S416 An example of person detection means S505 An example of person detection means

Claims (14)

Selection condition storage means for storing a selection condition for selecting an image to be arranged in the image list;
Image determination means for analyzing image information stored in a storage medium and determining whether an image formed based on the image information satisfies a selection condition stored in the selection condition storage means;
Among images formed on the basis of image information stored in the storage medium, an image determined by the image determination means to satisfy the selection condition or an index image composed of a reduced or enlarged image of the image is arranged. An image processing apparatus comprising: an output unit that outputs a list of images that have been processed. When the moving image information for reproducing a moving image is stored in the storage medium as the image information, the image determination unit is configured to detect each of a plurality of frame images extracted from the moving image corresponding to the moving image information. Determining whether the selection condition is satisfied,
The output means outputs an image list in which frame images determined by the image determination means to satisfy the selection condition or reduced or enlarged images of the frame images are arranged as the index images. Item 6. The image processing apparatus according to Item 1. When still image information for outputting a still image is stored as the image information in the storage medium together with the moving image information, the image determination unit is based on the still image information in addition to the frame image. For the still image to be formed, it is determined whether the selection condition is satisfied,
In addition to the frame image determined to satisfy the selection condition or the reduced or enlarged image of the frame image by the image determination unit, the output means determines the still image determined to satisfy the selection condition or the still image thereof The image processing apparatus according to claim 2, wherein a reduced or enlarged image is output as the index image. Analyzing the frame image information extracted from the moving image information, and comprising a person detecting means for detecting a person included in the frame image corresponding to the frame image information,
The image processing apparatus according to claim 2, wherein the image determination unit determines whether the frame image satisfies the selection condition based on a detection result by the person detection unit. When it is stored in the selection condition storage means as the selection condition that it is a person image including a person,
The image processing apparatus according to claim 4, wherein the image determination unit determines a frame image in which a person is detected by the person detection unit as an image satisfying the selection condition. When it is stored in the selection condition storage means as the selection condition that it is a person image including a person,
The image determining means determines, as an image satisfying the selection condition, an image that is included within a predetermined number from the top in the order in which the number of detected persons is large among the frame images in which a person is detected by the person detecting means. The image processing apparatus according to claim 4, wherein: When it is stored in the selection condition storage means as the selection condition that it is a person image including a person,
The image determining means determines, as an image satisfying the selection condition, an image that is included within a predetermined number from the top in a descending order of the detected person area among frame images in which a person is detected by the person detecting means. The image processing apparatus according to claim 4, wherein: When it is stored in the selection condition storage means as the selection condition that it is a landscape image including a landscape,
The image processing apparatus according to claim 4, wherein the image determination unit determines a frame image in which a person is not detected by the person detection unit as an image satisfying the selection condition. Analyzing the frame image information extracted from the moving image information and the still image information, and comprising color detection means for detecting a color included in the frame image and the still image,
The image processing apparatus according to claim 3, wherein the image determination unit determines whether the frame image and the still image satisfy the selection condition based on a detection result by the color detection unit.   The image judging means judges whether or not the selection condition is satisfied with respect to a frame image for every predetermined number of frames in a reproduction order of a series of frame images constituting a moving image. An image processing apparatus according to any one of the above. Analyzing the image information stored in the storage medium, and classifying means for classifying the image information into groups for each feature condition based on the feature conditions satisfied by the image corresponding to each image information;
A group determining unit that determines a group in which the number of pieces of image information classified by the classifying unit is maximized among the groups for each feature condition;
The image processing apparatus according to claim 3, wherein the selection condition storage unit stores the group characteristic condition determined by the group determination unit as the selection condition. The classification means includes
Moving image information classification means for classifying moving image information stored in a storage medium into groups for each of the feature conditions;
Still image information classification means for classifying still image information stored in the storage medium into groups for each feature condition,
The group determination means has a maximum sum of the number of moving image information classified by the moving image information classification means and the number of still image information classified by the still image information classification means among the plurality of groups. The image processing apparatus according to claim 11, wherein a group to be determined is determined. The moving image information classification means includes:
13. The image processing apparatus according to claim 12, wherein among the frame images constituting the moving image corresponding to the moving image information, the moving image information is classified into a group of feature conditions satisfied by the largest number of frame images. .   The classification means classifies each piece of image information into either a group based on a feature condition of being a person image including a person or a group based on a feature condition of being a landscape image including a landscape. The image processing apparatus according to claim 11.

Images