JP2017130109A - Program, information processor and control method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a program capable of ensuring satisfaction of a user.SOLUTION: The program is configured, with respect to a piece of image data, to acquire a first a scene which has a highest evaluation value from a plurality of scenes based on an evaluation relevant to the scene included in the plural scenes. The program is also configured to determine whether the image data should be classified in the first scene or in a scene other than the first scene in the plurality of scenes based on the acquired evaluation of the first scene.SELECTED DRAWING: Figure 16

Description

  The present invention relates to a program, an information processing apparatus, and a control method.

  A technique for classifying image data into semantic events (hereinafter referred to as scenes) such as travel, daily life, and ceremony is known. Patent Document 1 describes an apparatus that assigns a score for each scene to image data and classifies the image data into a scene having the highest score.

Special table 2011-525012 gazette

  However, in the evaluation of image data for each scene, accurate evaluation is not always performed so that the score of the scene intended by the user is the highest depending on the characteristics of the image data and the evaluation method. Therefore, in the method of classifying image data into a scene having the highest evaluation as in Patent Document 1, the image data is classified into a scene different from the user's intention, and the user's satisfaction may be lowered. There are challenges.

  Since the above-described problems occur, depending on the evaluation result, it may be possible to suppress the user satisfaction from being lowered by classifying the image data into a scene that is not the highest evaluated scene. An object of this invention is to provide the program, information processing apparatus, and control method which can suppress the fall of a user's satisfaction.

  In order to solve the above problems, a program according to the present invention provides an information processing apparatus that classifies image data into one of a plurality of scenes, and evaluates scenes included in the plurality of scenes with respect to the image data. Based on the evaluation by the evaluation means, an acquisition means for acquiring the first scene having the highest evaluation among the plurality of scenes, and based on the evaluation of the first scene acquired by the acquisition means , And functioning as determining means for determining whether to classify the image data into the first scene or to classify the image data into a scene that is not the first scene of the plurality of scenes.

  ADVANTAGE OF THE INVENTION According to this invention, when exact evaluation with respect to image data is not performed, it can suppress that a user's satisfaction falls.

It is a block diagram which shows the hardware constitutions of information processing apparatus. It is a block diagram which shows the software structure of an album creation application. It is a figure which shows the user interface screen of an album creation application. It is a flowchart which shows the procedure of an automatic layout process. It is a figure which shows the analysis information of image data. It is a figure for demonstrating the division | segmentation of an image data group. It is a figure for demonstrating the classification of a scene. It is a figure for demonstrating scoring about a main slot and a subslot. It is a figure for demonstrating selection of image data. It is a figure for demonstrating the layout of image data. It is a flowchart which shows the procedure of a frame extraction process. It is a flowchart which shows the procedure of an image data acquisition process. It is a figure which shows the layout compatibility between scenes. It is a figure for demonstrating scoring of image data. It is a figure for demonstrating the classification of a scene. It is a flowchart which shows the procedure of a scene classification | category process. It is a figure for demonstrating the classification of a scene. It is a flowchart which shows the procedure of a scene classification | category process.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the present invention according to the claims, and all combinations of features described in the embodiments are not necessarily essential to the solution means of the present invention. . The same constituent elements are denoted by the same reference numerals, and the description thereof is omitted.

[First Embodiment]
In the present embodiment, a procedure until an album creation application is operated on an information processing apparatus and an automatic layout is generated will be described. Unless otherwise specified, the images described below include still images, moving images, frame images in moving images, and still images, moving images, and frame images in moving images on SNS (Social Networking Service).

<Hardware configuration>
FIG. 1 is a block diagram illustrating a hardware configuration of the information processing apparatus 100 according to the present embodiment. Examples of the information processing apparatus include a PC, a smartphone, a tablet terminal, a camera, and a printer. In this embodiment, the information processing apparatus is a PC.

  In FIG. 1, a CPU (central processing unit / processor) 101 is a processing unit that executes a control method described in the present embodiment in accordance with a program. Although the number of CPUs in the drawing is one, a plurality of CPUs are also represented as a part of the CPU in this embodiment. The ROM 102 is a general-purpose ROM, and stores programs executed by the CPU 101, for example. The RAM 103 is a general-purpose RAM, and provides a memory for temporarily storing various types of information when the CPU 101 executes a program, for example. An HDD (hard disk) 104 is a storage medium for storing a still image file, a moving image file, a database that holds processing results such as image analysis, and the like. The ROM 102, RAM 103, and HDD 104 constitute a storage unit.

  The display 105 is a device that presents a user interface (UI) and image layout results of the present embodiment to the user, and constitutes a display unit. The display 105 may have a touch sensor function. The keyboard 106 receives input from the user such as the number of spreads of the album to be created on the UI displayed on the display 105. The pointing device 107 is a device for designating coordinates on the display 105, and is used, for example, to click a button on the UI displayed on the display 105. A keyboard 106 and a pointing device 107 are input units and constitute a user interface together with the display 105.

  The data communication unit 108 transmits data to be automatically laid out to a printer or server (not shown) connected to the PC. Further, the data communication unit 108 receives a still image or a moving image on a server (not shown) or SNS. The data bus 109 connects the above-described units and the CPU 101 so that they can communicate with each other. The album creating application in the present embodiment is stored in the HDD 104, and the user double-clicks the application icon displayed on the display 105 as will be described later, and is activated.

<Automatic layout processing section>
FIG. 2 is a software block diagram related to the automatic layout processing unit 216 of the album creation application. The album creating application has various functions. In the present embodiment, an automatic layout function provided by the automatic layout processing unit 216 will be described. The automatic layout function is a function for laying out photographed still images and moving images based on the contents and attributes of the still images and moving images in a template prepared in advance and displaying them on the display 105.

  In FIG. 2, the condition designation unit 201 accepts designation of album creation conditions in accordance with a UI operation described later by an input unit, for example, the pointing device 107, and outputs it to the automatic layout processing unit 216. The designated conditions include, for example, designation of image data to be processed, main character ID, number of album spreads, template information, ON / OFF conditions for image correction, ON / OFF conditions for using moving images, and the like. The designation of image data may be, for example, designation by accompanying information or attribute information of individual image data such as shooting date and time, or based on the structure of a file system including image data, such as designation of a device or directory (folder). It may be specified. Note that the spread is equivalent to one display window in display, and corresponds to a pair of adjacent pages printed on different sheets (or pages) in printing.

  The image acquisition unit 202 acquires the image data group designated by the condition designation unit 201 from a server or SNS on the network via the HDD 104 or the data communication unit 108. The image conversion unit 203 converts image data used for subsequent processing into a desired number of pixels and color information. In the present embodiment, the number of pixels on the short side is 420 pixels, and converted into analysis image data (hereinafter referred to as analysis image data) of sRGB color information. The image analysis unit 204 executes processing such as feature amount acquisition, face detection, facial expression recognition, and personal recognition described later from the analysis image data. Further, the image analysis unit 204 acquires the shooting date and time from data attached to the image data acquired from the server via the HDD 104 or the data communication unit 108, for example, Exif information. The shooting date / time is not limited to the acquisition from the Exif information, and the date / time when the image data is created or updated may be used. The date and time when image data is uploaded to the server or SNS or image data is downloaded from the server or SNS may be used. These dates are also treated as shooting date information below.

  The image classification unit 205 executes scene division and scene classification, which will be described later, on the image data group using the shooting date / time information, the number of images, and the detected face information. A scene is a shooting scene such as “travel”, “daily life”, or “wedding”. A scene can be said to be a collection of image data shot at a shooting opportunity for a period of time, for example. The image scoring unit 207 scores each image data so that image data suitable for the layout has a high score. As will be described later, when scoring, information from the image analysis unit 204 and information from the image classification unit 205 are used. Further, other information may be used additionally or alternatively.

  The main character information input unit 206 inputs the main character ID (identification information) input from the condition specifying unit 201 to the image scoring unit 207. The image score unit 207 is configured to add a higher score to image data including the main character ID input from the main character information input unit 206.

  The spread assignment unit 209 divides (groups) the image data group and assigns it to each spread. The spread number input unit 208 inputs the number of spreads of the album input from the condition specifying unit 201 to the spread allocation unit 209. The spread allocation unit 209 divides the image data group according to the input spread number and allocates a part (divided portion) of the image data group to each spread. The image selection unit 210 selects a part of the image data group assigned to each spread by the spread allocation unit 209 based on the score given by the image scoring unit 207.

  The image layout unit 212 determines the layout of image data. The template input unit 211 inputs a plurality of templates corresponding to the template information input from the condition specifying unit 201 to the image layout unit 212. The image layout unit 212 selects a template suitable for the image data selected by the image selection unit 210 from a plurality of templates input from the template input unit 211, and determines the layout of the image data. The layout information output unit 215 outputs layout information to be displayed on the display 105 in accordance with the image data layout determined by the image layout unit 212. The layout information is, for example, bitmap data obtained by laying out the image data selected by the image selection unit 210 on the selected template.

  The image correction unit 214 executes correction processes such as dodging correction (luminance correction), red-eye correction, and contrast correction. The correction condition input unit 213 inputs the image correction ON / OFF condition input from the condition specifying unit 201 to the image correction unit 214. Image correction ON / OFF may be specified for each type of correction processing, or all types may be specified collectively. The image correction unit 214 executes correction processing on the image data when the image correction condition is ON, and does not execute correction processing when the image correction condition is OFF. Note that the image correction unit 214 performs correction processing on the image data input from the image conversion unit 203 in accordance with ON / OFF of image correction. The number of pixels of the image data input from the image conversion unit 203 to the image correction unit 214 can be changed according to the layout size determined by the image layout unit 212. The image data output from the layout information output unit 215 is set on a setting screen displayed on the display 105 in a format as shown in FIG. 3, for example.

<User operation>
When the album creation application of the present embodiment is installed in the information processing apparatus 100, a start icon is displayed on a top screen (desktop) of an OS (operating system) that runs on the information processing apparatus 100. When the user double-clicks on the desktop startup icon displayed on the display 105 with the pointing device 107, the album creation application program stored in the HDD 104 is loaded into the RAM 103. Then, the program in the RAM 103 is executed by the CPU 101, and the album creation application is activated. Note that the program may be stored in the ROM 102.

<Display example>
FIG. 3 is a diagram illustrating an example of a UI configuration screen 301 provided by the activated album creation application. The UI configuration screen 301 is displayed on the display 105. When the user sets an album creation condition to be described later via the UI configuration screen 301, the condition designating unit 201 acquires the setting content from the user. A path box 302 on the UI configuration screen 301 indicates a storage location (path) in the HDD 104 of an image data group to be created as an album. A folder selection button 303 displays, in a tree configuration, a user including a folder including image data groups that are targets of album creation when the user clicks on the pointing device 107. Then, a folder path including the image data group selected by the user is displayed in the path box 302.

  The hero designation icon 304 is an icon for the user to designate the hero, and a person's face image is displayed as an icon. A plurality of different face image icons are displayed side by side on the main character designation icon 304, and can be selected by the user by clicking with the pointing device 107. Each face image is associated with a personal ID. The hero designation icon 304 is used to specify the hero who is the central person from the person shown in the image represented by the image data to be analyzed. For example, the face image is detected in advance from the image data and registered in the face database in association with the personal ID. The protagonist designation icon 304 is, for example, a face image of a person selected by a user or a face image of a person determined by a method described later among face images registered in the face database. The main character can be automatically set by the procedure shown in FIG.

  A spread number box 305 receives the setting of the number of spreads of the album from the user. The user inputs a number directly into the spread number box 305 through the keyboard 106 or inputs a number from the list into the spread number box using the pointing device 107. The template designation icon 306 displays an illustration image for each template taste (pop style, chic style, etc.). A plurality of template icons are displayed side by side on the template designation icon 306, and can be selected by the user by clicking with the pointing device 107. The image correction check box 307 accepts an image correction ON / OFF setting from the user. A state in which the image correction check box 307 is checked is a state in which image correction ON is designated, and a state in which the image correction check box 307 is not checked is a state in which image correction OFF is designated. The UI configuration screen 301 has a configuration in which ON / OFF is set for all correction processes at once, but ON / OFF may be set for each individual correction process.

  When the user clicks the OK button 308 with the pointing device 107, the condition specifying unit 201 acquires the content set on the UI configuration screen 301. Then, the condition specifying unit 201 outputs the acquired setting information to the automatic layout processing unit 216. At this time, the path input in the path box 302 is transmitted to the image acquisition unit 202. The personal ID of the hero selected by the hero designation icon 304 is transmitted to the hero information input unit 206. Further, the spread number input in the spread number box 305 is transmitted to the spread number input unit 208. Further, the template information selected by the template designation icon 306 is transmitted to the template input unit 211. Also, the image correction ON / OFF setting of the image correction check box 307 is transmitted to the correction condition input unit 213. A reset button 309 is a button for resetting each setting information on the UI configuration screen 301.

  Next, the designation of a moving image and an image on a server or SNS will be described in detail. A server name box 310 on the UI configuration screen 301 indicates a server name or SNS name including an image data group for which an album is to be created. The server selection button 311 displays a server name and an SNS name including an image group for which an album is to be created in a list configuration so that the user can select it. Then, the server name or SNS name selected by the user is displayed in the server name box 310. When the OK button 308 is clicked with the pointing device 107 while the server name or SNS name is displayed in the server name box 310, a login screen for the designated server or SNS is displayed. When the login to the designated server or SNS is completed by the user's operation via the displayed login screen, the CPU 101 can acquire image data from the designated server or SNS.

  The moving image use check box 312 receives from the user a setting as to whether or not a folder specified in the path box 302 or a moving image on the server or SNS specified in the server name box 310 is an object of album creation. A state in which the use check box 312 is checked is a state in which the album creation target is specified, and a state in which the movie use check box 312 is not checked is a state in which the album creation target is not specified.

  The target period box 313 accepts the setting of the shooting date / time period of the image that is the target of album creation from the user. The target period box 313 displays the designated shooting date and time period. When the OK button 308 is clicked with the pointing device 107 while the shooting date / time period is displayed in the target period box 313, an image data group corresponding to the specified shooting date / time period is extracted as an album creation target. .

<Automatic layout processing>
FIG. 4 is a flowchart showing a processing procedure in the automatic layout processing unit 216 of the album creating application in the present embodiment. The flowchart shown in FIG. 4 is realized, for example, when the CPU 101 reads out a program stored in the HDD 104 or the ROM 102 to the RAM 103 and executes it. The automatic layout process will be described below with reference to FIG.

  In step S <b> 401, the CPU 101 sequentially converts the image data acquired by the image acquisition unit 202 by the image conversion unit 203 to generate analysis image data. Here, image acquisition in the image acquisition unit 202 will be described with reference to FIG.

  FIG. 12 is a flowchart illustrating a procedure of processing in which the CPU 101 acquires a still image and a moving image from the folder, server, or SNS specified in the path box 302 and the server name box 310 by the image acquisition unit 202. The flowchart shown in FIG. 12 is realized, for example, when the CPU 101 reads out a program stored in the HDD 104 or the ROM 102 to the RAM 103 and executes it.

  When the user clicks the OK button 308 with the pointing device 107, the CPU 101 starts acquiring image data. In the following, processing from a state in which login to the server or SNS has been completed and the CPU 101 can acquire image data will be described.

  In step S1201, the CPU 101 refers to the folder set in the path box 302. If no folder is set in the path box 302, the process advances to step S1206. In step S1202, the CPU 101 determines whether all selection target still image data in the designated folder has been acquired. Here, the selection target still image data is all the still image data in the designated folder. When the target period box 313 accepts the setting of the shooting date and time period, the still image data generated during the set period is set as the selection target still image data. If it is determined that all the selection target still image data has not been acquired, the CPU 101 proceeds to S1203, and if it is determined that all the selection target still image data has been acquired, the CPU 101 proceeds to S1204. . In step S1203, the CPU 101 acquires one unacquired selection target still image data in the designated folder. After S1203, the CPU 101 ends the image acquisition process shown in FIG. 12, and returns to the process in FIG.

  In step S1204, the CPU 101 determines whether all selection target moving image data in the designated folder has been acquired. Here, the selection target moving image data is all the moving image data in the designated folder. When the target period box 313 accepts the setting of the shooting date and time period, the moving image data generated during the set period is set as the selection target moving image data. If it is determined that not all selection target moving image data has been acquired, the CPU 101 proceeds to S1205. If it is determined that all selection target moving image data has been acquired, the CPU 101 proceeds to S1206. In step S1205, the CPU 101 acquires one unacquired selection target moving image data in the designated folder. Note that the CPU 101 does not select a moving image when the moving image use check box 312 is not checked. After S1205, the CPU 101 ends the image acquisition process shown in FIG. 12, and returns to the process in FIG.

  In step S1206, the CPU 101 refers to the server or SNS specified in the server name box 310. If the server name box 310 has not accepted the server or SNS setting, the CPU 101 ends the image acquisition process of FIG. 12 and returns to the process of FIG.

  In step S <b> 1207, the CPU 101 determines whether all selection target still image data in the designated server or SNS has been acquired. Here, the selection target still image data is all the still image data in the designated server or SNS. When the target period box 313 accepts the setting of the shooting date and time period, the still image data generated during the set period is set as the selection target still image data. If it is determined that not all selection target still image data has been acquired, the CPU 101 proceeds to S1208. If it is determined that all selection target still image data has been acquired, the CPU 101 proceeds to S1209. . In step S1208, the CPU 101 acquires one target image data that has not been acquired in the designated server or SNS. After S1208, the CPU 101 ends the image acquisition process shown in FIG. 12, and returns to the process in FIG.

  In step S <b> 1209, the CPU 101 determines whether all selection target moving image data in the designated server or SNS has been acquired. Here, the selection target moving image data is all moving image data in the designated server or SNS. Further, when the target period box 313 accepts the setting of the shooting date and time period, the moving picture data generated during the set period is set as the selection target moving picture data. If it is determined that not all selection target moving image data has been acquired, the CPU 101 proceeds to S1210. If it is determined that all selection target moving image data has been acquired, the image acquisition illustrated in FIG. The process ends, and the process returns to the process of FIG. In S1210, the CPU 101 acquires one selection target moving image data that has not been acquired in the designated server or SNS. After S1210, the CPU 101 ends the image acquisition process shown in FIG. 12, and returns to the process in FIG.

  Next, image data analysis processing in the image conversion unit 203 will be described. First, the CPU 101 generates analysis image data by converting the number of pixels and color information of the acquired still image data into a desired number of pixels and color information. In the present embodiment, the CPU 101 generates analysis image data having a size of 420 pixels on the short side and sRGB color information. For the acquired moving image data, a frame is extracted from the moving image data, and image data corresponding to the frame is converted into the analysis image data. Here, frame extraction from moving image data will be described with reference to FIG. FIG. 11 is a flowchart showing a procedure of frame extraction processing from moving image data. The flowchart shown in FIG. 11 is realized, for example, when the CPU 101 reads out a program stored in the HDD 104 or the ROM 102 to the RAM 103 and executes it.

  In step S1101, the CPU 101 sets a timing for extracting a frame from the acquired moving image data. For example, the CPU 101 sets the timing so as to extract a frame every predetermined second such as 1 second or 5 seconds. Note that a timing for extracting a fixed number of frames may be set. For example, when 30 frames are extracted from a one-minute video, every 2 seconds is set as the timing for extracting the frames. Note that the timing may be set by a table in which the extraction timing is specified for the moving image shooting time, and the number of extracted frames may be changed according to the moving image shooting time.

  In S1102, the CPU 101 extracts a plurality of frames from the moving image data at the set timing.

  In S1103, the CPU 101 adds identification information to the extracted frame. The identification information is information about which moving image data has been acquired from, for example, a specified folder, a specified server, or an SNS. The identification information may be moving image shooting date / time information, extracted frame size, moving image shooting time, and the like. After S1103, the CPU 101 ends the frame extraction process from the moving image shown in FIG. Note that the extracted frames are sequentially converted into the analysis image data by the image conversion unit 203.

  Reference is again made to FIG. In step S <b> 402, the CPU 101 acquires image information of analysis image data (image data acquired / analyzed in step S <b> 401) using the image analysis unit 204. For example, the CPU 101 acquires the shooting date and time from Exif information attached to each analysis image data. When the analysis date / time information is not attached to the analysis image data, as described above, the creation date / update date / time of image data before analysis, and the date / time of upload / download of image data before analysis may be used. In addition, the CPU 101 acquires a feature amount from the analysis image data. Examples of the feature amount include focus. For example, a general Sobel filter is used as an edge detection method for acquiring a focus feature amount.

  If the edge is detected by a Sobel filter and the luminance difference between the start point and the end point of the edge is divided by the distance between the start point and the end point, the inclination of the edge, that is, the luminance gradient can be calculated. From the result of calculating the average inclination of the edges in the image, an image with a large average inclination can be considered to be in focus than an image with a small average inclination. If a plurality of threshold values having different values with respect to the inclination are set, it is possible to output an evaluation value of the focus amount by determining which threshold value or more.

  In the present embodiment, two different threshold values are set in advance, and the focus amount is determined in three stages of ◯ Δ ×. The threshold is set in advance by experiment or the like so that the focus inclination to be adopted for the album is “◯”, the allowable focus inclination is “Δ”, and the unacceptable inclination is “X”. Note that the threshold setting may be provided by, for example, the creator of the album creation application, or may be set on the user interface.

  As another method of calculating the focus amount, the peripheral pixels are referenced with respect to the edge detected by the Sobel filter, the maximum and minimum pixels of the peak luminance are specified, and the width between the two pixels is determined. It may be the amount of focus. In this case, it can be considered that the focus is achieved when the inter-pixel width is narrow. Further, a plurality of threshold values may be set for the inter-pixel width, and the focus amount may be determined in three stages of “◯”, “Δ”, and “×”.

  In step S <b> 403, the CPU 101 performs face detection by the image analysis unit 204. Here, face detection is performed from the analysis image data generated in S401. A known method can be used for the face detection process. For example, Adaboost is used to create a strong classifier from a plurality of weak classifiers. In this embodiment, a face image of a person (object) is detected by a strong classifier created by Adaboost. The CPU 101 extracts a face image and obtains an upper left coordinate value and a lower right coordinate value of the detected face image position. By having these two types of coordinates, the CPU 101 can acquire the position of the face image and the size of the face image. It should be noted that a strong classifier may be created by Adaboost using not only the face but also animals such as dogs and cats, flowers, food, buildings, figurines and the like as detection targets instead of faces. As a result, the CPU 101 can detect an object other than the face.

  In step S <b> 404, the CPU 101 performs personal recognition using the image analysis unit 204. Specifically, the CPU 101 performs personal recognition by comparing the similarity between the face image extracted in S403 and the representative face image stored for each personal ID in the face dictionary database. The CPU 101 determines the similarity between each representative face image and the extracted face image, and sets the ID of the representative face image having the highest similarity with the similarity equal to or greater than a threshold as the ID of the extracted face image. If the similarity with all representative face images damaged in the face dictionary database is less than the threshold value, the CPU 101 registers the extracted face image in the face dictionary database as a new personal ID.

  The image analysis information 500 acquired in S402 to S404 is distinguished for each ID 501 for identifying each image and stored in a storage area such as the RAM 103 or the HDD 104. For example, as shown in FIG. 5, the shooting date / time information 502 and the focus determination result 503 acquired in S402, the number of face images 504 detected in S403, and the position information 505 are stored in a table format. The face image position information is stored separately for each personal ID acquired in S404.

  In step S <b> 405, the CPU 101 determines whether the processing in steps S <b> 401 to S <b> 404 has been completed for all image data in the image data group designated by the condition designation unit 201. Here, when it is determined that the processing has not ended (No), the CPU 101 repeats the processing from S401 for the image data that has not been subjected to the processing of S401 to S404. If the CPU 101 determines that the process has ended (Yes), the process proceeds to S406.

  In step S <b> 406, the CPU 101 performs scene division by the image classification unit 205. The scene division means that all images of the image group designated by the condition designation unit 201 are divided for each scene and managed as a plurality of image groups. Specifically, the CPU 101 divides the image data group into a plurality of scenes based on the time difference between the images calculated from the shooting date / time information acquired in S402. In the present embodiment, for example, scene division is performed by the following method. First, if the difference between the imaging date / time of the N-th captured image data and the imaging date / time of the (N + 1) -th captured image data in the order of the imaging date / time is one day or less, the following is performed. If the difference between the imaging date / time of the N-th captured image data and the imaging date / time of the (N + 1) -th captured image data in the order of the imaging date / time is 16 hours or more, the N-th captured image data and the (N + 1) -th image data A group boundary is set between the captured image data. In addition, it is assumed that the difference in the imaging date / time between adjacent captured image data in the order of the imaging date / time is less than 16 hours between any captured image data. In this case, if the difference between the imaging date / time of the first captured image data and the imaging date / time of the last captured image data in the captured image data group captured in one day is less than 4 hours, the imaging of the day is performed. A group of image data is defined as one group. On the other hand, it is assumed that the difference between the imaging date / time of the first captured image data and the imaging date / time of the last captured image data in the group of captured image data captured in one day is 4 hours or more. In this case, if the number of captured image data captured in the day is less than 50 (the number of captured images captured in the day is less than 50), the captured image data group of the day is set as one group. . An example of such grouping of captured image data is shown in FIG. 6A to 6C, the horizontal axis indicates the imaging date and time (older toward the left side and newer toward the right side), and the vertical axis indicates the number of captured image data. In FIG. 6A, the captured image data group is divided into eight groups 601 to 608. In FIG. 6A, an arrow indicates a group boundary. Note that the thresholds for the difference in imaging date and time and the threshold for the number of captured image data used in the description of the above scene division method are examples, and are not limited to these values. The divided image group is called a divided image group, and one divided image group is set for each scene.

  In step S <b> 407, the CPU 101 performs scene classification by the image classification unit 205. Specifically, the CPU 101 scores the divided image group obtained by dividing the scene in S406 for each type of scene, and the scene with the highest score is assigned to the scene with the highest score except when the reliability of scene classification described later is low. Classify divided image groups. In the following description, scoring in S407 is referred to as scene classification scoring. In the present embodiment, an example in which “travel”, “daily life”, and “ceremony” are assumed as types of scenes, and the divided image group is classified into one of these scenes will be described. For scene classification scoring, a scene classification table in which information on feature amounts corresponding to each type of scene is stored is used.

  Here, a method for creating a scene classification table stored in advance in the program will be described. First, a plurality of image data groups determined in advance as being classified as “travel”, “daily life”, and “ceremony” scenes are collected, and feature quantities are acquired for the collected image data groups. Examples of the feature amount acquired here are, for example, a shooting period, the number of shots, and the number of shots. The photographing period is a difference in photographing time from the first photographed image to the last photographed image in each image data group. The number of shots is the number of each image data group. The number of photographers is the average number of faces in image data in which each face is captured in each image data group. As a result, for example, for one image data group made up of a plurality of image data determined in advance as a travel scene, the shooting period, the number of shots, and the feature quantity of the number of shots are acquired. Note that the feature amount is not limited to the shooting period, the number of shots, and the number of shots, and other feature amounts may be used, or only one of them may be used.

  Then, the acquisition of the feature amount of the shooting period, the number of shots, and the number of shots as described above is also performed for other image data groups collected in advance. Next, based on feature values acquired from a plurality of image data groups collected for each scene, the average value and standard deviation of the shooting period, the average value and standard deviation of the number of shots, and the average number of shots per image Values and standard deviations are determined. By associating the value obtained in this way with each type of scene, a scene classification table indicating each average value and standard deviation for each type of scene is created.

  In the present embodiment, it is assumed that a table 700 shown in FIG. 7 is used as the scene classification table. In the table 700, the average value and the standard deviation of the shooting period 702, the number of shots 703, and the number of shots 704 are registered in association with the scene ID 701. The CPU 101 stores the table 700 in advance in a storage area such as the ROM 102 or the HDD 104 before scene classification processing. The table 700 may be incorporated in advance in the album creation application program, or may be incorporated in a program outside the album creation application.

  Here, scene classification scoring will be described. After the album creation application is activated, the CPU 101 performs the shooting period, the number of shots, the number of shots for each of the divided image data groups designated by the user in the pass box 302 or the server name box 310 in S406. The score of each feature amount is calculated. The CPU 101 calculates the shooting period for each division, the number of shots, the score of the number of photographers, and the average score thereof from equations (1) and (2) using the average value and standard deviation for each scene shown in FIG. To do. The average value of the predetermined feature amounts corresponding to the predetermined type of scene and the standard deviation of the predetermined feature amounts corresponding to the predetermined type of scene in the following equation are registered in the table 700. Also, the feature amount is acquired for three feature items of the shooting period, the number of shots, and the number of shots. Therefore, the number of feature items is 3 in this example.

  First, the CPU 101 acquires a score for each type of scene and each feature amount of the divided image data group according to the equation (1).

Score of a predetermined feature amount corresponding to a predetermined type of scene of a divided image data group = 50− | 10 × (average value of predetermined feature amounts corresponding to a predetermined type of scene−predetermined value of a divided image data group Feature amount) / standard deviation of a predetermined feature amount corresponding to a predetermined type of scene | (1)
Thereby, for example, in the divided image data group, the scores for the shooting period, the number of shots, and the number of shots corresponding to the “travel” scene are acquired. Note that the CPU 101 acquires scores not only for the “travel” scene but also for each of the other types of scenes.

  Next, the CPU 101 averages each score obtained by the equation (1) by the equation (2), and obtains a score regarding each type of scene of the divided image data group.

Score for a predetermined type of scene of the divided image data group = (Score for the shooting period corresponding to the predetermined type of scene of the divided image data group + Score for the number of shots + Score for the number of shots) / Number of feature items (2)
Thereby, for example, in the divided image data group, scores relating to each type of scene of “travel”, “daily life”, and “ceremony” are acquired. The CPU 101 performs scene classification scoring for each divided image data group.

  As described above, the score for each “travel”, “daily life”, and “ceremony” is calculated for each divided image data group. Then, the CPU 101 classifies the scene with the first score as the scene of the divided image data group, with the exception described later. For example, in the divided image data group 5 corresponding to the scene 5 after the scene division of FIG. 6A, the shooting period is 36 hours, the number of shots is 300, and the number of shots is 1.7. Then, in the divided image data group 5, the score for “travel” is 45.32, the score for “daily” is 18.38, the score for “ceremony” is −29.92, and the divided image data group 5 has the highest score. It is classified as a “travel” scene. Note that which divided image data group is classified into which scene is managed by a scene ID so that it can be identified.

  Note that, as the feature amount for classifying the scene, here, the shooting period, the number of shots, and the number of shots are focused, but the present invention is not limited to this. Classification accuracy may be increased by adding feature quantities such as image brightness, person, and number of images. The type of scene is not limited to “travel”, “daily life”, and “ceremony” described here, but may be other scenes such as parties and sports. When performing scene classification including other scenes, in the creation of the table 700, a plurality of image data groups of the other scenes are collected in the same manner as described above, and feature quantities for scene classification are collected. What is necessary is just to obtain an average value and a standard deviation.

  In S408, the CPU 101 performs the scene classification in S407 on the entire divided image data group divided in S406 (in this description, the divided image data group corresponding to each divided scene in FIG. 6A). It is determined whether or not it has been completed. If it is determined that the processing has not ended (No), the CPU 101 pays attention to a new divided image data group that has not been processed, and returns to S407. If the CPU 101 determines that the process has ended (Yes), the process proceeds to S409.

  In step S <b> 409, the CPU 101 performs hero setting by the image score unit 207. The main character setting is executed for an image group designated by the user, and is performed by one of two types of setting methods, automatic and manual. The CPU 101 determines the number of individual IDs appearing in the image data group, the number of individual IDs appearing in each scene, and the number of individual IDs appearing in each scene from the result of the personal recognition executed in S404 and the result of the scene division executed in S406. The number of scenes in which the personal ID appears can be acquired. Based on these pieces of information, the CPU 101 automatically sets the main character regardless of user designation. In the present embodiment, when the image data group designated by the user is divided into a plurality of divided image data groups for each scene, the CPU 101 sets a personal ID having a large number of appearances in a plurality of scenes as the main character ID. . Further, when the image data group is a single scene, the CPU 101 sets a personal ID having a large number of appearances in a single scene as the main character ID. When the hero designation icon 304 is designated by the user, the CPU 101 notifies the image scoring unit 207 of the personal ID designated via the hero information input unit 206. When the personal ID is specified by the user, the main character ID automatically set as described above is ignored, and the personal ID specified by the user is set as the main character ID. This setting is called manual setting.

  In S <b> 410, the CPU 101 performs image scoring by the image scoring unit 207. The image scoring in S410 is to assign a score evaluated from the viewpoint described later for each image data (scoring), and is referred to when selecting image data used for layout described later. FIG. 10 shows a group of templates used for image data layout. Each of the plurality of templates included in the template group corresponds to each spread. The template 1001 is a single template, the main slot 1002 indicates a main slot, and the sub slots 1003 and 1004 indicate sub slots. The main slot 1002 is a main slot (a frame for laying out (arranging) images) in the template 1001 and is larger in size than the sub slots 1003 and 1004. Note that image data classified into the same type of scene may be laid out in one template, or image data classified into different types of scenes may be laid out in one template. In S410, the CPU 101 performs image scoring, which is a process of assigning to the image data both the main slot score and the subslot score corresponding to the type of scene to which the image data belongs.

  In image scoring, a slot feature table is used in which information about image feature amounts to be used in main slots and sub slots is stored for each type of scene.

  Here, a method for creating the slot feature table will be described. FIG. 8A is a table in which the characteristics of the image data employed in the main slot and subslot are summarized for each type of scene of “travel”, “daily life”, and “ceremony”. First, in the image data classified into each type of scene, a plurality of image data determined to be suitable for the image feature to be employed in the main slot is collected. In addition, a plurality of image data determined to match the image features to be employed in the subslot are collected. That is, image data matching the features shown in FIG. 8A is collected. Then, the image analysis unit 204 extracts feature quantities such as the number of face images, the position of the face image, and the size of the face image in each collected image data. Then, the slot feature table is created by obtaining the average value and standard deviation of the extracted feature values for each scene type and each slot type (main slot and subslot). The slot feature table is stored in advance in a storage area such as the ROM 102 or the HDD 104 before scoring an image. The slot feature table may be incorporated in advance in the album creation application program, or may be incorporated in a program outside the album creation application.

  In image scoring, the CPU 101 scores image data for each template slot corresponding to the type of scene to which the image data belongs, using the following equation. Note that the type of scene to which each image data of the image data group designated by the user belongs can be known from the result of the scene classification in S407. The average value of the predetermined feature amounts corresponding to the predetermined type scene and the predetermined slot and the standard deviation of the predetermined feature amount corresponding to the predetermined type scene and the predetermined slot in the following equation are the slot characteristic table. It is registered in. In addition, the feature amount is acquired for three feature items of the number of face images of the main character ID, the position of the face image, and the size of the face image. Therefore, the number of feature items is 3 in this example.

Score for a predetermined feature amount corresponding to a predetermined slot of image data = 50− | 10 × (average value of predetermined feature amounts corresponding to a kind of scene to which image data belongs and a predetermined slot−predetermined image data Feature amount) / standard deviation of a predetermined feature amount corresponding to a scene of a type to which image data belongs and a predetermined slot | (3)
Thereby, for example, the score of “the number of faces of the main character ID” corresponding to the main slot of the image data classified as the “travel” scene is acquired. Note that the CPU 101 acquires scores not only for the main slot but also for the feature amount corresponding to the subslot. Further, the CPU 101 acquires scores not only for “the number of face images of the main character ID” but also for the feature amounts corresponding to “the position of the face image” and “the size of the face image”.

  Next, the CPU 101 averages each score obtained by the equation (1) by the equation (2), and obtains a score relating to a predetermined slot of the image data.

Score for a predetermined slot of image data = (Score of the type to which the image data belongs and the score for the number of faces corresponding to the predetermined slot + the score for the face position + the score for the face size) / number of feature items (4)
Thus, scoring for both the main slot and the subslot is performed on the image data. The CPU 101 performs image scoring on each image data in the image data group designated by the user. The score added by the image scoring becomes a selection criterion in the image selection process in S417 in the subsequent stage. As described above, even with the same image data, the selection criteria differ depending on the scene to be classified. Since it is preferable that the images used for the album are in focus, a predetermined score may be added to an image with an image ID whose focus feature is “◯” in the table shown in FIG. Good. In addition, since an image with a large number of pixels has less jaggy when laid out in an album, a predetermined score may be added to an image with a large number of pixels. FIG. 8B shows an example of a score result by layout scoring. The table 800 shows the result of scoring each of the main slot and subslot for each image ID 801, and a score 802 is registered for each image ID 801 image. For example, in FIG. 8B, image ID 1 is assigned 20 points for the main slot, and image ID 2 is assigned 45 points for the main slot. This means that for the main slot, the image ID 2 is closer to the user's criterion.

  In step S <b> 411, the CPU 101 determines whether layout scoring has been completed for all image data in the user-specified image data group by the image scoring unit 207. If it is determined that the processing has not ended (No), the CPU 101 focuses on the next unprocessed image and returns to S410. If the CPU 101 determines that the process has been completed (Yes), the process proceeds to S412.

  In the present embodiment, the CPU 101 causes the image layout unit 212 to lay out image data included in each of a plurality of scenes in each of a plurality of templates (a plurality of spreads). For this reason, the number of scenes and the number of templates (a predetermined number of spreads) need to match.

  Therefore, in S412, the CPU 101 causes the spread allocation unit 209 to determine whether the number of scene divisions in S406 by the image classification unit 205 is the same as the number of spreads of the album input from the spread number input unit 208. . If the CPU 101 determines that they are not the same (No), it proceeds to S413, and if it is determined that they are the same (Yes), it proceeds to S416. For example, if the number of scene divisions is 8 and the number of inputs of the spread number input unit 208 is 8 as shown in FIG. 6A, the CPU 101 proceeds to S416.

  In S <b> 413, the CPU 101 determines whether the spread allocation unit 209 determines that the number of scene divisions in S <b> 406 by the image classification unit 205 is smaller than the number of spreads of the album input from the spread number input unit 208. If the CPU 101 determines that the number is not small (large) (No), the process proceeds to S415. If the CPU 101 determines that the number is small (Yes), the process proceeds to S414. For example, if the number of scene divisions is 8 and the number of inputs of the spread number input unit 208 is 10 as shown in FIG. 6A, the CPU 101 proceeds to S414.

  In S <b> 414, the CPU 101 executes sub-scene division by the spread allocation unit 209. The sub-scene division is to further subdivide the divided scene when the number of scene divisions <the number of album spreads. Here, a case where the number of scene divisions is 8 as shown in FIG. 6A and the designated spread number is 10 will be described. FIG. 6B shows the result of dividing the sub-scene of FIG. Here, the division interval is set more finely, and the number of divisions is set to 10 by dividing the divided image group number at an arrow between 2 and 3 and an arrow between 6 and 7.

  The criteria for division will be described. In the division shown in FIG. 6A, a division portion having a large number of image data is searched. Here, in order to increase the number of divisions from 8 to 10, two locations with a large number of image data are determined. That is, the scene to be subdivided is specified in order from the scene corresponding to the divided image data group including the most image data. For the scenes corresponding to the divided image data groups having the same number of image data, the scene having the larger maximum value of the photographing date / time difference between the image data included in the corresponding divided image data group is selected. If selection is still not possible, for example, a scene corresponding to a divided image data group including image data generated at an earlier time may be pre-divided.

  In FIG. 6A, the scenes with the largest number of image data included in the corresponding divided image data group are scene 5, scene 1, and scene 2 in order from the largest. Although scenes 1 and 2 have the same number of data, scene 2 has a larger time difference between image data generated last from image data generated first in the corresponding divided image data group. To do. That is, in FIG. 6A, scene 5 and scene 2 are each divided.

  First, scene 2 division will be described. Scene 2 has two peaks of image data, and image data included in these two peaks has different shooting dates for each mountain. Therefore, the image is divided at the location indicated by the thick arrow between the divided image group numbers 2 and 3 in FIG. Next, the division of the scene 5 will be described. In scene 5, there are three peaks of image data. Like scene 2, the image data included in these three peaks has a different shooting date for each mountain. In scene 5, there are two places where the shooting date changes. In this case, the shooting date is divided in units of shooting days so that the difference in the number of data after division becomes small. That is, here, the image is divided at the location of the thick arrow between the divided image group numbers 6 and 7 in FIG.

  Thus, the number of divisions, that is, the number of scenes is changed from 8 to 10. Specifically, in the re-division of the scene, if the selected scene includes image data having different shooting dates, the image data groups having different shooting dates are divided into new scenes. If the shooting date exceeds three days, the difference in the number of image data included in each divided scene is minimized, and the images on the same shooting date are included in the same scene. Although an example of dividing at different locations on the shooting date has been described here, if the shooting date of the image data in the scene to be split is a single day, the time difference between shooting times is the largest within the single day. Divide at the point. The number of scenes is matched with the number of spreads by the above procedure. The scene generated by the division may be classified again, or the scene classification before the division may be taken over.

  In S415, the CPU 101 performs scene integration. The scene integration means that the divided scenes are integrated when the number of scene divisions> the number of album spreads. Here, a case where the number of scene divisions is 8 as shown in FIG. 6A and the designated spread number is 6 will be described as an example. FIG. 6C shows the result of scene integration of FIG. The number of divisions is set to 6 by integrating the scenes before and after the broken-line arrow in FIG.

  Explain the integration criteria. First, the CPU 101 detects a scene with a small number of image data among the divided scenes. Here, in order to reduce the number of scenes from 8 to 6, two scenes with a small number of image data are detected. In FIG. 6A, scenes with the largest number of image data included in the corresponding divided image data group are scenes 8, 3, and 7 in order from the smallest. Scenes 3 and 7 have the same number of data.

  Next, the CPU 101 sets the scene 8 having the smallest number of image data included in the corresponding divided image data group as an integration target. Next, since the scene 3 and the scene 7 have the same number of image data included in the corresponding divided image data group, which one is to be integrated is selected. Here, since the scene 8 adjacent to the scene 7 is the integration target (since the scene 7 and the scene 8 are integrated), the scene 3 is the integration target.

  Next, the CPU 101 determines whether to integrate the sub image data group to be integrated into the previous scene with the shooting date and time of the image data included in the corresponding divided image data group or with the later scene. To do. At this time, the CPU 101 sets a scene having a smaller time difference in shooting date / time of image data included in the corresponding divided image data group as a partner of integration among two scenes adjacent to the scene to be integrated. Therefore, in FIG. 6A, for scene 3, any of scenes 2 and 4 adjacent to it is set as the integration partner. Here, when the time difference between the shooting dates and times of the images included in the corresponding divided image data group between the scene 3 and the preceding and following scenes 2 and 4 is compared, the time difference between the scene 3 and the scene 4 is smaller. Therefore, the CPU 101 determines that the scene 3 is integrated into the scene 4. That is, here, integration is performed at the location of the broken line arrow of the divided image group number 3 in FIG.

  When there is only one adjacent scene such as the scene 8, the CPU 101 sets the only adjacent scene as the integration partner.

  In this way, the scenes are integrated in the scenes before and after the dotted arrow in FIG. The scene integration includes, for example, updating information indicating an image file included in the scene according to the integrated scene.

  In S <b> 416, the CPU 101 executes spread allocation by the spread allocation unit 209. By S412 to S415, the number of scene divisions and the number of specified spreads are the same. The CPU 101 assigns in order from the top divided image data group in the shooting date and time to the top of the spread.

  In S <b> 417, the CPU 101 performs image selection by the image selection unit 210. Here, an example in which four pieces of image data are selected from a divided image data group assigned to a certain spread will be described with reference to FIG.

  FIG. 9A shows the time difference (divided shooting period) between the shooting date and time of the image data having the earliest shooting date and the image data having the latest shooting date among the divided image data groups assigned to the spread. Here, selection of image data is performed in the order of the main slot and the subslot. Here, it is assumed that one main slot 1002 is included in the template corresponding to the spread. Therefore, the image data selected for the first sheet is image data for the main slot. The CPU 101 selects the image data (1) having the highest score for the main slot added in S410 from the image data corresponding to the divided shooting period shown in FIG. 9B as the image data for the main slot. .

  The image data selected after the second image is image data for the subslot. The selection of the second and subsequent image data is performed by the method described below so as not to concentrate on a part of the divided shooting period. First, as shown in FIG. 9C, the CPU 101 divides the divided shooting period into two. Next, as shown in FIG. 9D, the CPU 101 selects the second image from the image data generated during the divided shooting period in which the first image data is not selected (the period indicated by the solid line in the figure). Select the image data. As the second image data, the image data (2) having the highest score for the subslot is selected from the image data generated during the divided shooting period in which the first image data is not selected. . Next, as shown in FIG. 9E, the CPU 101 divides each divided shooting period shown in FIG. 9D into two. Then, as shown in FIG. 9F, the CPU 101 generates the divided image capturing period (period shown by a solid line in the figure) in which neither the first image data nor the second image data is selected. The third image data is selected from the obtained image data. As the third image data, the score for the subslot is the highest among the image data generated during the divided shooting period in which neither the first image data nor the second image data is selected. Image data (3) is selected.

  Next, an example in which there is no image generated during the divided shooting period as the selection target of image data and data cannot be selected from the image data generated during the divided shooting period is taken as an example of selecting the fourth image data. explain. Here, as shown in FIG. 9G, it is assumed that there is no image data generated in a divided shooting period (period indicated by hatching in the figure) in which image data is not yet selected. In that case, the CPU 101 divides each divided shooting period into two as shown in FIG. Next, as shown in FIG. 9 (I), the CPU 101 divides the divided shooting period other than the divided shooting period in which it is recognized that there is no image data generated within the period and the image data is not yet selected (in the drawing). The fourth image data is selected from the images generated during the period indicated by the solid line. The CPU 101 selects the highest point image data (4) for the subslot from the image data generated during the divided photographing period as the fourth image data.

  In S <b> 418, the CPU 101 determines the image layout by the image layout unit 212. In order to determine the image layout, a template is selected based on the selected image.

  An image layout method will be described. First, the CPU 101 acquires a template group input to the template input unit 211 as a selection candidate. Next, the CPU 101 selects, from the candidates, a template in which the position of the main slot in the template corresponds to the position of the main slot image data in the image data group selected in S417. For example, this determination is made on the assumption that the positions of the slots in the template are arranged in time series from the upper left to the lower right. However, since it is complicated to individually associate the position of each slot and the image, for example, the templates are grouped according to the positions of the main slot and the subslot. At this time, further grouping may be performed in accordance with the direction of the image orientation (vertically long or horizontally long). The template is narrowed down according to the position and direction of the main slot image in time series, and the subslots are similarly narrowed down. In this way, template candidates are narrowed down, and a template to be finally adopted is determined.

  The image layout method will be described in detail. Here, an example will be described in which (A) to (P) in FIG. 10 are input to the template input unit 211 for a certain spread according to the specified template information. Here, it is assumed that the number of slots of the input template is designated as 3. For example, the first three images selected from the four image data selected in the description of FIG. 9 are used as the images. Assume that the orientation of the three selected image data images is arranged in the vertical direction or the horizontal direction with respect to the shooting date and time as shown in FIG. Here, the image data 1005 is for the main slot, and the image data 1006 and the image data 1007 are for the subslot. That is, the image data 1005 for the main slot has the latest shooting date and time and the orientation is the vertical direction. In this embodiment, image data with an older shooting date and time is laid out in the upper left of the template, and image data with a newer shooting date and time is laid out in the lower right. Therefore, if the selected image data has a form as shown in FIG. 9J, the template whose orientation is vertical is located on the rightmost, that is, (I) to (I) in FIG. The template of L) is a candidate. Since the orientation of the old image data 1006 for the subslot is vertical and the orientation of the new image data 1007 is horizontal, the template (J) is the most suitable template for the selected image. The layout is determined. Thus, in S418, which image is laid out in which slot of which template is determined.

  In step S <b> 419, the CPU 101 performs image correction by the image correction unit 214. Note that the CPU 101 executes image correction when the setting that the image correction is ON is input from the correction condition input unit 213. As the image correction, for example, brightness correction, red-eye correction, and contrast correction are executed. The CPU 101 does not perform image correction when the setting that the image correction is OFF is input from the correction condition input unit 213. The image correction can be executed, for example, on image data converted into an sRGB color space with a short side of 1200 pixels.

  In S <b> 420, the CPU 101 creates layout information by the layout information output unit 215. The CPU 101 lays out the image data corresponding to each slot for which the image correction of S419 has been executed in each slot of the template determined in S418. At this time, the CPU 101 lays out an image to be laid out according to the size information of the slot by scaling or equal magnification. Then, the CPU 101 generates bitmap data in which an image is laid out on a template.

  In S421, the CPU 101 determines whether or not S417 to S420 have been completed for all spreads. If the CPU 101 determines that it has not ended (No), it returns to S417, and if it is determined that it has ended (Yes), it ends the automatic layout processing of FIG.

  By doing as described above, it is possible to automatically select image data from the image data group of each scene, and automatically create an album by laying out the selected image data in a spread.

  In the above procedure, the selection of the first image data is performed using the score for the main slot, and the selection of the second and subsequent image data is performed using the score for the subslot. However, it is not limited to this. When a template having a plurality of main slots is prepared in advance, a plurality of pieces of image data may be selected using a score for the main slot.

  Further, although the mode in which the number of image data to be selected is four has been described, the present invention is not limited to this. When a plurality of templates having different numbers of slots are prepared in advance, image data of a number other than four may be selected. Note that the number of divisions in the divided shooting period described in S417 may be increased or decreased according to the number of selected image data.

  Further, although the mode in which the image data to be laid out on the template is three has been described, the present invention is not limited to this. When a plurality of templates having different numbers of slots are prepared in advance, a number of image data other than three may be laid out.

  In the above description, in the scene classification, the score for each type of scene is calculated for each divided image data group, and the divided image data group is classified into the scene with the highest score. Here, the scene classification exception process will be described.

  In the present embodiment, after the scene classification, a score is assigned to the image data with a different scoring standard (FIG. 8A) for each classified scene, and the image data is assigned to the main slot and subslot according to the given score. Layout. However, in the scene classification, the divided image data group may be classified into a scene different from the user's intention. In that case, as described above, the scoring criteria of the image data are different for each scene, so that a layout result different from the layout result when the divided image data group is classified into an appropriate scene is output. As a result, a layout result with low user satisfaction is output. Therefore, in the present embodiment, control for suppressing the output of a layout result with low user satisfaction is executed. In the following description, the degree of user satisfaction between the layout when the divided image data group is classified into an appropriate scene and the layout when the divided image data group is classified by scene classification according to the above-described method. Is called layout compatibility (affinity).

  FIG. 13 is a table showing layout compatibility (hereinafter, layout compatibility table). In the layout compatibility table, when the divided image data group is classified into an appropriate scene, naturally, a layout result with high user satisfaction can be obtained. Therefore, the layout compatibility is indicated by “◯”. When the divided image data group is classified into a scene that is not an appropriate scene (an erroneous determination occurs), a layout result with high user satisfaction cannot be obtained. For this reason, in the case where an erroneous determination occurs and the user satisfaction of the layout result obtained when an erroneous determination occurs significantly, the layout compatibility is indicated by “x” in the layout compatibility table. If the user satisfaction does not decrease greatly even if an erroneous determination occurs, the layout compatibility is indicated by “Δ”.

  For example, consider a case where a divided image data group in which the actual scene is “travel” is erroneously determined as “daily” in the scene classification. When the actual scene is “travel”, the user's satisfaction increases when the image data in which the person and the landscape are photographed are laid out in the main slot. However, if an erroneous determination such as that described above occurs, the scoring criteria for the main slot and subslot are reversed as can be seen from FIG. Will be. That is, due to an erroneous determination of the scene classification, there is a high possibility that image data that does not show much scenery is laid out in the main slot regardless of whether the actual scene is “travel”. In such a case, user satisfaction is greatly reduced. Therefore, in the layout compatibility table, the layout compatibility when the scene classification result of the divided image data group whose actual scene is “travel” is “daily” is indicated by “x”.

  Next, consider a case where a divided image data group whose actual scene is “daily” is erroneously determined as “travel” in the scene classification. When the actual scene is “everyday”, the satisfaction of the user increases when image data showing an upside view of a child or a casual profile of a person is laid out in the main slot. However, even when an erroneous determination as described above occurs, as can be seen from FIG. 8A, the scoring criteria for the main slot and subslot are reversed, and a high score is assigned to image data in which a small person appears. End up. In other words, due to an erroneous determination of the scene classification, there is a high possibility that image data that does not satisfy the conditions of the main slot in the “daily” scene will be laid out in the main slot, even though the actual scene is “daily”. . In such a case, user satisfaction is reduced. However, some of the image data generated in the “daily” scene may be laid out in the main slot even though it satisfies the conditions of the main slot in the “travel” scene instead of the “daily” scene. Some users do n’t get much less satisfied. For example, even if image data that a child is playing casually with a landscape in a park or house is laid out in the main slot, the satisfaction level of the user is not necessarily greatly reduced. For this reason, in the layout compatibility table, the layout compatibility when the scene classification result of the divided image data group whose actual scene is “daily” is “travel” is indicated by “Δ”.

  Next, an example of the scene classification result will be described. In FIG. 14A, the actual scene is “travel”, the feature amount is 20 hours for the shooting period, the number of shots is 75, and the number of shots is 1.7. Shows the score for the scene. The score for each scene type of the image data group having the above-described feature amount is 36.13 points for “travel”, 36.69 points for “daily”, and 6.97 points for “ceremony”. Therefore, although the score difference is as small as 0.56 points, the score for “daily life” is 1st and the score for “travel” is 2nd. Judgment will occur.

  Further, FIG. 14B shows each of the divided image data groups in which the actual scene is “travel”, the feature amount is 20 hours for the shooting period, the number of shots is 120, and the number of shots is 1.7. Shows the score for each type of scene. The score for each scene type of the image data group having the above-described feature amount is 36.77 points for “travel”, 34.39 points for “daily life”, and 10.48 points for “ceremony”. Therefore, the score of “travel” is 1st and the score of “daily” is 2nd. If the divided image data group is classified into the highest score scene, the scene classification is performed accurately. However, in this example, the score difference between the first place and the second place is 2.38, and even when scene classification is performed accurately as in this example, the score difference may be small.

  It should be noted that in the divided image data group, each feature quantity is unlikely to greatly deviate from the numerical value corresponding to the actual scene, so even if an erroneous determination occurs, the actual scene (if an erroneous determination occurs) The score of the second scene is higher to some extent. That is, there is little possibility of erroneous determination when there is a large score difference between the scene with the highest score and the scene with the second highest score. Therefore, in the present embodiment, the CPU 101 determines that the scene classification is appropriately performed when the score difference between the scene with the first score and the scene with the second score is large, and the score is the first. The divided image data group is classified into scenes.

  Contrary to the above, if the score difference between the first-ranked scene and the second-ranked scene is small, the possibility of erroneous determination increases. That is, the reliability of the result of scene classification scoring is lowered. In addition, when the layout compatibility of the first-ranked scene is “x” with respect to the second-ranked scene, if a misjudgment occurs, the user's satisfaction level is greatly reduced. For this reason, if the reliability of the result of scene classification scoring is low and the layout compatibility of the first scene is “x” with respect to the second scene, the user's satisfaction is greatly reduced. It is highly possible.

  For this reason, in this embodiment, when the reliability of the result of scene classification scoring is low, the CPU 101 determines the layout compatibility between the first-ranked scene and the second-ranked scene. If the layout compatibility of the first scene is “x” with respect to the scene with the second highest score, and the layout compatibility of the second scene with respect to the first score is “△”, The divided image data group is classified into a scene with the second highest score. Thereby, when there is a high possibility that the evaluation of the image data is wrong, it is possible to classify the image data into scenes that can avoid the layout compatibility from being “x”.

  It should be noted that the determination of whether or not there is a large score difference between the scene with the highest score and the scene with the second highest score is, for example, the difference in scores between the scene with the highest score and the scene with the second highest score. Is compared with a threshold. The threshold value may be set arbitrarily so that the user's satisfaction with the layout result is high. In addition, the threshold value may not be fixed, but the threshold value may be changed according to, for example, a combination of a scene with the highest score and a scene with the second highest score.

  FIG. 15 is a table showing scene classification results in each case. In the table showing the scene classification result, the scene classification result in each case is set. In the present embodiment, the CPU 101 performs scene classification based on a table indicating scene classification results. Specifically, first, the CPU 101 determines which case the divided image data group to be classified is based on the combination of the scene with the highest score and the scene with the second highest score and the reliability of the layout scoring. To do. Then, the CPU 101 refers to the scene classification result shown in the table row corresponding to the determined case, and classifies the divided image data group. It should be noted that the CPU 101 stores the table 700 indicating the scene classification result in each case in advance in the storage area such as the ROM 102 before the scene classification process. The table 700 may be incorporated in advance in the album creation application program, or may be incorporated in a program outside the album creation application.

  When the score difference between the first-ranked scene and the second-ranked scene is large, the CPU 101 refers to the scene classification result in the column where “◯” is set in the “score difference evaluation result”. Further, when the score difference between the first-ranked scene and the second-ranked scene is small, the CPU 101 refers to the scene classification result in the column where “x” is set in the “score difference evaluation result”. .

  The case where the divided image data group is classified into a scene different from the scene with the highest score is Case No. 4 will be described as an example.

  Case No. No.4, the scene with the highest score is "Travel", the scene with the second highest score is "Ceremony", and the score difference between the scene with the highest score and the scene with the second highest score is small This is the case. In addition, when the actual scene of the divided image data group is the “ceremony” with the second highest score, when the scene is classified into “travel” with the first highest score, the layout compatibility is “x”. Therefore, in order to avoid that the layout compatibility becomes “x” due to the erroneous determination, case no. 4, it is desirable to classify the divided image data group into a scene different from the scene with the highest score. Here, referring to FIG. 13, when the actual scene is “ceremony”, the layout compatibility is “◯” when it is classified as “ceremony”, and when the actual scene is “travel”, it is changed to “ceremony”. When classified, the layout compatibility is “Δ”. That is, by classifying as “ceremony”, it is possible to prevent the layout compatibility from being “x” even if an erroneous determination occurs, unless the actual scene is “daily”. Case No. In No. 4, since the score of the “daily” scene is the lowest, the possibility that the actual scene is “daily” is low. Therefore, the CPU 101 determines the case number. In 4, the divided image data group is classified as “ceremony”. As a result, even if a misjudgment occurs, the possibility of layout compatibility “○” or “△” (possibility of avoiding layout compatibility “x”) increases, and the minimum number of users Satisfaction can be ensured.

  Case No. No. 6 is case no. As in the case of No. 4, if the scene is classified into the second highest score, the layout compatibility is “◯” or “Δ” even if it is erroneously determined. Therefore, Case No. 6 is also classified into a scene with the second highest score.

  Next, the case where the divided image data group is classified into the scene with the highest score is Case No. 2 will be described.

  Case No. 2, the scene with the highest score is “Travel”, the scene with the second highest score is “Daily”, and the score difference between the scene with the highest score and the scene with the second highest score is small This is the case. When the actual scene of the divided image data group is “travel”, the layout compatibility is “◯” when the scene is classified as “travel” with the highest score. If the actual scene is “daily” with a score of 2nd and the scene is classified as “travel” with the score of 1st, the layout compatibility is “Δ” as shown in the layout compatibility table. That is, by classifying as “travel”, it is possible to avoid the layout compatibility from being “x” even if an erroneous determination occurs unless the actual scene is “ceremony”. Case No. In 2, the score of the “ceremony” scene is the lowest, so the possibility that the actual scene is “ceremony” is low. Therefore, the CPU 101 determines the case number. In 2, the divided image data group is classified as “travel”. As a result, even if a misjudgment occurs, the possibility of layout compatibility “○” or “△” (possibility of avoiding layout compatibility “x”) increases, and the minimum number of users Satisfaction can be ensured.

  Case No. 8, 10 and 12 are also case no. As in the case of No. 2, if the scene is classified into the scene with the highest score, the layout compatibility is “◯” or “Δ” even if it is erroneously determined. Therefore, Case No. 8, 10, and 12 are also classified into the scene with the highest score.

  As shown in FIG. 13, when the actual scene is classified as “ceremony”, the layout compatibility is “Δ” or “◯”. Therefore, Case No. in which the “score difference evaluation result” is “×” (the reliability of the result of scene classification scoring is low). For 2, 4, 6, 8, 10, 12, all the scene classification results may be “ceremony”.

  FIG. 16 is a flowchart showing the procedure of the scene classification process in this embodiment. The flowchart shown in FIG. 16 is realized, for example, when the CPU 101 reads out a program stored in the HDD 104 or the ROM 102 to the RAM 103 and executes it. Note that the processing shown in this flowchart corresponds to S407 in FIG.

  In S1601, the CPU 101 acquires an image data group in which scene classification is not completed from the image data group (divided image data group) divided in S406.

  In step S1602, the CPU 101 determines feature items used for scene classification. The feature items are, for example, a shooting period, the number of shots, and the number of shots.

  In step S1603, the CPU 101 acquires a feature amount that has not yet been acquired from the image data acquired in step S1601.

  In step S1604, the CPU 101 determines whether all the feature values determined in step S1602 have been acquired from the image data group acquired in step S1601. If it is determined that it has been acquired, the process proceeds to S1605. If it is determined that it has not been acquired, the process returns to S1603 to acquire a feature quantity that has not yet been acquired. In step S1605, the CPU 101 determines the type of scene to be classified. Examples of the scene type include “travel”, “daily life”, and “ceremony”.

  In S <b> 1606, the CPU 101 scores the feature items that have not yet been scored for the image data group acquired in S <b> 1601 based on the feature values acquired in S <b> 1604.

  In step S1607, the CPU 101 determines whether scoring of all the feature items determined in step S1602 is completed for one type of scene for the image data group acquired in step S1601. Here, for example, it is determined whether scoring of the shooting period, scoring of the number of shots, and scoring of the number of shots have been completed. If it is determined that the process has been completed, the process proceeds to S1608. If it is determined that the process has not been completed, the process returns to S1606 to score the feature quantity that has not been scored yet.

  In step S <b> 1608, the CPU 101 calculates an average score of feature amount scores calculated for one type of scene.

  In step S <b> 1609, the CPU 101 determines whether the calculation of the average score of feature amount scores has been completed for all types of scenes determined in step S <b> 1605. Here, for example, it is determined whether or not the calculation of the average score of the feature amount scores has been completed for all types of scenes of “travel”, “daily life”, and “ceremony”. If it is determined that the processing has ended, the CPU 101 proceeds to S1610. If it is determined that the processing has not ended, the CPU 101 returns to S1606 and performs scoring in a scene that has not yet been scored.

  Note that the CPU 101 performs the above scoring using the scene classification table in the same manner as the method described in the description of S407.

  In step S <b> 1610, the CPU 101 acquires the scene types (scene classification candidates) having the first and second highest scores.

  In step S <b> 1611, the CPU 101 sets a threshold value to be compared with the score difference between the first and second scene types. The threshold value acquired here is stored in advance in the ROM 102 or the like. Note that the CPU 101 may change the threshold value to be acquired according to the types of scenes with the first and second scores.

  In step S <b> 1612, the CPU 101 calculates a score difference between the first and second scene types.

  In S1613, the CPU 101 compares the score difference between the first and second scene types with the threshold value acquired in S1611. As a result of the comparison, the CPU 101 proceeds to S1614 when it is determined that the score difference is greater than or equal to the threshold value, and proceeds to S1615 when it is determined that the score difference is less than the threshold value.

  In S <b> 1614, the CPU 101 classifies the divided image data group acquired in S <b> 1601 into the scene with the highest score. After S1614, the CPU 101 proceeds to S408.

  In step S <b> 1615, the CPU 101 refers to the layout compatibility table to determine the layout compatibility of the scenes with the first and second scores. If the layout compatibility when the scene classification result of the divided image data group in which the actual scene is the second highest score scene is the first highest score is “◯” or “Δ”, the CPU 101 proceeds to S1614. To do. On the other hand, if the layout compatibility when the scene classification result of the divided image data group in which the actual scene is the second highest score scene is the first highest score is “x”, the CPU 101 proceeds to S1616.

  In S <b> 1616, the CPU 101 classifies the divided image data group acquired in S <b> 1601 into a different scene type from the scene with the highest score. At this time, the type of scene into which the divided image data group is classified is determined to which case the divided image data group corresponds, and a table indicating the scene classification result in each case is referred to Determined by.

  As described above, according to the present embodiment, in the scene classification process, it is possible to suppress scene classification so that a layout result with low user satisfaction is output.

[Second Embodiment]
In the first embodiment, when the reliability of the result of scene classification scoring is low, an example of classifying divided image data groups into scenes determined according to the combination of the scene with the first score and the scene with the second score Explained. In the present embodiment, an example will be described in which divided image data groups are classified into “common” scenes when the reliability of the result of scene classification scoring is low. The description of the same configuration as that of the first embodiment is omitted.

  FIG. 17A is a table that summarizes the characteristics of the image data employed in the main slot and subslot of the template corresponding to each type of scene, which is used in the present embodiment. In any of the “travel”, “daily”, and “ceremony” scenes, there are many cases where a state where a plurality of people are together is photographed in commemoration. The image data (commemorative image data) taken in this way is often taken with the user's intention to leave it as a memorial. Therefore, even if the image data is laid out in the main slot of the template corresponding to any scene, Satisfaction is not greatly reduced. In addition, since commemorative photographed image data is often photographed including the surrounding situation, there are few cases where the photographed image data is a human image. Therefore, in this embodiment, a “common” scene is added as a scene type. The feature of the image data employed in the main slot of the “common” scene is that a plurality of people are included and the people are not up. Further, in any of the “travel”, “daily”, and “ceremony” scenes, there are cases in which image data of a person up or image data of a person's profile is captured. At this time, if only the image data having the image feature of the commemorative photograph data is laid out in one template, a monotonous layout result without change is obtained. Therefore, the characteristics of the image data adopted for the sub-slot of the “common” scene are that the person is up and that the person's profile is reflected. Note that the characteristics of the image data adopted for the “common” main slot and sub-slot are not limited to the above example, and arbitrary characteristics that do not greatly reduce the user satisfaction in any scene are set. May be.

  In the present embodiment, the table shown in FIG. 17B is used as a table indicating the scene classification result in each case. The CPU 101 stores the table shown in FIG. 17B in advance in a storage area such as the ROM 102 before scene classification processing. Note that the table shown in FIG. 17B may be incorporated in advance in the album creation application program, or may be incorporated in a program outside the album creation application.

  Next, the scene classification process executed in this embodiment will be described. In the present embodiment, the flowchart shown in FIG. 18 is executed. The flowchart shown in FIG. 18 is realized, for example, when the CPU 101 reads out a program stored in the HDD 104 or the ROM 102 to the RAM 103 and executes it. Note that the processing shown in this flowchart corresponds to S407 in FIG. The processes before S407 in FIG. 4 are the same as those in the first embodiment. Further, the processing of S1801 to S1815 is the same as the processing of S1601 to S1615, and thus the description thereof is omitted. Further, in S1806, the score of the divided image data group is assigned to “travel”, “daily life”, and “ceremony” as in the first embodiment. However, the score for the “common” scene is not calculated.

  In S1813, the CPU 101 compares the score difference between the first and second scene types with the threshold acquired in S1811. If the CPU 101 determines that the score difference is equal to or greater than the threshold value as a result of the comparison, the CPU 101 proceeds to S1814 and classifies the divided image data group acquired in S1801 into the scene with the highest score. If the CPU 101 determines that the score difference is less than the threshold value, the CPU 101 proceeds to S1815.

  In step S1815, the CPU 101 classifies the divided image data group acquired in step S1801 into “common” scenes. For a scene whose scene classification result is “common”, in S410 of FIG. 4, scoring is performed so that the score of the image feature image shown in FIG. Note that the average value and the standard deviation of the feature values of each slot of the template corresponding to the “common” scene set in the slot feature table are performed in the same manner as “travel”, “daily life”, and “ceremony”. That is, first, among the image data classified into each type of scene, a plurality of image data determined to match the characteristics of the image data to be employed in the main slot are collected. In addition, a plurality of image data determined to match the image features to be employed in the subslot are collected. That is, image data matching the features shown in FIG. Then, the image analysis unit 204 extracts the feature quantity of the number of face images, the position of the face image, and the size of the face image in each collected image data. Then, an average value and a standard deviation of these feature values are obtained for the “common” scene and each slot type (main slot and subslot), and stored together with the album creation application program.

  The steps after S408 in FIG. 4 are the same as those described in the first embodiment. As described above, according to the present embodiment, in the scene classification process, it is possible to suppress scene classification so that a layout result with low user satisfaction is output.

[Other Embodiments]
In each of the above-described embodiments, a case has been described in which the scene classification scoring result has a low reliability, but the scene is classified into a different scene from the scene with the highest score. However, the present invention is not limited to this. That is, when the reliability of the result of scoring the scene classification is low, the setting value of the slot feature table used for scoring the image data may be changed instead of changing the scene to be classified. At this time, the average value and the standard deviation of the feature values used in scoring the image may be calculated based on image data having image features that do not greatly reduce the user satisfaction. The average value and the standard deviation are set for each scene of the scene classification results shown in FIGS. 15 and 17, for example. In this way, the score of image data having an image feature that does not significantly lower the user's satisfaction level is increased, so that it is possible to suppress the output of a layout result having a low user satisfaction level.

  If the reliability of the result of scene classification scoring is low, the user's satisfaction is not greatly reduced even if an erroneous determination occurs (the layout compatibility does not become “x”). The image selection may be performed. In this case, the CPU 101 calculates not only the score corresponding to the classified scene but also the score corresponding to each scene for the image data.

  In each of the above-described embodiments, the layout compatibility has been described with an example in which the image data laid out in the main slot corresponding to each scene is compared. However, the image data laid out in the subslot is compared. You may evaluate by. Or you may evaluate by both.

  Moreover, although the example which determines the reliability of scene classification scoring by referring to the score difference between the first-ranked scene and the second-ranked scene has been described above, the present invention is not limited to this. For example, the determination may be made by referring to the absolute value of the score of the scene with the highest score. In this case, for example, the CPU 101 compares the score of the first scene with a threshold value set in advance in a program or the like. Then, the CPU 101 determines that the reliability of scene classification scoring is high if the first-ranked score is equal to or greater than the threshold value, and determines that the reliability of scene classification scoring is low if it is less than the threshold value. This is because there is a high possibility that a misjudgment has occurred if the score of the first-ranked scene is extremely low, regardless of the difference between the first-ranked scene and the second-ranked scene. Because. If it is determined that the reliability of scene classification scoring is low by this method, the scene to be classified may be determined according to the layout compatibility between scenes with higher scores as in the above-described embodiment. , May be classified as “common” scenes. It should be noted that a determination method that refers to the difference between the score of the first-ranked scene and the second-ranked scene may be combined with a determination method that refers to the absolute value of the score of the first-ranked scene.

  In the above-described embodiment, the example in which the scene classification processing according to the present invention is performed for each divided image data group has been described. However, the present invention is not limited to this mode. For example, the mode may be performed for each piece of image data. . In this case, after the individual image data classified into scenes are collected for each scene, the layout is executed.

  In the above-described embodiment, an example in which the scene classification process according to the present invention is applied to the automatic layout function has been described, but the present invention is not limited to this form. For example, the scene classification processing according to the present invention may be applied to a function or the like that automatically distributes scene classified image data to folders corresponding to each scene.

  In the above-described embodiment, evaluation is performed by assigning points in evaluations such as scene classification scoring and image scoring. However, the present invention is not limited to this form. That is, the evaluation may be performed without using the score. For example, the evaluations “◎”, “○”, “△”, and “×” are given in the descending order of the evaluation, and the scene classification or image is given by the attached evaluation. It may be a form in which selection or the like is performed.

  In the above-described embodiment, an example of creating an album by automatic layout has been described. However, the present invention can also be applied to a form of creating a single-leaf printed matter. In this case, for example, the CPU 101 may perform processing by regarding one spread in the above-described embodiment as one single-leaf printed matter. In addition, since a single-leaf printed material generally has only one image data layout, it is assumed that a single-leaf printed material template has only one slot. This slot may be regarded as a main slot or a subslot. As long as the number of printed materials for each scene can be set, the CPU 101 may perform processing by regarding one spread in the above-described embodiment as a set number of single-sheet printed materials. In this case, for example, the slot in the template of the first single-sheet printed material may be regarded as the main slot, and the slot in the template of the first single-sheet printed material as the sub-slot.

  The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  101 CPU: 102 ROM: 103 RAM: 216 Automatic layout processing unit

Claims (25)

An information processing device that classifies image data into one of a plurality of scenes,
Evaluation means for evaluating the scene included in the plurality of scenes with respect to the image data,
An acquisition unit that acquires a first scene having the highest evaluation among the plurality of scenes based on the evaluation by the evaluation unit;
Whether to classify the image data into the first scene or classify the image data into a scene that is not the first scene among the plurality of scenes based on the evaluation of the first scene acquired by the acquisition unit Determining means to determine,
Program to function as. The determining means determines the image data when the difference between the evaluation of the first scene and the evaluation of the second scene having the second highest evaluation among the plurality of scenes is greater than a predetermined threshold. If the difference between the evaluation of the first scene and the evaluation of the second scene is equal to or less than the predetermined threshold, the image data is classified as a scene that is not the first scene. Decide to do,
The program according to claim 1. The determining unit determines to classify the image data into the first scene when the evaluation of the first scene is larger than a predetermined threshold, and when the evaluation of the first scene is equal to or lower than the predetermined threshold. Determining to classify the image data into a scene that is not the first scene;
The program according to claim 1. The determining means is based on the evaluation of the first scene acquired by the acquiring means and the combination of the first scene and the second scene having the second highest evaluation among the plurality of scenes. Determining whether to classify the image data into the first scene or to classify the image data into a scene that is not the first scene of the plurality of scenes;
The program according to claim 1. The determining means determines the image data when the difference between the evaluation of the first scene and the evaluation of the second scene having the second highest evaluation among the plurality of scenes is greater than a predetermined threshold. Deciding to classify into the first scene,
When the difference between the evaluation of the first scene and the evaluation of the second scene is equal to or less than the predetermined threshold, the image data is converted into the first scene based on a combination of the first scene and the second scene. Determining whether to classify or classify the image data into a scene that is not the first scene of the plurality of scenes;
The program according to claim 4. The determining unit determines to classify the image data into the first scene when the evaluation of the first scene is larger than a predetermined threshold, and when the evaluation of the first scene is equal to or lower than the predetermined threshold. Further, based on a combination of the first scene and the second scene, the image data is classified into the first scene, or the image data is classified into a scene that is not the first scene among the plurality of scenes. To decide,
The program according to claim 4. The scene that is not the first scene is a second scene having the second highest evaluation after the first scene among the plurality of scenes, or a third scene that is not evaluated by the evaluation unit and is not the second scene.
The program according to any one of claims 1 to 6, wherein: An analysis means for analyzing the feature amount of the image data;
The evaluation means evaluates a scene included in the plurality of scenes with respect to the image data based on the feature amount analyzed by the analysis means.
The program according to any one of claims 1 to 7, characterized in that:   The program according to claim 8, wherein the feature amount includes at least one of a photographing period of the image data and a number of photographers included in the image data. The evaluation means evaluates the image data by assigning a score for each scene included in the plurality of scenes,
The acquisition unit acquires a first scene having the highest score assigned by the evaluation unit among a plurality of scenes.
The program according to any one of claims 1 to 9. The evaluation means performs an evaluation on scenes included in the plurality of scenes for an image data group including a plurality of image data,
The determining unit classifies the image data group into the first scene based on the evaluation of the first scene acquired by the acquiring unit, or classifies the image data group into the first scene of the plurality of scenes. Decide whether to classify into a non-scene,
The program according to any one of claims 1 to 10, wherein: From the plurality of image data classified into any one of the plurality of scenes according to the determination by the determining means, the selected image data is laid out in a template based on the scene into which the plurality of image data is classified. Layout means,
The program according to claim 1, further causing the information processing apparatus to function. An information processing apparatus for classifying image data into one of a plurality of scenes,
Evaluation means for evaluating the scene included in the plurality of scenes for the image data;
Based on the evaluation by the evaluation means, an acquisition means for acquiring a first scene having the highest evaluation among the plurality of scenes;
Whether to classify the image data into the first scene or classify the image data into a scene that is not the first scene among the plurality of scenes based on the evaluation of the first scene acquired by the acquisition unit A determination means for determining
An information processing apparatus comprising: The determining means determines the image data when the difference between the evaluation of the first scene and the evaluation of the second scene having the second highest evaluation among the plurality of scenes is greater than a predetermined threshold. If the difference between the evaluation of the first scene and the evaluation of the second scene is equal to or less than the predetermined threshold, the image data is classified as a scene that is not the first scene. Decide to do,
The information processing apparatus according to claim 13. The determining unit determines to classify the image data into the first scene when the evaluation of the first scene is larger than a predetermined threshold, and when the evaluation of the first scene is equal to or lower than the predetermined threshold. Determining to classify the image data into a scene that is not the first scene;
The information processing apparatus according to claim 13. The determining means is based on the evaluation of the first scene acquired by the acquiring means and the combination of the first scene and the second scene having the second highest evaluation among the plurality of scenes. Determining whether to classify the image data into the first scene or to classify the image data into a scene that is not the first scene of the plurality of scenes;
The information processing apparatus according to claim 13. The determining means determines the image data when the difference between the evaluation of the first scene and the evaluation of the second scene having the second highest evaluation among the plurality of scenes is greater than a predetermined threshold. Deciding to classify into the first scene,
When the difference between the evaluation of the first scene and the evaluation of the second scene is equal to or less than the predetermined threshold, the image data is converted into the first scene based on a combination of the first scene and the second scene. Determining whether to classify or classify the image data into a scene that is not the first scene of the plurality of scenes;
The information processing apparatus according to claim 16. The determining unit determines to classify the image data into the first scene when the evaluation of the first scene is larger than a predetermined threshold, and when the evaluation of the first scene is equal to or lower than the predetermined threshold. Further, based on a combination of the first scene and the second scene, the image data is classified into the first scene, or the image data is classified into a scene that is not the first scene among the plurality of scenes. To decide,
The information processing apparatus according to claim 16. The scene that is not the first scene is a second scene having the second highest evaluation after the first scene among the plurality of scenes, or a third scene that is not evaluated by the evaluation unit and is not the second scene.
The information processing apparatus according to claim 13, wherein the information processing apparatus is an information processing apparatus. An analysis means for analyzing the feature amount of the image data;
The evaluation means evaluates a scene included in the plurality of scenes with respect to the image data based on the feature amount analyzed by the analysis means.
The information processing apparatus according to claim 13, wherein the information processing apparatus is an information processing apparatus.   The information processing apparatus according to claim 20, wherein the feature amount includes at least one of a photographing period of the image data and a number of photographers included in the image data. The evaluation means evaluates the image data by assigning a score for each scene included in the plurality of scenes,
The acquisition unit acquires a first scene having the highest score assigned by the evaluation unit among a plurality of scenes.
The information processing apparatus according to any one of claims 13 to 21. The evaluation means performs an evaluation on scenes included in the plurality of scenes for an image data group including a plurality of image data,
The determining unit classifies the image data group into the first scene based on the evaluation of the first scene acquired by the acquiring unit, or classifies the image data group into the first scene of the plurality of scenes. Decide whether to classify into a non-scene,
The information processing apparatus according to claim 13, wherein the information processing apparatus is an information processing apparatus. From the plurality of image data classified into any one of the plurality of scenes according to the determination by the determining means, the selected image data is laid out in a template based on the scene into which the plurality of image data is classified. Layout means,
The information processing apparatus according to claim 13, further comprising: A control method executed in an information processing apparatus for classifying image data into one of a plurality of scenes,
An evaluation step for evaluating the scene included in the plurality of scenes with respect to the image data;
Based on the evaluation in the evaluation step, an acquisition step of acquiring a first scene having the highest evaluation among the plurality of scenes;
Whether to classify the image data into the first scene or classify the image data into a scene that is not the first of the plurality of scenes based on the evaluation of the first scene acquired in the acquiring step A determination step for determining
A control method characterized by comprising:

Images