JP2008271043A - Photo album preparation apparatus, photo album preparation method, program, and storage medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prepare a photo album by editing images according to preference while viewing how all objects are treated. <P>SOLUTION: In the photo album preparation apparatus, the display area of each object in each image is calculated by applying image recognition processing to all the images fetched into the photo album, and object information based on the display area in which each image is laid out, the display area of the object in the image and the number of the images wherein the object appears in the photo album is displayed. Thus, how each object is used in the photo album is displayed to a user during editing. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a photo album creation apparatus, a photo album creation method, a program, and a storage medium that electronically create a photo album using image data.

  In recent years, with the widespread use of digital cameras, it is possible to easily and smoothly edit and create a photo book using images taken by a photographer on a personal computer owned by the photographer. As a device for this purpose, many creation means such as an application program on a personal computer and a photo collection creation order system using a network service have been developed and proposed.

  In particular, many photo books are created for groups with multiple subjects, such as graduation photos and travel photos. Or ordered. However, in a photo book for this group, it is required to treat all subjects equally in the photo book without biasing to a specific subject. Further, in a photo book for such a group, there are cases where it is required to focus on a specific subject in the photo book.

  Conventionally proposed electronic photo album creating apparatuses detect a specific subject among a plurality of subjects of an image by image recognition processing. Thus, there is a method of calculating “the number of appearances of a specific subject” and “the total number of subjects appearing in the image”, and selecting an image including the specific subject or an image including many subjects. .

  In addition, in other conventionally proposed photo album creation devices, as a means of creating a photo album containing many images highlighting a specific subject, a specific subject is sent to a contractor who performs the photo album creation work. Is associated with the personal information of the specific subject. Then, there is a method of preferentially selecting an image related to a specific subject by an editor who is a contractor based on the association data (see, for example, Patent Document 1).

In addition, in another electronic photo book creation device that has been proposed in the past, a digital camera is distributed in advance to each individual subject as a means for automatically classifying images related to the specific subject. To do. There is a method of automatically selecting an image depending on a specific subject by selecting an image according to attribute information such as model information of a digital camera included in a photographed image taken by an individual who is a specific subject. (For example, refer to Patent Document 2).
JP 2002-042145 A JP 2003-018513 A

  As described above, in the conventionally proposed electronic photo album creation apparatus, means for creating a photo album that focuses on a specific subject is provided. However, it has not been possible to create a photo book that handles a plurality of subjects equally or a photo book that has been edited to select images in which each subject exists in a more nearly equal state.

  Further, when creating a photo book that focuses on a specific subject, the determination is made only by the number of images including the specific subject as a guideline for editing. Therefore, even if the number of images is large, there is a possibility that the display area of a specific subject in each image is small, and a photo book in which the desired specific subject is not so conspicuous is created.

  In addition, in the case of a photo album creation method in which all images to be captured in a photo album are determined in advance, the number of images including each subject is also determined in advance. Therefore, there is a problem that it is not possible to create a photo book that treats all subjects equally or a photo book that focuses on a specific subject without being biased toward a specific subject.

  Therefore, an object of the present invention is to create a photo book by editing it so as to be in a desired state while viewing the state of handling of all subjects.

In order to achieve the above object, a photo album creating apparatus according to an embodiment of the present invention includes:
An image selecting means for selecting an image, a template selecting means for selecting a photo album template, a layout means for applying the image selected by the image selecting means to the photo album template selected by the template selecting means, and the layout means An editing means for editing the generated photo book, and a photo book creating device comprising:
Image recognition means for recognizing a face image of a subject included in the image selected by the image selection means;
Feature extraction means for extracting features of the face image of the subject recognized by the image recognition means;
Comparing the features of the face image of the subject extracted by the feature extraction unit between the different images, and a subject recognition unit for recognizing the same subject,
Subject information management means for managing all subject information of the subject included in an image used in the photo album recognized by the subject recognition means, and all subject information of the subject managed by the subject information management means. A calculating means for calculating;
The image processing apparatus further includes display means for displaying all subject information of the subject.

In order to achieve the above object, a method for controlling a photo album creating apparatus according to another embodiment of the present invention includes:
An image selecting means for selecting an image, a template selecting means for selecting a photo album template, a layout means for applying the image selected by the image selecting means to the photo album template selected by the template selecting means, and the layout means An editing means for editing the generated photo album, and a control method of the photo album creating device comprising:
An image recognition step for recognizing a face image of a subject included in the image selected by the image selection means;
A feature extraction step of extracting features of the face image of the subject recognized in the image recognition step;
Comparing the features of the face image of the subject extracted in the feature extraction step between the different images, and a subject recognition step of recognizing the same subject,
A subject information management step that manages all subject information of the subject that is recognized in the subject recognition step and that is present in an image used in the photo book; and all subject information of the subject that is managed in the subject information management step. A calculation process of subject information to be calculated;
And a display step for displaying all subject information of the subject.

  According to the photo album creation apparatus according to the present invention, it is possible to create a photo album by editing it so as to be in a desired state while viewing the handling state of all subjects.

  Hereinafter, a photo album creating apparatus according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a diagram showing a hardware configuration of a photo album creating apparatus 100 according to an embodiment of the present invention. The photo album creating apparatus 100 can be configured by an information processing apparatus such as a personal computer, for example. The photo album creating apparatus 100 manages images in association with photo albums and plays a role of creating photo albums using these images.

  In FIG. 1, reference numeral 101 denotes an LCD (Liquid Crystal Display) used as a display device. The display screen of the LCD 101 displays, for example, a document being edited, a graphic, an image, and other editing information, icons, messages, menus, and other user interface information. 1 shows an example in which the LCD 101 is used in the photo album creating apparatus 100 according to the embodiment of the present invention. However, the present invention is not limited to this, and is a CRT display, a plasma display, an EL (Electro Luminescent) display, or the like. Also good.

  Reference numeral 102 denotes a VRAM (Video RAM) that accumulates and generates an image to be displayed on the display screen of the LCD 101. The image generated in the VRAM 102 is transferred to the LCD 101 in accordance with a predetermined rule, whereby the image is displayed on the LCD 101.

  Reference numeral 103 denotes a BMU (Bit Move Unit) which controls data transfer between memories, for example, between the VRAM 102 and another memory, and data transfer between the memory and each I / O device, for example, the network interface 111.

  A keyboard 104 has various keys for inputting documents, instructions, and the like. Reference numeral 105 denotes a PD (Pointing Device), which is used, for example, for designating icons, menus, and other objects displayed on the display screen of the LCD 101.

  Reference numeral 106 denotes a CPU (Central Processing Unit). The CPU 106 controls each block connected via the bus 112 based on a control program stored in a ROM (Read Only Memory) 107 or an HDD (Hard Disk Drive) 109. The ROM 107 holds various control programs and control data. The CPU 106 executes a photo album creating apparatus 100 shown in FIGS. 2A and 2B, which will be described later, and various program modules included therein.

  Reference numeral 108 denotes a RAM (Random Access Memory), which has a work area of the CPU 106, a data save area during error processing, a control program load area, and the like.

  The HDD 109 stores control programs executed by the photo album creation apparatus 100 and content files such as image files and text files. For example, the HDD 109 stores an OS (Operating System), photobook data, image files, a photobook creation program, various tables for managing various types of information, and the like.

  Reference numeral 110 denotes an FDD (Flexible Disk Drive), which controls access to the flexible disk. Reference numeral 111 denotes a NETI / F (Network Interface), which can communicate with other servers, information processing apparatuses such as personal computers, and peripheral devices such as printers. The provision of the control program to the CPU 106 can be performed from any of the ROM 107, the HDD 109, and the FDD 110, and can also be performed from another information processing apparatus or the like via the NETI / F 111.

<Embodiment 1>
FIG. 2A is a block diagram showing a program configuration corresponding to the basic functions of the photo album creating apparatus 100 according to Embodiment 1 of the present invention. In FIG. 2A, reference numeral 200 denotes an editing program that is an application program body of the photo album creating apparatus 100. Reference numeral 201 denotes photo album template data, which is a set of photo album template data for determining a layout used for layout processing executed by the editing program 200, and is stored in the HDD 109, for example. Then, when the editing program 200 is executed, it is read from the HDD 109 as needed, expanded in the RAM 108 and used.

  An image DB (Data Base) 202 that captures an image used in the editing program 200 is prepared by being recorded in the same HDD 109, FDD 110, and the like. An image recognition module 203 is a program module having a facial image recognition function. The image recognition module 203 recognizes a face from the read image data and performs feature extraction under the control of the editing program 200. The image recognition module 203 is a subject image recognition module 203 and is also called a specific subject recognition module.

  Reference numeral 204 denotes an image unit subject information table, which is prepared for each image unit to be captured, and is developed and used on the RAM 108 while the editing program 200 is activated. Reference numeral 205 denotes a subject information management table used by the editing program 200. One subject information management table is developed on the RAM 108 and used by the editing program 200 for one photo book. Reference numeral 206 denotes photo album data. During editing of a photo album, one photo album data 206 for each photo album is developed on the RAM 108 and updated as needed by editing processing by the editing program 200. Finally completed photo album data 206 is stored.

  Reference numeral 207 denotes a photo album ordering module for ordering photo albums. The photo album order module 207 converts the completed photo album data 206 that has been completed and stored in accordance with an instruction from the editing program 200 into a data format that can be transmitted, and uploads it to the photo album order server 209 via the Internet 208. The photo album order data uploaded to the photo album order server 209 is sent to the print site 210 after processing such as necessary charging and delivery arrangement, and an actual photo album output process is performed. The output photo album is delivered to the designated destination by the delivery means designated for the photo album ordering server 209. When an editor who is a contractor executes the photo album creating apparatus 100, it may be distributed to the end user as photo album data 206 instead of printed matter.

  FIG. 2B shows basic modules included in the editing program 200. That is, the editing program 200 basically includes an image selection module 251, a display module 252, a template selection module 253, a layout module 254, a subject information management module 255, and a calculation module 256. The editing program 200 further includes an editing module 257, a page layout module 258, an image processing module 259, and a subject designation module 260. Not only the first embodiment but also the second and third embodiments described below are shown in FIG. 2B together with the image recognition module 203 and the photo album order module 207 when the editing program 200 is executed. Each module operates in cooperation with each other. Naturally, each of the modules described above is for executing functional processing such as page designation, page layout selection, and page layout generation as processing functions in the operations of the following embodiments. These modules function according to a program that controls the CPU 106.

  Next, a procedure for creating an electronic photo book by executing the editing program 200 stored in the HDD 109 under the control of the CPU 106 of the photo book creating device 100 will be described in detail with reference to FIG.

  When the user starts a new photo album creation process in step S301, the CPU 106 causes the LCD 101 to display an image selection screen 401 described later with reference to FIG. While viewing the image selection screen 401, the user operates the keyboard 104 and the PD 105 to execute the capture of the image file recorded in the HDD 109 or the FDD 110. In this case, of course, it is possible to import from another information processing apparatus. This process is executed by the image selection module 251.

  When the CPU 106 finishes taking in the image file, the process proceeds to step S302, and the subject information management table 205 is created on the RAM 108. This is executed by the subject information management module 255. Subsequently, in step S303, the CPU 106 reads the photo album template data from the photo album template data 201, which is a template used for the photo album, and displays the photo album template selection screen 801 on the LCD 101. The user operates the keyboard 104 and PD 105 while viewing the photo album template selection screen 801 as shown in FIG. 8 to select a desired photo album template from the photo album template data 201 registered in the HDD 109. This is controlled by the template selection module 253 and the display module.

  When the selection of the images constituting the photo album and the photo album template is completed, the process advances to step S304, and the CPU 106 executes automatic layout processing to determine the temporary page layout of all the pages of the photo album. The result of the temporary page layout is developed on the RAM 108 and recorded on the HDD 109 for temporary storage as an intermediate file. This is performed by the layout module 254.

  Next, in step S305, an editing process operation by the user is accepted, and the editing process instructed by the user is executed. Here, the editing processing that can be performed on the photo album includes image change processing such as page layout change processing, image addition processing, image deletion processing, and image replacement processing on each page, and image display. For example, area resize processing, image trimming processing, and the like. These are executed by the editing module 257 and the image processing module 259.

  When editing of the photo album is completed and the photo album data 206 is completed, the process proceeds to step S306 to generate photo album order data for print output, and the photo album order data is transmitted in step S307. For the transmitted photo album order data, the billing mode, transmission method, etc. are determined by the processing of the photo album order server 209 connected to the Internet 208. This process is executed in association with the photo album order module 207. Thereafter, photo album order data is transmitted to the print site 210 for output, and print output is executed. After that, a printed photo book, which is a product of the photo book, is delivered to a designated destination. However, the method is not particularly limited here. As described above, it is possible to distribute photo book data instead of printed matter.

  FIG. 4 is a diagram showing an example of the image selection screen 401 displayed on the display screen of the LCD 101 and used in step S301 in the first embodiment. The image selection screen 401 displays a file selection field 402 for displaying image file candidates to be selected and a file registration field 403 for displaying the selected image file.

  The file selection field 402 displays thumbnails 404 and 405 of image files from the contents of the designated folder location recorded in the HDD 109 or the FDD 110. The thumbnails 404 and 405 of these image files can be selected by the PD 105. For example, the thumbnail 404 of the selected image file is distinguished from the unselected thumbnail 405, as shown in FIG.

  In the file registration field 403, thumbnails 406 and the like registered as image files used in the photo album are displayed side by side. In order to register the thumbnail 405 of the image file selected in the file selection field 402 in the file registration field 403, it is executed by clicking the registration button 407 using the PD 105 after selecting the thumbnail 405 of the image file. The

  The button 408 is used to move to the next processing step. When the button 408 is clicked by the PD 105 or the like, the image capturing is terminated, and the process proceeds to step S302, which is the next processing step.

  FIG. 5 is a schematic diagram for explaining the process of creating the subject information management table 205 in step S302. In FIG. 5, reference numeral 501 denotes an arbitrary image to be captured. When the image 501 is captured, the CPU 106 controls the image recognition module 203 to perform the image recognition processing of the faces of the subjects A to D and create the image unit subject information table 502. This image unit subject information table 502 corresponds to the image unit subject information table 204 of FIG. In the image unit subject information table 502, the facial feature extracted feature value for each subject A to D and the ratio of the display area in the image are registered.

  By creating all similar image unit subject information tables 504 for all the images 503 captured in step S301, a subject information management table 505 for the entire photo album to be created is created. The subject information management table 505 corresponds to the subject information management table 205 in FIG. In the subject information management table 505, the feature values obtained by extracting facial features used as a reference for determining whether or not each of the subjects A to D is equivalent between different images, the number of included features, and images having different display areas are used. The sum is registered in the total display area 506. At the same time, the number of images 501 in which a subject exists is also registered in the appearance number 507. These are executed by the subject information management module 255 and the calculation module 256.

  Next, the detailed operation of the photo album creating apparatus 100 according to the first embodiment of the present invention will be described with reference to the flowchart of FIG. According to the flowchart of FIG. 6, the editing program 200 executes a process of determining whether or not a specific subject is photographed equally between different images included in the photo album.

  In step S601 after the start of the processing, the check loop is repeated for the image unit subject information table 502, and whether or not the image unit subject information table 502 has been created for all subjects A to D is determined. Check against. If there is no image unit subject information table 502 including an unrecognized subject, the processing here ends. However, if there is an image unit subject information table 502 including an unrecognized subject, the process proceeds to step S602, and all images 503 are examined with respect to all image unit subject information tables 504 to check whether or not there is an unrecognized subject. .

  In step S603, whether or not the subject subject has already been registered in the subject information management table 505 is determined based on whether or not the difference between the feature values extracted by the face image recognition processing is within an allowable range. If this feature value is within a certain allowable range, it is treated as the same subject in step S604, and information on the subject is added and registered as information on the subject information management table 505 as an image change. If it is outside the allowable range, it is treated as a different subject in step S605, and is newly registered in the subject information management table 505 as a new subject. In this way, by comparing the feature values for all the images 503, it is possible to recognize the equivalence of all the subjects, for example, A to D in the photo album.

  As described above, the subject information management table 505 related to all the captured images A to D can be created. However, when actually laying out, it is necessary to consider the difference in display area of each image.

  FIG. 7 is a schematic diagram for explaining a method of calculating the total display area between different images after layout. In FIG. 7, reference numeral 701 denotes an example of a temporary layout state of a certain page, and the respective images are arranged at different display magnifications.

  In the image unit subject information table 702 for this image, a display area converted using the display magnification of the image for all subjects A to D existing in the corresponding image is additionally recorded as a converted area 703. . By performing similar calculation processing on other images on the page and images on other pages, the total subject information management table 704 through all the images 503 is corrected.

  Next, the process of selecting a photo album template to be a template for a photo album in step S303 in FIG. 3 will be described in detail using a photo album template selection screen 801 shown in FIG. In FIG. 8A, a photo album template selection screen 801 is a screen for selecting a photo album template, and is displayed on the LCD 101 by the CPU 106. While viewing the photo album template selection screen 801, the user can select a desired photo album template data 201 recorded in the HDD 109 and registered, for example, by operating the keyboard 104 or PD 105.

  A selection area 802 displays thumbnails and names of selectable photo album templates. A thumbnail of each photo album template displayed in 802 can be selected by the PD 105 or the like. Reference numeral 803 is a display example of thumbnails of a photo album template TA in a selected state, 804 is a display example of thumbnails TF of a photo album template in a non-selected state, and 803 is visually highlighted to be in a selected state. expressing. The thumbnail TA of the selected photo album template is changed to a non-selected state by selecting another thumbnail.

  Reference numeral 805 denotes a mode switching button for performing detailed mode display on a photo album template selection screen as a photo album template selection result to be described later. When the mode switching button 805 is pressed, the screen display state is changed to a detailed mode indicating the result of the temporary layout shown on the photo album template selection screen 807 in FIG. Reference numeral 806 denotes a button for transitioning to the next processing step. The button 806 is used to execute an automatic layout process using the photo album template 803 selected in the photo album template selection area 802 by designating in the PD 105. The process proceeds to step S304.

  As described above, reference numeral 807 in FIG. 8B is a detailed mode display state of the photo album template selection screen displayed when the mode switching button 805 is selected. Reference numeral 808 denotes a detailed display area that displays the number of used images of the subjects A to F and the total display area of each subject when the thumbnails TA to TE of each photo album template are selected. In the table 809, subjects A to F are designated on the vertical axis, and thumbnails TA to TE of the photo album template are designated on the horizontal axis. The numerical values displayed in the table 809 are relative values normalized in the photo album template, and it is not necessary to specify the unit system. Table 810 shows the calculation result of the degree of variation in the total display area of each subject when each photo album template is selected, and uses standard deviation or the like. As the standard deviation value is smaller, the variation is smaller, that is, the degree of uniformity in which a specific subject is used is higher, and a guideline is provided to the user.

  The previous candidate button 811 and the next candidate button 812 are buttons for displaying a temporary layout result of another photo album template. When selected, the state of the result of automatic layout using another photo album template is displayed. To do.

  Reference numeral 813 denotes a mode switching button similar to 805, and when pressed, the screen display state is changed to a standard mode photo album template selection screen indicated by reference numeral 801 in FIG. Reference numeral 814 denotes a button for transitioning to the next step, as in 806. By selecting the button 814, the selected photo album template is applied, and the process proceeds to step S304 of the automatic layout process. FIG. 9 shows an example of the data structure of the photo album template data 201 in this embodiment. In FIG. 9, reference numeral 901 denotes header information for managing the photo album template data 201. The header information stores information to be prepared for each photo album template, such as the number of layouts prepared for the photo album template, the number of images that can be captured in the photo album, and the number of images that can be arranged for each page.

  Reference numeral 902 denotes cover layout management information, which stores information necessary for determining the cover layout of a photo album. Reference numerals 903, 904, and 905 denote information blocks for managing the layout in which one, two,..., N images are arranged on each page in the photo book. I manage. Reference numerals 906, 907, 908, and 909 denote data blocks for storing a data group of layout information for determining the layout of the cover layout, the one-body layout, the two-sheet layout, and the N-sheet layout, respectively. By changing the layout information used here, the layout in the page of the photo album can be changed.

  The photo album template data 201 having the data structure shown in FIG. 9 can be used as data temporarily expanded on the RAM 108 or data loaded from the HDD 109.

  Next, an editing screen displayed on the LCD 101 after the automatic layout process is executed in step S304 will be described.

  FIG. 10 shows an example of an editing screen of the editing program 200 of the photo album creating apparatus according to the first embodiment of the present invention. In FIG. 10, reference numeral 1001 denotes a main window which is a display screen of an edit screen displayed on the LCD 101. Reference numeral 1002 denotes an editing area, which displays the state of a double-page spread during editing of a photo book and can accept an editing operation instruction from the user's PD 105. An operation area 1003 provides a GUI for designating an operation desired by the user to the editing area 1002. Reference numeral 1004 denotes a layout change button, which displays a layout change screen to be described later and accepts a layout change of the corresponding page.

  Reference numeral 1005 denotes an image addition button, which enables an image to be added and inserted on a page with a photo album by opening an image addition screen described later. Reference numeral 1006 denotes an image deletion button, which makes it possible to delete a designated image on a page of a photo album by opening an image deletion screen described later. Reference numeral 1007 denotes an image editing button, which enables editing processing such as trimming processing and resizing processing of a designated image by opening an image editing screen described later. In this way, the image change process is executed.

  Reference numerals 1008 and 1009 denote a previous page display button and a next page display button, respectively, and the display page of the photo album can be changed by selecting each with the PD 105. The page number of the photo album displayed in the editing area 1002 is displayed at 1010. In the illustrated example, it is displayed that pages 4 and 5 of the photo book being edited are open.

  Reference numeral 1011 denotes an order button, which is selected by the PD 105, and transitions to step S306 in FIG. Then, photo album order data is created, and the process further proceeds to step S307 to start transmitting photo album order data to the photo album order server 209. Here, description of the processing in the subsequent steps is omitted.

  For convenience, the editing functions described in the first embodiment are layout change, image addition, image deletion, and image editing. However, editing functions provided by general document editing applications such as character editing and graphic editing are provided. It is also possible to apply. Note that these processes are performed mainly by the editing module 257, and necessary modules included in the editing program 200 are selected and executed.

  FIG. 11 is an example of a page layout change screen displayed on the display screen of the LCD 101 when the layout change button 1004 in FIG. 10 is pressed. This process is executed by the page layout module 258. In FIG. 11A, reference numeral 1101 denotes a page layout change screen, which is a sub-window of the edit screen 1001 that is a main window. The user can operate the keyboard 104 and PD 105 according to the page layout change screen 1101 to select and change the desired layout from the thumbnails LA to LJ of the page layout candidates of the corresponding photo book.

  A selection area 1102 displays thumbnails and names of layout candidates that can be selected. A thumbnail of each layout candidate displayed in 1102 can be selected by the PD 105 or the like. 1103 is a display example of a thumbnail LA of a layout candidate that is in a selected state, 1104 is a display example of a thumbnail LF of a layout that is in a non-selected state, and 1103 is visually highlighted to be in a selected state. Yes. The thumbnail of the selected layout is changed to a non-selected state by selecting another thumbnail.

  Reference numeral 1105 denotes a mode switching button for performing a detailed mode display of a layout selection result to be described later. Reference numeral 1106 denotes an apply button for actually applying the layout candidate selected in the selection area 1102 and executing layout change processing. Reference numeral 1107 denotes a cancel button for canceling application of the layout candidate. When the apply button 1106 and the cancel button 1107 are selected on the PD 105, each process is executed, the page layout change screen 1101 is closed, and the user's screen operation returns to the edit screen 1001.

  Reference numeral 1108 in FIG. 11B is a detailed mode display state of the page layout change screen displayed when the mode switching button 1105 is selected. A detail display area 1109 displays the number of used images of the subject and the total display area of the subject when each layout is selected. The subject A to F is plotted on the vertical axis of 1110, and the thumbnail thumbnail LA of the layout candidate is plotted on the horizontal axis. Thru LE names are displayed. The numerical value displayed in 1110 is a relative value normalized in the photo album template, and there is no need to specify the unit system. Reference numeral 1111 denotes a calculation result of the degree of variation in the total display area of each subject when each layout candidate is selected, and uses a standard deviation or the like. As the standard deviation value is smaller, the variation is smaller, that is, the degree of uniformity in which a specific subject is used is higher, and a guideline is provided to the user.

  The previous candidate button 1112 and the next candidate button 1113 are buttons for displaying the result when another layout is used, and the state of the result of automatic layout using another layout candidate when selected is set to 1110 indicate.

  Reference numeral 1114 denotes a mode switching button similar to 1105. When pressed, the screen display is changed to the page layout change screen 1101. The apply button 1115 and the cancel button 1116 have the same functions as the apply button 1106 and the cancel button 1107, respectively.

  FIG. 12 is an example of an image addition screen displayed on the LCD 101 when the image addition button 1005 is selected on the editing screen 1001. In FIG. 12, a screen 1201 is an image addition screen and a subwindow of the editing screen 1001.

  An image selection field 1202 displays thumbnails 1203 and 1204 of image files from the contents of designated folder positions in the HDD 109 or the FDD 110 and can be selected by the PD 105. The thumbnail 1203 of the selected image file is highlighted so as to be distinguished from that of the unselected thumbnail 1204.

  Reference numeral 1205 denotes an apply button for actually applying the image selected in the image selection field 1202 and executing layout change processing. By selecting with the PD 105, the image addition screen 1201 is closed and an image number change screen 1301, which will be described later. Is displayed.

  Reference numeral 1206 denotes a cancel button for canceling the application of the image file, and the image addition screen 1201 is closed by selecting with the PD 105, and the user's screen operation returns to the edit screen 1001.

  FIG. 13 shows an example of the image number change screen displayed after the image change calculation process is executed when the apply button 1205 in FIG. 12 or the image delete button 1006 in FIG. 10 is pressed. In the former case, the page layout change state after adding the image selected in the image selection field 1202 can be determined, and in the latter case, the page layout change state after deleting the image selected in the editing area 1002 can be determined. The image change calculation process is executed by the calculation module 256 under the control of the CPU 106, reflecting the changed result after each process.

  In the main window of FIG. 13A, reference numeral 1301 denotes an image number change screen, which is a subwindow of the edit screen 1001. The user can select and change a desired layout from among the layout candidates after changing the number of images of the corresponding page of the photo album by operating the keyboard 104 and PD 105 according to the image number change screen 1301. A selection area 1302 displays thumbnails and names of layout candidates that can be selected. The user can select a thumbnail of each layout displayed in the selection area 1302 using the PD 105 or the like. 1303 is a display example of a thumbnail of a layout in a selected state, 1304 is a display example of a thumbnail of a layout in a non-selected state, and 1303 is highlighted to visually express that it is in a selected state. The thumbnail of the selected layout is changed to a non-selected state by selecting another thumbnail.

  Reference numeral 1305 denotes a mode switching button for performing a detailed mode display of a layout selection result to be described later. Reference numeral 1306 denotes an apply button for actually applying the layout candidate selected in the selection area 1302 and executing the image number changing process. Reference numeral 1307 denotes a cancel button for canceling the application of the layout candidate. Each process is executed by selecting the apply button 1306 and the cancel button 1307 on the PD 105, the image number change screen 1301 is closed, and the user's screen operation returns to the edit screen 1001.

  Reference numeral 1308 in FIG. 13B denotes a detailed mode display state of the image number change screen displayed on the LCD 101 when the mode switching button 1305 is selected.

  A detail display area 1309 displays the number of images including each subject before and after the layout change, and the total display area of the subject. Reference numeral 1310 denotes a state display before the layout change. Table 1311 shows a state display after the layout change. The vertical axis indicates subjects A to F, and the horizontal axis indicates the names of layout candidates thumbnails LA to LC. The numerical values displayed as the total display areas 1310 and 1311 are relative values normalized in the photo album template being edited, and it is not necessary to specify the unit system.

  1310 also includes a calculation result of the degree of variation in the total display area of each subject when each layout candidate is selected. Numerical values such as standard deviation are used to indicate the degree of variation. The smaller the standard deviation value, the smaller the variation, that is, the higher the degree of uniformity that the subject is used, and provides a guide to the user. These are calculated by the calculation module 256.

  The previous candidate button 1312 and the next candidate button 1313 are buttons for displaying a result when another layout is used, and the state of the result of automatic layout using another layout candidate by selection is displayed in a table 1311. To display.

  Reference numeral 1314 denotes a mode switching button similar to 1305, and when pressed, changes the screen display from the image number change screen 1308 to the image number change screen 1301. The apply button 1315 and the cancel button 1316 have the same functions as the apply button 1306 and the cancel button 1307, respectively.

  FIG. 14 is an example of an image editing screen 1401 displayed on the display screen of the LCD 101 when the image editing button 1007 in FIG. 10 is pressed. The image editing described here is limited to image display size changing processing and image display area changing processing (so-called trimming processing), but it is possible to use other image editing applications such as image effect processing such as color tone change and character input. Possible processing can also be performed.

  In FIG. 14, reference numeral 1401 denotes an image editing screen, which is a subwindow of the editing screen 1001. The image editing screen 1401 includes two areas, a status display area 1402 and an image editing area 1403, and several buttons.

  The state display area 1402 is divided into a display area 1404 in a state before the image editing process and a display area 1405 in a state after the image editing process, and the subject is displayed by the image editing calculation process before and after the image editing process. Information can be calculated and its status compared. In display areas 1404 and 1405, subjects appearing in the photo album, the number of images in which the subject appears, the total display area occupied by each subject in the photo album, and variations in the current layout state (standard deviation values) are displayed. The The image editing calculation process is executed by the calculation module 256 according to the control of the CPU 106.

  The image editing area 1403 displays a target image and provides a resize frame 1406 and a trimming frame 1407 for the same image. By operating the PD 105 for each frame, image resize processing and Trimming processing can be executed. This processing is executed by the image processing module 259, and the result of the resizing and trimming image editing processing performed in the image editing area 1403 is immediately displayed in the status display area 1402. Therefore, the user can simultaneously check changes in values before and after the process by an operation of the image editing process. These are also executed in the image change calculation process under the control of the CPU 106.

  Reference numeral 1408 denotes an apply button for actually applying the result of the image editing process operated in the image editing area 1403. Reference numeral 1409 denotes a cancel button for canceling the image editing process. The application button 1408 and the cancel button 1409 are selected by the PD 105 to execute each process, the image editing screen 1401 is closed, and the user operation screen returns to the editing screen 1001.

  According to the photo album creating apparatus according to the first embodiment of the present invention described above, a photo album in which all subjects are treated equally while monitoring the degree of variation displayed, that is, the uniformity of the subject being used. Can be easily created.

<Embodiment 2>
The photo album creation apparatus according to the second embodiment of the present invention relates to an “equal mode” for facilitating the creation of a photo album in which subjects are treated equally, and the creation of a photo book in which specific subjects are handled with priority. A photo album creation apparatus having an “emphasis mode” for promoting the above will be described.

  In the second embodiment, a mode between “equal mode” and “priority mode” on the photo album template selection screen 807, the page layout change screen 1108, and the image number change screen 1308 of the photo album creating apparatus according to the first embodiment. It is set as the structure which can be switched. The selection candidates are rearranged according to each mode.

  FIGS. 15A and 15B are examples when the “emphasis mode” and the emphasis mode are designated on the photo album template selection screen 807, respectively. In FIG. 15A, the photo album template selection screen 1501 is a display mode in the equal mode, and is equivalent to the photo album template selection screen 807 in FIG. The display order of the photobook template candidates 1503 of 1502 is arranged in the order of the small value of the variation 1504.

  Reference numeral 1505 in FIG. 15A denotes a switching button, which can be switched from the “equal mode” to the “priority mode” by selecting with the PD 105. The switching button 1505 on the photo album template selection screen 1501 displays “equal mode”. Is selected.

  15B is a display state of the photo album template selection screen 1506 in the priority mode, and the display order of the photo album template candidates 1508 in the display content of 1507 is the display total area 1509 of the designated subject B. They are arranged in the most order. Reference numeral 1510 denotes a switching button for switching from the “priority mode” to the “equal mode”.

  Although description is omitted, the page layout change screen 1108 and the image number change screen 1308 can be similarly provided with a display screen including a switching button for switching from the “priority mode” to the “equal mode”.

  As described above, according to the photo album creating apparatus according to the second embodiment, it is possible to create a photo album that is used evenly or preferentially for an arbitrary subject designated by the user. it can.

<Embodiment 3>
As an example of a photo album creation apparatus according to Embodiment 3 of the present invention, an example of an embodiment that assists a user's editing operation by making a subject appear more visually will be described.

  Reference numeral 1601 in FIG. 16 denotes a display screen displaying thumbnails of face images of the subjects A to F on the photo album template selection screen 807 in FIG. A face thumbnail 1603 is displayed corresponding to the subject A 1602, thereby enabling the user to visually determine which subject is the subject.

  Although description is omitted, thumbnails of face images of the subjects A to F can be displayed in the same manner on the page layout change screen 1108, the image number change screen 1308, and the image edit screen 1401.

  As described above, according to the photo album creation apparatus according to the third embodiment, when editing a photo album, an editing operation can be executed in consideration of a specific subject.

<Embodiment 4>
As an example of a photo album creation apparatus according to a fourth embodiment of the present invention, an embodiment capable of giving a warning when the number of used specific subjects and the total display area occupied in the photo album are small will be described. .

  FIGS. 17A and 17B are display screens used when the subject information management table is created in step S302 after selecting an image in step S301 in FIG. That is, when there is an object whose display area is extremely smaller than the predetermined range value as a result of the editing operation in step S305 and the number of used sheets is extremely smaller than the predetermined range value, a subject warning screen is displayed. Are displayed on the display screen of the LCD 101.

  In FIG. 17A, reference numeral 1701 denotes a subject warning screen, which is displayed as a subwindow such as the editing screen 1001. Reference numeral 1702 denotes a warning area, which displays a subject with an extremely small number of images or a total display area. Reference numeral 1703 denotes a subject name, 1704 denotes the number of images including the subject, and 1705 denotes a column that displays the total area of the subject. These are calculated by the calculation module 256. When the confirmation button 1706 is selected, the subject warning screen 1701 is closed, and the screen display returns to the editing screen 1707 in FIG. On the editing screen 1707, an image 1708 including the subject is selectively displayed, which can be used as a guide for subsequent editing operations.

  It should be noted that by using a display screen having a configuration similar to that in FIGS. 17A and 17B, a subject whose number of used sheets or total display area is extremely small as a result of image selection in step S301 and editing operation in step S305. Similar warning processing can be realized even when there is. However, this detailed description is omitted here. The subject designation is executed by the subject designation module 260.

  As described above, according to the photo album creating apparatus according to the fourth embodiment, when editing a photo album, the number of specific subjects used and the total display area occupied in the photo album are more extreme than a predetermined range of values. It is possible to present warnings when there are few and many cases. As a result, it is possible to create a photo book with a high degree of uniformity that includes each identified subject.

<Embodiment 5>
In the photo album creating apparatus according to the fifth embodiment of the present invention, a method of using personal information such as a list of specific subjects and face information prepared in advance in a database will be described.

  FIG. 18 is a block diagram for explaining the configuration of the photo album creating apparatus according to the fifth embodiment. FIG. 18 is almost the same as the block of FIG. 2 except that a personal information DB (Data Base) 1811 is added. Reference numerals 200 to 210 are the same as those in FIG. 2, and detailed description thereof is omitted.

  In the personal information DB 1811, name list information of each person to be captured as a subject in the photo album to be created, a face image of each subject to be recognized by the image recognition module 203, and the like are recorded. The personal information DB 1811 determines whether the subject extracted from the image used in the photo book when the subject information management table is created in step S302 in the flowchart of FIG. Recognize in the recognition process. If it is determined that the subject name is included, the subject name can be determined. If it is determined that the subject name is not included, the subject information is deleted from the subject information management table so that extra subjects can be considered. It is possible to edit a photo book without having to.

  According to the photo album creating apparatus according to the fifth embodiment of the present invention described above, it is possible to edit a photo album without considering extra subjects.

  The present invention can also be implemented by supplying a storage medium storing software program codes for realizing the functions of the first to fifth embodiments to a system or apparatus. That is, the object of the present invention can also be achieved by reading and executing a program code stored in a storage medium by a computer (or CPU or MPU) of the system or apparatus. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. As a storage medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS or the like running on the computer performs actual processing based on an instruction of the program code. Some or all may be done. Thus, the present invention includes a case where the functions of the above-described embodiments are realized by the processing.

  Furthermore, in the present invention, the program code read from the storage medium can be written in a memory provided in a function expansion board inserted in the computer or a function expansion unit connected to the computer. Therefore, the function of the above-described embodiment is realized by performing part or all of the actual processing by the CPU provided in the function expansion board or function expansion unit based on the written program code instruction. Is also included.

It is a block diagram which shows the hardware constitutions of the electrophotographic photograph collection production apparatus which concerns on embodiment of this invention. It is a block diagram which shows the program structure of the electrophotographic photograph collection production apparatus which concerns on embodiment of this invention. It is a flowchart figure which shows the outline | summary of the process sequence of the electrophotographic photograph collection production apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the image selection screen displayed on the display apparatus of the electrophotographic album production apparatus which concerns on embodiment of this invention. It is a schematic diagram for demonstrating the procedure which produces the to-be-photographed object information management table of the electrophotographic photo album preparation apparatus which concerns on embodiment of this invention. It is a flowchart for demonstrating the procedure which produces the to-be-photographed object information management table of the electrophotographic photo album preparation apparatus concerning embodiment of this invention. It is a schematic diagram for demonstrating the procedure which calculates the display area of a to-be-photographed object from the to-be-photographed object information management table of the electrophotographic photo album preparation apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the photobook template selection screen displayed on the display apparatus of the electrophotographic photobook creation apparatus which concerns on embodiment of this invention. It is a block diagram for demonstrating an example of the data structure of the photograph book template of the electrophotographic book creation apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the edit screen displayed on the display apparatus of the electrophotographic album production apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the page layout change screen displayed on the display apparatus of the electrophotographic album production apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the image addition screen displayed on the display apparatus of the electrophotographic album production apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the image number change screen displayed on the display apparatus of the electrophotographic album production apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the image edit screen displayed on the display apparatus of the electrophotographic album production apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the photobook template selection screen displayed on the display apparatus of the electrophotographic photobook creation apparatus which concerns on the 2nd Embodiment of this invention. It is a figure which shows an example of the photobook template selection screen displayed on the display apparatus of the electrophotographic photobook creation apparatus which concerns on the 3rd Embodiment of this invention. It is a figure which shows an example of the subject warning screen and edit screen which are displayed on the display apparatus of the electrophotographic photo album preparation apparatus which concerns on the 4th Embodiment of this invention. It is a block diagram which shows the program structure of the electrophotographic album creating apparatus which concerns on the 5th Embodiment of this invention.

Explanation of symbols

101 LCD (Liquid Crystal Display)
102 VRAM (Video RAM)
103 BMU (Bit Move Unit)
104 Keyboard 105 PD (Pointing Device)
106 CPU (Central Processing Unit)
107 ROM (Read Only Memory)
108 RAM (Random Access Memory)
109 HDD (Hard Disk Drive)
110 FDD (Flexible Disk Drive)
111 NETI / F (Network Interface)
112 Bus 200 Editing Program 201 Photobook Template Data 202 Image DB (Data Base)
203 Image Recognition Module 204 Image Unit Subject Information Table 205 Subject Information Management Table 206 Photobook Data 207 Photobook Order Module 1811 Personal Information DB (Data Base)

Claims (22)

An image selecting means for selecting an image, a template selecting means for selecting a photo album template, a layout means for applying the image selected by the image selecting means to the photo album template selected by the template selecting means, and the layout means An editing means for editing the generated photo book, and a photo book creating device comprising:
Image recognition means for recognizing a face image of a subject included in the image selected by the image selection means;
Feature extraction means for extracting features of the face image of the subject recognized by the image recognition means;
Comparing the features of the face image of the subject extracted by the feature extraction unit between the different images, and a subject recognition unit for recognizing the same subject,
Subject information management means for managing all subject information of the subject included in an image used in the photo album recognized by the subject recognition means, and all subject information of the subject managed by the subject information management means. A calculating means for calculating;
A photo album creating apparatus, further comprising display means for displaying all subject information of the subject. All subject information of the subject is
Information on the number of images including each of the subjects used in the photo book;
Information on each display area of the subject included in the image used in the photo book;
The photo album creating apparatus according to claim 1, further comprising information on a total display area that is a sum of the display areas of each of the subjects.   3. The photo album creating apparatus according to claim 2, wherein the display unit rearranges and displays the subject information of each of the subjects based on the information on the total display area or the information on the number of sheets. The template selection means includes
The layout unit generates a photo album having a temporary layout using the candidate photo album template, and the calculation unit includes the subject of the subject included in the photo album having the temporary layout. Let the information be calculated,
3. The photograph according to claim 1, wherein the display unit displays the subject information included in the photo album having the temporary layout in association with the candidate photo album template. 4. Collection device. A designation unit for designating any one of the subjects;
5. The photo album creation apparatus according to claim 4, wherein the display unit rearranges and displays the candidate photo album templates based on information on the total display area of the designated subject. A designation unit for designating any one of the subjects;
The template selection unit causes the calculation unit to calculate variation in information of the display total area of the designated subject,
5. The photo album creating apparatus according to claim 4, wherein the display unit rearranges and displays the candidate photo album templates based on the variation in the information on the total display area. Page designation means for designating a page of the photo album to be edited;
Page layout selecting means for selecting a page layout of the photo album;
Page layout generation means for generating a plurality of temporary page layouts in which the page layout is changed without changing the number of images included in the page specified by the page specification means;
The editing means includes
The layout unit generates the page of the photo album using the temporary page layout, and the calculation unit acquires the subject information of the subject included in the page generated by the temporary page layout. Let me calculate
4. The photo album according to claim 1, wherein the display unit displays the subject information calculated by the calculation unit in correspondence with the plurality of temporary page layouts. 5. Creation device. A designation unit for designating any one of the subjects;
8. The photo album creating apparatus according to claim 7, wherein the display unit rearranges and displays the temporary page layout based on information on the total display area of the designated subject. A designation unit for designating any one of the subjects;
The editing unit causes the calculation unit to calculate variation in information of the display total area of the designated subject,
8. The photo album creating apparatus according to claim 7, wherein the display unit rearranges and displays the temporary page layout based on the variation in the information on the total display area of the designated subject. An additional means for adding an image to be used for the photo album;
A deletion means for deleting an image used for the photo album;
An image changing means for executing any processing by the adding means or the deleting means,
The editing means causes the calculation means to calculate the subject information from the photo album generated as a result of the processing executed by the image changing means,
The photo album creating apparatus according to claim 1, wherein the display unit displays the subject information.   The said display means displays the information of the said display total area before the process by the said image change means, and the information of the said display total area after the process by the said image change means, The display means of Claim 10 characterized by the above-mentioned. Photo book creation device. The editing unit causes the calculation unit to calculate variation in information of the total display area,
12. The photo album creating apparatus according to claim 10, wherein the display unit displays the variation before processing by the image changing unit and the variation after processing by the image changing unit. Image selecting means for selecting a specific image from images used for the photo album;
Resizing means for resizing the image selected by the image selecting means;
Trimming means for trimming the image selected by the image selection means,
The editing unit causes the resizing unit or the trimming unit to execute processing, and causes the calculation unit to calculate the subject information based on the image processed by the resizing unit or the trimming unit,
4. The photo album creating apparatus according to claim 1, wherein the display unit displays the subject information calculated by the calculation unit. 5.   14. The photograph according to claim 13, wherein the display means displays information on the total display area before processing by the editing means and information on the total display area after processing by the editing means. Collection device. The editing unit causes the calculation unit to calculate variation in information of the total display area,
14. The photo album creating apparatus according to claim 13, wherein the display unit displays the variation before processing by the editing unit and the variation after processing by the editing unit.   The photo album creating apparatus according to claim 1, wherein the display unit displays a thumbnail corresponding to the subject. A subject selection means for selecting the specific subject;
The display unit highlights and displays a frame of an image including the specific subject selected by the subject selection unit, and highlights a region of the face image of the subject in the highlighted frame. The photo album creating apparatus according to claim 1, wherein the photo album creating apparatus displays the image. Subject warning means for extracting the subject whose information on the total display area is smaller than a predetermined range and the subject whose information on the total display area is larger than the predetermined range;
2. The photo album creating apparatus according to claim 1, wherein the display unit highlights and displays the subject extracted by the subject warning unit. The subject information management means includes
Personal information registration means for registering personal information of the subject;
Personal information managing means for managing the personal information, and the same subject from the characteristics of the subject extracted by the feature extracting means and the personal information registered in the personal information managing means is connected to the recognizing means. To identify
The photo album creating apparatus according to claim 1, wherein the display unit displays the personal information of the subject specified by the recognition unit. An image selecting means for selecting an image, a template selecting means for selecting a photo album template, a layout means for applying the image selected by the image selecting means to the photo album template selected by the template selecting means, and the layout means An editing means for editing the generated photo album, and a control method of the photo album creating device comprising:
An image recognition step for recognizing a face image of a subject included in the image selected by the image selection means;
A feature extraction step of extracting features of the face image of the subject recognized in the image recognition step;
Comparing the features of the face image of the subject extracted in the feature extraction step between the different images, and a subject recognition step of recognizing the same subject,
A subject information management step that manages all subject information of the subject that is recognized in the subject recognition step and that is present in an image used in the photo book; and all subject information of the subject that is managed in the subject information management step. A calculation process of subject information to be calculated;
And a display step for displaying all subject information of the subject.   A program for causing a computer to execute each step of the control method of the photo album creating apparatus according to claim 20.   A computer-readable storage medium storing a program for causing a computer to execute each step of the method for controlling a photo album creating apparatus according to claim 20.

Images