JP5629499B2 - Information processing apparatus, information processing method, and program - Google Patents

Description

  The present invention relates to an information processing apparatus, an information processing method, and a program for performing image layout.

  Conventionally, when an image is laid out, a layout is automatically generated using a template, and when an automatically generated layout is additionally corrected, the selected image is vertically and horizontally changed and enlarged / reduced. There exists a method (for example, refer patent document 1). Recently, a service “shuffle print (shuffle print is a trade name of Fuji Film Co., Ltd.)” for printing a plurality of images packed on one sheet with almost no gap has been started.

JP-A-9-190546

  There is a desire to print photos taken with a digital camera on a page of paper with a frame aligned for each folder created for each date, with no gaps.

  However, in Patent Document 1, a layout is automatically generated using a given template. However, an arbitrary number of images are not laid out with a frame aligned on one page with no gaps. There are also cases where there are gaps between photographs or layout frames are not aligned.

  In addition, there may be a case where a page can be laid out with no gaps only when the number of frames of a photo is a fixed number such as 12, 16, 48, 54, 64, and 70. Furthermore, in the preceding example, a desired layout cannot be generated by editing a layout automatically generated by an easy operation while an arbitrary number of images are packed in one page without any gaps. In addition, the automatically generated layout alone may not match the user's preference. Further, since the number of layout candidate combinations becomes enormous as the number of photographs increases, it is difficult for the user to obtain a desired layout only by automatically generating a layout.

  In view of the above problems, an object of the present invention is to obtain a layout candidate in which an arbitrary number of images having the same aspect ratio are arranged on one page without any gaps. It is another object of the present invention to obtain a desired layout by editing an automatically generated layout with an easy operation while maintaining the condition that the frames are aligned and packed into one page without a gap.

An information processing apparatus according to the present invention that achieves at least one of the above-described objects,
An information processing apparatus that performs image layout,
Counting means for counting the number of images of a plurality of images having the same aspect ratio;
The display area corresponding to the unit image having the minimum size and the aspect ratio when laid out without any gap in the display area having the aspect ratio, where n is an integer greater than or equal to the square root of the number of images counted by the counting means Setting means for setting a display area having a size of n × n as the length of each side of
Based on the number of images counted by the counting means and the integer value n set by the setting means, one or a plurality of size enlargements whose side lengths are the same integer multiple with respect to the unit image A calculating means for calculating the required number of each of the one or a plurality of enlarged images necessary for laying out the image and the unit image without gaps in the display area;
Layout means for laying out the enlarged image of the one or more sizes and the unit image in the display area based on the required number of each of the enlarged images of the one or more sizes calculated by the calculating means;
Characterized by comprising

  According to the present invention, it is possible to obtain a layout candidate in which an arbitrary number of images having the same aspect ratio are arranged on one page without any gaps. Furthermore, it is possible to edit the automatically generated layout with a simple operation and obtain a desired layout while maintaining the condition that the frames are aligned and packed into one page without a gap.

(A) The block diagram which shows the structure of the image layout editing apparatus 100, (b) The figure which shows the operation screen 150 of an image layout editing program. The figure which shows the operation | movement outline | summary of an image layout process. 6 is a flowchart showing the overall flow of image layout editing processing. The flowchart which shows the flow of the process A which calculates a layout candidate. The figure which shows the layout candidate (K, A, B) obtained by this algorithm when the number of layout images is N = 27. The flowchart which shows the flow of the process B which sets basic area | region P (I). The figure which shows the state in which basic area P (I) was set. The figure which shows the flow of the process C which draws an image in the set basic area | region P (I). The flowchart which shows the flow of the process D which selects an image. The figure which sets basic area number I to start area number SI. 10 is a flowchart showing a flow of processing “deletion / return” of an image. The figure which shows a mode that the process which "deletes" an image is performed. The figure which shows a mode that the process of "returning" an image is performed. The flowchart which shows the flow of the process F which calculates the position which can shift the selected image. The figure which shows the mode of the calculation of the position which can shift the selected image. The flowchart which shows the flow of the process G which drags and shifts the selected image Img [P (I)]. The figure which shows a mode that the selected image shifts. The flowchart which shows the flow of the process H which replaces | exchanges an image. The figure which shows a mode that an image is exchanged. The flowchart which shows the flow of the process A2 which is another Example which calculates a layout candidate. The figure which shows the example of the layout candidate (K, A, B, C, D) obtained by this algorithm when the number of layout images is N = 28. The figure which shows the example which applied this invention to the digital camera with a small screen.

  An information processing apparatus (image layout editing apparatus) that lays out all of a plurality of images having the same aspect ratio in a display area having the same aspect ratio without overlapping and without any gap will be described in detail below.

(First embodiment)
<Configuration of image layout editing device>
First, a block configuration showing a configuration applicable to the image layout editing device will be described with reference to FIG. The image layout editing apparatus 100 includes a CPU 101, a ROM 102, a RAM 103, an image storage unit 104, a display unit 105, an operation unit 106, and an input event processing unit 107. The image layout editing apparatus 100 operates when the CPU 101 reads and executes an image layout editing program stored in the ROM 102. The RAM 103 is used to read and write variables and image data used during the execution of the image layout editing program. The display unit 105 is a display or the like. The user performs an operation using the operation unit 106 (for example, a mouse), and the user's mouse operation is analyzed by the input event processing unit 107 as to whether the mouse is Down, Mouse Up, Mouse Drag, or Mouse Move. An image folder is formed for each date in the image storage unit 104, and several image images are stored in each image folder. <Operation screen of image layout editing device>
Next, the operation screen 150 of the image layout editing program displayed on the display unit 105 will be described with reference to FIG. The operation screen 150 includes an operation button area 151, an operation mode display area 152, a folder display area 153, a main layout display area 154, a main image number display area 155, a “delete / restore” area 156, a sub layout display area 157, a sub An image number display area 158 and a layout candidate display area 159 are arranged.

  In the operation button area 151, an “↑” button 151a, a “↓” button 151b, a “<” button 151c, a “>” button 151d, an “SFT” button 151e (moving / returning to the image shift mode), an “EX” button 151f (moving / returning to the image exchange mode) is arranged. In the operation mode display area 152, text describing the operation mode is displayed.

  In the folder display area 153, Folder-1, Folder-2,..., And Folder-N, which are folder names of image folders formed in the image storage unit 104, are displayed. In the main layout display area 154, an arbitrary number of images having the same aspect ratio are laid out and displayed without a gap.

  In the main image number display area 155, the number of images displayed in the main layout display area 154 is displayed. In the sub-layout display area 157, an image deleted from the main layout display area 154 is displayed. In the sub image number display area 158, the number of images displayed in the sub layout display area 157 is displayed.

  In the layout candidate display area 159, layout candidates Cand-1, Cand-2,... Cand-7 are displayed.

<Basic operation with mouse>
In selecting a folder, when the “↑” button 151a or the “↓” button 151b is pressed by operating the mouse, the selected folder is highlighted. The user selects a desired image folder from Folder-1, Folder-2,..., Folder-N. In selecting a layout candidate, when the “<” button 151c and the “>” button 151d are pressed by operating the mouse, the selected layout candidate is highlighted. The user selects a desired layout candidate from Cand-1, Cand-2,... Cand-7.

  Initially, “image selection mode” is displayed in the operation mode display area 152. When the “SFT” button 151e is pressed, the mode shifts from the image selection mode to the image shift mode. When the mode shifts to the image shift mode, “image shift mode” is displayed in the operation mode display area 152. In the image shift mode, only the “SFT” button 151e and the “EX” button 151f are active, and the “↑” button 151a, the “↓” button 151b, the “<” button 151c, and the “>” button 151d are locked. . To return from the image shift mode to the image selection mode, the “SFT” button 151e is pressed. To move from the image shift mode to the image exchange mode, the “EX” button 151f is pressed. When the mode is changed to the image exchange mode, “image exchange mode” is displayed in the operation mode display area 152. In the image exchange mode, only the “EX” button 151f is active, and the “↑” button 151a, the “↓” button 151b, the “<” button 151c, the “>” button 151d, and the “SFT” button 151e are locked. . To return from the image exchange mode to the image shift mode, the “EX” button 151f is pressed.

  In order to delete an image displayed in the main layout display area 154, the image to be deleted is selected with the mouse and dragged to move to the “delete / restore” area 156. Then, the selected image is deleted from the main layout display area 154, and the deleted image is displayed in the sub-layout display area 157. To return the image displayed in the sub-layout display area 157 to the main layout display area 154 and add it, the user selects the image to be returned with the mouse and drags it to the “delete / restore” area 156. Then, the image is returned from the sub-layout display area 157 to the main layout display area 154 and displayed. An image shift operation and an exchange operation in the main layout display area 154 are also performed by dragging the mouse.

<Outline of layout processing operations>
An outline of operation of layout processing in the image layout editing apparatus 100 will be described with reference to FIGS.

  When the image layout editing program is started, the image folders stored in the image storage unit 104 and the image files in each image folder are read. The read image folder name, the number of images included in each image folder, and the image file name are stored, and the image folder name is displayed in the folder display area 153.

  In Phase 1 shown in FIG. 2A, the user presses the “↑” button 151 a or the “↓” button 151 b with the mouse to select a desired image folder. When an image folder is selected, the selected image folder is highlighted.

  In Phase 2 shown in FIG. 2B, a process of laying out the images in the image folder without gaps and displaying the layout is executed. First, the number of images in the image folder is counted and displayed in the main image number display area 155. In the example shown in FIG. 2B, the number of images is 67. Next, the image files in the image folder are packed and displayed in the main layout display area 154 without any gaps. In addition, a maximum of seven layout candidates that can be stuffed without overlapping the enlarged images with the same number of images are displayed in the layout candidate display area 159.

  In Phase 3 shown in FIG. 2C, a process of rearranging images according to the priority specified by the priority specifying unit is executed. For example, the priority order designation unit designates the image including the face so as to give priority. Then, the images F1, F2, F3, F4, F5, and F6 are selected in descending order of likelihood of including the face by the face recognition included in the priority order specifying unit, and the read images are rearranged and displayed according to the priority order. The The designation of the priority order for the image may be performed based on time order, likelihood including a specific event, or the like, in addition to the facial appearance.

  In Phase 4 shown in FIG. 2D, processing for deleting unnecessary images such as failed photos from the main layout display area 154 is executed. When deleting an image, an unnecessary image in the main layout display area 154 is selected with the mouse and dragged to the “delete / return” area 156. Then, the selected image is deleted from the main layout display area 154 and moved to the sub layout display area 157. To restore the deleted image, an unnecessary image in the sub-layout display area 157 is selected with the mouse and dragged to the “delete / restore” area 156. Then, the selected image is returned from the sub-layout display area 157 to the main layout display area 154.

  In Phase 5 shown in FIG. 2E, processing for selecting another layout candidate is executed under the same number of images. The user presses the “<” button 151 c or the “>” button 151 d with the mouse, and selects a desired candidate from the layout candidates displayed in the layout candidate display area 159.

  In Phase 6 shown in FIG. 2F, processing for editing the arrangement of large-size images and exchanging images is executed. In the image shift mode, when a large size image is selected with the mouse, a shiftable position is displayed, and when the selected image is dragged to the shiftable position, the large size image is shifted. In the image exchange mode, when an image is selected and dragged, the drag source image and the drag destination image are exchanged.

  In Phase 7 shown in FIG. 2G, a process of printing an image with a completed layout is executed. Then, as shown in FIG. 2 (h), an arbitrary number of images are printed on one sheet without any gaps.

<Overall Flowchart of Image Layout Editing Process>
The overall flow of the image layout editing process will be described with reference to FIG. This corresponds to the operation outline of the layout processing of FIG.

  In step S301, an initial folder is set. In step S302, it is determined whether or not to end the process. If not completed (step S302; NO), the process proceeds to step S303. On the other hand, if it is finished (step S302; YES), the process is finished.

  In step S303, the number of image files in the folder is counted and set to a variable N0, and the image files are read into the arrays Img [0] to Img [N0-1].

  In step S304, N0 is set to the main image number MN, 0 is set to the sub image number SN, and N0 is set to the layout image number N.

  In process A of step S305, a set of layout candidates (K, A, B) [i] (i = 0, 1,...) Is obtained for a given number N of layout images. The process A will be described later with reference to FIGS.

  In process B of step S306, a basic area P (I) for arranging images in the main layout display area 154 and the sub-layout display area 157 is set. The process B will be described later with reference to FIGS.

  In process C of step S307, an image is drawn according to the set value of the basic area P (I). The process C will be described later with reference to FIG. In step S308, it is determined whether the currently selected image folder is a user's desired folder. That is, the user is confirmed whether the selected image folder is OK. When it is determined that the selected image folder is a folder desired by the user (step S308; YES), the process proceeds to step S310. On the other hand, when it is determined that the currently selected image folder is not the user's desired folder (step S308; NO), the process proceeds to step S309.

  In step S309, the next / previous folder is selected, and the process returns to step S303. In step S310, it is confirmed to the user whether the priority order set by the priority order setting unit is OK. When it is determined that the priority order set by the priority order setting unit is acceptable (step S310; YES), the process proceeds to step S314. On the other hand, when it is determined that the priority order set by the priority order setting unit is not good (step S310; NO), the process proceeds to step S311.

  In step S311, a priority order is set. The priority order setting unit sets whether to rearrange the images with priority given to the face, smile, time or place.

  In step S312, likelihoods are assigned to the images Img [0] to Img [N0-1] according to the set priority order, and the images are rearranged in the order of likelihood. In the process C of step S313, the images rearranged in the order of likelihood are drawn according to the value of the basic area P (I) as in step S307. Thereafter, the process returns to step S310.

  On the other hand, in step S314, N0 is set as the main image number MN, and 0 is set as the sub image number SN. In step S315, the user is confirmed whether the image set displayed in the main layout display area 154 is OK. If it is determined that the image set displayed in the main layout display area 154 is acceptable (step S315; YES), the process proceeds to step S321. On the other hand, if it is determined that the image set displayed in the main layout display area 154 is not acceptable (step S315; NO), the process proceeds to process D of step S316.

  In process D of step S316, the user selects an image with the mouse Down. The process D will be described later with reference to FIGS. In process E of step S317, the image selected in step S316 is moved (deleted) from the main layout display area 154 to the sub layout display area 157, or the selected image is moved from the sub layout display area 157 to the main layout display area 154. Move (return) to. The process E will be described later with reference to FIGS. 11, 12, and 13.

  As a result of deleting or returning the image in the process A of step S318, the layout candidate group (K, A, B) [i] (i = 0) for the new layout image number N as in step S305. , 1, ...).

  In process B of step S319, as in step S306, a basic area P (I) for arranging images in the main layout display area 154 and the sub-layout display area 157 is set. In process C of step S320, an image is drawn according to the set value of the basic area P (I), as in step S307. Thereafter, the process returns to step S315.

  On the other hand, in step S321, the layout candidates are rearranged according to the priority order set by the priority order setting unit. Here, based on the likelihood set for the entire layout candidate, the layout candidates are rearranged in the order of likelihood and displayed in the layout candidate display area 159.

  In step S322, the user is confirmed whether the layout candidate being selected is OK. If it is determined that the selected layout candidate is acceptable (step S322; YES), the process proceeds to step S326. On the other hand, when it is determined that the selected layout candidate is not acceptable (step S322; NO), the process proceeds to step S323.

  In step S323, the next / previous layout candidate is displayed and selected. In process B of step S324, as in step S306, a basic area P (I) for arranging images in the main layout display area 154 and the sub-layout display area 157 is set.

  In process C of step S325, an image is drawn according to the set value of the basic area P (I), as in step S307. Thereafter, the process returns to step S322.

  On the other hand, in step S326, the user is confirmed whether or not to print the layout image displayed in the main layout display area 154. If it is determined to print the layout image (step S326; YES), the process proceeds to step S327. In step S327, the image displayed in the main layout display area 154 is printed. On the other hand, if it is determined not to print the layout image (step S327; NO), the process proceeds to step S328.

  In step S328, confirmation is made with the user as to whether or not to shift the image. If it is determined not to shift the image (step S328; NO), the process returns to step S302. On the other hand, when it is determined to shift the image (step S328; YES), the process proceeds to step S329.

  In process F of step S329, the shiftable position of the currently selected image is calculated and displayed. The process F will be described later with reference to FIGS. In process G of step S330, the currently selected image is shifted by dragging the mouse (FIGS. 17A to 17B). Then, the value of the basic area P (I) is rewritten according to the mouse operation (FIGS. 16 and 17C to 17D). The process G will be described later with reference to FIGS.

  In process C of step S331, an image is drawn according to the set value of the basic area P (I), as in step S307 (FIG. 17E).

  In step S332, the user is confirmed as to whether or not to exchange images. If it is determined not to replace the image (step S332; NO), the process returns to step S326. On the other hand, when it is determined that the images are to be exchanged (step S332; YES), the process proceeds to step S333.

  In step D of step 333, as in step S316, the user selects an exchange source image with the mouse Down. In process H of step S334, the user drags the mouse to select an exchange destination image (FIG. 19 (a)), and exchanges the exchange source image and the exchange destination image with the mouse Up (FIG. 19). The process H will be described later with reference to FIGS.

  In the process C of step S335, an image is drawn according to the set value of the basic area P (I) as in step S307 (FIG. 19C). Thereafter, the process returns to step S332.

  This is the end of the description of the overall flow of the image layout editing process. Hereinafter, details of each of the processes A, B,..., G will be described.

<Calculation of layout candidates: Process A>
First, with reference to FIG. 4, the flow of process A for calculating layout candidates in step S305 will be described. In this algorithm, when N images having the same aspect ratio as the display area are packed in the display area without any gaps, what is the number of images each having an integer multiple (here, double or triple) of the minimum image? Calculate whether or not to use one. Here, the size of each basic area P (I) in the main layout display area 154 is the size of the minimum image.

  In step S401, the number N of layout images is counted. In step S402, a minimum natural number equal to or greater than √N is obtained and set in the variable K. Here, K is the length of one side of the display area when the length of one side of the minimum image is 1. That is, when the minimum image size is H × W, a display area having a size of KH × WKW is set. However, the variable K indicating the size of the display area is not necessarily a minimum integer equal to or larger than the square root of the number N of counted images, and may be an integer equal to or larger than the square root of the number N of images. When the integer is the value n, a display area having a size of n × n is set.

In step S403, based on the counted number N of images and the value of the set variable K, the length of one side of the unit image is the same integer k times (k = 2, 3,..., M). The necessary number of each of the enlarged images of one or a plurality of sizes necessary for laying out the enlarged image of one or a plurality of sizes and the unit image without a gap in the display area is calculated. Here, for example, consider a case where there are two types of enlarged image sizes of 2 × 2 (k = 2) and 3 × 3 (k = 3). Specifically, D = K × K-3A-8B-N is set. Here, D is a stuffing determination function, A is the number of images whose side length is twice the minimum image, and B is the number of images whose side length is three times the minimum image. That is, from the square of the above-mentioned integer, the length of one side of the unit image is three times (2 ² -1 times) the number of enlarged images, each of which is twice the length of the unit image. The value of 8 times (3 ² −1 times) the number of 3 times magnified images and the counted number of images are subtracted. A combination of a 2 × 2 (k = 2) enlarged image and a 3 × 3 (k = 3) enlarged image with a value of zero is calculated. There are natural numbers A and B for which the stuffing determination function D = 0 with respect to the length K of one side of the display area, and when the images are stuffed as shown in FIGS. If they do not overlap, the set (K, A, B) is a layout candidate. If there are still enlarged images of other sizes, the stuffing determination function D uses D = K × K−Σ [k = 2, 3,..., M] (k ² −1) G (k ) -N. In the example described above, G (2) corresponds to A and G (3) corresponds to B. This general form will be described later.

  In step S404, it is determined whether or not there are natural numbers A and B for which the stuffing determination function D = 0 with respect to the length K of one side of the display area. When it is determined that there are natural numbers A and B that satisfy the stuffing determination function D = 0 (step S404; YES), the process proceeds to step S405. On the other hand, when it is determined that there are no natural numbers A and B that satisfy the stuffing determination function D = 0 (step S404; NO), the process proceeds to step S408.

  In step S405, it is determined whether or not large-sized images (images each having a side length twice that of the smallest image: A sheet, three times image: B sheet) do not overlap each other. When it is determined that the large-size images overlap (step S405; NO), the process proceeds to step S407. On the other hand, when it is determined that the large-size images do not overlap (step S405; YES), the process proceeds to step S406. In step S405, the overlap determination is performed as follows. First, A sheets of images whose side length is twice the minimum image are arranged by a raster scan method from the upper left to the lower right of the display area. Next, B images having a side length three times the minimum image are arranged by a raster scan method from the lower right to the upper left of the display area. When these arrangements are made, the A image having twice the one side length of the minimum image and the B image having one side length three times the minimum image do not overlap. In step S405, YES is determined.

  In step S406, the set (K, A, B) is stored as a layout candidate. In step S407, it is determined whether more layout candidates are to be obtained. If it is determined that more layout candidates are desired (step S407; YES), the process proceeds to step S408. On the other hand, when it is determined that it is not necessary to obtain more layout candidates (step S407; NO), the process ends.

  In step S408, the length K of one side of the display screen is incremented, and the process returns to step S403. Note that it is not always necessary to increment, and a predetermined integer may be added.

  Here, FIGS. 5A to 5E show layout candidates (K, A, B) obtained by this algorithm when the number of layout images is N = 27. FIG. 5A shows a case where layout candidates (K, A, B) = (6, 3, 0). FIG. 5B shows a case where layout candidates (K, A, B) = (7, 2, 2). FIG. 5C shows a case where layout candidates (K, A, B) = (8, 7, 2). FIG. 5D shows a case where layout candidates (K, A, B) = (9, 10, 3). FIG. 5E shows a case where layout candidates (K, A, B) = (9, 2, 6).

  In this algorithm, an image having a side length twice or three times that of the minimum image is used as a large size image. However, when the number of images is 2, 3, or 5, the images cannot be packed without a gap. Therefore, when N <6, as an exception process, as shown in FIGS. 5 (f1) to (f5), as shown in FIGS. 5 (f1) to (f5), a number of images are rotated 90 degrees and packed, or (g2), (g3), ( The same image is used twice as in g5).

  The above is the details of the process A for calculating the layout candidate.

<Setting of Basic Area P (I): Process B>
Next, with reference to FIG. 6 and FIG. 7, the flow of the process B for setting the basic area P (I) will be described. The layout candidates (K, A, B) used in this process are calculated by setting the main image number MN to the layout image number N in the algorithm of the process A (FIG. 4).

  In step S601, the value of the basic area P (I) is initialized for all the basic area numbers I (P (I) ← −1). That is, P (I) is initialized with -1. In step S602, the main layout display area 154 is divided into K × K basic areas P (I) (FIG. 7A). The size of each basic area P (I) in the main layout display area 154 is the minimum size image. Next, as shown in FIG. 7C, values are set in the basic area P (I) in the main layout display area 154.

  First, in step S603, A is set for a quadruple area of 2 × 2 size by a raster scan method from the upper left to the lower right of the main layout display area 154, and numerical values are set in the quadruple area in order from 0. . In the four basic areas P (I) in the quadruple area, the same numerical values as those set in the quadruple area are set. In the present invention, “setting the area by the raster scan method from the upper left to the lower right of the display area” means, for example, P (0), P (1),..., P (6 in FIG. ), P (7), P (8),...

  Next, in step S604, B 9 × 3 × 3 size areas are set by the raster scan method from the lower right to the upper left of the main layout display area 154, and numerical values are set in the 9 × area in order. In the present invention, “setting the area by a raster scan method from the lower right to the upper left of the display area” means, for example, P (48), P (47),. ), P (41), P (40),...

  In step S605, numerical values are also set in order in the remaining MN-A-B basic areas P (I).

  In step S606, SN or more basic areas P (I) are set in the sub-layout display area 157 in order to arrange SN images (FIG. 7B). In step S607, as shown in FIG. 7D, numerical values are also set in order in the SN basic areas P (I) of the sub-layout display area 157.

  7 (c) and 7 (d), P (0) = 0, P (1) = 0, P (2) = 1, P (3) = 1, P (4) = 4, P (5) = 5, P (6) = 6, P (7) = 0, P (8) = 0, P (9) = 1, P (10) = 1, P (11) = 7,. The value of the basic area is set like

  The above is the details of the process B for setting the basic area P (I).

<Image drawing: Process C>
With reference to FIG. 8, the flow of process C for drawing an image in the set basic area P (I) will be described. First, an image is drawn according to the value of the basic area P (I) in the main layout display area 154 (FIGS. 8A to 8B). Here, first, an image Img [J] whose side length is three times the minimum image is applied to a 9 × region in which nine adjacent 3 × 3 basic regions P (I) have the same value J. draw. Next, an image Img [J] whose side length is twice the minimum image is drawn in a quadruple region in which four adjacent 2 × 2 basic regions P (I) have the same value J. . Finally, the image Img [J] having the minimum image size is drawn for the remaining basic area P (I) = J.

  Further, an image is drawn according to the value of the basic area P (I) in the sub-layout display area 157 (FIGS. 8C to 8D). Here, the image Img [J] is drawn for the basic region P (I) = J. In this manner, the enlarged image and the unit image can be laid out in the display area based on the required number of enlarged images of one or a plurality of sizes.

  The above is the details of the process C for drawing an image in the set basic region P (I).

<Image selection: Process D>
With reference to FIGS. 9 and 10, the flow of the process D for selecting an image will be described. First, in step S901, the position (X, Y) where the mouse is down is detected.

  In step S <b> 902, it is determined whether or not the position (X, Y) where the mouse is down is in the area P (I) in the main layout display area 154. If it is determined that the position (X, Y) is in the area P (I) in the main layout display area 154 (step S902; YES), the process proceeds to step S903. On the other hand, when it is determined that the position (X, Y) is not in the area P (I) in the main layout display area 154 (step S902; NO), the process proceeds to step S905.

  In step S903, the basic area number I is set as the start area number SI. In step S904, the size and range of the image at the position (X, Y) are acquired. Thereafter, the process ends.

  In step S 905, it is determined whether or not the position (X, Y) where the mouse is down is in the area P (I) in the sub-layout display area 157. When it is determined that the position (X, Y) is in the area P (I) in the sub-layout display area 157 (step S905; YES), the process proceeds to step S906. On the other hand, when it is determined that the position (X, Y) is not in the area P (I) in the sub-layout display area 157 (step S905; NO), the process ends. In step S906, the basic area number I is set as the start area number SI. Thereafter, the process ends.

  When the mouse Down is detected at the position of the arrow in the main layout display area 154 as shown in FIG. 10A, 32, which is the number indicated by the arrow, is set to the start area number SI. Similarly, when the mouse Down is detected at the position of the arrow in the sub-layout display area 157 as shown in FIG. 10B, 52, which is the number indicated by the arrow, is set to the start area number SI.

  The above is the details of the process D for selecting an image.

<“Delete / Restore” image: Process E>
With reference to FIGS. 11, 12, and 13, the flow of the process “deleting / returning” an image will be described.

  In step S1101, the position (X, Y) of the mouse Up is detected. In step S 1102, it is determined whether or not the position of the mouse Up is within the “delete / return” area 156. If it is determined that the position of the mouse Up is within the “delete / return” area 156 (step S1102; YES), the process proceeds to step S1103. . On the other hand, if it is determined that the position of the mouse Up is not within the “delete / return” area 156 (step S1102; NO), the process is terminated.

  In step S1103, it is determined whether or not the basic area P (SI) corresponding to the start area number SI obtained in the process D is in the main layout display area 154. When it is determined that the basic area P (SI) is within the main layout display area 154 (step S1103; YES), the process proceeds to step S1104. On the other hand, when it is determined that the basic area P (SI) is not in the main layout display area 154 (step S1103; NO), the process proceeds to step S1107.

  In step S1104, the image Img [P (SI)] is deleted from the main layout display area 154, and is set as the last image Img [MN + SN-1] in the sub-layout display area 157.

  In step S1105, the main image number MN is decremented and the sub image number SN is incremented. In step S1106, the main image number MN is set as the layout image number N. Thereafter, the process ends.

  In step S1107, it is determined whether or not the basic area P (SI) corresponding to the start area number SI obtained in process D is in the sub-layout display area 157. If it is determined that the basic area P (SI) is within the sub-layout display area 157 (step S1107; YES), the process proceeds to step S1108. On the other hand, if it is determined that the basic area P (SI) is not within the sub-layout display area 157 (step S1107; NO), the process is terminated.

  In step S1108, the image Img [P (SI)] is deleted from the sub-layout display area 157 and is set as the first image Img [0] in the main layout display area 154.

  In step S1109, the main image number MN is incremented and the sub image number SN is decremented. In step S1110, the main image number MN is set as the layout image number N. Thereafter, the process ends.

  FIG. 12 is a diagram illustrating a state in which the process of “deleting” an image is performed. As indicated by the arrows in the figure, the image Img [2] in the main layout display area 154 is selected with the mouse and dragged to the “delete / restore” area 156. Then, the image Img [2] in FIG. 12A is deleted from the main layout display area 154 and becomes the image Img [38] in the sub-layout display area 157 in FIG. At this time, the main image number MN in the main image number display area 155 is decremented from 27 to 26, and the sub image number SN in the sub image number display area 158 is incremented from 12 to 13.

  On the other hand, FIG. 13 is a diagram showing a state in which an image “returning” process is performed. As indicated by the arrows in the figure, the image Img [28] in the sub-layout display area 157 is selected with the mouse and dragged to the “delete / restore” area 156. Then, the image Img [28] in FIG. 13A is deleted from the sub-layout display area 157 and becomes the image Img [0] in the main layout display area 154 in FIG. At this time, the main image number MN in the main image number display area 155 is incremented from 19 to 20, and the sub image number SN in the sub image number display area 158 is decremented from 20 to 19.

  The above is the details of the process “deletion / return” of the image.

<Image Shift: Processing FG>
Image shift will be described with reference to FIGS. 14, 15, 16, and 17. FIG.

  First, with reference to FIG. 14 and FIG. 15, the flow of the process F for calculating the shiftable position of the designated image will be described. When one of the enlarged images or unit images of one or more sizes specified by the user is shifted in the vertical and horizontal directions, the position including all of the other enlarged images or unit images existing at the shift destination is shifted. Calculate as a possible position in the display area.

  In step S1401, the position (X, Y) of the mouse Down is detected, the basic area number I of the position (X, Y) is obtained, and the rectangle of the image Img [P (I)] is obtained (FIG. 15 (a), (B)).

  In step S1402, a rectangle including the boundary between the images, including the selected rectangle, is searched for a rectangle having the same width as the selected rectangle without intersecting with the other rectangles (FIG. 15C).

  In step S1403, a rectangle including the boundary between images and including the selected rectangle is searched for a rectangle having the same height as the selected rectangle without intersecting with other rectangles (FIG. 15D).

  In step S1404, among the sides of the rectangle obtained in steps S1402 and S1403, a side not included in the selected rectangle is set as a shiftable position (FIG. 15E).

  That is, when the mouse is down at the position shown in FIG. 15A, the image Img [3] can be shifted as L1, R1, U1, and U2 shown in FIG. 15F (shiftable position calculation).

  The above is the details of the process F for calculating the shiftable position of the image.

  Next, with reference to FIG. 16 and FIG. 17, the flow of the process G for dragging the selected image Img [P (I)] and executing a shift will be described.

  In step S1601, the image is dragged with the mouse as indicated by the arrow in FIG. If it is determined that the mouse has been dragged (step S1601; YES), the process proceeds to step S1602. On the other hand, if it is determined that the image is not dragged with the mouse (step S1601; NO), the process is terminated.

  In step S1602, it is determined whether or not the mouse position (X, Y) in the Drag has come within a certain range of any of L1, R1, U1, and U2 that are shiftable positions. When it is determined that the mouse position (X, Y) in the Drag has approached any one of the shiftable positions (step S1602; YES), the process proceeds to step S1603. On the other hand, if it is determined that the mouse position (X, Y) in the Drag is not approaching any one of the shiftable positions (step S1602; NO), the process proceeds to step S1604.

  In step S1603, the closest shiftable position within a certain range of the mouse position (X, Y) indicated by the arrow is identified and displayed with a thick line (FIG. 17B).

  In step S1604, it is determined whether or not there is a mouse Up. If it is determined that the mouse is up (step S1604; YES), the process proceeds to step S1605. On the other hand, if it is determined that the mouse is not up (step S1604; NO), the process returns to step S1601.

  In step S1605, the position of the mouse Up is calculated (shiftable position selection process). Here, it is assumed that the mouse is up at the position of FIG.

  In step S1606, the value of the basic area P (I) is switched according to the position of the mouse Down and the position of the mouse Up. Here, the values [3] of the nine basic regions P (I) corresponding to the image Img [3] that were initially at the position of the mouse down are shifted up by two, and the values of the six basic regions P (I) [11], [12], [13], [18], [19], [20] are shifted down by three. As a result, the value of the basic area P (I) is switched as shown in FIG. As described with reference to FIG. 3, since the process C for drawing an image is executed after the process G, an image is drawn by the process C based on the value of the basic area P (I) (FIG. 17E). . Thereafter, the process ends.

  The above is the details of the process G for performing the shift by dragging the selected image Img [P (I)].

<Image exchange: Process H>
As described with reference to FIG. 3, the process H (image exchange process) for exchanging images is performed after the process D (image designation process) for selecting an image. The flow of processing H for exchanging images will be described with reference to FIGS. Here, for example, as shown by the arrow in FIG. 19A, the image Img [8] is dragged to the position of the image Img [3] with the mouse.

  In step S1801, it is determined whether the image Img [P (SI)] in the basic area P (SI) is being dragged with the mouse (FIG. 19B). When it is determined that dragging is in progress (step S1801; YES), the process proceeds to step S1802. On the other hand, if it is determined that the drag is not being performed (step S1801; NO), the process is terminated.

  In step S1802, it is determined whether the mouse is up. When it is determined that the mouse is up (step S1802; YES), the process proceeds to step S1803. On the other hand, when it is determined that the mouse is not up (step S1802; NO), the process returns to step S1801.

  In step S1803, the position (X, Y) of the mouse Up is calculated. In step S 1804, it is determined whether or not the position (X, Y) is in any basic area P (I) in the main layout display area 154. If it is determined that the position (X, Y) is within the basic region P (I) (step S1804; YES), the process proceeds to step S1805. On the other hand, when it is determined that the position (X, Y) is not within the basic region P (I) (step S1804; NO), the process is terminated.

  In step S1805, the basic area number I is set as the end area number EI. In step S1806, the image Img [P (SI)] and the image Img [P (EI)] are exchanged. Here, as shown in FIG. 19C, the image Img [8] and the image Img [3] are exchanged. The above is the details of the processing H for exchanging images.

  In the first embodiment, given the number of images, the length of one side of the screen, the minimum number of images, and the length of the sides for packing a plurality of images on one screen without gaps are integer multiples of the minimum image ( 2 times and 3 times) of images. As a result, an arbitrary number of images can be packed in one screen without any gaps, so that it is possible to generate a beautiful layout with space and no waste of screen or paper and with a uniform frame.

  In the first embodiment, the selected image can be deleted or returned from the selected layout candidate. If the number of images changes by the deletion or return operation, the images are again packed into one screen without any gaps. To do. As a result, it is possible to delete the failed image and the failed photo and to redo the deletion.

  Further, in the first embodiment, all the rectangles that include the selected image, have the same width as the selected image and do not intersect with the other rectangles, and have the boundary between the images, and obtain the upper side of the obtained rectangle And find the bottom. Further, all the rectangles including the selected image and having the same height as the selected image and including the boundary between the images are obtained without intersecting with other rectangles, and the left side and the right side of the obtained rectangle are obtained. Then, the shiftable positions of the selected image are set as the upper side, the lower side, the left side, and the right side obtained in this way, and are displayed in an easy-to-understand manner for the user. Thus, the user can generate a desired layout by shifting the image to a position that can be shifted with a simple operation.

  In the first embodiment, when the selected image is moved, the image at the destination position is exchanged with the selected image. This makes it possible to lay out a desired image at a desired position.

(Second Embodiment)
<Another example of calculating layout candidates: Process A2>
With reference to FIGS. 20 and 21, the flow of process A2, which is another example of calculating layout candidates, will be described. In this algorithm, when N images having the same aspect ratio as the display area are packed in the display area without any gaps, one side is an integral multiple of the minimum image (here, 2 times, 3 times,..., M times). It calculates how many of each image should be used. Here, the size of each basic area P (I) in the main layout display area 154 is the size of the minimum image.

  In step S2001, the number N of layout images is set.

  In step S2002, the minimum natural number equal to or greater than √N is obtained and set in the variable K. Here, K is the length of one side of the display area when the length of one side of the minimum image is 1.

  In step S2003, the stuffing determination function D = K × K−Σ [L = 2,... M] (L × L−1) G (L) −N is set. Here, Σ is the sum, M is the length of one side of the maximum image that may be used, and G (L) is the number of images whose side length is L times the minimum image. There exists a set of natural numbers G (L: L = 2,... M) for which the stuffing determination function D = 0 with respect to the length K of one side of the display area, and FIGS. ) If the images with large L × L (L = 2, 3,..., M) size do not overlap when packed (K, G (L: L = 2,... M) ) Is a layout candidate.

  In step S2004, it is determined whether or not there is a set of natural numbers G (L: L = 2,... M) for which the stuffing determination function D = 0 with respect to the length K of one side of the display area. When it is determined that there is a set of natural numbers G (L: L = 2,... M) that satisfies the stuffing determination function D = 0 (step S2004; YES), the process proceeds to step S2005. On the other hand, if it is determined that there is no set of natural numbers G (L: L = 2,... M) that satisfy the stuffing determination function D = 0 (step S2004; NO), the process proceeds to step S2008.

  In step S2005, it is determined whether large-sized images can be arranged in the display area without overlapping with a predetermined layout. When it is determined that large-sized images can be arranged without overlapping in a predetermined layout (step S2005; YES), the process proceeds to step S2006. On the other hand, when it is determined that large-sized images overlap (step S2005; NO), the process proceeds to step S2007. Here, the overlap determination in step S2005 is performed as follows (S1) to (S3). (S1): When L (≧ 2) is an even number, a vacant display area is scanned by a raster scan method from the upper left to the lower right to search for an area where an L × L size area can be set; G (L) areas having a size of L × L are set in the searched area by a raster scan method from the upper left to the lower right. (S2): When L (≧ 3) is an odd number, a vacant display area is scanned by a raster scan method from the lower right to the upper left to search for an area where an L × L size area can be set; G (L) areas having a size of L × L are set in the searched area by a raster scan method from the lower right to the upper left. (S3): The above processes (S1) and (S2) are repeated for the variables L = 2, 3,..., M, and all L × L (L = 2, 3,. If the areas of size M) do not overlap, YES is determined in step S2005.

  In step S2006, a set of (K, G (L: L = 2,... M)) is stored as a layout candidate.

  In step S2007, it is determined whether more layout candidates are to be obtained. If it is determined that more layout candidates are desired (step S2007; YES), the process proceeds to step S2008. On the other hand, when it is determined that it is not necessary to obtain more layout candidates (step S2007; NO), the process ends.

  In step S2008, the length K of one side of the display screen is incremented, and the process returns to step S2003.

  FIGS. 21A to 21J show layout candidates (K, G (2), G (3), G (4), G (5) obtained by this algorithm when the number of layout images is N = 28. )) = Some examples of (K, A, B, C, D) are shown. FIG. 21A shows a case where (K, A, B, C, D) = (6, 0, 1, 0, 0). FIG. 21B shows the case where (K, A, B, C, D) = (7, 7, 0, 0, 0). FIG. 21C shows the case where (K, A, B, C, D) = (7, 2, 0, 1, 0). FIG. 21D shows a case where (K, A, B, C, D) = (8, 12, 0, 0, 0). FIG. 21E shows the case where (K, A, B, C, D) = (8, 7, 0, 1, 0). FIG. 21F shows the case where (K, A, B, C, D) = (8, 4, 3, 0, 0). FIG. 21G shows the case where (K, A, B, C, D) = (8, 4, 0, 0, 1). FIG. 21H shows the case where (K, A, B, C, D) = (9, 15, 1, 0, 0). FIG. 21 (i) shows a case where (K, A, B, C, D) = (9, 7, 4, 0, 0). FIG. 21J shows the case where (K, A, B, C, D) = (9, 7, 1, 0, 1). In the second embodiment, when the number of images is given, the length of one side of the screen, the number of minimum images, and the length of the sides are an integral multiple of the minimum image for packing a plurality of images on one screen without gaps ( 2 times, 3 times, 4 times, and 5 times the number of images. As a result, the selected image can be displayed in a particularly large size, that is, the side length is four times or five times that of the minimum image.

(Third embodiment)
<Application example to small screen>
An example in which the present invention is applied to a digital camera with a small screen will be described with reference to FIG. In this digital camera, an input screen 2201 and operation buttons 2202 with a touch panel arranged on the surface are arranged on a main body 2200.

  First, as shown in FIG. 22A, the operation buttons 2202 are operated to select a photo folder created for each date.

  When a photo folder is selected, as shown in FIG. 22B, the photos in the selected folder are read, packed into the input screen without any gaps, laid out and displayed. Here, the user touches the position of the image IA on the screen with a finger.

  Since the size of the finger is larger than the minimum image size on the screen, the center of the position touched with the finger is calculated, and the image IA at the center of the touch position and the operation menu (“Delete”, “Shift”, “Exchange”) are displayed. An enlarged display is displayed (FIG. 22C). Here, the user touches the “delete” menu.

  Then, the image IA is deleted, and layout candidates are calculated with a new number, as shown in FIG. Next, the user touches the position of the image IB on the screen with a finger (FIG. 22D).

  Then, the center of the position touched by the finger is calculated, and the image IB at the center of the touch position and the operation menu (“delete”, “shift”, “exchange”) are enlarged and displayed (FIG. 22E). Here, the user touches the “shift” menu.

  Then, a position where the image IB can be shifted is displayed as shown in FIG. Here, the user drags the image by tracing the position indicated by the arrow in FIG. Then, the image IB shifts as shown in FIG. Next, the user touches the position of the image IC on the screen with a finger (FIG. 22 (g)). Then, the center of the position touched with the finger is calculated, and the image IC at the center of the touch position and the operation menu (“delete”, “shift”, “exchange”) are enlarged and displayed (FIG. 22 (h)). Here, the user touches the “exchange” menu. Then, as shown in FIG. 22 (i), the image IC is enlarged and displayed in a size that can be operated with a finger. As shown by the arrow in FIG. 22 (i), the user drags the image IC to the position of the image IB by tracing with a finger. Then, the image IC and the image IB are exchanged as shown in FIG.

  In the third embodiment, when an image is selected, the selected image and the operation menu for the image are enlarged and displayed. The present invention can be applied to a small-screen device by dragging a selected image with a finger or touching an operation menu with a finger.

(Fourth embodiment)
In addition, when layout editing is performed by applying the present invention to a device such as a digital camera or printer with a small touch panel screen, a large image is displayed, but a small image is not displayed and only a layout frame is displayed. Also good. Further, only large-size images may be shifted or exchanged. At this time, the touch operation may be accepted only for the large-size image area, and the touch operation may not be accepted for the small-size image area.

  In the fourth embodiment, by displaying only a layout frame without displaying a small-sized image and allowing only a large-sized image to be shifted or exchanged, it is possible to prevent an erroneous operation with a small-screen touch panel.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

Claims (12)

An information processing apparatus that performs image layout,
Counting means for counting the number of images of a plurality of images having the same aspect ratio;
The display area corresponding to the unit image having the minimum size and the aspect ratio when laid out without any gap in the display area having the aspect ratio, where n is an integer greater than or equal to the square root of the number of images counted by the counting means Setting means for setting a display area having a size of n × n as the length of each side of
Based on the number of images counted by the counting means and the integer value n set by the setting means, one or a plurality of size enlargements whose side lengths are the same integer multiple with respect to the unit image A calculating means for calculating the required number of each of the one or a plurality of enlarged images necessary for laying out the image and the unit image without gaps in the display area;
Layout means for laying out the enlarged image of the one or more sizes and the unit image in the display area based on the required number of each of the enlarged images of the one or more sizes calculated by the calculating means;
An information processing apparatus comprising: Adding means for adding a predetermined integer to an integer value n equal to or larger than the square root of the number of images counted by the counting means;
The setting means re-sets the size of the display area with the new integer value added by the adding means as the length of each side of the display area for the unit image. Item 4. The information processing apparatus according to Item 1.   The setting means sets the size of the display area with the minimum integer value equal to or larger than the square root of the number of images counted by the counting means as the length of each side of the display area for the unit image. The information processing apparatus according to claim 1, wherein the information processing apparatus is an information processing apparatus. The calculating means includes
From the square of the integer value n set by the setting means,
The length of one side of the unit image is k2-1 times the number of enlarged images each having k (k = 2, 3,..., M ≦ the integer value set by the setting means) times. The sum of the values for each k from k = 2 to k = M for the value,
The number of images counted by the counting means;
The required number of each of the one or more enlarged images is calculated by calculating a combination of the number of enlarged images in which the length of one side is k times each of the unit images. The information processing apparatus according to claim 1, wherein When the size of the enlarged image is twice the same length of each side of the unit image, and when the length of one side of the unit image is the same three times,
The calculation means calculates a value that is three times the number of enlarged images, each of which is twice the length of one side of the unit image, from the square of the integer set by the setting means, and the unit image. One side of the unit image in which the value obtained by subtracting the value of eight times the number of enlarged images each having a side length of three times and the number of images counted by the counting means is zero. By calculating a combination of the number of enlarged images each having a length of 2 times and the number of enlarged images having a side length of 3 times each of the unit images. The information processing apparatus according to claim 4, wherein a necessary number of each of the enlarged images is calculated. The layout means includes
When laying out one or more sizes of the enlarged image and the unit image calculated by the calculating unit in the display area, an arrangement in which the enlarged images do not overlap is generated as a layout candidate. Generating means;
Selecting means for selecting one layout candidate among the layout candidates,
6. The one or more sizes of the enlarged image and the unit image are laid out in the display area according to a layout candidate selected by the selection unit. Information processing device. Deleting means for deleting the enlarged image or the unit image of the one or more sizes laid out by the layout means from the display area;
Adding means for adding again the enlarged image or the unit image of the one or more sizes deleted by the deleting means to the display area;
The information processing apparatus according to claim 6, wherein the counting unit counts the number of images again when deleted or added by the deleting unit or the adding unit. Designating means for allowing a user to designate one of the one or a plurality of sizes of the enlarged image or the unit image laid out by the layout means;
When one of the enlarged image or the unit image of one or a plurality of sizes designated by the designation unit is shifted in the vertical and horizontal directions, all other enlarged images or unit images existing at the shift destination are included. Shiftable position calculating means for calculating the position to be shifted as the position in the display area that can be shifted;
Shiftable position selection means for allowing the user to select one of the positions in the shiftable display area calculated by the shiftable position calculation means;
Shift means for shifting the enlarged image or the unit image of the one or more sizes to a position in the shiftable display area selected by the shiftable position selection means;
The information processing apparatus according to claim 6, further comprising: Image designation means for allowing a user to designate one of the enlarged image or the unit image of the one or more sizes laid out by the layout means;
Exchange means for exchanging the position and size in the display area of the enlarged image or unit image of the one or a plurality of sizes designated by the image designation means and another enlarged image or unit image in the display area; ,
The information processing apparatus according to claim 6, further comprising: An information processing method for performing image layout,
A counting step in which the counting means counts the number of images of a plurality of images having the same aspect ratio;
The unit image having the minimum aspect ratio and the aspect ratio when the setting means lays out an integer value equal to or larger than the square root of the number of images counted in the counting step and lays out the display area having the aspect ratio without any gap. A setting step of setting a display area having a size of n × n as the length of each side of the display area with respect to
The calculating means is based on the number of images counted in the counting step and the integer value n set in the setting step, and the length of one side of the unit image is the same integer multiple or A calculation step of calculating a necessary number of each of the one or a plurality of size enlarged images necessary for laying out a plurality of size enlarged images and the unit image without gaps in the display area;
A layout unit lays out the enlarged image of one or more sizes and the unit image in the display area based on the required number of enlarged images of the one or more sizes calculated by the calculating step. Layout process;
An information processing method characterized by comprising:   A program for causing a computer to execute the information processing method according to claim 10. When the vertical length of the unit image is H and the horizontal length of the unit image is W, the vertical length of the display area is an integer value n × H set by the setting unit, The information processing apparatus according to claim 1, wherein a horizontal length of the display area is an integer value n × W set by the setting unit.

Images