JP2014158071A - Image layout device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To set a template for frame division which easily reflects size restriction in which cut-off of an important area is designated to be small by a small amount of calculation.SOLUTION: An order range setting part 5 sets an order range when selecting additional images to complement a shortage of favorite images. A first setting part 7a sets an allowable shape range for each image. A second setting part 7b sets a representative allowable shape range for a virtual additional image selected from each order range. A template evaluation part 10 evaluates each template candidate on the basis of the degree of divergence between each frame shape and the allowable shape range or representative allowable shape range of each image when each favorite image and the virtual additional image selected from each order range are pasted to each frame of a plurality of different template candidates by keeping relative order of each image. An additional image selection part 11 selects an additional image within each order range on the basis of a frame shape corresponding to each order range of the template determined based on the evaluation.

Description

  The present invention relates to an image layout apparatus that selects a plurality of images from a plurality of images whose order is specified in advance and lays them out on a frame-divided output page, and in particular, more important than the prior art while maintaining the image layout order. Even if an image to be displayed in large / small is specified in advance by setting a template for frame division that reduces area clipping with a small amount of calculation, and large / small frames can be assigned to the image, it can be displayed in large / small. The present invention relates to an image layout apparatus capable of setting a frame allocation template.

  As one of the methods for creating thumbnails of video information and photo summaries, a key area image or representative photo is extracted from the video, and the rectangular order is used like a comic manga frame by keeping the time order. Attention has been focused on a method of displaying thumbnails by placing them inside. Comic-style frame division has the advantage that if you are interested in comic manga, you can understand the sequence of frames without assigning ordinal numbers to each frame.

  In order to improve the subjective quality of such comic-style frame layout display, an important area is set for the image to be pasted on each frame, and the order of the pages to the output page (rectangular area) with few important areas cut off is maintained. A split setting is required. It is also required to display important images on a large frame.

  Patent Document 1 includes frame number information, which is information indicating the maximum number of frames in a page, gap information, which is information indicating a gap between frames, and margin information indicating margins between an inner frame and an outer frame. When set as frame allocation condition information, a method for automatically generating frame allocation that satisfies all conditions has been proposed. As a result, the photos can be pasted in the same order as a comic, and a photo to be enlarged can be designated and displayed in a large size.

  Japanese Patent Application Laid-Open No. 2004-228688 discloses a frame layout evaluation apparatus that evaluates frame layout in which a plurality of images are arranged in a predetermined order on a predetermined output page in a predetermined order, and a large number of templates that match the size of the output page and the number of pasted images. For each template, the degree of divergence between the shape of each frame and the shape of the image pasted on the frame is calculated, and the frame of the output page is divided based on the template that has the smallest total divergence for one page. Is determined. In addition, in order to further reduce the degree of divergence, a technique is disclosed in which an important area is extracted from each image, and trimming within the range including the important area is allowed to deform the image according to the frame shape. Has been.

  Non-Patent Document 1 discloses a technique for creating a collage by pasting an image to be displayed on a large frame.

  In Non-Patent Document 2, photos can be pasted in a comic style in a pre-specified order, important areas are not easily cut off even during trimming, various layouts are easy to exist, In addition, a layout method that can specify a photograph to be enlarged is disclosed.

Japanese Patent No. 4111948 JP 2010-50528 A

Jun Xiao, Xuemei Zhang, Phil Cheatle, Yuli Gao, C. Brian Atkins, "Mixed-initiative photo collage authoring", Intl. Conf. On ACM MM'08, 2008. (ISBN: 978-1-60558-303-7 ) "Automatic Template Creation Method for Arbitrary Rectangular Image Group Layout" Proceedings of the IEICE Technical Committee Conference Vol. 255, pp.61-68, 2011-03-03. Emi Meido, Koichi Takagi, Shigeyuki Sazawa,

  In Non-Patent Document 1, the order of photographs cannot be maintained during layout. Therefore, it is not suitable for a group of images that are frequently requested to be browsed in sequence, such as when viewing a large number of event photos such as weddings.

  In Patent Document 1, the order of photographs is maintained at the time of layout, but important areas are not detected, and there is no algorithm for maintaining important areas. Therefore, important areas tend to be cut off and subjective image quality tends to be low. Moreover, even if the content is such that the story can be understood in the order of the images if the important area is displayed, the understanding of the story is likely to be impaired because the important area is easily cut off.

  In Non-Patent Document 2, in order to maintain the order of images and important areas, the problem is solved by giving a width to the shape of the area that can be cut out. Regions are often clipped, resulting in lower subjective image quality.

  Patent Document 2 is basically the same as the method of Non-Patent Document 2, but differs in that an image to be enlarged / reduced can be designated. However, the number of frames is not limited, and there are many cases where even if an image to be displayed larger / smaller is specified, it cannot be displayed in a larger / smaller frame.

  In order to solve the above technical problem, an object of the present invention is to provide an image layout apparatus that can cut off important areas as compared with the conventional technique while maintaining the order of image layout. Furthermore, a template for performing an image layout that satisfies the above is set with a small amount of calculation. Furthermore, when an image to be displayed in large / small is designated in advance, a frame allocation template that can display large / small images by assigning large / small frames to the image while keeping the important area cut off small is reduced in computational complexity It is to provide an image layout apparatus that can be set by the above.

  In order to achieve the above object, the present invention is characterized in that an image layout apparatus that divides an output page into frames and lays out a plurality of images while maintaining the relative order of the images has the following configuration. There is.

  (1) From the set of candidate images, an image designation accepting unit that accepts designation of a smaller number of favorite images than the number of frames on the output page, and an additional image that compensates for the shortage of favorite images with respect to the number of frames on the output page. Based on the order range setting means for setting the order range when selecting from the set, the allowable shape range setting means for setting the allowable shape range for each image, and the allowable shape range of the candidate images included in each order range, Means for setting a representative allowable shape range representing the allowable shape range of an additional image selected from the order range, and virtual images selected from each favorite image and each order range for each frame of a plurality of template candidates with different frame divisions Based on the degree of divergence between the shape of each frame and the allowable shape range or representative allowable shape range of each image when the additional images are pasted in the relative order of the images A template evaluation unit that evaluates each template candidate, and an additional image selection unit that selects an additional image from each order range based on the shape of the frame corresponding to each order range of the template determined based on the evaluation result It was equipped with.

  (2) A template selection means for excluding template candidates including frames having shapes deviating from the allowable shape range of each favorite image or image within each order range from the target of template evaluation is provided.

  (3) The image processing apparatus further includes means for imposing a size constraint on at least a part of the favorite image, and the template selection unit selects a template candidate whose frame size corresponding to the favorite image with the size constraint does not satisfy the size constraint. It was excluded from the target of the template evaluation.

According to the present invention, the following effects are achieved.
(1) From the image group of pasting candidates with order, conventionally, templates were selected after selecting images for the number of frames on the output page in advance, but in the present invention, the number of favorite images is smaller than the number of frames on the output page. After specifying the template and narrowing down the template, an additional image is determined and laid out. It is acceptable to specify the extraction range in the order of additional images by specifying the position of the favorite image when the number of favorite images is less than the number of frames on the output page, and to allow each extraction range to have a width. The width of the frame shape that can be created and the extraction position of the additional image that compensates for the shortage of the image can be changed to increase the degree of freedom of image layout. By determining, it is possible to realize an image layout in which important regions are cut off more than before without losing the order of all pasted images including additional images that compensate for the shortage of images.

  (2) In order to reduce clipping of important areas, template candidates whose aspect ratio of each frame greatly deviates from the range of representative acceptable aspect ratios of images are excluded from the evaluation target in advance from among a large number of template candidates. The number of template candidates to be subjected to template evaluation is narrowed down in advance, and the amount of calculation for template evaluation can be reduced.

  (3) The order of additional images when laying out images including size-constrained images that you want to display especially large / small from the group of images with pasting candidates that are smaller than the number of frames on the output page. The extraction range above is defined by the position of the favorite image, and by allowing each extraction range to have a width, the allowable frame shape is widened, and the extraction position of the additional image that compensates for the shortage of the image By making the image variable, the degree of freedom of image layout increases, and after template evaluation, an additional image with little difference from the frame shape or size constraint is determined, so that the order of the image pasted image is not impaired, It is possible to realize an image layout in which an image that is desired to be displayed in a large / small manner is pasted on a large / small frame while reducing the number of clippings.

  (4) In order to reflect the specified size constraint on the layout, out of many template candidates, candidates whose frame area corresponding to the size constraint image is larger / smaller than the predetermined value are excluded from the evaluation target in advance. Therefore, the number of template candidates to be subjected to template evaluation is narrowed down in advance, and the amount of calculation for template evaluation can be reduced.

It is a functional block diagram of the frame division device concerning a 1st embodiment of the present invention. It is the figure which showed an example of the template candidate. It is a figure expressing the setting method of the order range of an additional image. It is a figure expressing the extraction method of an important area | region. It is a figure expressing the setting method of the allowable aspect ratio range. It is a figure expressing the selection method of an additional image. It is the flowchart which showed operation | movement of 1st Embodiment of this invention. 8 is a flowchart of template candidate narrowing-down processing in FIG. It is the figure which showed the example of a setting of the order range of an additional image. It is a functional block diagram of the frame division device concerning a 2nd embodiment of the present invention. It is the figure which showed an example of the template candidate whose frame number is "4". It is the flowchart which showed the operation | movement of 2nd Embodiment of this invention. 13 is a flowchart of a template candidate narrowing-down process in FIG.

[First embodiment]
FIG. 1 is a functional block diagram of an image layout method and apparatus according to the first embodiment of the present invention. Here, illustrations of components unnecessary for the description of the present invention are omitted.

  The image input unit 1 includes the shooting date and time, sequential file names, and the like, such as still images captured by a digital camera or read by a scanner, or key frames extracted from a number of frame images constituting a video. A number of still images including information on the order of the images are input and temporarily stored. Hereinafter, these images to be pasting candidates are collectively referred to as “candidate images”.

  The image designation accepting unit 2 accepts designation of a favorite image to be actually pasted on the framed output page from among the large number of candidate images. In the present embodiment, a number smaller than the number of images (number of frames) to be actually pasted is designated as a favorite image.

  The frame allocation condition receiving unit 3 receives the specification of the output page size S and the frame number N as conditions required for the frame allocation template. For example, if four images are pasted on A4 size (vertical) paper, “A4 (vertical)” as size S and “4” as frame number N are accepted.

  The template candidate output unit 4 outputs a number of frame division candidates (templates) that divide the output page of the set size S into the set number N of frames. That is, the number of frames is N, and a large number of template candidates having different shapes, positions, and sizes are output. In each template candidate, an image pasting order is set for each frame, and details thereof are disclosed in, for example, Non-Patent Document 2 and Patent Document 2.

  The template candidate may be generated each time by the template candidate output unit 4 itself based on the frame allocation condition, or the template candidate output based on the frame allocation condition from a large number of template candidates stored in advance. The unit 4 may select. The template candidate storage location is not limited to the device, but is stored in a server on the network, so that the template candidate output unit 4 obtains the template candidate that satisfies the frame allocation condition from an external server each time. May be.

  FIG. 2 is a diagram showing an example of a template candidate. Various templates satisfying the conditions of size S and number of frames N (here, “4”) are generated. Ordinal numbers 1 to 4 corresponding to the number of frames are virtually allocated in the same order as in mamanga.

  The order range setting unit 5 includes an additional image number calculation unit 5a that calculates the number of additional images extracted from the candidate images in order to compensate for the shortage of the number of favorite images with respect to the number N of frames, A range when extracting additional images from the set of candidate images while maintaining the relative order of the images is set as the order range.

  That is, as shown in FIG. 3, if the number of frames on the output page is “4” and there are only two favorite images Ia and Ib, two additional images are selected as the candidates. It is necessary to extract from a set of images. At this time, the order range from which each additional image is extracted includes “order range before favorite image Ia”, “order range between favorite images Ia and Ib”, and “order after favorite image Ib”. There are three "ranges". Therefore, here, one of the combinations for selecting two from the three order ranges is randomly selected. In the present embodiment, selection of extracting a plurality of additional images from the same order range is also permitted.

  In the above, one of the combinations for selecting two from the order range is selected at random. However, the selection method is not random, and all combinations may be selected in order. Alternatively, selection may be made preferentially from an extraction range in which the number of order ranges and the shooting time interval are wide. This makes it easy to obtain additional images of various scenes different from the favorite image.

  The important area extraction unit 6 extracts an important area Rt from the designated favorite image and all candidate images belonging to the selected order range. This important area Rt is an area that is considered to be particularly important in the image. For example, in the case of a snapshot including a person, the area including the face is set as the important area Rt. Such an important region Rt may be set manually by a user via a user interface (not shown) or automatically using a known face recognition application. In the present embodiment, the entire image is allowed to be an important region.

  Note that the important areas are generally set using Visual Attention Model (Itti, L. Koch, C., and Niebur, E., "A model of saliency based visual attention for rapid scene analysis.IEEE Trans. On Pattern. Analysis and Machine Intelligence no. 11, pp. 1254-1259, Nov., 1998), or Ming-Ming Cheng, Guo-Xin Zhang, Niloy J. Mitra, Xialei Huang, Shi-Min Hu, "Global Contrast based Salient Region Detection ", CVPR 2011 is used to set the target object area, and a rectangular important area may be set to include it.

  The allowable shape range setting unit 7 determines the range of shape values allowed for the image to be pasted. As the shape value, a value calculated from numerical values related to the width and height of the image, such as a value obtained by dividing the width of the image by the height or a value obtained by multiplying them by a logarithm, can be used. In this embodiment, the aspect ratio A indicating the shape is a value obtained by dividing the horizontal width w of the rectangle set as the important region by the height h of the rectangle (A = w / h).

  In the present embodiment, the allowable shape range setting unit 7 calculates the allowable shape aspect ratio range of each favorite image based on the important region Rt of each image, and the selected order range. A second setting unit 7b that calculates a representative value of the allowable aspect ratio range of all candidate images belonging to the candidate image based on the important region Rt of each candidate image is included.

  FIG. 4 is a diagram schematically showing a method for setting the allowable aspect ratio range. In the present embodiment, the rectangular aspect region Rt extracted from the favorite image [FIG. (A)] [FIG. (B)] is allowed to have an aspect ratio of a shape that extends in the left-right direction to the left and right sides of the original image. The upper limit value Amax of the aspect ratio range, and the aspect ratio of the shape in which the important area is extended vertically to the upper and lower sides of the original image is the lower limit value Amin of the allowable aspect ratio range [(c), (d)] Is done.

  FIG. 5 is a diagram schematically showing a method of setting an allowable aspect ratio range by the second setting unit 7b. For each candidate image I1 to Im belonging to the range for each order range, the allowable aspect range is set. The range from the maximum aspect ratio to the minimum aspect ratio in all the allowable aspect ratio ranges obtained in the same manner as described above is the representative value of the allowable aspect ratio ranges of all candidate images belonging to the order range, that is, the representative allowable shape. Scope.

  Returning to FIG. 1, the template selection unit 9 selects only the template candidates suitable for the current designated content from among a large number of template candidates output from the template candidate output unit 4, and selects the other candidates from the evaluation targets. By excluding in advance, the template candidates to be evaluated are narrowed down to a small number.

  In this embodiment, when a virtual additional image selected from each favorite image and candidate images within each order range is associated with each frame of the current template candidate while maintaining the relative order of the images, Template candidates whose aspect ratio is out of the representative allowable shape range set in the allowable shape range of the corresponding favorite image or the order range to which the additional image belongs are excluded from the evaluation target with a small amount of calculation.

  For this purpose, in the present embodiment, the allowable aspect ratio range of all favorite images and all candidate images within each order range is checked in advance, and templates including frames outside the range are excluded in advance. In addition, the range of the allowable aspect ratio is likely to be narrower for the favorite image than the candidate for the additional image.

  Therefore, in this embodiment, an image having the narrowest allowable aspect ratio range is selected from the favorite images whose allowable aspect ratio tends to be narrow, and only the frames corresponding to the image are considered without considering the aspect ratio of other images. Evaluation is made based on the degree of shape divergence, and template candidates whose evaluation values are below a predetermined threshold are excluded without evaluating other frames.

  Then, by repeating such processing in the order of images with a narrow aspect ratio range and excluding template candidates, it is not necessary to evaluate all frames for a template that is unlikely to be selected, so the calculation amount is saved. It becomes possible.

  The template evaluation unit 10 stores virtual favorite images selected from the favorite images and candidate images within the order ranges in association with the frames of the current template candidate while maintaining the relative order of the images. Based on the degree of deviation between the frame shape and the allowable shape range (favorite image) or representative allowable shape range (additional image) of each image, the evaluation value of each template candidate is calculated.

  In this embodiment, as disclosed in a patent application (Japanese Patent Laid-Open No. 2012-019387) by the inventors of the present invention, the aspect ratio of each frame is determined from the allowable aspect ratio range of an image pasted on the frame. The degree of divergence between each frame and the image is evaluated using an evaluation function that gives a lower evaluation as it deviates, and each template candidate is evaluated by the sum of one page. The template determination unit 10a determines the template candidate with the highest evaluation as the current template.

  The additional image selection unit 11 selects and adds candidate images satisfying the aspect ratio of each frame from each order range with reference to the aspect ratio of each frame corresponding to each of the order ranges, for the determined template. An image. When there are a plurality of additional image candidates having the same aspect ratio, they can be selected by an arbitrary image selection method. For example, it is possible to select at random, or to calculate the degree of similarity with an image already selected in a color layout or the like and select the image that is not the most similar. FIG. 6 is a diagram schematically illustrating the function of the additional image selection unit 11. Here, the candidate image I2 is selected as the additional image.

  The image layout unit 12 includes a cutout unit 12a that cuts out an image area including the important region Rt from each image according to the shape of each frame of the template, and a scaling unit 12b that scales each image according to the size of the corresponding frame. A predetermined range including the important region Rt from each favorite image and additional image is cut out in accordance with the aspect ratio of each corresponding frame of the adopted template, and further laid out after each image is enlarged / reduced according to the frame size.

  Next, a template evaluation method according to the present invention will be described in detail with reference to the flowcharts of FIGS. Here, an example will be described in which five favorite images are designated and pasted from 15 candidate images to an output page having a frame number N of “7”.

  In step S1, the frame division condition accepting unit 3 accepts an input of “7” as the number N of frames. In step S2, the image designation accepting unit 2 accepts an operation for designating five favorite images from 15 candidate images I1 to I15. In this embodiment, it is assumed that I4, I7, I9, I11, and I15 are designated as favorite images.

  In step S3, in the additional image number output unit 5a, the difference (= 2) of the number of favorite images (= 5) with respect to the number of frames N (= 7) is obtained as the number of additional images to compensate for the number of images that are insufficient. In step S4, the order range setting unit 5 sets an order range for extracting the two additional images from the remaining set of candidate images.

  In the present embodiment, as shown in FIG. 9, the first order range [I1, I2, I3] is from the top to the favorite image I4, and the second order range [I5, I6] is between the favorite images I4 and I7. The third order range [I8] is between the favorite images I7 and I9, the fourth order range [I10] is between the favorite images I9 and I11, and the fifth order range [I12, I3, I14 is between the favorite images I11 and I15. ].

  In step S5, an order range for reading the two additional images is randomly determined. Here, the description is continued assuming that the second order range and the fifth order range are determined. In step S6, the important area extracting unit 6 extracts important areas from each of the five favorite images I4, I7, I9, I11, and I15, and all of them belonging to the second and fifth order ranges. An important region is extracted from the candidate images.

  In step S7, the first setting unit 7a of the allowable shape range setting unit 7 obtains the maximum value APmax and the minimum value APmin of the aspect ratio including the important area for each favorite image, and the range from APmax to APmin is determined. , The allowable aspect ratio range of each favorite image is set.

  In step S8, the maximum value APmax and the minimum value APmin of the aspect ratio including the important region are obtained for all candidate images belonging to the second and fifth order ranges. Furthermore, the range from the maximum value of all APmax to the minimum value of all APmin is set as a representative allowable shape range that represents the allowable aspect ratio range of the candidate images belonging to each order range.

  In step S9, the template candidate output unit 4 sequentially provides the frame allocation template candidates of the frame number N (= 7). In step S11, it is determined whether or not the current template candidate is suitable for evaluation, and the template selection unit 9 performs a narrowing process for excluding inappropriate templates from the evaluation target.

  FIG. 8 is a flowchart showing the template narrowing-down process. In step S51, the aspect ratios of the frames corresponding to the five favorite images I4, I7, I9, I11, and I15 are set to the respective images I4. , I7, I9, I11, and I15 are determined. It should be noted that the allowable aspect range may be expanded by a predetermined allowable error.

  If this condition is satisfied, the process proceeds to step S53. If not satisfied, the current template candidate is excluded from the evaluation targets, and the process ends. Note that, as described above, the templates may be evaluated in order from images with strong restrictions on the allowable aspect ratio, or only frames corresponding to favorite images may be evaluated for each template. Here, a method for evaluating each template will be described.

  In step S53, the allowable aspect ratio range of each frame corresponding to the additional image extracted from each order range or a range obtained by adding the allowable error to the allowable aspect ratio range is set as the representative range set in the order range. It is determined whether it is within the allowable shape (aspect) range. If it is within the range, the process proceeds to step S54, and the current template candidate is registered as an evaluation target. If it is out of the range, the current template candidate is excluded from the evaluation target, and the process ends.

  At this time, when there are a plurality of aspect ratio ranges of one order section represented by a virtual additional image, it is preferable to evaluate each range and record an evaluation value for each range. The case where there are a plurality of aspect ratio ranges in one order section means that, for example, there are three images in the order range, and the allowable aspect ratio of each image is 0.2 to 0.3, 0.25 to 0.4, 0.5 to 1. In this case, it means having a plurality of representative allowable shape ranges, such as having two representative allowable shape ranges of 0.2 (Amin) to 0.4 (Amax) and 0.5 (Amin) to 1 (Amax). Thereafter, the one or more template candidates having the highest evaluation value are determined as the current template.

  Returning to FIG. 7, in step S12, it is determined whether or not the number of registered template candidates to be evaluated has reached the predetermined number. If not reached, the process returns to step S9 to acquire the next template, and the above processes are repeated for the template candidate.

  Thereafter, when the number of registered template candidates to be evaluated reaches a predetermined number, the process proceeds to step S13, and each additional candidate image and a virtual additional image extracted from each order range are stored in each template candidate while maintaining the relative order of the images. The evaluation value of each template is calculated based on the degree of divergence between the shape of each frame when associated with each frame and the shape of the image pasted on the frame.

In step S14, one additional image having an aspect of the range recorded in the range having a high evaluation value is selected one by one from the second and fifth order ranges. When there are a plurality of images having a desired aspect ratio, the images may be selected at random, or other appropriate image selection methods may be applied.
As described above, in the present embodiment, additional images other than favorite images are determined after the order range and the aspect ratio range are obtained, so that the important area can be cut off.

  In step S15, the cutout unit 12a cuts out thumbnail images from each image in accordance with the shape of each frame of the template. Further, in the enlargement / reduction unit 12b, the clipped thumbnail image is enlarged / reduced according to the size of the pasting frame. In step S16, the scaled thumbnail images are laid out at the corresponding frame positions.

  According to the present invention, it is possible to specify a plurality of favorite images from an image group of pasting candidate images having an ordering number smaller than the number of frames on the output page, and to change the extraction position of an additional image that compensates for the shortage of images. As a result, the allowable shape of the additional image has a degree of freedom, and the degree of freedom of the layout increases. Therefore, an image layout in which important areas are further cut off can be realized with a small amount of calculation without impairing the order of the image. become.

[Second Embodiment]
FIG. 10 is a functional block diagram of an image layout method and apparatus according to the second embodiment of the present invention. The same reference numerals as those in FIG. 1 represent the same or equivalent parts, and therefore, the description thereof may be omitted. . The present embodiment is characterized in that when a pasted image is designated to the image designation receiving unit 2, a restriction condition related to the image size can be designated.

  Similar to the first embodiment, the image designation accepting unit 2 is larger than the other images when accepting designation of a favorite image to be actually pasted on a framed output page from among a large number of candidate images. A first accepting unit 2a that accepts designation of a favorite image (size constrained image) that has a size constraint such as wanting to be displayed or displayed small, and a first accepting unit that designates a favorite image without such size constraint (non-size constrained image) 2 reception part 2b is included. Also in this embodiment, the number of favorite images (total number of size constraint images and non-size constraint images) is smaller than the number of images (number of frames) to be actually pasted.

  The frame size determination unit 8 calculates the area of each frame included in each template candidate. The template selection unit 9 includes first and second selection units 9a and 9b, and evaluates only template candidates suitable for the specified content including the current size constraint from among a large number of template candidates output by the template candidate output unit 4. The candidate candidates for evaluation are narrowed down to a small number in advance by selecting them as targets and excluding others from the evaluation targets. Here, a size constraint for increasing the frame shape to which a predetermined image is pasted will be described as an example, but the same applies to a size constraint for decreasing the size.

  When the first selection unit 9a associates each favorite image and virtual additional image with each frame of the current template candidate while maintaining the relative order of the images, the first selection unit 9a selects the frame corresponding to the size constraint image. Template candidates whose area is smaller than the average value of all frames are excluded from the evaluation target. It should be noted that template candidates in which frames smaller than the median or maximum value when the frames in the page are arranged in size order may be excluded.

  FIG. 11 is a diagram schematically illustrating the function of the first selection unit 9a. When the first and fourth images in the relative order among the four images to be pasted are size-constrained images, all of the template candidates shown in FIGS. Since the size of each frame corresponding to the eye and the fourth image is larger than the average size of all frames, it is not excluded. On the other hand, the template candidate shown in FIG. 10C is excluded from the evaluation target because the size of the frame corresponding to the fourth frame is smaller than the average size.

  FIG. 12 is a flowchart showing the operation of the second embodiment of the present invention. Since the same or equivalent processing is executed in the step having the same number as the flowchart of FIG. 7, the description thereof may be omitted or simplified. is there. This embodiment is characterized in that step S10 is newly added.

  In step S <b> 1, the frame division condition reception unit 3 receives “7” as the number N of frames. In step S2, the image designation accepting unit 2 accepts an operation for designating five favorite images from 15 candidate images I1 to I15. In this embodiment, two candidate images I4 and I11 are designated as favorite images (size constraint images) including size constraints to be displayed in large size, and three candidate images I7, I9, and I15 are favorites not including size constraints. Assume that it is specified as an image (non-size restricted image).

  In step S3, in the additional image number output unit 5a, the difference (= 2) of the number of favorite images (= 5) with respect to the number of frames N (= 7) is obtained as the number of additional images to compensate for the number of images that are insufficient. In step S4, the order range setting unit 5 sets an order range for extracting the two additional images from the remaining set of candidate images. Here, the description will be continued assuming that each order range is set as shown in FIG.

  In step S5, an order range for reading the two additional images is randomly determined. Here, the description will be continued assuming that the second order range and the fifth order range are determined. In step S6, the important area extracting unit 6 extracts important areas from each of the five favorite images I4, I7, I9, I11, and I15, and all of them belonging to the second and fifth order ranges. An important region is extracted from the candidate images.

  In step S7, the first setting unit 7a of the allowable aspect ratio range setting unit 7 obtains the maximum value APmax and the minimum value APmin of the aspect ratio including the important area for each favorite image, and the range from APmax to APmin. Is set as the allowable aspect ratio range of each favorite image.

  In step S8, the maximum value APmax and the minimum value APmin of the aspect ratio including the important region are obtained for all candidate images belonging to the second and fifth order ranges. Further, a range from the maximum value of all APmax to the minimum value of all APmin is set as a representative value of the virtual additional image allowable aspect ratio range of the images belonging to each order range. In step S9, the template candidate output unit 4 sequentially generates the frame allocation template candidates having the frame number N (= 7).

  In step S10, the frame size determination unit 8 calculates the area of each frame for the current template candidate. In step S11, it is determined whether or not the current template candidate is suitable for evaluation, and the template selection unit 9 performs a narrowing process for excluding inappropriate templates from the evaluation target.

  FIG. 13 is a flowchart showing a template narrowing-down process. Since the same or equivalent processes are executed in the steps having the same numbers as those in the flowchart of FIG. 8, the description thereof may be omitted or simplified.

  In step S51, it is determined whether or not the current template candidate satisfies the following two conditions for the two size constraint images I4 and I11.

  Condition 1 is that the aspect ratios of the frames corresponding to the size constraint images I4 and I11 are within the allowable aspect ranges of the size constraint images I4 and I11, respectively. When narrowing down the template, an allowable error may be set in the allowable aspect range to widen the allowable aspect ratio range.

  Condition 2 is that the area of the frames corresponding to the size constraint images I4 and I11 is larger than the average area of the areas of all the frames. If both conditions 1 and 2 are satisfied, the process proceeds to step S52. If any of the conditions is not satisfied, the current template candidate is excluded from the evaluation target, and the process is terminated. In the present embodiment, attention is paid to the fact that favorite images including size constraints have a low degree of freedom, and the amount of calculation can be reduced by narrowing down templates from such images.

  In step S52, the aspect ratio of each frame corresponding to each of the non-size constrained images I7, I9, and I15 is within the allowable aspect range of the non-size constrained images I7, I9, and I15, or a range obtained by adding an allowable error thereto. If it is within the range, the process proceeds to step S53. If it is out of the range, the current template candidate is excluded from the evaluation target, and the process ends.

  Returning to FIG. 12, in step S12, it is determined whether or not the number of registered template candidates to be evaluated has reached the predetermined number. In step S13, the shape of each frame when the favorite image and the virtual additional image are associated with each frame of each template candidate while maintaining the relative order of the images, and the shape of the image pasted on the frame. An evaluation value for each template is calculated based on the degree of deviation.

  At this time, the value is calculated without considering the allowable error. When there are a plurality of aspect ratio ranges of one extraction section represented by a virtual additional image, each range is evaluated, and an evaluation value is recorded for each range. Then, the one or more template candidates having the highest evaluation value are determined as the current template.

In step S14, an additional image having an aspect of a range recorded in a range having a high evaluation value is selected one by one from the second and fifth order ranges. When there are a plurality of images having a desired aspect ratio, they may be selected randomly as described above, or other image selection methods may be applied.
As described above, the selected image other than the favorite image is determined after obtaining the order range of the additional images and the range of the aspect ratio that can be added, thereby reducing the cutout of the important region.

  In step S15, the cutout unit 12a cuts out thumbnail images from each image in accordance with the shape of each frame of the template. Further, in the enlargement / reduction unit 12b, the clipped thumbnail image is enlarged / reduced according to the size of the pasting frame. In step S16, the scaled thumbnail images are laid out at the corresponding frame positions.

  According to the present embodiment, a plurality of favorite images including size constraints that are particularly desired to be displayed in a large size are designated from the image group of pasting candidate images that are ordered, and the number of images that are smaller than the number of frames on the output page is specified. By making the extraction position of supplementary images variable, the degree of freedom of layout increases, so image layout that pastes the image you want to display on a large frame without losing the order of images and reducing the clipping of important areas Can be realized with a small amount of calculation.

  DESCRIPTION OF SYMBOLS 1 ... Image input part, 2 ... Image designation | designated reception part, 3 ... Frame allocation condition reception part, 4 ... Template candidate output part, 5 ... Order range setting part, 6 ... Important area extraction part, 7 ... Allowable shape range setting part, 8 ... Frame size determination unit, 9 ... Template selection unit, 10 ... Template evaluation unit, 11 ... Additional image selection unit, 12 ... Image layout unit

Claims (7)

In an image layout apparatus that divides an output page into frames and lays out a plurality of images while maintaining the relative order of the images,
Image designation accepting means for accepting designation of a smaller number of favorite images than the number of frames on the output page from a set of candidate images;
An order range setting means for setting an order range when selecting an additional image that compensates for a shortage of favorite images with respect to the number of frames of the output page from the set of candidate images;
An allowable shape range setting means for setting an allowable shape range for each image;
Means for setting a representative allowable shape range representing the allowable shape range of the additional image selected from each order range based on the allowable shape range of the candidate image included in each order range;
The shape of each frame and each image when a virtual additional image selected from each favorite image and each order range is pasted to each frame of a plurality of template candidates with different frame divisions while maintaining the relative order of each image Template evaluation means for evaluating each template candidate based on the degree of deviation from the allowable shape range or the representative allowable shape range;
Template determining means for determining a template based on the evaluation result;
An image layout apparatus comprising: additional image selection means for selecting an additional image from each order range based on the shape of the frame corresponding to each determined order range of the template.   2. The image layout apparatus according to claim 1, further comprising a frame division condition receiving unit that receives an input operation relating to the number of frames of the output page.   The image layout apparatus according to claim 1, wherein the allowable shape range setting unit sets the entire or important region of each image as the allowable shape range.   4. The method according to claim 1, further comprising template selection means for excluding, from the template evaluation target, a template candidate including a frame having a shape deviating from an allowable shape range of each favorite image or image within each order range. The image layout apparatus according to any one of the above. Further comprising means for imposing a size constraint on at least a portion of the favorite image;
5. The image according to claim 4, wherein the template selection unit excludes, from the template evaluation target, a template candidate whose frame size corresponding to a favorite image with a size constraint does not satisfy the size constraint. Layout device.   The size constraint imposed on the favorite image is that the size of a frame corresponding to the favorite image is larger than a predetermined threshold, and the template selection unit is configured to select a frame corresponding to the favorite image on which the size constraint is imposed. 6. The image layout apparatus according to claim 5, wherein template candidates whose sizes are smaller than the average value, median value, or maximum value of all frames are excluded from the template evaluation targets.   The size constraint imposed on the favorite image is that the size of a frame corresponding to the favorite image is smaller than a predetermined threshold, and the template selection unit is configured to select a frame corresponding to the favorite image on which the size constraint is imposed. The image layout apparatus according to claim 5, wherein a template candidate whose size is larger than an average value, a median value, or a minimum value of all frames is excluded from the template evaluation targets.