JP2016058868A - Image processing system and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem in which, when seam carving is applied without any condition to a layout in which a page is condensed, distortion may be generated in an image or even important information in an image may be made to be a target of elimination.SOLUTION: When an image is disposed in a page aggregated layout, image data representing an image to be disposed is reduced with linear scaling. By removing a seam with low importance from the reduced image data, the reduced image data is further reduced.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an image processing apparatus and an image processing method for scaling an image.

  Seam carving is known as an image scaling technique that is conscious of content in an image. Seam carving is a technique that realizes zooming that preserves important points in an image when the image is reduced or when zooming without maintaining the aspect ratio of the image. In seam carving, an importance map (energy map) that represents the importance of each pixel of an image is generated based on brightness gradient, edge information, visual saliency, and the like. Then, the seam (seam) having the lowest importance in the vertical and horizontal directions is searched from the importance map, and the image is scaled while maintaining the rectangle by deleting the seam. In particular, since the important information of the image is retained when the image is reduced, it can be regarded as a technique for selecting important information in the image. As a technique in which seam carving is applied to a document, a technique for deleting a seam from a document background in a well-balanced manner is known (see, for example, Patent Document 1).

JP 2010-187376 A

  Seam carving is a technique for retaining important parts in an image. However, if the reduction ratio is large, the image may be distorted or important information in the image may be reduced.

  An image processing apparatus according to the present invention includes an acquisition unit that acquires an instruction to arrange an image in a page-aggregated layout, a first reduction unit that reduces image data indicating an arrangement target image by linear scaling, And a second reduction means for further reducing the image data reduced by the first reduction means by deleting a seam having low importance from the image data reduced by the first reduction means. To do.

  According to the present invention, by applying an appropriate seam carving process, it is possible to provide a layout that retains important portions.

It is a block diagram which shows the example of the hardware used by description of embodiment. It is a block diagram which shows the example of the software used by description of embodiment. FIG. 5 is a diagram illustrating a flow of an image layout used in the description of Embodiment 1. It is a figure explaining the seam carving used by description of Embodiment 1. FIG. It is a figure which shows the example of the page aggregation completed image used by description of Embodiment 2, and the example which applied seam carving to this image. FIG. 10 is a diagram illustrating a flow of a re-layout process of print data that has been subjected to page aggregation layout and is used in the description of Embodiment 2. FIG. 10 is a diagram illustrating a flow of layout processing when increasing the number of page aggregations used in the description of Embodiment 3.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.

<Hardware configuration diagram of image processing apparatus>
FIG. 1 is a hardware block diagram illustrating the configuration of the image processing apparatus according to the present exemplary embodiment. In this embodiment, the image processing apparatus will be described using a form that functions as a printer apparatus. However, the image processing apparatus can also be realized as an independent image processing apparatus.

  The image processing apparatus 100 in FIG. 1 includes a controller unit 150, an operation unit 114, and a printer engine 115. The controller unit 150 includes a CPU 101, RAM 102, ROM 103, HDD 104, system bus 105, network I / F 106, operation unit I / F 107, image companding unit 111, image rotating unit 112, RIP 109, and printer image processing unit 110.

  The controller unit 150 controls input / output of image signals and device information. The CPU 101 reads the program stored in the ROM 103 or the HDD 104 into the RAM 102 and executes it. Further, the CPU 101 comprehensively controls each device connected to the system bus 105. A RAM 102 is a main memory of the CPU 101 and is used as a work area for a control program of the image processing apparatus. The ROM 103 stores a boot program that is executed when the power is turned on, and the HDD 104 stores an operating system and a control program body of the image processing apparatus. The HDD 104 is also used for temporarily or long-term holding large-capacity data such as print data and image data.

  A network I / F 106 is connected to the local area network 113 and performs input / output of print data and device information. The operation unit I / F 107 is an interface unit with the operation unit 114, and outputs bitmap data to be displayed on the operation unit 114 to the operation unit 114. Also, it serves to transmit information input by the user of the image processing apparatus from the operation unit 114 to the CPU 101. The operation unit 114 includes, for example, a liquid crystal panel and a sound source as output devices, and a touch panel and hard keys as input devices. The controller unit 150 is connected to the printer engine 115 via the device I / F 108. The device I / F 108 transmits an image signal, instructs a device operation, and receives device information based on an instruction from the CPU 101. The printer engine 115 is an output device that outputs an image signal from the controller unit 150 onto a medium, and may be either an electrophotographic system or an inkjet system.

  The Raster Image Processor (RIP) 109 is dedicated hardware for expanding a display list (hereinafter abbreviated as DL), which is intermediate data, into image data. The RIP 109 processes the DL generated on the RAM 102 by the CPU 101 at high speed and in parallel with the execution of the CPU 101. The printer image processing unit 110 performs image correction such as gamma correction and halftoning on the image data. The image rotation unit 112 rotates image data. The image companding unit 111 performs compression / decompression processing such as JPEG method for multi-valued image data and JBIG, MMR, or MH method for binary image data.

<Software module configuration diagram of image processing apparatus>
FIG. 2 is a software module configuration diagram in the controller unit 150 of the image processing apparatus. Each software module described in FIG. 2 is stored as a program in the HDD 104, loaded into the RAM 102, and executed by the CPU 101. More specifically, each software module is loaded into the RAM 102 by an OS (operating system) operating on the CPU 101, and an execution right is given and executed in units of threads.

  The image processing apparatus includes a data reception unit 201, a job control unit 202, a PDL interpreter 203, a renderer 204, a print driver 205, a user interface 206, a layouter 207, a seam carving processing unit 208, and a job data management unit 209.

  The data receiving unit 201 receives print data transmitted from an external host computer. The received print data is held in the job data management unit 209 via the job control unit 202. The job control unit 202 is responsible for overall job control from data reception to printing. The PDL interpreter 203 interprets PDL data described in a page description language (PDL) included in the print data, and generates a DL that is intermediate data. The generated DL is held in the job data management unit 209 via the job control unit 202. The renderer 204 is a module that generates image data from the DL, and many processes are executed by the dedicated hardware RIP 109. The generated image data is held in the job data management unit 209 via the job control unit 202.

  The print driver 205 sends a print instruction and image data to the printer engine 115 via the device I / F 108. At this time, the print driver 205 controls image correction and halftoning by the printer image processing unit 110.

  The user interface 206 is a module that controls the operation unit 114 via the operation unit I / F 107. Data to be displayed on the liquid crystal panel of the operation unit 114 is generated, and the display on the liquid crystal panel is updated in accordance with an input from the touch panel. If the input from the touch panel is any job execution instruction, the instruction is transmitted to the job control unit 202.

  A layouter 207 is a module that controls a print layout, and generates image data representing a final layout image to be output on a sheet. In particular, scaling processing such as image reduction and enlargement for page aggregation layout and image arrangement control are performed. The actual image scaling process is executed by the layouter 207 instructing the PDL interpreter 203 or the seam carving processing unit 208.

  The seam carving processing unit 208 is a module that reduces an image to a specified size by seam carving processing. Seam carving is a function that realizes image reduction by leaving important parts of an image and reducing non-important parts as described above. In the present embodiment, the seam carving process is described as being executed by software, but may be configured to be processed by hardware.

<Image layout basic flow diagram>
FIG. 3 is a flowchart showing an example of image layout processing for print data executed by the controller unit 150. In this description, it is described so as to be executed by each software module of FIG. Also, this flowchart is realized by reading each program in each software module of FIG. 2 stored in the HDD 104 into the RAM 102 and executing it by the CPU 101.

  First, in step S301, the data receiving unit 201 receives print data, takes out PDL data and a job ticket, and stores them in the job data management unit 209. A job ticket is a file that describes print settings such as duplex output, output paper, and page aggregation, and is used to notify the print settings for each job. Details of the job ticket will be described later.

  In step S302, the job control unit 202 determines whether page aggregation is designated with reference to the job ticket. The page aggregation is a function for reducing a plurality of continuous pages in PDL data to a designated sheet, allocating and printing. For example, it is possible to specify 1 to 5 as attribute values, “1” indicates 1 in 1, “2” indicates 2 in 1, “3” indicates 4 in 1, and “4” indicates 8 in 1. For example, when the page aggregation attribute value included in the job ticket is “2” (that is, 2 in 1), two consecutive pages are assigned to one side of one output sheet. Here, the attribute value “1” (1 in 1) is synonymous with not specifying page aggregation. In step S <b> 302, the job control unit 202 determines whether page aggregation is designated based on the page aggregation attribute value included in the job ticket. If it is determined in step S302 that page aggregation is designated, the process proceeds to step S303. If it is determined that page aggregation is not designated, the process proceeds to step S311.

  In step S303, the layouter 207 determines “image size after simple reduction” and “image size after seam carving” from the number of pages allocated to one sheet. In order to explain these sizes, an outline of the processing of this embodiment will be briefly described. In this embodiment, for example, when two pages of images are allocated to one sheet of paper, such as 2 in 1, each image to be arranged is first simply reduced. Simple reduction is a general linear reduction process rather than a reduction process according to the importance as in seam carving. Then, reduction processing by seam carving is performed on each simply reduced image. Each image subjected to the reduction process by seam carving becomes an image arranged in an area allocated to one sheet of paper, such as 2in1. That is, the image of each page that has been subjected to the reduction process by seam carving (hereinafter, the image of each allocated page is referred to as a page image) is then arranged in a predetermined area at the same magnification. In step S303, the “image size after seam carving” is first determined from the number of pages allocated in this way. Then, the image size before the seam carving process of “image size after seam carving” is determined as “image size after simple reduction”. Then, as described in the later-described steps, the page image of each page is simply reduced so as to be the “image size after simple reduction”.

  Step S303 will be further described. As described above, in this embodiment, the page image of each page generated by the seam carving is arranged in the page aggregation arrangement while maintaining the same magnification to generate the output image. Accordingly, the “image size after seam carving” is calculated from the relationship between the number of pages allocated, the output paper, and the output resolution. Here, the image size after seam carving is a pixel value. In step S303, the layouter 207 first calculates the “image size after seam carving”. Next, the layouter 207 determines the reduction ratio by seam carving based on the following seam carving reduction ratio correspondence table. The reduction ratio by seam carving is the ratio between the image size after seam carving and the image size before seam carving. The reduction rate by seam carving is defined in advance as shown in the table shown in Table 1 according to the number of page allocations.

  The reason why the reduction ratio as shown in Table 1 is set will be described. In Table 1, the larger the number of page allocations, the higher the reduction ratio (that is, the smaller the image size after reduction), and the information selection by seam carving works strongly. This is because the smaller the number of assignments, the more noticeable the distortion of the image, and the larger the number of assignments, the greater the need to increase the visibility of the image. That is, a large number of allocations reduces the size of a page image for one page on one sheet of each image. Therefore, in order to increase the visibility of highly important parts by seam carving, the reduction rate is increased so that information selection by seam carving works strongly. On the other hand, there is a possibility that unintended information selection may be performed to increase the reduction ratio, and image distortion may become conspicuous. Therefore, when the allocation number is small, the reduction ratio is set lower than when the page allocation number is large. In addition, Table 1 shows an example to the last, and it does not necessarily have to be based on such a reduction ratio. For example, the same reduction ratio may be used when the page allocation number is “2” and “4”. Moreover, it may be configured such that the user can arbitrarily set such a reduction ratio.

  The layouter 207 calculates the image size (pixel value) after simple reduction from the image size after seam carving calculated previously and the seam carving reduction rate as shown in Table 1. That is, the image size obtained by enlarging the image size after seam carving according to the seam carving reduction rate is obtained as the “image size after simple reduction”.

  The next processing from step S304 to step S308 is processing for each page collected on one sheet. In step S304, the PDL interpreter 203 takes out a page in the PDL data and calculates a reduction ratio by simple reduction. The page size in the PDL data is defined as a logical size that does not depend on the output resolution. The PDL interpreter 203 first calculates a pixel size considering the output resolution from this logical size. Then, the reduction ratio is calculated so that the calculated pixel size becomes the same size as the image size (pixel value) after the simple reduction determined in step S303.

  Next, in step S305, the PDL interpreter 203 interprets the PDL data of the page, and renders it by reducing it at the reduction ratio calculated in step S304. The PDL interpreter 203 holds the generated image data in the job data management unit 209. In this embodiment, the reduction process is performed when the logical coordinates specified by the drawing command in the PDL data are converted into pixel coordinates. Specifically, a transformation matrix reflecting the reduction ratio is connected to a coordinate transformation matrix from logical coordinates to pixel coordinates. Then, a drawing process using the connected matrix is performed. However, this does not limit the reduction processing in the present embodiment, and a configuration in which image data after rendering at the same magnification is reduced and converted may be used. Further, the reduction processing at this time may be reduction with image interpolation instead of simple reduction by thinning. Any reduction processing method may be used as long as it is a linear reduction processing that maintains the occupation ratio of the content in the image.

  Next, in step S306, the seam carving processing unit 208 generates an importance map (energy map) of the rendered image generated in step S305. The importance map is data representing the importance corresponding to each pixel of the image in 8 bits, that is, 256 levels. The importance map is generated by applying, for example, an edge enhancement filter to the image. As the edge enhancement filter, an existing method such as a Laplacian filter, a Prewitt filter, or a Sobel filter is applied. However, the importance map generation method is not limited to this, and an L1 norm, an L2 norm based on a luminance gradient, a visual saliency map, Hog (Historm of Gradient), or the like may be used.

  Next, in step S307, the seam carving processing unit 208 searches for the low importance seam using the importance map generated in step S3065 and deletes it from the image generated in step S305. A seam is a one-pixel pass, and each pixel in the seam is connected in one of eight directions: up, down, left, right, upper left, upper right, lower left, and lower right. There are two types, a vertical seam that traverses the image and a horizontal seam that traverses the image, and each direction is connected within an oblique range of the traveling direction. For example, a pixel in a vertical seam is connected to the next pixel in the lower left, lower, or lower right. The seam search method will be described later. In step S307, the seam carving processing unit 208 finds and deletes one vertical seam and one horizontal seam. That is, the image is reduced by one pixel vertically and horizontally. However, when the target size is reduced in either the vertical or horizontal direction, the search and deletion of the seam in that direction is not performed.

  In step S308, the seam carving processing unit 208 determines whether the image has been reduced to the target size, that is, the “image size after seam carving” calculated in step S303. If it is determined that the image has been reduced, the process proceeds to step S309. If it is determined that the image has not been reduced, the process returns to step S306, and the image reduction process by seam carving is continued.

  In step S309, the PDL interpreter 203 determines whether all pages have been processed. If it is determined that all pages have been processed, the process proceeds to step S310. If it is determined that all the pages have not been processed, the process returns to step S304, and the remaining pages are processed.

  In step S310, the layouter 207 arranges each page that has undergone image reduction processing by seam carving in the page aggregation layout. For example, in the case of 4 in 1, four images that have been subjected to image reduction processing by seam carving are combined into one image in the order of upper left, upper right, lower left, and lower right. If there are not enough pages, a blank image is placed instead. The layouter 207 stores image data indicating the synthesized image in the job data management unit 209.

  If it is determined in step S302 that page aggregation is not specified in the job ticket, in step S311, the PDL interpreter 203 renders all pages at the same magnification.

  As described above, in the present embodiment, when realizing a page aggregation layout, each page is reduced in two stages. First, simple reduction is performed, and then reduction by seam carving is performed. This is because, when only the reduction by seam carving is performed, the image detriment and the uncomfortable feeling felt by the operator due to image processing become too strong. Also, the seam carving is performed only when the page aggregation layout is designated because the intention of the operator can be reflected. In other words, even if the visibility is somewhat reduced due to page aggregation, it can be determined that the operator gives priority to resource saving and browsing. In other words, since the operator can determine that there is no strong attachment to the original image, it can be determined that application of seam carving giving priority to visibility is appropriate.

  Further, by changing the reduction rate by seam carving according to the setting of image processing, it is possible to optimally control the information selection effect and the balance of image processing.

<List of print settings by job ticket>
Next, details of the job ticket will be described. The job ticket detail table shown in Table 2 is a list of print settings that can be specified by a job ticket that constitutes print data together with PDL data. The print settings include, for example, “double-sided”, “page reduction”, “toner saving”, “paper size”, and “page aggregation”. “Double-sided” is a function for printing a continuous page only on the front side of the paper or printing alternately on the front side, the back side, the front side, and the back side, and it is possible to specify 1: single side, 2: double side as attribute values. “Page reduction” is a function for reducing a page at a specified reduction ratio, and 1: None, 2: 75%, and 3: 50% can be specified as attribute values. “Toner saving” is a function for reducing toner used in printing, and the attribute value can be specified as 1: No, 2: Yes. 'Paper size' is a setting for designating the output paper size, and 1: A4, 2: A3, 3: B4 can be designated as attribute values. 'Page aggregation' is as described above. In this embodiment, the applicability of seam carving is determined based on whether or not page aggregation is specified in step S302 of FIG. 3, but another method is also conceivable. For example, it can be determined that seam carving is applied when page reduction or toner saving is specified, or in accordance with complex conditions.

<Seam search process>
Next, seam search processing by the seam carving processing unit 208 in step S307 will be schematically described with reference to FIG. Seam search is a method for searching for a route having the lowest cumulative importance in the importance map, and uses the concept of dynamic programming.

  FIG. 4A shows an example of an importance map corresponding to an image of 4 pixels × 4 pixels. Each cell corresponds to one pixel of the image, and the larger the numerical value, the higher the importance. The seam carving processing unit 208 searches for a path having the minimum cumulative importance among the vertical seams, that is, paths from any pixel in the first column to any pixel in the fourth column. .

  First, the path with the smallest cumulative importance from the first column to each pixel in the second column is examined. As shown in FIG. 4B, [1A] → [2A], [1C] → [2B], [1C] → [2C], [1C] → [2D] are from the first column to the second column. It can be seen that this is the path with the least cumulative importance to each pixel of the eye. The cumulative importance in the second column is 4, 5, 5, and 4 in order as shown in the right diagram of FIG.

  Next, the path with the smallest cumulative importance reaching each pixel in the third row is examined. At that time, the cumulative importance in the second column is referred to. As shown in FIG. 4 (c), [2A] → [3A], [2A] → [3B], [2D] → [3C], and [2D] → [3D] are paths with the minimum cumulative importance. I understand that there is. At this time, it is not necessary to store all the paths from the first column, and the cumulative importance to each pixel in the third column is 9, 6, in order as shown in the right diagram of FIG. What is necessary is just to memorize | store 8 and 6.

  Next, the route with the smallest cumulative importance reaching the fourth column is examined. As shown in FIG. 4D, [3B] → [4A], [3B] → [4B], [3B] → [4C], [3D] → [4C], [3D] → [4D] It can be seen that the route has the minimum cumulative importance. There are two minimum routes from [3B] and [3D] to [4C], but it is sufficient to find one of the routes with the minimum cumulative importance, so either one should be adopted. It ’s fine. The cumulative importance levels reaching each pixel in the fourth column are 7, 8, 9, and 8 in order as shown in FIG. Therefore, it is found that the lower end of the vertical seam is [4A] pixel, and its cumulative importance is 7.

  Finally, tracing the path with the minimum cumulative importance going back from [4A], the vertical seam is [1A] → [2A] → [3B] → [4C] as shown in FIG. It turns out that there is. Similar processing is performed for the horizontal seam, and the vertical seam and the horizontal seam searched in this way are deleted in step S307.

  As described above, according to this embodiment, it is possible to provide a layout that retains important portions by applying an appropriate seam carving process. That is, by performing a linear reduction process on the image of the layout page first and then performing a seam carving process, it is possible to provide an image with a layout with less distortion while retaining important portions. Seam carving is a process with a large processing load, and the processing time can be shortened by reducing the processing by seam carving.

  Further, by continuously applying a combination of linear reduction of an image and reduction by seam carving, it is possible to suppress image adverse effects and image processing due to seam carving within a certain range. On the other hand, since the function of selecting important information by seam carving works, it is possible to generate an image with better visibility and appeal than linear reduction.

  In addition, by determining whether or not reduction of image data is specified, processing based on the operator's intention can be performed. In many cases, the explicit reduction can be regarded as a resource saving by reducing the number of output sheets or reducing the image area. That is, it can be determined that some image defects and image processing due to seam carving are allowed.

  That is, by appropriately selecting important information and determining the operator's intention, it is possible to perform image processing that appropriately incorporates the operator's needs. Further, by combining linear reduction and reduction by seam carving, a secondary effect of reducing the processing load associated with seam carving can be obtained.

<Print data with page aggregation layout>
In the first embodiment, the example in which the page aggregation process is performed in the image processing apparatus has been described. In the second embodiment, an example in which seam carving processing is performed when page aggregation processing has already been performed in another apparatus will be described. That is, in the second embodiment, an image processing method in which selection of important information by seam carving is applied to print data that has been subjected to page aggregation layout will be described. Most print data is generated by a printer driver installed on the user terminal. In a page aggregation layout using a printer driver, print data with a page aggregation layout already generated is often generated by the printer driver. When seam carving is applied to print data that has been page-gathered, the boundary of each page included in the gage layout is distorted, and the original page section cannot be maintained. For example, in the case of 4in1, even if the original data is equally divided into four as shown in FIG. 5A, the page boundary is eroded as shown in FIG. 5B. As a result, it becomes unclear to which page an object in the page originally belongs. In the second embodiment, a method for applying seam carving while maintaining the original section will be described.

<Re-layout flow of print data with page aggregation layout>
FIG. 6 is a flowchart showing a flow for performing the re-layout process of the print data having been subjected to the page aggregation layout executed in the controller. Since this flow has many parts that are similar to the basic flow of the image layout in FIG. 3, different points will be mainly described.

  First, in step S601, the data reception unit 201 receives print data, extracts PDL data and a job ticket, and stores them in the job data management unit 209. Information indicating whether the print data has been subjected to the page aggregation layout is added to the job ticket. For example, “5” (2 in 1 already completed), “6” (4 in 1 already completed), and “7” (8 in 1 already completed) are attributes for the page aggregation setting instead of the attributes of the first embodiment. Shall be added.

  In step S <b> 602, the job control unit 202 determines whether page aggregation has been specified in the job ticket. If it is determined that page aggregation has been designated, the process proceeds to step S603, and if No, the process proceeds to step S610.

  The processes from the next step S603 to step S607 are performed for each page when there are a plurality of pages that have been consolidated. In step S603, the PDL interpreter 203 interprets the PDL data of the page, executes rendering, generates image data, and stores the image data in the job data management unit 209.

  In step S604, the layouter 207 determines a section on the image of each page based on the number of pages allocated by page aggregation. For example, when 4 in 1 has been completed, four sections are determined as shown in FIG.

  In step S605, the seam carving processing unit 208 generates an importance map (energy map) of the rendered image. This process is the same as step S306 in FIG.

  In step S606, the seam carving processing unit 208 searches for a seam with low importance (energy) for each section and deletes it from the image. Here, the same processing as step S307 in FIG. 3 is executed for each section. Seams are connected within each compartment, but are not necessarily connected between compartments and are often unconnected. Since the vertical seam and the horizontal seam are simultaneously deleted from the plurality of sections, the image is reduced by a plurality of pixels at a time. For example, in the case of 4 in 1, the vertical and horizontal directions are reduced by 2 pixels at a time.

  In step S607, the seam carving processing unit 208 determines whether the size has been reduced to the target size. The target size is determined based on the seam carving reduction ratio correspondence table in Table 1 as in the first embodiment. If it is determined that the image has been reduced to the target size, the process proceeds to step S608. If it is determined that the image has not been reduced to the target size, the process returns to step S605, and the image reduction process by seam carving is continued.

  In step S608, the PDL interpreter 203 determines whether or not all pages have been processed. If it is determined that all pages have been processed, the process proceeds to step S609. If it is determined that all pages have not been processed, the process returns to step S603, and the remaining pages are processed.

  In step S609, the layouter 207 interpolates and enlarges each page to the original size. That is, since the area corresponding to each section (that is, the image of the allocated page) is reduced by the seam carving process, the area corresponding to each section is interpolated and enlarged to the original size. This process is performed for each page.

  If it is determined in step S602 that the print data is not page-gathered, the PDL interpreter 203 renders all pages of the PDL data at the same magnification in step S610. Instead of step S610, the process may proceed to step S302 in FIG. That is, if it is determined that the print data has not been page consolidated, it may be determined in step S302 in FIG. 3 whether page consolidation is designated in the job ticket, and then the processing in FIG. 3 may be performed.

  As described above, it is possible to select and emphasize important information by interpolating and enlarging the original image size after seam carving on the page-combined image. Further, by performing seam search and deletion for each aggregated page section, it is possible to apply the seam carving process while maintaining the page section of the original print data. Note that the image processing shown in the present embodiment can be applied not only to print data but also to scanned image data, and the effect is not limited to print data. The layout page number of the scanned image data may be acquired by the layouter 207 based on input information from the user, or the image data scanned by the layouter 207 is analyzed and estimated according to the position of the blank area. It may be acquired from the layout. When applied to scan image data, for example, a scene where a document on which a page-gathered image is printed on paper is copied can be considered. At this time, according to the processing of the present embodiment, even when a copy instruction is issued at the same magnification, the image on the printed paper is as follows. That is, the size of the section corresponding to a plurality of page images is the same as that of the document, and the scaling factor of the first object included in each page image of the image printed on the paper, and the first object Will have different scaling factors for different second objects. In other words, when the printed paper and the original are compared, the object corresponding to the important information is enlarged on the printed paper, whereas the low importance object corresponding to the seam (part) Will be deleted and reduced.

  In the present embodiment, an example is described in which seam carving processing is performed on print data that has been page-combined, and then enlargement processing is performed to return the page image reduced by seam carving to the original size. It may be the process. That is, the page image of each section is enlarged for the print data that has been page-gathered in consideration of the reduction rate of seam carving. And the form which performs a seam carving process with respect to the page image of each division by which the expansion process was carried out may be sufficient.

<Increase in the number of pages allocated in page aggregation>
In the third embodiment, a method for increasing the page allocation number in page aggregation will be described.

  FIG. 7 is a flow chart of layout processing when the number of page aggregations is increased. The processing content of step S303 in FIG. 3 of the first embodiment is changed. In step S703, the layouter 207 increases the number of allocated images. For example, 2in1 is changed to 4in1, and 4in1 is changed to 8in1. If it was originally 1 in 1, it is not changed to 2 in 1. This is because the intention for the aggregate layout is recognized in 2in1, but the intention is not recognized in 1in1. Thereafter, the layouter 207 determines the image size after simple reduction and the image size after seam carving based on the increased number of allocations. The other processes are the same as those described with reference to FIG.

  Since the selection and emphasis of important information by seam carving processing works, even if the number of allocations is increased, it is possible to suppress a decrease in readability. It is possible to further improve the listability by further resource saving and information aggregation.

<Other examples>
In the above embodiment, a printer is described as an example of the image processing apparatus. However, for example, a computer may be used as the image processing apparatus. For example, a printer driver mounted on a computer may be configured to output data (for example, PDF data or bitmap data) having the layout described above.

  The object of the present invention can also be achieved by executing the following processing. That is, a storage medium in which a program code of software that realizes the functions of the above-described embodiment is supplied to a system or apparatus, and a computer (or CPU, MPU, etc.) of the system or apparatus is stored in the storage medium. Is a process of reading.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code and the storage medium storing the program code constitute the present invention.

Claims (18)

An acquisition means for acquiring an instruction to arrange an image in a page-aggregated layout;
First reduction means for reducing image data indicating an arrangement target image by linear scaling;
A second reduction means for further reducing the image data reduced by the first reduction means by deleting a seam having low importance from the image data reduced by the first reduction means. An image processing apparatus.   The image processing apparatus according to claim 1, wherein the second reduction unit determines a reduction ratio to be used by the second reduction unit according to the number of pages to be aggregated.   3. The second reduction unit determines a higher reduction rate as a reduction rate used by the second reduction unit when the number of pages to be aggregated is larger than when the number of pages to be aggregated is small. An image processing apparatus according to 1.   4. The method according to claim 2, wherein the first reduction unit determines a reduction ratio used in the first reduction unit based on an image size before being reduced by the second reduction unit. The image processing apparatus described.   The image size before being reduced by the second reduction means is determined based on the image size after being reduced by the second reduction means and the reduction ratio used by the second reduction means. The image processing apparatus according to claim 4.   The image processing apparatus according to claim 5, wherein the image size after being reduced by the second reduction unit is determined based on an output sheet, an output resolution, and a number of pages to be aggregated. The acquisition means further acquires an instruction to increase the number of pages to be aggregated included in the layout,
The image processing apparatus according to claim 1, wherein the first reduction unit and the second reduction unit perform reduction according to a layout in which the number of pages is increased. The second reducing means includes
Generate an importance map that represents the importance of each pixel in the image data,
Search the importance map to determine a low importance seam,
The image processing apparatus according to claim 1, wherein the image data is reduced by repeatedly deleting the determined seam until the image has a predetermined size. An acquisition means for acquiring image data indicating a page-aggregated image;
Determining means for determining a section on the image according to the number of pages allocated to the image;
Reduction means for reducing the image data by deleting a seam of low importance for each determined section from each section;
An image processing apparatus comprising: an enlarging unit that enlarges the reduced image data by linear scaling until a section reduced by seam deletion becomes a section of the same size as before the reduction. An acquisition means for acquiring image data indicating a page-aggregated image;
First determining means for determining a section on the image according to the number of pages allocated to the image;
Second determining means for determining a reduction ratio for reducing the image data by deleting a seam having a low importance from each of the determined sections;
An enlargement means for enlarging the acquired image data by linear scaling based on the determined reduction ratio; and a reduction means for reducing the image data enlarged by the enlargement means by deleting the seam. An image processing apparatus. Acquisition means for acquiring image data indicating an image in which a plurality of page images are aggregated;
Printing means for printing the aggregated image on a sheet of paper,
The image printed on one sheet by the printing unit has the same size of the section corresponding to the plurality of page images, and the first object included in each page image of the printed image An image processing apparatus characterized in that a scaling factor of a second object different from the first object is different. Acquisition means for acquiring image data indicating an image in which a plurality of page images are aggregated, and an instruction to print the image data at an equal magnification;
Printing means for printing the aggregated image on a sheet of paper at an equal magnification according to the instruction acquired by the acquisition means;
The image printed on one sheet by the printing unit has the same size of the section corresponding to the plurality of page images, and the first object included in each page image of the printed image An image processing apparatus characterized in that a scaling factor of a second object different from the first object is different. An acquisition step of acquiring an instruction to arrange an image in a page-aggregated layout;
A first reduction step for reducing image data indicating an image to be arranged by linear scaling;
A second reduction step of further reducing the image data reduced in the first reduction step by deleting a seam having low importance from the image data reduced in the first reduction step. An image processing method. An acquisition step of acquiring image data indicating an image that has been page-aggregated;
A determination step of determining a section on the image according to the number of pages allocated to the image;
For each determined partition, a reduction step of reducing the image data by deleting a less important seam from each partition;
An image processing method comprising: an enlarging step of enlarging the reduced image data by linear scaling until a section reduced by seam removal becomes a section of the same size as before the reduction. An acquisition step of acquiring image data indicating an image that has been page-aggregated;
A first determination step of determining a section on the image according to the number of pages allocated to the image;
A second determination step for determining a reduction ratio for reducing the image data by deleting a low-importance seam from each of the determined sections;
An enlargement step of enlarging the acquired image data by linear scaling based on the determined reduction ratio; and a reduction step of reducing the image data enlarged in the enlargement step by deleting the seam. An image processing method. An acquisition step of acquiring image data indicating an image in which a plurality of page images are aggregated;
A printing step for printing the aggregated image on a sheet of paper,
The images printed on one sheet in the printing step have the same section size corresponding to the plurality of page images, and the first object included in each page image of the printed images And a scaling factor of a second object different from the first object is different from the first object. An acquisition step of acquiring image data indicating an image in which a plurality of page images are aggregated, and an instruction to print the image data at an equal magnification;
A printing step of printing the aggregated image on a sheet of paper at an equal magnification according to the instruction acquired in the acquisition step;
The images printed on one sheet in the printing step have the same section size corresponding to the plurality of page images, and the first object included in each page image of the printed images And a scaling factor of a second object different from the first object is different from the first object.   A program for causing a computer to function as the image processing apparatus according to any one of claims 1 to 12.

Images