JP2011065295A - Apparatus, system and program for processing image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To more easily arrange a plurality of content in one page. <P>SOLUTION: In an image processing apparatus, a content acquisition part obtains content output from an image input device, and obtains the attributes of the content. A layout generation part generates layout information for arranging content whose attributes are coincident or similar in one page. In this case, the layout of the content by the coincident or similar attributes (attribute class) units is performed by efficiently using the space of one page by using, for example, the densest packing layout method so that content is prevented from striding the page. An image generation part generates an output image for forming an image based on the content based on the layout information generated by the layout generation part. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, an image processing system, and an image processing program that generate a layout for a page of content.

  In recent image forming apparatuses such as copiers and MFPs (Multi Function Printers), a function (Nin1 function) for forming an image by arranging N originals in one page is provided.

  Japanese Patent Laid-Open No. 2004-228561 displays and outputs the output possible area and the original size of the original to be copied, arranges the original to be copied in the output possible area in the editing means, and outputs the original output format, for example, movement, A technique is disclosed in which editing such as enlargement / reduction, rotation, merging, and overlay is performed, and an editing process result is executed as a job.

  Japanese Patent Application Laid-Open No. 2005-228561 describes a technique in which an output paper template and an input image template are displayed and the template arrangement of the input image is designated on the output paper template on the panel. According to Patent Document 2, the input image template can also be designated for reduction / enlargement / rotation. In addition, the output form can be freely set by a direct operation.

  On the other hand, Patent Document 3 proposes an allocation method in units of content when only necessary content from a page is used as a material. That is, in Patent Document 3, operability at the time of allocating a plurality of images is taken into consideration by displaying a window (plate) indicating an allocation area and a list window indicating thumbnail images of images that are materials.

  Patent Document 4 discloses a technique in which a frame is generated for each document structure element for document data in a magazine layout of printed matter, and the frame is automatically arranged on the magazine. Here, the elements of the document structure refer to headings, texts, columns, images, captions, and the like. According to this patent document 4, it is possible to easily lay out the work conventionally performed by a magazine designer skilled in work even by a user who does not have experience or know-how.

  In Patent Document 1 and Patent Document 2 described above, it is possible to compose a new document by combining a plurality of elements. Here, when the input is image data read by the scanner, the size of the document is a value detected by a sensor or the like in the scan, and the image of the entire read document is handled as image data. . Therefore, when the document size is very large with respect to the output possible area at the time of image formation, the image data has to be reduced as a whole, and each part in the document cannot be adjusted individually. .

  On the other hand, according to Patent Document 3 described above, it is possible to designate trimming of the input image on the display. Therefore, for example, it is possible to selectively extract and assign only necessary portions from the image data to be allocated.

  However, when the number of contents to be handled increases, the labor for the above-described trimming designation and allocation work increases, and the load on the user increases. Further, in the MFP or the like, the above-described trimming designation and assignment work are performed using a touch panel as an input device.

  In this case, the burden on the user increases due to the small display screen of the input device and the unfamiliar user interface. Therefore, it is desirable that content extraction (trimming) and assignment be automated. In particular, a layout generation function that allows a layout having a high degree of freedom and a layout that is easy for the user to be easily compared with an allocation process for allocating a plurality of pages to one page is desired.

  Further, in Patent Document 4 described above, the document data format to be handled is assumed to be a text data format such as an XML (Extensible Markup Language) format, an HTML (Hyper Text Markup Language) format, or a CSV format, and is read from a scanner. However, the point of using image data as material is not taken into consideration.

  Generally, when creating a document or a material, a newspaper, a magazine, a book, or other printed material is used as a material. It is possible to obtain the same document structure as a document in a predetermined format by analyzing the image data obtained by reading the material of the printed matter from the scanner. However, it is conceivable that the accuracy is remarkably lowered as compared with the case where the material is directly provided by electronic data.

  In addition, image data obtained by reading a document from a scanner varies in image quality depending on the document reading environment. This means that, for example, when contents having different reading densities are arranged and printed on the same page, the contents cannot be output after appropriate image processing. That is, Patent Document 4 does not determine the content page allocation and layout method in consideration of the image quality and convenience at the time of final print output.

  The present invention has been made in view of the above, and an object of the present invention is to provide an image processing apparatus, an image processing system, and an image processing program capable of more easily arranging a plurality of contents in one page. To do.

  In order to solve the above-described problems and achieve the object, the present invention provides a content acquisition unit that acquires content based on image data, and layout information that arranges content having the same or similar attributes among the content on the same page A layout generation unit that generates the output image, and an image generation unit that generates an output image in which contents having the same or similar attributes are arranged on the same page based on the layout information generated by the layout generation unit .

  In addition, the present invention provides an image input device having an image data acquisition unit that acquires image data and a content extraction unit that extracts content from the image data acquired by the image data acquisition unit, and the attributes of the content match each other. Or, a layout generation unit that generates layout information for arranging similar content on the same page, and an output image in which content that matches or is similar to each other is arranged on the same page based on the layout information generated by the layout generation unit. An image processing apparatus having an image generation means for generating, and an image forming apparatus for forming an image based on an output image are provided.

  In addition, the present invention includes a content acquisition step for acquiring content based on image data, a layout generation step for generating layout information for arranging content having the same or similar attributes among the content on the same page, and a layout generation step. Based on the generated layout information, a computer is caused to execute an image generation step of generating an output image in which contents having the same or similar attributes are arranged on the same page.

  According to the present invention, there is an effect that a plurality of contents can be easily arranged in one page.

FIG. 1 is a block diagram showing the configuration of an example of an image processing system applicable to the present invention. FIG. 2 is a schematic diagram for explaining a layout generation method applicable to the present invention. FIG. 3 is a flowchart of an example conceptually showing image processing by the layout generation method. FIG. 4 is a flowchart schematically showing an example of a content arrangement method using FFDH. FIG. 5 is a schematic diagram illustrating an example of the close-packed layout generated by FFDH. FIG. 6 is a schematic diagram for explaining blocks and rows. FIG. 7A is a schematic diagram for explaining an example in which blocks are arranged across pages. FIG. 7B is a schematic diagram for explaining an example in which blocks are arranged without straddling pages. FIG. 8 is a flowchart showing an example of a process for generating a page layout according to the first embodiment of the present invention. FIG. 9A is a schematic diagram for explaining an example where an appropriate page break is not found even in an efficient arrangement. FIG. 9-2 is a schematic diagram for explaining the effect of the content arrangement method according to the first embodiment of the present invention. FIG. 10 is a flowchart showing an example of a page layout generation method according to the second embodiment of the present invention. FIG. 11 is a flowchart illustrating an example of a process for adjusting the position of the block itself. FIG. 12 is a flowchart illustrating an example of a process for adjusting the position of content within a block. FIG. 13 is a functional block diagram illustrating an example of functions of the image processing apparatus according to the first embodiment. FIG. 14 is a flowchart illustrating an example of basic image processing according to the first embodiment. FIG. 15A is a schematic diagram illustrating an example of content. FIG. 15B is a schematic diagram illustrating an example of content. FIG. 15C is a schematic diagram illustrating an example of content. FIG. 16 is a flowchart illustrating an example of processing for generating layout information based on whether an image based on content is a monochrome image or a color image. FIG. 17 is a flowchart illustrating an example of processing for generating layout information based on a plurality of image characteristics. FIG. 18 is a functional block diagram illustrating an example of the functions of the image processing apparatus according to the third embodiment. FIG. 19 is a flowchart illustrating an example of processing for combining a plurality of contents. FIG. 20 is a block diagram illustrating a configuration of an example of an image processing system including an image processing apparatus applicable to the fourth embodiment. FIG. 21 is a functional block diagram illustrating an example of functions of the image processing apparatus according to the fourth embodiment. FIG. 22 is a schematic diagram illustrating a display example of a content viewer that is a GUI for confirming content according to the fourth embodiment. FIG. 23 is a schematic diagram illustrating a display example of a content viewer that is a GUI for confirming content according to the fourth embodiment. FIG. 24 is a schematic diagram illustrating an example of an editor display screen which is a GUI for designating content arrangement. FIG. 25 is a schematic diagram illustrating a display example of an editor in which a plurality of layout processing modes can be selected. FIG. 26 is a schematic diagram illustrating an example of a preview screen for confirming the layout.

  Hereinafter, an embodiment of an image processing apparatus according to the present invention will be described in detail with reference to the accompanying drawings. FIG. 1 shows a configuration of an example of an image processing system applicable to the present invention. As illustrated in FIG. 1, the image processing system applicable to the present invention includes an image input device 11, an image processing device 100, and an image forming device 12.

  The image input device 11 has a function of capturing an image and outputting it as image data, such as a scanner, a copier, or a multifunction device having a complex scanning function and printing function. The image input device 11 may include an ADF (Auto Document Feeder) so that a plurality of images can be automatically captured and output continuously. However, the present invention is not limited to this, and the image input device 11 may be provided with a communication function for performing communication via a network such as the Internet or a LAN (Local Area Network), and image data may be captured from a storage medium connected to the network.

  The image input device 11 has a function of performing processing such as trimming on the captured image data, cutting out a predetermined area of the image data, and outputting the content data as content data.

  Further, the image input device 11 can analyze image data obtained by reading a document or content data obtained by trimming the image data, and acquire attributes of the content data. Details of the content data analysis will be described later.

  The attribute of the content data is, for example, an image characteristic of an image based on the content data. This image characteristic indicates, for example, whether the image based on the content data is a monochrome image or a color image, a text or an image, a natural image or an artificial image. The image characteristics indicate the image quality level of the content data, the size of characters included in the content data, and the like.

  The attribute of content data is not limited to image characteristics. For example, the attribute of content data may indicate the degree of association between content data. Also, information indicating the original document source that is the source of the content data may be used as the content data attribute. Furthermore, information indicating the time when the original image data of the content data is acquired may be used as the attribute of the content data.

  The image processing apparatus 100 receives content data and analysis results output from the image input apparatus 11. The image processing apparatus 100 can generate layout information in which one or more pieces of input content data are arranged in units of pages. At this time, the image processing apparatus 100 can arrange content data having the same or similar attribute on the same page based on the attribute of the content data described above.

  The image forming apparatus 12 has a function of forming and outputting an image from content data, such as a printer or a display. The image forming apparatus 12 performs printing and display output of content data in units of pages according to the output data supplied from the image processing apparatus 100.

  The configuration of the image processing apparatus 100 will be described in more detail. In the image processing apparatus 100, a CPU (Central Processing Unit) 101, an I / O (Input / Output) unit 102, a ROM (Read Only Memory) 103, a RAM (Random Access Memory) 104, and a hard disk drive (with respect to the internal bus 110) HDD) 105 is connected. Each unit connected to the internal bus 110 can exchange data with each other via the internal bus 110.

  The HDD 105 stores various programs and data. For example, an OS (Operating System) and an image processing program for executing the image processing method according to the present invention are stored in the HDD 105. Further, image data input from the image input device 11 is stored in the HDD 105. The CPU 101 controls the overall operation of the image processing apparatus 100 using the RAM 104 as a work memory in accordance with a program stored in advance in the ROM 103 or a program stored in the HDD 105. The I / O unit 102 manages input / output of data to / from external devices connected to the image processing apparatus 100 including the image input apparatus 11 and the image forming apparatus 12.

  Image data as content data processed by the image processing apparatus 100 is input from the image input apparatus 11 to the image input apparatus 100 and stored in the HDD 105 via the I / O unit 102 or supplied to the CPU 101. The

  The image processing apparatus 100 can also process text data as content data. The text data is input to the image processing apparatus 100 via, for example, a keyboard (not shown) connected to the image processing apparatus 100 or an external device via the I / O unit 102.

  Output data generated by the image processing apparatus 100 is output to the image forming apparatus 12 via the I / O unit 102.

  In the above description, the image input device 11, the image processing device 100, and the image forming device 12 configuring the image processing system applicable to the present invention have been described as independent devices, but this is limited to this example. Not. For example, the image input device 11 and the image forming device 12 may be integrally configured to be a multifunction device. Furthermore, the image input device 11, the image processing device 100, and the image forming device 12 can be configured integrally.

<Layout Generation Method Applicable to the Present Invention>
Here, in order to facilitate understanding, a layout generation method applicable to the present invention will be described. In this layout generation method, as illustrated in FIG. 2, a layout when a plurality of rectangular contents having various sizes (width × height) are arranged on an arrangement region 200 having a width W and a height H. Is generated. Here, the width W has a finite value, and the height H has an infinite value. The layout indicates the content arrangement on the arrangement area 200, and the layout information includes information (for example, coordinate information) indicating the content arrangement in the layout. A page refers to an area for displaying at one time. In the following, each area obtained by dividing an arrangement area having a width W at a predetermined height h for displaying at one time is called a page.

  FIG. 3 is a flowchart of an example conceptually showing image processing by the layout generation method. Each process in the flowchart of FIG. 3 is executed by the CPU 101 in accordance with the image processing program according to the first embodiment.

  In step S1, a plurality of pieces of content data are acquired by the image processing apparatus 100. In the next step S2, the image processing apparatus 100 generates a close-packed layout based on the plurality of content data acquired in step S1. In the next step S3, the final layout for each page is determined based on the closest packed layout generated in step S2. Then, layout information indicating the final layout determined in step S3 is generated.

<Generate close-packed layout>
The close-packed layout generation process in step S2 of FIG. 3 will be described in more detail. In addition, although the content data actually input can assume various shapes, in the following, it is assumed that an image based on the content data is a rectangle for explanation. Here, a minimum rectangle including an image based on content data may be assumed, or a rectangle considering a margin or the like may be assumed for an image based on content data. Hereinafter, in order to avoid complications, “images based on content data” are simply referred to as “contents”.

In the close-packed layout in step S2, there is a problem that a plurality of contents are arranged so that the following conditions are satisfied and the height is minimized for an area having a predetermined width and an infinite height. And decide the layout of each content.
(1) The content has a rectangular shape, and the width and height are rational values.
(2) Do not allow rotation.
(3) Each content does not overlap each other.

Here, the condition that the contents in (3) do not overlap each other is equivalent to that at least one of the following expressions (1) to (4) holds for the two contents C _i and C _j. . In Expressions (1) to (4), value w and value h are the width and height of the content, respectively, value x and value y are the x coordinate and y coordinate of one end of the content (for example, the lower left corner), respectively. It is.
x _i + w _i ≦ x _j (1)
x _j + w _j ≦ x _i (2)
y _i + h _i ≦ y _j (3)
y _j + h _j ≦ y _i (4)

When deciding which one of the formulas (1) to (4) is to be adopted for each content, it can be said that there are 4 ^{(n (n-1) / 2)} possibilities in simple estimation. This problem is generally referred to as a boxing problem, belongs to NP difficulty, and is considered extremely difficult to solve in a practical calculation time. Therefore, in the present embodiment, a close-packed layout is generated using a technique for obtaining an optimal approximate solution that approximates the optimal solution with the simplest possible procedure.

  In order to solve such a problem, FFDH (First-Fit Decreasing-Height) or NFDH (NFDH) is used as a basic algorithm that satisfies the above-described conditions and has an evaluation value, that is, a degree of deviation from the optimal solution. Next-Fit Decreasing-Height) is known. In the first embodiment of the present invention, the closest packed layout is generated using FFDH. For FFDH, see “EGCoffman, JR., MRGarey, DSJohonson, RETarjan (Bell Lab. & Stanford Univ.)“ Performance bounds for level-oriented two-dimensional packing algorithms ”, SIAM J. Comput, vol.9. , pp808-826, No. 4, 1980 ”(Non-Patent Document 1).

  FIG. 4 is a flowchart schematically showing an example of a content arrangement method using FFDH. Each process in the flowchart of FIG. 4 is executed by the CPU 101 according to a predetermined program. According to FFDH, first, the contents are sorted in descending order, and the first content (that is, the highest content) in the sort order is arranged at the arrangement start position of the arrangement area, and the processing is started. To do. Hereinafter, for the sake of explanation, the arrangement start position is assumed to be the upper left corner of the arrangement area.

  Prior to the start of the processing according to the flowchart of FIG. 4, it is assumed that some content (for example, the first content in the sort order) is arranged in the arrangement area. With this as the last placed content, the processing according to the flowchart of FIG. 4 is started. In step S10, it is determined whether or not the arrangement has been completed for all the contents. If it is determined that the arrangement has not been completed, the process proceeds to step S11.

  In step S11, the next content is selected in the sort order with respect to the content selected immediately before. In the next step S12, it is determined whether or not the content selected in step S11 is the last content in the sort order. If it is determined that it is not the last content, the process proceeds to step S13.

  In step S13, it is determined whether or not the content selected in step S11 can be arranged on the right side of the content arranged last. That is, if the width from the right end of the content arranged last to the right end of the arrangement area is equal to or larger than the width of the content selected in step S11, it is determined that the selected content can be arranged. If it is determined that the content cannot be arranged, the process returns to step S10. If the arrangement has not been completed for all contents, the next content is selected in the sort order (step S11), and the process is repeated. .

  On the other hand, if it is determined in step S13 that the content can be arranged, the process proceeds to step S14, and the content selected in step S11 is arranged to the right of the content arranged last. Information indicating the arrangement position of the content is stored in the RAM 104, for example. Then, the process returns to step S10.

  If it is determined in step S12 described above that the content selected in step S11 is the last content in the sort order, the process proceeds to step S15. In step S15, among the unarranged contents, the contents having the earliest sort order (that is, the highest height) are arranged below the contents arranged at the left end in the width direction with respect to the contents arranged last. Are arranged left justified. Information indicating the arrangement position of the content is stored in the RAM 104, for example. Then, the process returns to step S11.

  FIG. 5 shows an example of the close-packed layout generated by FFDH as described above. In this example, eight contents are arranged in an arrangement area having a width of “10”. The close-packed layout generated in step S2 of FIG. 3 is referred to as an initial page layout.

  As can be seen from the processing according to the flowcharts of FIGS. 5 and 4, in the close-packed layout by FFDH, content having the same or lower height as the content is arranged on the right side of the content. As for the content at the left end, content having the same height as or lower than the content is arranged below the content. Hereinafter, the area that has the width of the placement area, the height and the position in the height direction in the placement area are defined at the upper and lower ends of the content at the left end, is referred to as “block”, and the block delimiter is defined as “row”. (Refer to FIG. 6).

  Here, in the example of FIG. 5, the contents are arranged in a line with the upper ends aligned. Not limited to this, a series of blocks arranged in a line in the right direction with respect to the leftmost block may be arranged in other ways as long as the upper and lower ends do not exceed the range of the upper and lower ends of the leftmost block.

In FFDH, it is known that the evaluation value for the optimal solution is as shown in the following equation (5). Expression (5) is an example when the height and width of the content are normalized to “1”, respectively. In the equation (5), the value H _alg represents the height obtained by FFDH, and the value H _opt represents an optimal solution.
H _alg ≦ 1.7 × H _opt +1 (5)

<First Layout Generation Method>
Next, generation of a final layout for each page in step S3 of FIG. 3 described above will be described. In step S2 of FIG. 3, after generating the initial page layout by the close-packed layout in accordance with the procedure described with reference to FIG. Divide into units. At this time, as illustrated in FIG. 7A, a certain block may be arranged across the page. In the example of FIG. 7A, among the blocks 301 to 304 arranged in the arrangement area 300, the block 303 is arranged across the pages. This is not preferable because the content is divided on the way. Therefore, in the first layout generation method, the layout is adjusted so that each block does not cross the page, and the final layout for each page is generated.

  In the first layout method, a block arranged across pages is moved to the head of the next page. Here, the next page refers to the next page viewed from the page to which the upper end of a block arranged across the pages belongs. That is, in the example of FIG. 7A, the block 303 arranged across the page is moved to the top of the next page, and each block arranged below the block 303 is also moved downward as the block 303 moves. Shift the position to. Thereafter, by repeating this process sequentially for each page, it is possible to finally generate a layout in which all the contents are arranged not to cross the page.

  FIG. 8 is a flowchart showing an example of a page layout generation method according to the first layout generation method. Each process in the flowchart of FIG. 8 is executed by the CPU 101 according to a predetermined program. First, in step S20, the page layout of the target page is acquired. Note that the page layout acquired in step S20 is a layout including blocks related to the target page to be processed. When the target page is the first page, the page layout of the target page is acquired from the initial page layout.

  In the next step S <b> 21, it is determined whether or not there is a block straddling the page in the page layout of the target page. If it is determined that there is no block straddling the page, the process proceeds to step S23.

  On the other hand, if it is determined in step S21 that the layout of the target page includes a block straddling the page, the process proceeds to step S22. In step S22, the block and each block arranged below the block are moved downward so that the block arranged across the page in the layout of the target page is arranged at the head of the next page of the target page. Shift and rearrange. When the block rearrangement is completed, the process proceeds to step S23.

  In step S23, a page layout in the target page is generated according to the current block arrangement state, and the page layout of the target page is determined. Then, the process proceeds to step S24, and it is determined whether or not the page layout has been determined for all pages. If it is determined that there is a page for which the page layout is not determined, the process returns to step S20, and the process is repeated with the next page for which the page layout is determined in step S23 as the target page.

  On the other hand, if it is determined in step S24 that the page layout has been determined for all the pages, a series of processing ends. In this case, when there is no unarranged block, it is determined that the page layout has been determined for all pages.

  As described above, according to the first layout generation method, when content of various sizes is arranged in an arrangement area extending over a plurality of pages, the number of output pages is as small as possible, and the content is not divided into pages. A simple layout can be generated automatically.

  In addition, when content of various sizes is placed in the placement area, there is no guarantee that page breaks can be provided so that all content does not cross pages, and content that is placed across pages occurs. Probability is high. In other words, in order to draw a line (page break) in the width direction so as not to divide the content in a state where the content is arranged, there is a condition that the upper side or the lower side of the content is aligned when viewed in the width direction. Necessary. FIG. 9A illustrates an example in which an appropriate page break is not found even in an efficient arrangement.

  According to the first layout generation method, when content is arranged, the width direction of the arrangement region does not protrude from the height of the content 201 arranged at the left end as illustrated in FIG. Since the contents that fall within the range 202 are searched, the arrangement can be changed in units of blocks. Therefore, it is possible to easily determine the layout of each page from which content is not divided between pages from the arrangement of the initial page layout by the close-packed layout.

<Second Layout Generation Method>
Next, a second layout generation method will be described. In the second layout generation method, in addition to the first layout generation method described above, when there are blocks straddling the page, a block that fits in the target page is searched from the blocks after the block. Then, rearrangement is performed by exchanging the searched block and the block over the page. Thereby, an efficient page layout can be easily generated.

  According to the method for generating the close-packed layout described with reference to FIG. 4, when a new block is formed, content having the maximum height among the unallocated content at that time is arranged at the left end of the block. become. Therefore, the height of the block monotonously decreases (or does not increase) as the block is disposed below. Therefore, when a block is placed across a page, there is a possibility that there is a block under the block that is high enough to eliminate the situation where the block crosses the page by exchanging the placement with the block. There is.

  Therefore, in the second layout generation method, when a certain block is arranged across the page, whether or not the block fits in the page when the arrangement is exchanged with the target block sequentially from the block located below the block. Determine. If there is a block having a height that fits within the page, the arrangement of the block and the arrangement of the block across the page are switched. By repeating the same process every time a block is placed across pages, a page layout can be generated so that all content does not cross pages.

  This will be described more specifically with reference to FIG. In the example shown in FIG. 7A, blocks 301 to 304 are arranged in the arrangement area 300, and among them, the block 303 is arranged across the pages, and the content included in the block 303 is divided by page breaks. I'm stuck. Such division of content is not preferable particularly when the content is a figure or a table.

  Here, by the above-described processing, the arrangement exchange with the block 303 is tried sequentially from the block arranged immediately below the block 303 arranged across the pages. In the example of FIG. 7A, blocks 301 to 303 whose heights are “6”, “3”, and “3” from the top with respect to the height “11” of one page are arranged, and the block 303 has a height. Only "1" protrudes from the page. When the block 303 is removed, a margin of height “2” is generated between the block 302 immediately above the block 303 and the lower end of the page. On the other hand, since the block 304 arranged immediately below the block 303 has a height of “2”, the situation where the block straddles the page is solved by replacing the block 302 and the block 303 and performing the rearrangement. You can see that

  FIG. 7-2 illustrates an example in which the blocks 303 and 304 are replaced and rearranged from the state illustrated in FIG. It can be seen that the block 303 is arranged without straddling the target page. By performing such rearrangement, it is possible to eliminate a situation in which blocks straddle pages while maintaining efficiency.

  FIG. 10 is a flowchart illustrating an example of processing of a page layout generation method according to the second layout generation method. Each process in the flowchart of FIG. 10 is executed by the CPU 101 in accordance with the image processing program according to the second embodiment. First, in step S30, the page layout of the target page is acquired in the same manner as in step S20 described above. Then, in the next step S31, it is determined whether or not there is a block straddling the page in the page layout of the target page. If it is determined that there is no block straddling the page, the process proceeds to step S37.

  On the other hand, if it is determined in step S31 that there is a block that crosses the page in the layout of the target page, the process proceeds to step S32. In step S32, the height y from the lower end of the block arranged immediately above the block determined to straddle the page to the lower end of the target page is obtained.

  In the next step S33, it is determined whether or not a block is further arranged below the target block. If the determination in step S33 is a determination immediately after it is determined in step S31 that there is a block straddling the page, the block straddling the page is set as the target block. If it is determined that more blocks are arranged immediately below the target block, the process proceeds to step S34.

In step S34, when the block arranged immediately below the target block is replaced with a block straddling the page, it is determined whether or not the block fits in the target page. For example, the height y obtained in step S32, is compared with the height h of the block which is disposed immediately below the current block, if the height y is the height h _b above, the block is the target page Is determined to be within the range.

Wakashi, in step S34, higher than the height h _b is the height y, when placed interchanging the block across the target block and page, if the target block is determined not to fit in the page, the process It returns to step S33. Then, the block that has been attempted to be replaced with the block that straddles the page immediately before is set as a new target block, and the determination is performed in the same manner for a block arranged immediately below the target block.

On the other hand, in step S34, the height h _b is not more than the height y, when placed interchanging the block across the target block and page, if the target block is determined to fit on the page, the process steps The process proceeds to S35. In step S35, the target block and the block straddling the page are switched, and the blocks are rearranged. In this case, the target block is arranged immediately below the block immediately above the block straddling the page, and the block straddling the page is arranged at the head of the next page of the target page. Then, from the position immediately below this block, the blocks arranged from directly below the target block are arranged without changing the order. When block rearrangement is performed in step S35, the process proceeds to step S37.

  If it is determined in step S33 described above that no block is placed below the target block, the process proceeds to step S36. In this case, this means that there is no block that can be replaced with a block that straddles the page so as to fit within the target page. Therefore, in step S36, the block and each block arranged below the block are shifted and rearranged so that the block straddling the page is arranged at the head of the next page of the target page. When the block rearrangement is completed, the process proceeds to step S37.

  In step S37, a page layout in the target page is generated according to the current block arrangement state, and the page layout of the target page is determined. Then, the process proceeds to step S38, and it is determined whether or not the page layout has been determined for all pages. If it is determined that there is a page for which the page layout is not determined, the process returns to step S30, and the process is repeated with the next page for which the page layout is determined in step S37 as the target page.

  On the other hand, if it is determined in step S38 that the page layout has been determined for all the pages, the series of processes is terminated. In this case, when there is no unarranged block, it is determined that the page layout has been determined for all pages.

  As described above, according to the second layout generation method, when contents of various sizes are arranged in an arrangement area extending over a plurality of pages, the number of output pages is as small as possible and the contents are not divided by pages. A simple layout can be generated automatically.

  Further, according to the second layout generation method, when a new block is formed by generating the close-packed layout, content having the maximum height among the unallocated contents at that time is arranged at the left end of the block. Will be. Therefore, the height of the block monotonously decreases (or does not increase) as the block is disposed below. Therefore, if a block is placed across a page, the block height is checked sequentially from the block immediately below, and if there is a block that fits within the page, that block and the block that crosses the page By replacing and rearranging, it is possible to easily generate an efficient page layout.

<Third Layout Generation Method>
Next, a third layout generation method will be described. In the third layout generation method, a more preferable page layout is generated by further adjusting the final page layout generated by the first or second layout generation method described above. In this case, a method of adjusting the position of the block itself and a method of adjusting the position of the content within the block are conceivable.

  First, as a first method, a method for adjusting the position of the block itself within the page will be described. That is, in the page layout generated by the above-described first and second layout generation methods, only one block may be arranged for the final page, and all the remaining areas of the page may be blank. Therefore, the page layout is generated by the method described in the first or second layout generation method described above, and after the total number of pages is determined, the position of the block itself is adjusted within the page and between pages.

  FIG. 11 is a flowchart illustrating an example of a process for adjusting the position of the block itself. Each process in the flowchart of FIG. 11 is executed by the CPU 101 in accordance with an image processing program according to the third layout generation method. First, in step S40, the closest packed layout is acquired as described with reference to FIG. 4, and in the next step S41, an initial page layout is generated from the closest packed layout. In this initial page layout, as already explained, there is a possibility that there are blocks arranged across the pages. In the next step S42, rearrangement is performed in units of blocks based on the initial page layout so that there are no blocks across the pages across all pages. The process in step S42 follows the procedure described with reference to FIG. 8 or FIG. 9 and moves the block across the page to the top of the next page or replaces it with another block, so that the block changes the page. Eliminate the situation of straddling.

  When the layout of all pages is determined by the processing up to step S42 and the total number of pages necessary for the layout is determined, the processing proceeds to step S43. In step S43, the position of the block is adjusted.

As the processing in step S43, for example, it is conceivable to rearrange the blocks so that the variation between the blank portions at the bottom of the page is reduced. As an example, when the processing of step S42 is completed, the total height h _avg of the total height of blocks arranged on one page is calculated by dividing the total height of all blocks by the number of pages. Then, the blocks are moved to the final page one by one from each page so that the total height of the blocks arranged on the final page is close to the average value _havg .

In addition, the layout of each page can be adjusted by rearranging blocks within the page. For example, it is conceivable to assign a blank generated at the bottom of the page as a margin between blocks. As an example, as illustrated in the following formula (6), in each page, the height M of the lower blank area is divided by the number of blocks n−1 and the inter-block margin is equally divided between the blocks. Allocate BM.
BM = M / (n-1) (6)

  Of course, the block rearrangement within this page may be combined with the above-described block movement between pages. That is, after the blocks are rearranged so that the variation between the blank portions at the bottom of the page is reduced, the blocks are rearranged in the page to make the blank portions uniform.

  Next, as a second method, a method for adjusting the position of content in a block will be described. Even if the content is moved within the block, the number of pages does not increase unless the block itself is moved. Therefore, a method of adjusting the left / right balance by moving the content in the block in the horizontal direction, for example, can be considered. At this time, it is preferable to move the content in consideration of the size and shape (aspect ratio) of the content arranged in the block.

  FIG. 12 is a flowchart illustrating an example of a process for adjusting the position of content within a block. Each process in the flowchart of FIG. 12 is executed by the CPU 101 according to the image processing program according to the third embodiment. First, in step S50, the closest packed layout is acquired as described with reference to FIG. 4, and in the next step S51, an initial page layout is generated from the closest packed layout. In this initial page layout, as already explained, there is a possibility that there are blocks arranged across the pages. In the next step S52, rearrangement is performed in units of blocks based on the initial page layout so that there are no blocks across the pages across all pages. The processing in step S52 follows the procedure described with reference to FIG. 8 or FIG. 9 and moves the block across the page to the top of the next page or replaces it with another block, so that the block changes the page. Eliminate the situation of straddling.

  When the layout of all pages is determined by the process up to step S52, the process proceeds to step S53. In step S53, the position of the content within the block is adjusted.

In step S53, the visual center of the block and the visual center of the content are, for example, “Steven J. Harrington, J. Femando Naveda, Rhys Price Jones, Paul Roetling, Nishant Thakker,“ Aesthetic Measures for Automated Document Layout ”. (Non-Patent Document 2). For example, the horizontal position of the entire in-block content can be calculated by the following formulas (7) to (9), and the left / right balance degree of the in-block content can be calculated by the following formula (10).

In addition, the meaning of each variable used by above-mentioned Formula (7)-Formula (10) is as follows.
(x _i , y _i ): xy coordinates of content _i
(x _c , y _c ): Visual center of block M _i : Visual center of content
(d _x , d _y ): Maximum distance from the visual center of the block d _h : Block height

Further, since the content can be freely moved within the block, for example, in step S53, the position of the content can be adjusted so that the center position of the content in the block approaches the center of the block. The center position of the in-block content can be calculated by, for example, the following equations (11) and (12), and the balance can be calculated by the equation (13).

  The block and content layout adjustment method is not limited to the above-described example. For example, the arrangement of content between blocks can be adjusted using some indices described in Non-Patent Document 2 described above.

  In the close-packed layout described above, the content is basically arranged with left justification (or right justification). On the other hand, as a layout, it is generally easy to see when the width direction is balanced. According to the third layout generation method, it is possible to realize a more beautiful layout while maintaining the efficiency of the layout by adjusting the left / right balance of the contents inside the block.

  In the above-described close-packed layout, when viewed in units of blocks, the content is basically arranged in an upper left alignment (or an upper right alignment). On the other hand, as the layout, it is said that the center of content is located near the center of the block, the balance is better, and it is generally easier to see. According to the third layout generation method, it is possible to realize a more beautiful layout while maintaining the efficiency of the layout by adjusting the position of the center of gravity of the content inside the block.

<First Embodiment>
Next, a first embodiment of the present invention will be described. In the first embodiment, when content is arranged on a page, content layout information is generated so that content having the same or similar attributes is arranged in one page.

  FIG. 13 is a functional block diagram illustrating an example of functions of the image processing apparatus 100 according to the first embodiment. The image processing apparatus 100 is realized by an image processing program that operates on the CPU 101 described with reference to FIG. For example, each unit of the image processing apparatus 100 is realized as a module in the image processing program.

  In FIG. 13, the image processing apparatus 100 includes a content acquisition unit 401, a layout generation unit 402, and an image generation unit 403. The content acquisition unit 401 acquires the content output from the image input device 11 and acquires the attribute of the content. The layout generation unit 402 generates layout information for arranging content having the same or similar attributes on one page. At this time, the layout of the content in units of matching or similar attributes (attribute classes) uses the above-described closest packing layout method to efficiently use the space of one page, and the content spans the pages. To be done. The image generation unit 403 generates an output image for forming an image based on content based on the layout information generated by the layout generation unit 402.

<Process according to the first embodiment>
FIG. 14 is a flowchart illustrating an example of basic image processing according to the first embodiment. First, as a user operation, in step S <b> 110, the user sets a document on the image input device 11 as an image reading device that reads the document as an image, and operates the image input device 11. The image input device 11 reads the set document and acquires an image (step S100).

  In the next step S101, the image input device 11 analyzes the acquired image and extracts the content. Here, the content means a component of a document. For example, in the case of a school textbook, a group having independent meanings such as diagrams, graphs, practice questions and answers is handled as one content. The process of analyzing the image and extracting the content will be described later.

  15A to 15C illustrate examples of content. The content here refers to all data used as a material when a new document is created. For example, data stored locally in a computer or the like, or an image obtained by scanning a paper document such as a newspaper or magazine.

  FIG. 15A shows an example of data 11A and 11B stored in the folder 10 of the computer. FIG. 15-2 shows an example of images 21A and 21B obtained by scanning and trimming the newspaper 20. FIG. 15C illustrates an example of images 31A and 31B obtained by scanning and trimming a certain page of the magazine 30. In this example, the images 31A and 31B are composed of text. In addition to these, for example, data obtained by copying a part of Web data is also included in the content.

  The image input device 11 further analyzes the extracted content to obtain an attribute (step S102). The content and the attribute of the content are supplied to the image processing apparatus 100.

  These contents and the attributes of the contents are acquired by the content acquisition unit 401 in the image processing apparatus 100. It should be noted that the processing for extracting content from image data and the processing for obtaining attributes by analyzing content may be performed by the image processing apparatus 100.

  When content and attributes are supplied to the image processing apparatus 100, in step S111, the user performs a predetermined operation on the image processing apparatus 100 and is supplied from the contents supplied from the image input apparatus 11. Select the content you want to place on the selected page. In the image processing apparatus 100, in step S103, the layout generation unit 402 selects content in accordance with this operation, and generates layout information for arranging the selected content on the page.

  Here, the layout generation unit 402 generates layout information by arranging content having matching or similar attributes of the selected content on the same page. At this time, by applying the above-described closest packing layout method in attribute class units with matching or similar attributes, the content is arranged so as not to cross the page, and the content is efficiently arranged within one page. . Then, the layout information of each attribute class is integrated into final layout information. Thus, it is possible to realize page allocation and layout that do not include content with different attributes in the same page efficiently.

  When the layout information is generated, the image generation unit 403 in the image processing apparatus 100 generates an output image for forming an image based on the content based on the layout information (step S104). If the image output device 12 is a printer device, this output image is generated, for example, in the format of PDL (Page Description Language). In step S105, the output image is supplied to the image forming apparatus 12. The image forming apparatus 12 forms an image according to the output image and outputs the image.

  A method for extracting content from image data will be described. Various techniques such as a method using projection, a method using black pixel integration, and a method using a blank separator and projection have been proposed as automatic region segmentation techniques. These known segmentation methods can be applied. It is.

  For example, in Japanese Patent No. 3187895, a plurality of small regions including a character string are extracted from a document image, a blank portion or ruled line is detected from the plurality of small regions, and the detected blank portion or ruled line is used as a basis. Discriminating column types including one column, multiple columns, and free columns, integrating the small areas using the blanks according to the column types, and dividing the document image into predetermined areas A document image region dividing method is disclosed.

  Further, for example, in Japanese Patent Application Laid-Open No. 2007-166541, characters, photographs, tables, line drawings and the like mixed on an image are analyzed, and image processing is performed according to content attributes or converted into data in an output format. Techniques to do this are disclosed. According to Japanese Patent Laid-Open No. 2007-166541, a set of black pixel blocks and white pixel blocks in an image is extracted, and characters, photos, figures, tables, frames are extracted from the feature quantities such as shape, size, and set state. , Attributes such as line drawings, and their areas are extracted.

  With respect to the work of collecting documents (contents) required by a manuscript from originals and image data and creating a document, the first embodiment uses content information obtained by analyzing an image read from the manuscript. The function of automatically generating the content layout (including the meaning of page assignment) in the output image is realized. As a result, the user can configure the output document with simple processing.

  At that time, in order to determine the layout of the content so that the content with the same or similar content is allocated on the same page, for example, when copyright information is used as the content attribute, it is easy from the viewpoint of copyright protection. Content that should not be printed can be laid out on a specific page. In this way, it is possible to easily perform printing in consideration of copyright protection by selecting a page including content permitted to be output.

  Note that the layout information generated in step S103 in the flowchart of FIG. 14 can be stored as a layout generation history. Specifically, a layout processing mode, which will be described later, and attributes used for layout generation are stored in, for example, the HDD 105. At this time, the history may be held in association with the profile of the user who performed the operation.

  By maintaining the layout method frequently used by the user, for example, the user prefers to use the layout based on which content attribute is prepared from a plurality of prepared content attributes, or the user can select the layout method among a plurality of layout processing modes. It is possible to analyze which layout processing mode is most likely to be used. Based on the analysis result, it is possible to devise such as displaying frequently used selection items at easy-to-select positions on an attribute selection screen, a layout processing mode selection screen, or the like displayed on a display unit (not shown).

  By maintaining a layout method frequently used by the user, for example, the user prefers to use a layout based on which content attribute is prepared from a plurality of prepared content attributes, or, among a plurality of layout processing modes, the user can Analyzing which layout mode is most likely to be used, etc., and displaying high-frequency options at easy-to-select positions on the attribute selection screen and layout mode selection screen in the display control means It becomes possible.

<Second Embodiment>
Next, a second embodiment of the present invention will be described. In the second embodiment, a layout is generated based on attributes relating to image characteristics of content.

  The first example is an example in a case where the image characteristics indicate whether the image based on the content is a monochrome image or a color image. FIG. 16 is a flowchart illustrating an example of processing for generating layout information based on whether an image based on content is a monochrome image or a color image.

  Here, it is assumed that an order is given in advance to the content designated by the user. For the order, for example, the selection order when selecting contents or the numbering designation by the user may be input, and the result may be used as it is. If no order is given to the contents, the contents may be sorted according to the area or height of the contents.

  First, in step S120, the content acquisition unit 401 acquires an attribute indicating whether the image of the designated content is a monochrome image or a color image. Next, in step S121, the layout generation unit 402 determines whether the image based on the specified content is a monochrome image or a color image based on the attribute. If it is determined that the image is a monochrome image, the process proceeds to step S122, and the content is arranged on the monochrome-dedicated page. On the other hand, if it is determined that the image based on the content is a color image, the process proceeds to step S123, and the content is arranged on the color-dedicated page.

  In step S124, it is determined whether or not processing has been completed for all the specified contents. If it is determined that the process has not ended, the process returns to step S120. On the other hand, if it is determined that the process has been completed for all the specified contents, the process proceeds to step S125, and the layout generation unit 402 generates layout information by integrating the layouts of the monochrome-only page and the color-only page.

  In this way, by arranging the content based on the monochrome image and the content based on the color image separately on different pages, the page on which the content based on the monochrome image is arranged can be printed in monochrome on the image forming apparatus 12. For this reason, it is possible to prevent the deterioration of the image quality of the content due to the monochrome image, which occurs when the color printing is performed on the page where the content of the monochrome image and the content of the color image are mixed. Furthermore, the number of color prints can be reduced, and color toner can be saved.

  The second example is an example in a case where the image characteristics indicate whether the image based on the content is a text or an image. In this case as well, processing can be performed in the same manner as in the first example described above. That is, if the image based on the content is text, the content is arranged on the text-dedicated page. If the image based on the content is an image, the content is arranged on the image-dedicated page.

  According to the second example, in an image processing apparatus that does not include an image area separation processing unit that separates a text area and an image area in image data and a parameter setting unit associated with the image area separation processing unit, The content can be laid out so that the content by the image and the content by the image are not mixed in the same page. By this. Even when a uniform image correction method is applied within a page, image quality degradation is unlikely to occur.

  On the other hand, in an image processing apparatus having an image area separation processing unit, there is a problem in the adaptability of processing for a text area and an image area even when a scanner γ, a filter, gradation processing, and the like are performed using the image area separation result. In many cases, image quality problems may occur in edges and graininess. By using the method of the second example to unify the attributes of the content in the page into text or image, an advantage in image quality is expected.

  The third example is an example in a case where the image characteristics indicate whether the image based on the content is a natural image or an artificial image. A natural image is represented by an image of a photograph or a painting, and is an image that has a low correlation between adjacent pixels and is not suitable for predictive coding. On the other hand, an artificial image is typified by computer graphics. Pixels are arranged according to a simple rule, and the correlation between adjacent pixels is high, which is suitable for predictive coding. In this case as well, processing can be performed in the same manner as in the first example described above. That is, if the image based on the content is a natural image, the content is arranged on the natural image dedicated page. If the image based on the content is an artificial image, the content is arranged on the artificial image dedicated page.

  The natural image and the artificial image differ in image processing (such as contour correction gain) related to image quality improvement. For this reason, a natural image and an artificial image are divided and assigned to pages in advance, so that a uniform image correction method can be applied within the page, and the image quality of content can be improved.

  The fourth example is an example in the case where the image characteristic indicates the image quality level of the image according to the content. Here, the image quality level is defined by providing a threshold for numerical data obtained from image features relating to the ground state, noise, density, and the like of the image. It is desirable to set the threshold value reflecting the characteristics of each model of the image input device 11.

  Next, an example of generating layout information based on a plurality of image characteristics will be described with reference to the flowchart of FIG. In this example, an image characteristic indicating whether the content is an image based on text or an image based on an image, an image characteristic indicating a level of the size of characters constituting the text included in the content, and an image indicating the type of object included in the content Layout information is generated based on the characteristics. Note that the types of objects included in the content are, for example, the above-described natural images and artificial images.

  First, in step S130, the content acquisition unit 401 acquires the attribute of the designated content. Next, in step S131, the layout generation unit 402 determines from the attribute whether the content image is a text or an image. If it is determined that the image is a text image, the process proceeds to step S132, and the content acquisition unit 401 acquires which level the character size belongs to. In the next step S133, the layout generation unit 402 arranges the content on a text-dedicated page for each character size level. Then, the process proceeds to step S136.

  On the other hand, if it is determined in step S131 that the image based on the content is based on an image, the process proceeds to step S134, and the content acquisition unit 401 determines whether the image type of the content is a natural image or an artificial image. To get. Then, in the next step S135, the layout generation unit 402 arranges content on a dedicated page provided for each object type in a predetermined manner. Then, the process proceeds to step S136.

  In step S136, it is determined whether or not processing has been completed for all the specified contents. If it is determined that the process has not ended, the process returns to step S130. On the other hand, if it is determined that the process has been completed for all the specified contents, the process proceeds to step S137, and the layout generation unit 402 generates layout information by integrating the layouts of the dedicated pages.

  First, in an image processing apparatus that does not include an image area separation processing unit and a parameter setting unit that accompanies the image area separation processing unit, a layout is made so that text content and image content are not mixed in the same page. An image correction method can be applied.

  On the other hand, even when the scanner γ, filter, gradation processing, etc. are performed using the image area separation result, there are many problems in the adaptability of processing for the area, and image quality problems may occur in edges and graininess. Sometimes. For this reason, by unifying the attributes (text or image) of the content in the page, an advantage in image quality is expected.

  In addition, it is possible to apply a uniform image correction process within a page by using image quality levels such as image ground state, noise, density, etc., as attributes, and not mixing contents with different image quality levels in the same page. it can.

  Furthermore, since image processing (such as contour correction gain) related to image quality improvement differs between natural images and artificial images, a natural image and an artificial image are allocated to pages in advance, so that a unified image within the page can be obtained. A correction method can be applied.

  Furthermore, from the viewpoint that it is easier to read if the character sizes of the same page are as uniform as possible, for the content by text, the content having the same character size as possible is allocated to the same page.

<Third Embodiment>
Next, a third embodiment of the present invention will be described. In the third embodiment, a process for determining page allocation and layout based on attributes acquired at the time of image acquisition, content analysis, or image designation will be described. That is, in the third embodiment, the characteristics of the image itself are not used as attributes.

  For example, if the content source is a magazine or newspaper, one content may straddle the next page to save space. In such a case, the part straddling the page becomes another content, and there is a possibility that it is distributed and arranged when the layout information is generated. In addition, as a result of performing image region division for content extraction, there may be a case where it is desired to combine content extracted by being divided into two content into one. For such a situation, image data read from a document is not always analyzed in units of image data, but is analyzed over a plurality of image data. As a result, content that spans multiple pages or content that is highly relevant can be joined and handled as one content.

  FIG. 18 is a functional block diagram illustrating an example of functions of the image processing apparatus 100 according to the third embodiment. In FIG. 18, the same reference numerals are given to the portions common to FIG. 13 described above, and detailed description thereof is omitted. As illustrated in FIG. 18, the image processing apparatus 100 according to the third embodiment is a content combining unit that combines a plurality of contents into one content with respect to the image processing apparatus 100 according to FIG. 13 described above. 404 is added.

  For example, as described with reference to FIG. 14, the user sets a document on the image reading device as the image input device 11 (step S110), and the image input device 11 acquires an image based on the document (step S100). Content is extracted and content attributes are obtained (steps S101 and S102). Thereafter, content is selected by a user operation (step S111). The layout generation unit 402 arranges the selected content on the page and generates layout information (step S103).

  At this time, if the plurality of selected contents are one content on the original document, it is conceivable to combine the selected plurality of contents into one content.

  FIG. 19 is a flowchart illustrating an example of processing for combining a plurality of contents. First, in step S140, the content combining unit 404 acquires position information on the original document of a content (content B) designated for a certain content (content A). The position information of the content on the original document is obtained as an attribute when the image input device 11 reads the document, for example, and is supplied to the image processing device 100. Further, it is assumed that the position information of the content A on the original document is acquired in advance. The content combining unit 404 determines based on the acquired position information of the content B whether these two content A and content B are one content on the original document.

  For example, if a distance between two contents is obtained based on the position information of the contents, and the distance is equal to or less than a threshold value, and the position in the height direction in the document of the two contents corresponds, for example, It is determined that the content is one linked content on the original document.

  If it is determined that the content A and the content B are one content on the original document, the content combining unit 404 combines the content A and the content B into one content (step S141). The process proceeds to step S142, and it is determined whether or not the above-described process has been performed for all the designated contents. If it is determined that the processing has been performed for all the specified contents, the series of processing ends. On the other hand, if it is determined that there is content that has not yet been processed, the process returns to step S140.

  When the document sources of a plurality of contents are the same, the relevance between the plurality of contents is often strong. In this case, it is preferable to arrange a plurality of contents having the same original document on the same page. In addition, it is highly possible that a plurality of contents having the same document source have similar image quality levels (background, noise, density, etc.), and are arranged in the same page as in the above-described various effects. The merit is high. The identity of the original document can be determined from image information such as the similarity of background colors and the uniformity of headings. However, the present invention is not limited to this, and when the user designates an image, a method of designating information on the original document is also possible.

  It is also conceivable to generate layout information using information such as the timing of image reading in the image input device 11 and the order of designated image content. For example, consider the case of copying a manuscript from a plurality of magazines, newspapers, textbooks, etc. using a copying machine. In this case, it can be expected that continuously input contents are highly related to each other or have a high possibility of being the same type of document source.

  More specifically, it is conceivable that the image input time is recorded in the image input device 11 and if the difference between the image read times is smaller than a threshold value in a plurality of contents, it is determined that the attributes of the plurality of contents match. Thereby, layout information can be generated based on the time when image data that is the source of content is generated, and content with a high degree of relevance can be arranged in the same page.

<Fourth Embodiment>
Next, a fourth embodiment of the present invention will be described. In the fourth embodiment, a preview screen indicating the progress of content placement processing or an edit screen for instructing placement of content on a page is displayed on the display unit, and content designation or layout confirmation is performed. Provide user support.

  FIG. 20 shows a configuration of an example of an image processing system including an image processing apparatus 100 ′ applicable to the fourth embodiment. Note that, in FIG. 20, the same reference numerals are given to portions common to FIG. 1 described above, and detailed description thereof is omitted. In the image processing apparatus 100 ′ illustrated in FIG. 20, a graphics unit 106 and an input unit 108 are added to the image processing apparatus 100 illustrated in FIG. 1 described above, and a display unit 107 is connected.

  The display unit 107 includes, for example, a display device such as an LCD (Liquid Crystal Display) and a drive circuit that drives the display device. The graphics unit 106 changes the display control signal output from the CPU 101 according to the program to a signal that can be handled by the display unit 107 and supplies the signal to the display unit 107. Thereby, the screen according to the program can be displayed on the display unit 107.

  The input unit 108 is connected to a pointing device such as a mouse or an input device that outputs a control signal corresponding to a user operation, such as a keyboard or a touch panel.

  FIG. 21 is a functional block diagram illustrating an example of functions of the image processing apparatus 100 ′ according to the fourth embodiment. Note that, in FIG. 21, the same reference numerals are given to portions common to FIG. 13 described above, and detailed description thereof is omitted. As illustrated in FIG. 21, the image processing apparatus 100 ′ according to the fourth embodiment has a display control unit 405 and an arrangement setting unit 406 added to the image processing apparatus 100 according to FIG. 13 described above. . The content combining unit 404 in FIG. 18 described above may be further added to the image processing apparatus 100 ′.

  The display control unit 405 generates a display control signal for performing display on the display unit 107. The arrangement setting unit 406 generates coordinate information and operation information based on a control signal corresponding to a user operation on the input unit 108, and performs content arrangement setting. A GUI (Graphical User Interface) can be configured by using the display on the input unit 108 and the display unit 107 described above, the arrangement setting unit 405, and the display control unit 406.

  FIG. 22 shows an example of a content viewer that is a GUI for confirming content according to the fourth embodiment. The content viewer 40 illustrated in FIG. 22 is an example in which images 41A, 41B, 41C, and 41D based on content acquired by reading an image from a document are displayed grouped for each attribute.

  FIG. 23 shows another example of the content viewer. 23 displays the content information 43, 43,... Arranged in the document reading order (the generation order of image data read from the document). As the content information 43, 43,..., For example, the content 44 and attribute information 45 corresponding to the content are displayed together.

  As illustrated in FIGS. 22 and 23, when displaying content, the attribute information 45 serving as a selection index is displayed in association with the content 44 so that the user can select the necessary content. Tips can be given.

  For example, it is possible to separately display whether the content is a text content or an image content. Further, with respect to the content displayed in the content viewer, it is possible to extract the content of the same document from the identity of the background color, etc., and display them together. As a result, the user can quickly find the necessary content from the content generated from the scanned image.

  Furthermore, a GUI can be configured in which an output paper template is displayed on the display screen of the display unit 107 and the user determines the initial arrangement of content by a drag-and-drop operation or the like. FIG. 24 shows an example of a display screen of the editor 50 which is a GUI for displaying the output paper template and the content on one screen and designating the arrangement of the content.

  In the example of FIG. 24, output paper templates 51A and 51B are displayed in units of pages, and contents 52A to 52D are displayed below the output paper templates 51A and 51B. Further, by designating the button 53 by operating the input unit 108 in the editor 50, another output paper template can be displayed. Similarly, by designating the button 54, other contents can be displayed.

  The user operates the input unit 108 while referring to the display screen of the editor 50, for example, to place a desired content among the displayed contents 52A to 52D at a desired position of the output paper templates 51A and 51B. The initial layout of pages can be set by dragging and dropping. The layout generation unit 402 generates layout information based on this initial arrangement.

  There may be cases where it is difficult for the user to imagine the output layout, such as when the input / output paper sizes are special. In the conventional integrated printing and integrated copying, in this case, it is necessary to set the final layout by trial and error by repeating the content arrangement, the printing process, and the layout confirmation, which is very troublesome.

  In order to reduce such a burden on the user, in the fourth embodiment, in the image processing apparatus 100, the input unit 108, the display unit 107, the display control unit 405, and the arrangement setting unit 406 constitute a GUI. Then, the user can set a rough arrangement of contents and page allocation (that is, initial arrangement) while imaging an output result image on the GUI screen. As a result, the document composition process from the content can be visually confirmed, and the load of the content layout processing can be reduced.

  By preparing a plurality of layout processing modes in advance and selecting them by the user, it becomes possible to easily set a layout according to the purpose. FIG. 25 shows a display example of the editor 60 in which a plurality of layout processing modes can be selected. Note that, in FIG. 25, the same reference numerals are given to portions common to the above-described FIG. 24, and detailed description thereof is omitted.

  In the editor 60 illustrated in FIG. 25, buttons 65A and 65B for selecting a layout processing mode are added to the editor 50 illustrated in FIG. FIG. 26 shows an example of a preview screen 70 for confirming the layout when these buttons 65A and 65B are designated.

  The button 65 </ b> A is preset with an arrangement that emphasizes the design (the balance of appearance). When the button 65A is designated, for example, an arrangement in which the above-described third layout generation method is applied to the page is performed. As an example, when the user arranges desired contents from the contents 52A to 52D on the output paper template 51A by drag and drop or the like, the position of the arranged contents is automatically adjusted according to the third layout generation method. . An example of the arrangement when the button 65A is designated is shown in the preview display 72 on the left side of FIG.

  On the other hand, the button 65B allows content to be arranged in a compact manner to efficiently use output paper. When the button 65B is designated, for example, the above-described first layout generation method is applied, and the content is arranged in the closest packing arrangement in the page. An example of the arrangement when the button 65B is designated is shown in the preview display 71 on the right side of FIG.

  The layout required by the user varies depending on the application. For example, if it is a personal note, there is little need to consider the appearance balance in the content layout, so that it is efficient so that output paper can be saved. It is thought that there are many cases where they are arranged. On the other hand, in the case of materials to be distributed to other people, it is desirable to arrange the content in consideration of the alignment and the balance of the arrangement. Thus, by preparing a plurality of layout processing modes, it is possible to easily set a layout according to the application. For example, by using the technique disclosed in Japanese Patent Application Laid-Open No. 2007-242021, layout as described above can be performed in consideration of design.

  Furthermore, in the image processing apparatus according to the present invention, work efficiency is improved by automating the layout portion. Here, when automating the layout of the designated content, the user cannot always correctly infer the layout result. Therefore, as in the fourth embodiment, displaying the output image of the layout processing result (see FIG. 26) has an effect of preventing output with a layout contrary to the user's expectation.

<Other embodiments>
The image processing program executed by the image processing apparatus 100 according to each of the embodiments described above is a file in an installable format or an executable format, such as a flexible disk (FD), a CD (Compact Disk), a DVD (Digital Versatile Disk), or the like. And recorded on a computer-readable recording medium.

  Further, the image processing program executed by the image processing apparatus 100 according to each embodiment described above may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. good. Further, the image processing program executed by the image processing apparatus 100 or the image processing apparatus 100 ′ according to each embodiment described above may be provided or distributed via a network such as the Internet.

  Further, the image processing program executed by the image processing apparatus 100 or the image processing apparatus 100 ′ according to each embodiment described above may be provided by being incorporated in advance in a ROM or the like.

  The image processing program executed by the image processing apparatus 100 or the image processing apparatus 100 ′ according to each embodiment described above is the image processing apparatus 100 described with reference to FIG. 13 or FIG. 18, or the image described with reference to FIG. The module includes a module that generates each unit of the processing apparatus 100 ′. As actual hardware, the CPU 101 (processor) reads out and executes an image processing program from the HDD 105, for example. Each module is loaded on the main storage device (RAM 104) and generated on the main storage device.

11 Image Input Device 12 Image Forming Apparatus 100, 100 ′ Image Processing Device 101 CPU
107 display unit 108 input unit 401 content acquisition unit 402 layout generation unit 403 image generation unit 404 content combination unit 405 display control unit 406 arrangement setting unit

Japanese Patent No. 3430592 JP 2007-110392 A Japanese Patent Laid-Open No. 2001-14479 JP 2005-190099 A

Claims (19)

Content acquisition means for acquiring content by image data;
Layout generating means for generating layout information for arranging contents having the same or similar attributes among the contents on the same page;
An image processing apparatus comprising: an image generation unit configured to generate an output image in which contents having the same or similar attributes are arranged on the same page based on the layout information generated by the layout generation unit. The attribute includes information indicating image characteristics of content,
The layout generation means includes
The image processing apparatus according to claim 1, wherein layout information for arranging the contents having the same or similar image characteristics among the contents on the same page is generated. The attribute includes the image characteristic indicating whether the image of the content is a monochrome image or a color image,
The layout generation means includes
Among the contents, the layout information is generated such that the content whose image characteristic indicates a monochrome image is arranged on the same page, and the content whose image characteristic indicates a color image is arranged on another same page. The image processing apparatus according to claim 2. The attribute includes the image characteristic indicating whether the image of the content is text or image,
The layout generation means includes
The layout information is generated such that content whose image characteristics indicate text is arranged on the same page, and content whose image characteristics indicate image is arranged on another same page. The image processing apparatus according to claim 2. The attribute includes the image characteristic indicating the image quality level of the content image,
The layout generation means includes
5. The image processing according to claim 2, further comprising: generating layout information that arranges, on the same page, content having the same image quality level indicated by the image characteristics among the content. 6. apparatus. The attribute includes the image characteristics indicating whether a content image is a natural image or an artificial image,
The layout generation means includes
Among the contents, the layout information is generated such that content whose image characteristic indicates the natural image is arranged on the same page, and content whose image characteristic indicates the artificial image is arranged on another same page. The image processing apparatus according to any one of claims 2 to 5. The attribute includes the image characteristic indicating the size of characters included in the content,
The layout generation means includes
7. The layout information according to any one of claims 2 to 6, wherein the layout information for arranging the content having the similar character size indicated by the image characteristics among the content on the same page is generated. Image processing device. The attribute includes information indicating a degree of association between content corresponding to the attribute and other content,
The layout generation means includes
The image processing apparatus according to claim 1, wherein, among the contents, the layout information for combining the contents having the relevance level equal to or higher than a threshold value and arranging them on a page is generated. The attribute includes information indicating the original document source of the content,
The layout generation means includes
The image processing apparatus according to claim 1, wherein layout information for arranging the same original content among the contents on the same page is generated. The attribute includes time information indicating a time when the image data is generated,
The layout generation means includes
For the first content and the second content, the interval between the time indicated by the time information corresponding to the first content and the time indicated by the time information included in the attribute corresponding to the second content 2. The image processing apparatus according to claim 1, wherein if the value is equal to or less than a threshold value, layout information for arranging the first content and the second content on the same page is generated. Display control means for controlling display on the display unit;
An arrangement setting means for setting the initial arrangement of the content according to a user operation;
The display control means includes
Based on the output available area where the image can be output in the output image generated by the image generating means, the output paper template is displayed on the display unit in units of pages, and the content is set by the arrangement setting means. Arranged in the initial arrangement and displayed on the display unit,
The layout generation means includes
The image processing apparatus according to claim 2, wherein the layout information is generated based on the initial arrangement selected on the output possible area by the arrangement selection unit. The display control means includes
The image processing apparatus according to claim 11, wherein the content is displayed in association with the attribute. The display control means includes
The image processing apparatus according to claim 11, wherein the display unit is caused to perform display based on the layout information generated by the layout generation unit. The layout generation means includes
Having a plurality of layout generation modes in which different arrangements are made for the same content, and generating the layout information according to the layout generation mode designated in accordance with a user operation among the plurality of layout generation modes. The image processing apparatus according to claim 1, wherein the image processing apparatus is characterized. The image processing apparatus according to claim 1, further comprising a layout information storage unit that stores the layout information generated by the layout generation unit. A display control means for controlling display on the display unit;
The image processing apparatus according to claim 1, wherein the display unit is caused to perform display based on the layout information generated by the layout generation unit. The display control means includes
The image processing apparatus according to claim 16, wherein the content is displayed in association with the attribute. Image data acquisition means for acquiring image data;
An image input device having content extraction means for extracting content from the image data acquired by the image data acquisition means;
Layout generating means for generating layout information for arranging contents having the same or similar attributes among the contents on the same page;
An image processing apparatus having an image generation means for generating an output image in which contents having the same or similar attributes are arranged on the same page based on the layout information generated by the layout generation means;
An image processing system comprising: an image forming apparatus that forms an image based on the output image. A content acquisition step of acquiring content by image data;
A layout generating step for generating layout information for arranging content having the same or similar attributes among the contents on the same page;
An image processing program for causing a computer to execute an image generation step of generating an output image in which contents having the same or similar attributes are arranged on the same page based on the layout information generated in the layout generation step.

Images