JP2014174900A - Image processor, image processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem that when a preview of a piece of electronic document data in the reflow format is displayed before printing, the display of a reflow layout result of all pages bothers a user to confirm the contents thereof, resulting in bad usability.SOLUTION: A preview of a layout result of electronic document data in the reflow format is displayed by extracting the less number of pages than the number of all pages but including a unique layout attribute.

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program for previewing a print image of reflow type electronic document data.

  As the performance of mobile information terminals increases, the number of scenes where documents are shared among multiple terminals is increasing. As a document shared by portable information terminals, there is an electronic document data format called EPUB defined by IDPF (International Digital Publishing Forum). EPUB is excellent as electronic document data when a document is shared and browsed by various information portable terminals, particularly various smartphones and tablet terminals. This is because the “reflow type” feature of EPUB is appropriate for displaying on the terminal screens of portable information terminals of various sizes. The reflow type is a format that does not have a fixed layout. In the reflow format, the layout displayed in the page can be changed according to the screen size and the character size. In the reflow type electronic document data, for example, the number of characters displayed in the page is changed by changing the character size. As described above, it is assumed that the reflow-type electronic document data is mainly displayed on the terminal screen of the portable information terminal. However, there is a usage form in which the reflow-type electronic document data is output to some medium, for example, a paper medium. Conceivable.

  As a technique for performing a preview display when outputting predetermined data to a paper medium, there is a technique of Patent Document 1. Patent Document 1 is not a technique for printing reflow-type electronic document data but a technique related to variable printing. The variable print is a form of printing in which individual data items are changed for each print page in the same print layout. The technique of Patent Document 1 is a technique in which when a print image is displayed as a preview, layout is performed, and a standard page and a page with a large change are extracted and previewed based on a specific layout scale called an area of content. is there.

JP 2005-216183 A

  In the reflow type electronic document data (which will be described as EPUB below), a fixed layout is not determined. Therefore, when printing EPUB content, it is necessary to apply some page layout to the EPUB content. Since a fixed layout is not fixed, the user generally confirms whether or not the page layout is appropriate using the preview display before printing the content in the EPUB format.

  However, there is a problem that a technique for performing a preview display suitable for confirming a print image of reflow type electronic document data does not exist. The technique disclosed in Patent Document 1 is not suitable for preview display of data in a format for which a fixed layout is not determined, and the extraction of pages to be previewed may occur.

  The present invention is an image processing apparatus that generates a preview image of document data including a plurality of pages, an acquisition unit that acquires information about a layout of a plurality of pages, and each page of document data based on the acquired information Specifying means for specifying a page having a change between pages in the layout, generating means for generating a preview image of the document data of the specified page, and transmitting means for transmitting the generated preview image to a transmission destination It is characterized by having.

  According to the present invention, it is possible to display an appropriate preview when reflow type electronic document data is printed.

It is a figure which shows the hardware constitutions of a printing apparatus. It is a figure which shows the hardware constitutions of a server apparatus, a printing apparatus, and a connection structure. It is a figure which shows the module of a server apparatus. It is a figure which shows the flowchart in which the server apparatus and the printing apparatus cooperated. It is a figure which shows the format of EPUB. It is a figure which shows the format of the content of an EPUB format. It is a figure which shows the example of CSS of EPUB, and the example of an application display. It is a figure which shows the UI example of template selection. It is a figure which shows the example of a display of template application. It is a figure which shows the example of a layout of the document to which the template is applied. It is a figure which shows the example of UI display of the document to which the template is applied. It is a figure which shows extraction of the feature page of the document to which the template is applied, and a UI display example. It is a flowchart which shows the extraction process of a feature page. It is a figure explaining attribute distribution and a processing procedure. It is a figure explaining the processing procedure of attribute distribution. It is a figure explaining the processing procedure of attribute distribution. It is a figure which shows the example of a layout of the document to which the template is applied. It is a flowchart which shows the extraction process of a feature page. It is a figure explaining the processing procedure of attribute distribution. It is a figure which shows the flowchart of the production | generation of a preview image, and the extraction process of a feature page. It is a figure explaining the processing procedure of attribute distribution.

  Prior to the description of a specific embodiment, points that may occur when a reflow-type electronic document format such as EPUB is printed and previewed will be supplementarily described. Hereinafter, the entire specification will be described by taking as an example the case of using EPUB as the reflow type electronic document format.

  Since EPUB does not have a fixed layout, it is necessary to specify a parameter for specifying the page layout when printing content in the EPUB format (hereinafter referred to as EPUB content). For example, the page layout parameters include “font face”, “font size”, “page margin”, and the like. However, even if only these values are known, the user usually cannot imagine the final page layout of the printed material. Therefore, it is necessary to preview the print image of the print page in which the page layout parameter is applied to the EPUB content.

  Here, it is assumed that the EPUB content data is directly input to the printing apparatus using a medium such as a USB memory, and the EPUB content is printed directly from the operation panel of the printing apparatus without using an external device such as a PC.

  Since a printing apparatus is often used as a shared apparatus in an office, it is required that the time for occupying the printing apparatus is short. In general, the operation system of the printing apparatus cannot often expect a high-speed response. Therefore, when printing EPUB content directly on the printing apparatus, it is required to efficiently confirm the page layout and proceed with the printing operation.

  As described above, in a reflow type EPUB content that does not have a fixed layout in advance, a print preview is required on the printing apparatus. In general, in print preview, all pages are displayed as preview targets. However, from the viewpoint of shortening the time required to occupy the printing apparatus, it is inefficient to preview all pages. This inefficiency is particularly noticeable in a print preview of EPUB content having a large number of pages. For example, if the layout result is 100 pages, there may be a large number of pages in which most of the text content is followed by text. It is not a good idea to operate such a similar page by performing a full-flicking operation and confirm the contents as an operation method on a printing apparatus having a relatively small display screen. When building EPUB content with an undefined layout according to a specific layout, if only the characteristic pages are extracted from all the pages and the print image can be confirmed in the preview display, the print processing can be performed with the minimum necessary confirmation and operation time. It is possible to proceed to.

  Hereinafter, various embodiments for carrying out the present invention will be described with reference to the drawings. In addition, the structure shown in the following Examples is only an example, and this invention is not limited to the structure shown in figure.

<System configuration of printing device>
A system according to the present embodiment will be described. FIG. 1 is a block diagram illustrating the configuration of the controller 100 of the printing apparatus. The controller 100 includes a CPU 101, ROM 102, RAM 103, internal storage device 104, operation unit I / F 105, network I / F 106, compression unit 107, external storage device I / F 108, and RIP 109. The controller 100 also includes an expansion unit 111, an image processing unit 112, and a printer I / F 113. The controller 100 is connected to the operation unit 110, the external storage device 120, the network 130 such as WAN or LAN, and the printer unit 140 through each I / F. The operation unit 110 has a display screen, as represented by a touch panel display.

  In such a printing apparatus, the user instructs the start of printing preview of EPUB content from the operation unit 110. Then, the EPUB content data designated by the user is read from the external storage device (USB memory or the like) 120 or the internal storage device (hard disk or the like) 104 connected to the controller 100.

  The controller 100 is electrically connected to a scanner unit and a printer unit 140 (not shown). On the other hand, the controller 100 is connected to an external device or the like via a network 130 such as a LAN or a WAN. As a result, image data and device information can be input and output.

  The CPU 101 comprehensively controls access to various connected devices based on a control program or the like stored in the ROM 102, and also comprehensively controls various processes performed in the controller. A RAM 103 is a system work memory for the operation of the CPU 101 and also a memory for temporarily storing image data. The ROM 102 stores a boot program for the apparatus. The internal storage device 104 is generally a hard disk drive and can store system software and image data.

  The operation unit I / F 105 is an interface unit for connecting the system bus 220 and the operation unit 110. The operation unit I / F 105 receives image data to be displayed on the operation unit 110 from the system bus 220 and outputs the data input from the operation unit 110 to the system bus 220.

  A network I / F 106 is connected to a network 130 such as a WAN or a LAN and a system bus 220 to input / output information. The compression unit 107 converts the resolution of the image data before transmission into a resolution that matches a predetermined resolution or printer capability. For compression and expansion, methods such as JBIG (Joint Bi-lebel Image experts Group) and MMR (Modified Modified READ) are used. Further, methods such as MR (modified READ: relative element address designate) and MH (modified huffman) are used. The image bus 230 is a transmission path for exchanging image data, and is configured by a PCI bus or IEEE1394.

  The decompression unit 111 decompresses the compressed image data, raster-expands it, and sends it to the image processing unit 112.

  The image processing unit 112 receives the image data sent from the decompression unit 111, and performs image processing on the image data with reference to attribute data attached to the image data. The image data after the image processing is output to the printer unit 140 via the printer I / F 113.

  The printer unit 140 is an image forming device that forms image data received from the controller 100 on a sheet. The printer unit 140 is provided with a plurality of paper cassettes that allow selection of different paper sizes or different paper orientations. The printed paper is discharged to the paper discharge tray.

  FIG. 2 is a block diagram showing a state in which the controller 100 of the printing apparatus and the server apparatus 200 on the cloud are connected via a network 130 that is the Internet. In the present embodiment, the server apparatus 200 on the cloud, which is the transmission destination of the printing apparatus, is described as being connected through the Internet, but may be connected through a local network such as a LAN.

  In this embodiment, a mechanism for generating print image data indicating a print image for confirming a preview is provided on a cloud server device. By delegating a part of the functions of the printing device to the cloud side, it is possible to easily expand the function only on the cloud.

  The server apparatus 200 is connected to the network 130 via the network I / F 201, and the controller 100 of the printing apparatus is present beyond the network 130.

  In addition, the server device 200 executes system software read from an internal storage device (such as a hard disk) 204 and a print preview generation program described in the present embodiment on the CPU 202. The ROM 205 stores a boot program of the server device 200 and the like, and the RAM 203 is used as a work memory of the server device 200.

  With such a configuration, the controller 100 of the printing apparatus requests the cloud-side server apparatus 200 to generate preview image data for printing a document file including EPUB content, and receives the result again via the network. Is possible.

  FIG. 3 is a block diagram illustrating an example of modules included in the server device 200 on the cloud side. The server device 200 includes a request reception unit 301, a request analysis unit 302, a layout unit 303, a layout feature analysis unit 304, a feature page determination unit 305, and a preview image generation unit 306.

  The request reception unit 301 receives a preview image generation request for document data including a plurality of pages to be printed from the printing apparatus. The request reception unit 301 acquires document data included in the transmitted request and stores it in the internal storage device 204. The request analysis unit 302 analyzes the received request. The request analysis unit 302 analyzes the information described in the file development of the EPUB content document data and the layout setting added to the EPUB content document data.

  The layout unit 303 lays out the EPUB document data expanded by the analysis by the request analysis unit 302 in the page area according to the layout setting obtained by the analysis by the request analysis unit 302. The layout feature analysis unit 304 analyzes information that is a layout feature in parallel with the layout unit 303. The feature page determination unit 45 determines whether or not the page is a feature page, and extracts a feature page to be previewed. The feature page is a page having a feature in the layout. Details will be described later.

  The preview image generation unit 306 expands the EPUB content according to the layout generated by the layout unit 303, and generates data converted into a format readable by the printing apparatus as a preview image.

  3 described above is realized by the CPU 202 of the cloud-side server device 200 executing a program stored in the internal storage device 204.

  Next, an exchange flow between the controller 100 of the printing apparatus and the server apparatus 200 will be described with reference to FIG. The process on the controller 100 side of the printing apparatus corresponds to steps S401 to S406, and the process of the server apparatus 200 on the cloud side corresponds to steps S411 to S415.

  In step S401, the controller 100 receives designation of a document file and a print layout from the user via the operation unit I / F 105. Print layout information indicating the print layout is determined based on the designated print layout. The print layout will be described later.

  In step S402, the printing apparatus determines whether the document file that has received the designation is EPUB content data. This determination is performed, for example, by referring to the extension of the document file. When the controller 100 determines that the data is EPUB content data, the controller 100 transmits a request for generating a preview image for printing to the server apparatus 200 via the network. This request includes a document file and print layout information.

  The request reception unit 301 of the server apparatus 200 receives the preview image generation request transmitted from the controller 100 in step S411. In step S412, the request analysis unit 302 analyzes the received request and acquires a document file and layout information included in the request.

  In step S413, the layout unit 303 lays out the EPUB content document file acquired in step S412 according to the layout information acquired in step S412. The preview image generation unit 306 generates preview image data for each page of EPUB content arranged according to the layout. The layout unit 303 generates information indicating the distribution of each attribute when laying out a document file of EPUB content. Information indicating each attribute distribution is used in the process of step S414. Details will be described later. In step S414, the feature page determination unit 305 extracts a feature page in cooperation with the layout unit 303 and the layout feature analysis unit 304. Details of step S414 will be described later. In step S415, the server apparatus 200 returns the generated preview image data and characteristic page information indicating the characteristic page to the controller 100 of the printing apparatus. In step S415, the generated preview image data for all pages may be transmitted to the printing apparatus, or only the preview image data for the page identified by the feature page information may be transmitted. When transmitting only the preview image data for the page identified by the feature page information, the server apparatus 200 may perform the following processing. That is, in response to a request for transmission of preview image data for pages other than the feature page from the controller 100, preview image data for untransmitted pages may be transmitted.

  Next, the description of the processing on the controller 100 side will be continued. If it is determined in step S402 that the data is not EPUB content data, in step S403, the controller 100 generates preview image data in the printing apparatus. In step S402, whether or not it is EPUB content has been described as a criterion for simplifying the description, but step S402 determines whether or not the printing apparatus itself has a document format that can generate a preview image. It can be said that it is processing. Whether or not the printing apparatus itself can generate a preview image can be comprehensively determined based on whether the controller 100 supports the data format or the data amount of the document. In addition, regarding a reflow type document in a predetermined format such as an EPUB format, a setting may be made so that processing is delegated to the server device side even if a preview image can be generated by the printing device itself.

  If it is determined in step S402 that the content is not an EPUB content, that is, if it is determined that the printing apparatus itself is in a document format that can generate a preview image, a preview image is generated in the printing apparatus in step S403. As a typical example, in the case of a non-reflow type document format in which the layout of each page is fixed, such as a PDF file, the printing apparatus generates preview image data inside the printing apparatus in step 403. Good.

  In either case of passing through step S403 or passing through the processing on the server device 200 side, in step S404, the controller 100 receives the generated preview image data. Then, the controller 100 displays a preview image on the operation unit 110 according to the received preview image data. In step S <b> 405, the controller 100 determines whether “extraction preview” is selected by a user input via the operation unit 110. When the extraction preview is selected, the controller 100 displays the extraction preview on the operation unit 110 in step S406. Details of the extraction preview will be described later.

  In the above processing, an example in which extraction preview is performed when the extraction preview is designated by the user has been described, but processing by which extraction preview is performed by default may be used. In this case, step S406 may be a process of switching the preview screen.

<Print layout>
Next, a print layout of document data in the EPUB format will be described.

  FIG. 5 is a diagram schematically showing the minimum structure of the EPUB file which is document data in the reflow format.

  A mimetype 502 describing the text encoding is included under the root 501. Further, under the root 501, a folder META-INF 503 for storing container.xml 504 indicating the location of the content and a folder OEBPS 505 for storing the document content are included. Further, under the folder OEBPS 505, package.opf 506 (file name is arbitrary) indicating the structure of the content and nav.xhtml 507 holding the table of contents information are included. Also, under the folder OEBPS 505 are included content.xhtml 508 (the file name is arbitrary) that holds the contents of the document itself, and a CSS style sheet stylesheet.css 509 that holds style information for display.

  FIG. 6 is an example of the contents of XHTML 508 corresponding to the content part of FIG. Reference numeral 601 denotes the entire XHTML document. 602 describes a chapter name with <h1> to </ h1>, and 603 describes body text delimited with <p> to </ p>. In the EPUB format described above, content is described in the XHTML format in this way. For example, the character string of “chapter name” 602 is described as “<h1> Lorem ipsum </ h1>”. The character string of “text text” in 603 is described as “<p> Dolor sit amet ... </ p>”. As other reflow format documents, the HTML format, the Microsoft word format, and the like are applicable, and are described in a format defined by each. In the present embodiment, description is made with EPUB as a main body, but the same idea can be applied to these other formats as long as the reflow display format is adopted.

  FIG. 7 is a diagram showing an example of style information CSS (Cascading Style Sheets) used in EPUB. CSS is described in stylesheet.css509 of FIG. As the base font size information is further added to the CSS, the EPUB layout changes as will be described later. The purpose of CSS is to define the display layout of XHTML. The defined style is interpreted by an XHTML display system (in this embodiment, conversion processing to an EPUB preview image), and is used for rasterizing a document on an actual display image or print image. Reference numeral 701 denotes the entire CSS file. “P” 702 defines the display style of “body text” in the EPUB described in the XHTML of FIG. 6 described above. “Font-family” 703 indicates a font type, and “font-size” 704 indicates a font size. Here, “font-size: 100%;” means that the text is displayed at 100% of a default font size determined by a viewer that displays EPUB content (hereinafter referred to as EBPU viewer). “H1” 705 defines the display style of “chapter name” described in the XHTML of FIG. Reference numeral 706 denotes a font type as in 703, and reference numeral 707 denotes a font size as in 704. “Font-size: 1.6em;” means that the corresponding item (here, the chapter name) is displayed at 1.6 times the height of the “M” character in the default font size determined by the EPUB viewer. It means that. In addition, various styles can be defined for CSS, but are omitted here.

  It should be noted that the font size is not yet determined only by giving CSS 701 to the EPUB content in FIG. This is because the font size is designated as a relative value in CSS, such as “font-size” 704 and 707. It is necessary to determine the font size, type the characters based on the font size, and determine the page layout as a result.

  Therefore, in the EPUB display system, it is necessary to determine the base font size. A base font size determined by the display system itself is given to an EPUB that has been designated with a relative font size as described above (many good EPUBs are created as such). In FIG. 7, an example in which the base font size is 14 points and an example in which the base font size is 10 points are depicted for comparison.

  When a base font size of 14 points is given, as shown in page 711, characters are flowed in the page with larger characters, and the layout is determined.

  When a base font size of 10 points is given, as shown on page 721, characters are flowed in the page with characters smaller than 14 points, and the layout is determined.

  Such determination of the base font size is one of the determination items of the EPUB print layout. That is, when generating a preview image on the server device 200 side, it is necessary to convey not only CSS but also print layout information indicating a print layout such as the base font size. As other information indicating the print layout, for example, the page size, the page orientation, and the presence / absence of printing of the page number are also unique designations not included in the EPUB document file having the structure shown in FIG.

  It is necessary to specify print layout information not included in these EPUB document files from the operation unit 110 of the printing apparatus. In general, in order to avoid detailed numerical setting in a printing apparatus, a template in which various parameters are set is often used. The user selects a template that is a set of parameters on the operation unit 110. Then, the parameters defined in the selected template are decomposed into the values of the respective parameters, and transmitted as print layout information from the controller 100 of the printing apparatus to the server apparatus 200.

  FIG. 8 is an example of a template selection screen by the operation unit 110. Reference numeral 801 denotes an area displayed on the screen of the operation unit. Reference numeral 802 denotes the appearance of the template 1, and reference numeral 803 denotes a name “Template 1” for specifying the template. Reference numeral 904 indicates the appearance of the template 2, and reference numeral 905 indicates a name “Template 2” that identifies the template. In both cases, A4 size paper is used as the output size.

  FIG. 9 shows an example in which the template of FIG. 8 is applied to the EPUB content 601 as the sample. When Template 1 is applied to the content 601, a screen 901 is displayed on the operation unit. When Template 1 is applied, the first page of the content becomes 902, and the second page of the content becomes 903. Further, when Template 2 is applied to the content 601, a screen 911 is displayed on the operation unit. When Template 2 is applied, the first page of the content becomes 912 and the second page of the content becomes 913. In this way, a template as a page expression can be given to the reflow type content to obtain a print preview screen having a different appearance.

  FIG. 10 shows the result of laying out one document using Template 2 in FIG. As a result of the layout, this document is expanded to 10 pages.

  When the layout result of 10 pages in FIG. 10 is previewed for printing by the operation unit 110 of the printing apparatus, it is as shown in FIG. Since the display area of the printing measure operation unit 110 is limited, an example in which two pages are simultaneously displayed is the limit. A screen 1011 is an initial display in which the print preview is started, and the first Page 1 and Page 2 are displayed. Using the “display previous page” button 1102 and the “display next page” button 1103, it is possible to preview while moving successive pages. The screen 1111 is where the subsequent pages Page 3 and Page 4 are displayed by pressing the button 1103 on the screen 1101. Similarly, the screen 1112 is where the button 1103 is pressed in the state of the screen 1111 and the subsequent pages Page5 and Page6 are displayed.

  In this way, EPUB reflow-type data is often centered on text content. If content such as the one shown in this example, pages 1 to 7 are continuously displayed with almost no change in appearance. Yes. For this reason, previewing while turning one page at a time requires less time and effort, but requires less information. The information here is feedback to the user by looking at the preview. For example, the font size of the text is too large, the size of the figure is too large, or the balance between the figure and the character size is bad. This is information on the appearance of the layout.

  Here, when the document shown in FIG. 10 is previewed, it is sufficient for the user to obtain “information on the appearance of the layout”. Therefore, as shown in FIG. If you look at the section, the role can be fulfilled. In FIG. 12A, Page 1, Page 8, and Page 10 are extracted as characteristic pages.

  Further, as a method of previewing the extracted page on the operation unit 110, a method as shown in FIG. Here, a switch button 1202 for switching between print preview mode and extraction preview mode for all pages in FIG. 11 is provided. If this button 1202 is pressed, the display can be switched to the all-page preview if the extraction preview is being displayed, and to the extraction preview if the all-page preview is being displayed.

  The screen 1201 displays the extracted Page 1 and Page 8, and when the button 1103 is pressed here, the display shifts to the screen 1211 and the subsequent extracted Page 10 is displayed.

  Here, attributes of contents for extracting “characteristic pages” are listed.

The attributes are divided into the following two types of groups.
1. Attributes given independently to content representation (hereinafter referred to as “content layout attributes”)
This is a set of attributes given by the CSS to each tag of the XHTML content included in the EPUB when displaying the content. Examples include the following (in parentheses <HTML tag name>, CSS style name):
・ Font style (font-family)
・ Font size (font-size)
-Paragraph appearance (quotes, paragraph spacing) (blockquote)
・ Vertical writing / Horizontal writing (text-orientation)
-Text wrap around image (float)
・ Column-count
・ Writing-mode
2. Attributes given to page representation (hereinafter referred to as “page layout attributes”)
This is an additional layout attribute given to the page when the content is expanded into the page layout. One feature is that one pattern is given to one page and a difference occurs between pages. Examples include the following (in parentheses <HTML tag name>, CSS style name):
・ Presence / absence of images / tables / enclosed articles (<img>, <tb>, border, column-rule)
・ Symmetrical page decoration. For example, "Pole, separator, header / footer, page number"
-New page breaks and page breaks (break-after, break-before)
・ Difference between lines with and without ruby / neighbors (<ruby>, <strong>)
-Difference in page orientation (height / width)
・ Text-orientation, column-count

  As for the above-mentioned “symmetric page decoration”, when the HTML that is EPUB content is expanded to the page layout, the page layout processing system independently draws the page decoration as the page decoration separately from the contents described in XHTML. Attribute. For example, this is realized by a method in which a page number character string is additionally drawn at the lower end of a page after layout development of one page into a JPEG image by the EPUB layout processing system. This realization means can be achieved by various methods, but details thereof are omitted in this document.

  The attributes listed here are merely examples of elements that constitute an EPUB or independently constitute a page layout, and it goes without saying that content attributes having similar characteristics are also subject to extraction. .

<Feature page extraction process>
The process for executing the feature page extraction as shown in FIG. 12A will be described with reference to the flowchart of FIG.

  FIG. 13 is a flowchart showing a specific process of “feature page extraction” in step S414 of FIG. In this example, description will be made with reference to a state where a preview image in which content is laid out on page 10 shown in FIG. 10 is generated by the preview image generation processing in step S413 of FIG.

  FIG. 14A shows how characteristic attributes are distributed in the content laid out on page 10 shown in FIG. The state where each of the above-mentioned attributes is distributed on each page is indicated by a black circle. For example, the attribute of <h1>, which is the title of the chapter, exists on the first page as a character string “Lorem ipsum”, and <p>, which is the body, exists on all pages. Also, the graphic image in the sentence is present on the eighth page as <img>, and the other pages are pages without <img>. The <float> attribute that causes text to wrap around <img> is also present on the eighth page. Finally, the table contents are present on the 10th page as <tb>, and the other pages are pages without <tb>.

  In addition, page numbers are allocated to the left and right on the page area by even pages and odd pages, which are shown to appear alternately.

  The “attribute distribution” information shown in FIG. 14A is generated in step S413 with the result of which content is laid out on which page in the process of EPUB content being page-laid.

  A method for extracting a feature page of content having such an attribute distribution will be described with reference to the flowchart of FIG.

  The processing shown in FIG. 13 is executed by the layout unit 303, the layout feature analysis unit 304, and the feature page determination unit 305 in the server apparatus of FIG. More specifically, it is realized by the CPU 202 of the server device 200 executing a program stored in the internal storage device 204.

  In step S1301, the layout unit 303 first counts how many times each attribute appears in all pages while developing EPUB content for a desired layout. In the table of FIG. 14A, the numerical value at the right end indicates the number of appearances of each attribute. For example, <h1> is 1 time, <p> is 10 times, and so on.

  In step S <b> 1302, the layout feature analysis unit 304 sorts the attribute arrangement by the number of appearances of the attribute. FIG. 14B shows a state where the arrangement of FIG. 14A is sorted. Attributes with fewer appearances (<h1>, <img>, etc.) are placed in the upper row of the table.

  In step S <b> 1303, the layout feature analysis unit 304 represents the attribute distribution of each sorted page as a bit flag with “1” indicating that the attribute is present and “0” indicating that the attribute is not present. When sorting is performed as shown in FIG. 14B, the numerical value (binary number) at the lower end of FIG. 14B is obtained. Each bit flag is associated with each page.

  In step S1304, if a certain attribute does not appear independently of the other attributes, the layout feature analysis unit 304 invalidates the upper bits and inserts the page in the bit flag list in duplicate. FIG. 15 shows a state in which the upper bits are invalidated and pages are duplicated and inserted into the list of bit flags. For example, the <float> attribute appears only on the eighth page. However, the bit of the attribute <img>, which is positioned higher than the <float> attribute of the eighth page, has become 1. As will be described later, in this embodiment, grouping is performed when the most significant bit is 1. Therefore, the <float> attribute on the eighth page can be conveniently referred to as an attribute hidden in the <img> attribute. . In order to prevent omission of attribute extraction, it is necessary to treat the <float> attribute as an attribute for convenience. Therefore, the <img> attribute positioned at the top of the eighth page is deleted (the white circle in the table), and the deleted eighth page is duplicated and reinserted as a separate page.

  In step S1305, the layout feature analysis unit 304 groups pages having the same most significant bit that is “1” in the bit flag. FIG. 16 shows this state. The eighth page inserted redundantly in step S1304 is regarded as having the <img> bit set to 0, and the pages are grouped for each leading bit position.

  In step S <b> 1306, the layout feature analysis unit 304 sorts each group in the order in which many “1” s are included in the bit flags, that is, in the descending order of the number of attributes included.

  In step S1311, the feature page determination unit 305 indexes each group in FIG. 16 and starts a loop from the first group.

  In step S1312, the feature page determination unit 305, after taking out the first group, takes out the first page in the group. This is equivalent to retrieving the page that contains the most attributes in the group. By performing sorting and grouping in this way, taking out the first page in the same group and performing the characteristic page determination process described later, the number of pages necessary for covering all attributes can be reduced.

  In step S1313, the feature page determination unit 305 adds the bit flag associated with the page extracted in step S1312 to the extracted flag (initial value is 0) by OR operation. The extracted flag is a flag having the same digit when the bit flag is expressed in step S1303. That is, it can be said that the process of step S1313 is a process of determining whether all attributes have been extracted. In step S1313, the feature page determination unit 305 determines whether or not all the bits of the extracted flag have become “1”.

  If it is determined in step S1313 that all the bits of the extracted flag are not “1”, the feature page determination unit 305 stores the page number extracted in step S1312 in the list L in step S1323. The list L stores page numbers of candidate pages that are candidates for feature pages. Next, in step S1324, the feature page determination unit 305 advances the process to the next page group.

  If it is determined in step S1314 that all bits are “1”, the feature page determination unit 305 removes duplicates from the page numbers stored in the list L in step S1321. For example, in the example of FIG. 16, since the eighth page stores two page numbers in the list L according to the presence or absence of the <img> attribute, one is removed as a duplicate page.

  In step S1322, the feature page determination unit 305 sorts the page numbers stored in the list L in ascending order in the order of page numbers. Then, the process is completed.

  The page number stored in the list L by the above processing becomes the page displayed in the extraction preview. That is, in step S415 shown in FIG. 4, the page numbers stored in the list L are transmitted to the printing apparatus as characteristic page information. The page numbers stored in the list L can be said to be page numbers of a page group including the extracted layout attributes as a whole.

  As described above, according to the present embodiment, even when the reflow type document data is directly printed by operating the printing apparatus, only the page having a characteristic layout can be previewed on the printing apparatus side. Become. Therefore, it is possible to confirm the print preview in a short time.

  In the second embodiment, processing corresponding to a layout different from the example described in the first embodiment will be described.

  FIG. 17 is an example of an EPUB layout different from the layout described in FIG. In this example, the <h1> element appears twice. The first page is “Lorem ipsum” and the fourth page is “Exception sint”. Since this <h1> element has the CSS “break-before” attribute, a page break is performed immediately before. For this reason, the third page has a page break even though there is sufficient margin, and the fourth page has a new <h1> element and the body text from the top.

  In the case of an attribute such as a page break, if two consecutive pages where the page breaks are not seen, the effect of the page break may not be determined even when previewing. That is, it can be said that the pages have layout attributes highly correlated with each other. Therefore, in the example of FIG. 17, the extraction preview should be displayed with the third and fourth pages paired.

  In order to display such a pair of pages, a description will be given with reference to FIG. FIG. 18 is a diagram illustrating detailed processing of the feature page extraction processing in step S414 in FIG. 4 according to the first embodiment.

  The processing flow of the flowchart of FIG. 18 is executed by each module in the server apparatus of FIG. Specifically, it is realized by the CPU 202 of the server device 200 executing a program stored in the internal storage device 204.

  FIG. 19A is a table showing the distribution of attributes in the layout of FIG. Here, the above-mentioned “content layout attribute and page layout attribute” are divided into two tables. Note that description of the page number attribute is omitted here. What should be noted here is the presence or absence of a page break of the page layout attribute before the chapter name (<h1>) of the content layout attribute. In FIG. This attribute is effective between page 3 and page 4. In the table, this attribute is enclosed in a rectangle to indicate this effect. The process of extracting and previewing will be described.

  In the flowchart of FIG. 18, in step S1801, the layout feature analysis unit 304 identifies a pair of pages based on the attribute characteristics. The process surrounded by the rectangle in FIG. 19A corresponds to the process of step S1801. In this embodiment, attention is paid to chapter breaks, but a combination of pages to be paired can be set as appropriate in accordance with attribute characteristics.

  In step S1802, the layout feature analysis unit 304 combines the attributes of the pair of pages specified in step S1801 so that they can be handled as attributes of one page. The attribute distribution information including the combined information is referred to as pair page information. That is, the layout feature analysis unit 304 generates pair page information in step S1802. FIG. 19B shows the result of merging the paired pages in FIG. 19A, that is, the third page and the fourth page. The information on the third page and the fourth page in FIG. 19A is combined into one to generate a pair page “3-4”. These two pages are always handled in pairs. The pair page “3-4” is in a state where a bit regarding the attribute included in any of the pages is set. In addition, the page break attribute that became the pairing factor is converted into a single new attribute “Chapter Page Break” by combining two pages that change from “Chapter Page Break” to “No Chapter Page Break”. The bit is set to 1.

  By creating pair page information including such a pair page as attribute distribution information, the subsequent processing includes steps such as S1301 to S1324 in accordance with the flow of the first embodiment. It is possible to perform page extraction. Step S1803 is the same as the processing of steps S1301 to S1324.

  In step S1804, the layout feature analysis unit 304 and the feature page determination unit 305 once generate an extraction page in step S1803, and then decompose the pair page into the original page based on the pair page information generated in step S1802. That is, “page 3-4” is divided into “third page” and “fourth page”, and each page is extracted as a feature page.

<Effect of Example 2>
Even if a pair of pages is required to confirm the appearance of the layout attributes, it can be included in the extraction preview as a pair of pages. Therefore, it becomes easy to confirm and grasp the attribute that changes between pages.

  In the first and second embodiments, the process of first generating preview image data according to the layout for all pages and determining the page to be extracted by calculating the flag indicating the presence or absence of each attribute in each page has been described.

  Here, a simpler method for extracting pages will be described with reference to the flowcharts of FIGS.

  A flowchart in which the server apparatus and the printing apparatus in the third embodiment cooperate with each other is a process in which steps S413 and S414 in FIG. 4 are combined into one. The other processes are the same as those in FIG. 4 described in the first embodiment, and a description thereof will be omitted.

  The flowchart of FIG. 20 is a diagram illustrating detailed processing of preview image generation and feature page extraction processing according to the third embodiment.

  In step S2001, the layout unit 303 and the preview image generation unit 306 first start processing from the first page of the EPUB content, and generate one page as preview image data along the layout. In step S2002, the layout unit 303 extracts characteristic attributes used for the page layout. In step S2003, the layout feature analysis unit 304 represents the combination of characteristic attributes extracted in step S2002 as a bit flag.

  An example in which characteristic attributes and combinations thereof are expressed as bit flags will be described. The layout example of FIG. 10 will be described again as a sample. In the layout of this document, when the layout is started from the first page, the attributes of each page appear as shown in the table of FIG. For example, on the first page, <h1> appears first as a content layout attribute, and then <p> appears. Furthermore, a layout element “page number is added to the right” appears as a page layout attribute. On the next second page, <p> exists as a content layout attribute, and a layout element “page number is given on the left” appears as a page layout attribute. As the page progresses, <img> appears as a new page layout attribute on page 8, and <float> appears as a content layout attribute. Further, on the 10th page, <tb> appears as a page layout attribute.

  In step S2003 described above, the extracted attribute combinations are sequentially expressed by bit flags. Then, as shown in the lower end of FIG. 21, it becomes a binary representation flag in which the number increases as the attribute increases. The first page in which the attribute <h1>, the attribute <p>, and the attribute “page number is added to the right” exist is represented by “111”. The second page is similarly expressed as “0101”. In this way, the number of digits increases as the attribute increases.

  In step S2004, the layout feature analysis unit 304 ORs an attribute combination bit flag acquired during the sequential layout of pages with an attribute summation flag (initial value is 0) held in the system. Then, it is determined whether there is a change before and after OR ORing. For example, the OR of the first page “111” and the second page “0101” is “1111”. In this state, the attribute sum flag for the third page does not change even if ORed. Therefore, the next page processing is advanced through steps S2008 and S2009 described later, and the subsequent pages proceed to the eighth page without changing the attribute summation flag. In the eighth page, the process proceeds such that the new attribute is ORed and the attribute sum flag becomes “111111”.

  If there is a change in step S2004, in step S2006, the layout feature analysis unit 304 ORs the flag expressed in step S2003 and the attribute sum flag and holds it as a new attribute sum flag. In step S2007, the feature page determination unit 305 determines that the current processing target page is an extraction page, and adds the page number to the extraction page list held by the system.

  If the layout feature analysis unit 304 determines in step S2004 that there is no change in the attribute sum flag, the process advances to step 2508.

  In step S2008, the feature page determination unit 305 determines whether the page that has been processed is the last page, and if it is the last page, the process ends. If it is not the last page, in step S2009, the feature page determination unit 305 advances the page to be processed to the next page, and repeats the processing from step S2001.

<Effect of Example 3>
In the third embodiment, the feature page can be extracted by a simple process without considering the calculation in which complicated flags are accumulated and the attribute distribution state in all pages. The method of the third embodiment is effective for implementation in an environment where the amount of calculation for processing and the amount of memory are restricted.

  In the first and third embodiments, the process of extracting pages by purely calculating attribute flags has been described. However, it is often useful to display the first page (first page) unconditionally as the beginning of a document for a preview to get an overview of the document. In addition, it may be better to display the last page of the document unconditionally.

  Processing corresponding to such a case will be described. In the flowchart of FIG. 13, the feature page determination unit 305 inserts the first page and the last page of the document into the list L in the initial state during the processing from step S1312 to step S1315. Then, the extracted flag may be processed with the OR state of the bit flags of the attributes included in those pages as an initial state.

<Effect of Example 4>
The first page and the last page, which are often important when grasping the outline of a document, can always be included in the extraction preview target, and the outline of the document can be easily grasped when the extraction preview is displayed.

  In the first embodiment, as an apparatus for processing EPUB content, an example is described in which a cloud-side server apparatus is used instead of the printing apparatus itself. However, a server device on the cloud side is not indispensable for processing EPUB content, and it is also possible to perform equivalent EPUB content preview generation and extraction preview selection processing in a controller in the printing apparatus.

<Effect of Example 5>
By processing in the controller, communication time with the cloud can be saved and processing can be performed at higher speed.

<Other examples>
The object of the present invention can also be achieved by executing the following processing. That is, a storage medium that records a program code of software that realizes the functions of the above-described embodiments is supplied to a system or apparatus, and a computer (or CPU, MPU, etc.) of the system or apparatus is stored in the storage medium. This is the process of reading the code. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code and the storage medium storing the program code constitute the present invention.

Claims (23)

An image processing apparatus for generating a preview image of document data including a plurality of pages,
An acquisition means for acquiring information on a layout of a plurality of pages;
A specifying means for specifying a page having a change between pages in a layout for each page of document data based on the acquired information;
Generating means for generating a preview image of the document data of the identified page;
Transmitting means for transmitting the generated preview image to a transmission destination;
An image processing apparatus comprising:   2. The image processing apparatus according to claim 1, wherein the specifying unit extracts a layout attribute included in each page, and specifies a page group including the extracted layout attribute as a page having the change. .   The image processing apparatus according to claim 1, wherein the specifying unit extracts a distribution of layout attributes included in each page, and specifies a page having the change by a combination of the distribution of layout attributes. .   The specifying means sorts the order of each page according to the distribution of the layout attribute, groups pages having the same attribute, and specifies a page with the change using the first page of the group. The image processing apparatus according to claim 3.   5. The page according to claim 1, wherein the specifying unit specifies a page having the change by handling a page including a plurality of attributes as a separate page having each attribute. 6. Image processing device.   The image processing apparatus according to claim 1, wherein the specifying unit specifies a plurality of pages having layout attributes highly correlated to each other as the page having the change.   The identification means treats the plurality of pages having layout attributes highly correlated as a single page that is combined, and adds a new attribute to the combined single page. 6. The image processing apparatus according to 6.   The image processing apparatus according to claim 1, wherein the specifying unit specifies in advance the first page and the last page as pages having a change.   The specifying unit stores a layout attribute included in the first page in the storage unit, and a page having a layout attribute different from the stored layout attribute is changed with the first page for subsequent pages. The image processing apparatus according to claim 1, wherein the different layout attributes are stored in the storage unit. The generating unit generates a preview image of the document data including a page not specified by the specifying unit;
10. The transmission unit according to claim 1, wherein the transmission unit transmits the generated preview image and information identifying a page having a change specified by the specification unit to a transmission destination. An image processing apparatus according to 1.   The image processing apparatus according to claim 1, wherein the document data is reflow type document data.   The image processing apparatus according to claim 1, wherein the document data is document data having no fixed layout.   The image processing apparatus according to any one of claims 1 to 12, wherein a layout of the document data is specified by a relative value from a value serving as a reference of an apparatus that performs processing. Transmission means for transmitting the document data to the image processing apparatus according to any one of claims 1 to 13,
Receiving means for receiving the generated preview image;
Display means for displaying the received preview image;
A printing apparatus comprising: The display means displays a layout template,
The printing apparatus according to claim 14, wherein the transmission unit transmits information relating to a layout according to selection of the displayed template to the image processing apparatus.   The switching device according to claim 14, further comprising a switching unit that switches between a mode for displaying the received preview image and a mode for displaying preview images of all pages of the transmitted document data. Printing device. The receiving unit receives a preview image of a page including a page specified by the specifying unit of the image processing apparatus and a page not specified, and information for identifying the specified page;
The printing apparatus according to claim 16, wherein the display unit displays a preview image corresponding to the mode switched by the switching unit based on the information received by the receiving unit and the preview image. An image processing apparatus for generating a preview image of document data including a plurality of pages,
An acquisition means for acquiring information on a layout of a plurality of pages;
A specifying means for specifying a page having a change between pages in a layout for each page of document data based on the acquired information;
Generating means for generating a preview image of the document data of the identified page;
Display means for displaying the generated preview image;
An image processing apparatus comprising: An input means for inputting reflow type document data to be printed;
Selecting means for selecting a print layout of the document data;
A display unit that displays a print preview screen of the document data in accordance with the print layout selected by the selection unit; and a print preview screen of all pages is displayed by the display unit, or the number of pages is smaller than the total pages. A printing apparatus comprising: switching means for switching whether to display a print preview screen of a page having a layout characteristic. An image processing method for generating a preview image of document data including a plurality of pages,
An acquisition step for acquiring information about a layout of a plurality of pages;
A specifying step for specifying a page having a change between pages in a layout for each page of document data based on the acquired information;
Generating a preview image of the document data of the identified page;
A transmission step of transmitting the generated preview image to a transmission destination;
An image processing method comprising: An image processing method for generating a preview image of document data including a plurality of pages,
An acquisition step for acquiring information about a layout of a plurality of pages;
A specifying step for specifying a page having a change between pages in a layout for each page of document data based on the acquired information;
Generating a preview image of the document data of the identified page;
A display step for displaying the generated preview image;
An image processing method comprising: An input step for entering reflow type document data to be printed;
A selection step of selecting a print layout of the document data;
A display step of displaying a print preview screen of the document data according to the print layout selected in the selection step; a print preview screen of all pages in the display step; or a print preview of only pages having a layout characteristic And a switching step for switching whether to display a screen.   A program for causing a computer to function as the image processing apparatus according to any one of claims 1 to 13.

Images