JP5194995B2 - Document processing apparatus, document summary creation method, and document summary creation program - Google Patents

Description

  The present invention relates to a document processing device, a document summary creation method, and a document summary creation program, and more particularly, to a document processing device, a document summary creation method, and a document summary creation program for creating a summary of a document image from a document image.

  2. Description of the Related Art A document management system that converts a document described on a paper manuscript into an electronic document and manages it has been put into practical use. In such a document management system, a document image is generated by reading a document using a scanner device or the like, and an electronic document is generated from these document images.

  However, when scanning a paper document such as a thick manual with no table of contents or a table of contents lost, it is common for reasons of user interface such as poor resolution or limited viewable range at a time. Specifically, it is difficult to find a target location from an electronic document compared to the original paper document. However, there is also a demand for storing such document images as electronic documents in order to save space. For this reason, various methods for quickly finding a target location from a document image have been proposed.

  For example, Patent Document 1 discloses an invention for creating a summary document image or a table of contents document image configured by extracting only a desired document element portion from a document image of a document. In this document, an input document image is divided into a plurality of rectangular regions (meaningful chunks as document elements), and a region corresponding to a desired element is extracted from the plurality of divided regions. And the partial image of the extracted area | region is arrange | positioned in order at a predetermined coordinate position.

Patent Document 2 discloses an invention that performs OCR (Optical Character Recognition) analysis of various types of printed documents and generates a structured document tagged with XML or HTML from such a printed document with high accuracy. Is disclosed. In this document, the hierarchical level of headings is determined by analyzing the character strings of numbers and symbols given to the heads of chapter headings and bullets.
JP-A-5-342326 JP-A-2005-43990

  However, in the invention of Patent Document 1, the clipped partial images (titles and abstracts) are only arranged in order, and no consideration is given to the hierarchy level of the headings.

  In the invention of Patent Document 2, although the hierarchy level of the headline is taken into consideration, the hierarchy level of the headline has to be determined, and the processing becomes complicated.

  Alternatively, it is possible to easily determine the hierarchical level of a heading or the like based on the indent width. In such a case, if the scan position is shifted between pages, the hierarchical level may not be accurately determined.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to generate a summary image reflecting a hierarchy level such as a heading without performing a hierarchy level determination process such as the heading. It is to provide a document processing apparatus, a document summary creation method, and a document summary creation program that can be used.

A document processing apparatus according to an aspect of the present invention is a document processing apparatus that creates a summary of a document image from a document image, an extraction unit for extracting a document element of a predetermined type from the document image, and a document image Detecting means for detecting a stage for each page; line setting means for setting at least one layout reference line in a direction perpendicular to the row direction of the stage for each page based on the layout of the stage; A correction processing unit that performs processing for correcting a shift of the layout reference line between pages, and a partial area including a summary area corresponding to the extracted document element of a predetermined type are cut out using the layout reference line as a reference. And a summary for generating a summary image, which is an image representing a summary, by arranging a partial region in a predetermined image with reference to a predetermined element reference line in the predetermined image And a forming means.

  Preferably, the cutout unit cuts out the partial region with the layout reference line as an end, and the summary generation unit arranges the partial regions in order along a predetermined element reference line.

  Alternatively, the cutting means cuts out the minimum rectangle including the summary area as a partial area, detects the distance between the partial area and the layout reference line for each partial area, and the summary generating means determines the position of the predetermined element reference line. Therefore, it is preferable to shift the partial area according to the detected distance and arrange it in the image.

  Preferably, the correction processing means includes alignment means for aligning the reference position of the text area in the stage between pages.

  Preferably, the image processing apparatus further includes a rectangle setting unit configured to group at least one virtual rectangle and set at least one virtual rectangle for each page based on a predetermined condition, and the line setting unit includes a layout reference line for each virtual rectangle. Set.

  Preferably, the line setting means sets the side on the line head side perpendicular to the line direction in the virtual rectangle as a layout reference line.

  Preferably, document generation means for separately generating a first electronic document including a document image and a second electronic document including a summary image, and the first electronic document and the second electronic document are stored. Storage means.

  Preferably, the image forming apparatus further includes an adding means for generating browsing support information for associating with a corresponding portion of the document image for each partial region, and adding the browsing support information to the summary image. Document specifying information for specifying one electronic document and position information indicating a corresponding location in the document image for each partial area are included.

  Preferably, the apparatus further includes document generation means for generating an electronic document including a document image and a summary image, and storage means for storing the electronic document.

  Preferably, the image forming apparatus further includes an adding unit for generating browsing support information for associating with a corresponding portion of the document image for each partial area, and adding the browsing support information to the summary image. The position information indicating the corresponding location in the document image of is included.

  Preferably, the image processing apparatus further includes correction means for correcting a document element of a predetermined type extracted by the extraction means based on an instruction from the user, and the extraction means summarizes the region of the document element corrected by the correction means. Cut out as an area.

  Preferably, the correcting unit includes a specifying unit that receives specification of a user-desired region from the document image, and the cutting unit cuts out the region specified by the specifying unit as a summary region.

  Preferably, selection means for accepting a selection from the user to output whether the summary image is output separately from the document image or the summary image is combined with the document image is selected to be output separately by the selection means. In this case, it is selected that the first electronic document including the document image and the first electronic document generating unit for generating the second electronic document including the summary image and the selecting unit are combined and output. In this case, the apparatus further includes second document generation means for generating a third electronic document by combining the document image and the summary image.

  Preferably, the predetermined type includes at least one of a title, a headline, a figure, a table, a photograph, a predetermined line at the beginning of a paragraph, a target of handwritten annotation, and a caption.

A document summary creation method according to another aspect of the present invention is a method for creating a summary of a document image from a document image in a document processing apparatus including a storage unit and an arithmetic processing unit. A step of extracting a predetermined type of document element, a step of the arithmetic processing unit detecting a step for each page from the document image, and a step of the arithmetic processing unit for each page based on the step layout. A step of setting at least one layout reference line in a direction perpendicular to the row direction, a step of an arithmetic processing unit performing a process for correcting a shift of a layout reference line between pages, and an arithmetic processing unit, A step of cutting out a partial area including a summary area corresponding to the extracted document element of a predetermined type with reference to a layout reference line; By arranging the partial area relative to the reference line, and generating a summary image.

  A document summary creation program according to still another aspect of the present invention causes a computer to execute the document summary creation method described above.

  According to the present invention, it is possible to generate a summary image reflecting a hierarchical level such as a heading without performing a determination process of the hierarchical level such as a heading.

  Embodiments of the present invention will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals and description thereof will not be repeated.

<About configuration>
(Overall system configuration)
FIG. 1 is a schematic configuration diagram of a system including a document processing apparatus according to an embodiment of the present invention. In the present embodiment, an MFP (Multi Function Peripheral) equipped with the document processing apparatus according to the present invention will be typically described. The document processing apparatus according to the present invention is not limited to an MFP, but can be applied to a PC (Personal Computer), a copier, a facsimile apparatus, a scanner apparatus, and the like.

  Referring to FIG. 1, MFP 1 according to the present embodiment includes an image reading unit 104 for reading a document 300 and a printing unit 106 for performing a printing process on a paper medium or the like.

  In particular, MFP 1 according to the present embodiment acquires a document image by reading original document 300 with image reading unit 104, and generates an electronic document including the document image. At this time, the MFP 1 generates a summary image by cutting out a predetermined type of document element from among the document elements included in the read document image. The generated summary image may be integrated with the document image or may be included in an electronic document different from the document image. These electronic documents 400 typically employ a format such as PDF (Portable Document Format).

  Note that “summary” represents a summary or table of contents of a document image, and “summary image” is an image (image functioning as a summary) representing a summary of a document image. A “document element” is a description element that constitutes a document.

  The MFP 1 stores the generated electronic document 400 in its own storage unit (not shown in FIG. 1). In addition, the stored electronic document 400 is transmitted to personal computers PC1, PC2 and PC3 (hereinafter also collectively referred to as “personal computer PC”) via a network. As a typical usage pattern, the MFP 1 sends an electronic document 400 to the personal computers PC1 and PC2 connected to a local area network (LAN) that is a network laid in the same office where the MFP 1 is installed. Is sent directly. On the other hand, a server SRV is provided at a connection point between a LAN and a WAN (Wide Area Network). A personal computer PC3 or the like located in an office remote from the MFP 1 is connected from the MFP 1 via the server SRV. An electronic document 400 is transmitted. Here, the server SRV typically includes a mail server, an FTP (File Transfer Protocol) server, a Web server, an SMB server, and the like.

  The image reading unit 104 includes a loading table for setting a document, a document table glass, a conveyance unit that automatically conveys the documents set on the loading table one by one to the document table glass, and a scanned document. And a discharge stand for discharging (both not shown). As a result, a plurality of documents can be continuously read and generated as one electronic document.

(Schematic configuration of MFP 1)
FIG. 2 is a block diagram showing a schematic functional configuration in MFP 1 according to the embodiment of the present invention.

  Referring to FIG. 2, the functional configuration of MFP 1 includes a control unit 100, a memory unit 102, an image reading unit 104, a printing unit 106, a communication interface unit 108, a first operation panel unit 110, A second operation panel unit 111 and a storage unit 112 are included.

  The control unit 100 is typically configured by an arithmetic device such as a CPU (Central Processing Unit), and implements document processing according to the present embodiment by executing a program. The memory unit 102 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory), and holds a program executed by the control unit 100 and data necessary for executing the program. The communication interface unit 108 is typically a part for transmitting and receiving data to and from a personal computer PC (FIG. 1) via a network (for example, the LAN shown in FIG. 1). Including driver software to control The print unit 106 is a part for performing print processing, and includes a control device for controlling the operation of each unit in addition to the hardware configuration related to print processing. The storage unit 112 is typically a nonvolatile storage device such as a hard disk device or a flash memory, and stores a program for the operation of the control unit 100, the electronic document 400 generated by the control unit 100, and the like.

  An example of the appearance of the first and second operation panel sections 110 and 111 is shown in FIG. 3A is a diagram showing an example of the appearance of first operation panel unit 110 of MFP 1 according to the embodiment of the present invention, and FIG. 3B is a diagram of MFP 1 according to the embodiment of the present invention. It is a figure which shows an example of the external appearance of the 2 operation panel part. The first operation panel unit 110 is a part used in basic processing of the MFP 1, and the second operation panel unit 111 is a part used in document summary creation processing described later.

  Referring to FIG. 3A, the first operation panel unit 110 includes a display panel 110a composed of a liquid crystal display device, a touch panel, etc., a stop button 110b, a start button 110c, an alphanumeric key, and the like. And an operation button 110d.

  Referring to FIG. 3B, the second operation panel unit 111 is a display panel configured by a liquid crystal display device, a touch panel, or the like. The second operation panel unit 111 includes a display area 111a for displaying summary images and the like, and an input area 111b for displaying operation buttons such as cursor buttons.

  In this embodiment, the second operation panel unit 111 for document summary creation processing is provided separately from the first operation panel unit 110, but an integrated operation panel unit may be employed.

  Alternatively, the MFP 1 includes the first and second operation panel units 110 and 111 having both a display function and an instruction input function. Instead, at least one display unit and hardware buttons are provided. It is good also as providing both the input part containing.

(Configuration of personal computer)
FIG. 4 is a schematic diagram showing a schematic hardware configuration of personal computer PC according to the embodiment of the present invention.

Referring to FIG. 4, a personal computer PC is a CPU (Central Processing Unit) that executes various programs including an operating system (OS).
) 201, a memory unit 213 that temporarily stores data necessary for execution of the program by the CPU 201, and a hard disk unit (HDD: Hard Disk Drive) 211 that stores the program executed by the CPU 201 in a nonvolatile manner. . The hard disk unit 211 stores a browsing application for displaying the electronic document 400 generated by the MFP 1, and such a program is stored in the flexible disk 217 a by the FDD drive 217 or the CD-ROM drive 215, respectively. Alternatively, it is read from a CD-ROM (Compact Disk-Read Only Memory) 215a or the like.

  The CPU 201 receives an instruction from the user via the input unit 209 including a keyboard and a mouse, and outputs a screen output generated by executing the program to the display unit 205. Further, the CPU 201 acquires the electronic document 400 from the MFP 1 or server SRV (FIG. 1) connected to the LAN or WAN via the communication interface unit 207 including a LAN card and stores the electronic document 400 in the hard disk unit 211 or the like. Further, the above-described units exchange data with each other via the internal bus 203.

(Functional configuration of MFP)
FIG. 5 is a functional block diagram showing a functional configuration of MFP 1 according to the embodiment of the present invention.

  Referring to FIG. 5, the control unit 100 includes an output form selection unit 12, an image buffer unit 14, a summary element extraction unit 16, a stage detection unit 18, an element correction unit 20, and a partial region extraction unit 22. A summary image generation unit 24, an output data generation unit 26, a bookmark assignment unit 28, a data storage processing unit 30, and a transmission processing unit 32.

  The image reading unit 104 described above reads the document 300 to obtain a document image, and outputs the document image to the image buffer unit 14. The image buffer unit 14 is a part that temporarily stores document image data sequentially output by the image reading unit 104, and outputs the temporarily stored document image to the summary element extraction unit 16 and the output data generation unit 26. . In this embodiment, the compression processing of the document image is omitted, but the compression processing may be performed. In this case, the degree of compression may be changed according to the size of the generated electronic document, the required resolution of the document image, and the like. The compression process may be irreversible conversion.

  The summary element extraction unit 16 extracts a predetermined type of document element included in the document image output from the image buffer unit 14.

  The “predetermined type” is a type to be included in the summary, and includes at least one of “title (title)” and “headline”. Preferably, in addition to “title” and “headline”, “drawing”, “table”, “photograph”, “predetermined line at the beginning of paragraph”, “target of handwritten annotation”, and “caption” are further included. In the present embodiment, the “predetermined type” includes all the types described above. The “handwritten annotation target” indicates text, a figure, a table, or the like indicated by the handwritten annotation.

  Here, it is assumed that the types to be included in the summary are preliminarily determined at the time of manufacture. However, the user-desired types may be selected in advance before the document summary creation process described later. .

Hereinafter, “a predetermined type of document element” is referred to as a “summary element”.
The summary element extraction unit 16 extracts document elements other than a predetermined type prior to extraction of summary elements. That is, a plurality of types of document elements including a predetermined type are extracted. At that time, the summary element extraction unit 16 also extracts the position of each document element in the document image. The document elements to be extracted include, for example, those having types such as “text”, “handwritten characters”, and “header / footer” in addition to those having the above-mentioned predetermined types.

  The summary element extraction unit 16 outputs the extracted summary element type and its position information to the element correction unit 20.

  The stage detection unit 18 detects “stage (column set)” from the document image output from the image buffer unit 14. Further, the stage detection unit 18 detects the “row direction” of each stage and the “connection information” of each stage. The dan can be detected by a known technique disclosed in, for example, “Document Recognition Technology for Books Electronic (Hiro Kamada, Katsuhito Fujimoto, Koji Kurokawa), Magazine FUJITSU, VOL.49, No.6, 1998”. . The connection information of each stage is information indicating which stage should be connected to each stage from the context or the like (that is, information indicating the order of the stages), and is disclosed in, for example, Japanese Patent Laid-Open No. 10-40248. It can be detected with the technology. The detection in the row direction can be detected by a known method. The stage detected by the stage detection unit 18, the row direction of each stage, and connection information of each stage are output to the line setting unit 19.

  The line setting unit 19 sets at least one “layout reference line” for each page based on the detected layout (assignment) of the stage. The “layout reference line” corresponds to one side of a minimum rectangle (called “stage layout frame”) including the stages grouped on each page, and is a line on the line head side perpendicular to the line direction in the grouped stages. It is. The conditions for grouping the stages will be described later. Information on the set layout reference line is output to the partial region cutout unit 22.

  The element correction unit 20 displays the extracted summary elements on the second operation panel unit 111. In addition, the element correction unit 20 corrects the summary element based on an instruction from the user input to the second operation panel unit 111. In the present embodiment, “correction” includes changing the range of summary elements, deleting summary elements, and adding summary elements. Specifically, the element correction unit 20 corrects the position information of the summary element in accordance with an instruction from the user. The position information of the summary element corrected (or not corrected) by the element correction unit 20 is output to the partial region extraction unit 22.

  The partial area cutout unit 22 cuts out a partial area (partial image) including the corrected area (hereinafter referred to as “summary element area”) based on the set layout reference line. Specifically, the partial region cutout unit 22 cuts out the partial region including the summary element region so that the layout reference line becomes the end portion on the line head side (that is, specifies the position information of the partial region). The cut out partial area is output to the summary image generation unit 24. The partial area to be cut out is preferably a rectangular area. Hereinafter, the cut out partial area is referred to as a “rectangular area”.

  In this way, the summary element area is cut out based on the layout reference line, so that a summary of the document image can be created while maintaining the hierarchical structure existing in the document image to be processed.

  In the present embodiment, the partial area cutout unit 22 further performs a process (hereinafter referred to as “correction process”) for correcting a shift of the layout reference line between pages before the rectangular area cutout process. Do. As the correction process, a process of matching the reference position of the text area between pages is performed. A specific example of such correction processing will be described in detail later.

  The summary image generation unit 24 generates a summary image by arranging a rectangular area in a predetermined image with reference to a predetermined “element reference line” in the image. That is, based on the position information of the rectangular area, the rectangular area is arranged in the output image with reference to a predetermined element reference line. The rectangular regions are preferably arranged at a predetermined interval. The image for arranging the rectangular area may be a plain page or the same size as the original document image. Further, when arranging the rectangular areas, all the rectangular areas may be enlarged or reduced by a predetermined ratio.

  The predetermined element reference line is a line perpendicular to the row direction of the rectangular region to be arranged, and in the case of horizontal writing, it is a vertical line set at a predetermined distance from the left end of the output image. In this case, the summary image generation unit 24 arranges the rectangular areas at predetermined intervals in order from the top. On the other hand, in the case of vertical writing, the predetermined element reference line is a horizontal line set at a predetermined distance from the upper end of the output image. In this case, the summary image generation unit 24 arranges the rectangular areas at predetermined intervals in order from the right. The element reference line may be a straight line perpendicular to the row direction, and may coincide with the left end or upper end of a predetermined image. The predetermined distance may be determined in advance according to the size of the page of the image.

The summary image generated in this way is output to the output data generation unit 26.
The output form selection unit 12 causes the user to select whether the output form of the summary image is “separation” or “combination”. Specifically, the output form selection unit 12 displays an “output form selection screen” on the second operation panel unit 111. The output form selection unit 12 selects an output form based on an instruction from the user input to the second operation panel unit 111. The output form selected by the output form selecting unit 12 is output to the output data generating unit 26 and the bookmark assigning unit 28.

  The output data generation unit 26 generates output data based on the output form selected by the output form selection unit 12. More specifically, when “separation” is selected by the user, an electronic document including a summary image (hereinafter referred to as “electronic summary document”) separate from an electronic document including a document image (hereinafter referred to as “electronic manuscript document”). Generated). On the other hand, when “composite” is selected by the user, one electronic document including a document image and a summary image (hereinafter referred to as “electronic composite document”) is generated. In other words, the output data generation unit 26 adds summary image data to the original document image data. The output data generated in this way is output to the bookmark giving unit 28. The electronic document 400 described above includes at least one of an electronic manuscript document, an electronic summary document, and an electronic composite document.

  The bookmark giving unit 28 generates browsing support information based on the output mode selected by the output mode selection unit 12, and adds (provides) the generated browsing support information to the summary image. The “browsing support information” is information for associating each rectangular area with a corresponding portion of the document image. In this specification, “browsing support information” is “bookmark” data including at least information indicating the “position in the original document image” of the rectangular area. In addition to “bookmarks”, supplementary comments about the rectangular area It may be an “annotation” or the like further including

  The bookmark assigning unit 28 outputs the output data (any one of the electronic manuscript document, the electronic summary document, and the electronic composite document) after giving the bookmark to the data storage processing unit 30.

  The data storage processing unit 30 stores the output data input from the bookmark giving unit 28 in the storage unit 112.

  The transmission processing unit 32 performs processing for transmitting the electronic document stored in the storage unit 112 to the outside. Specifically, a list of electronic documents stored in the storage unit 112 is displayed on the first operation panel unit 110 or the second operation panel unit 111, and the user is allowed to select a desired electronic document. Then, the selected electronic document is output to the communication interface unit 108. If the selected electronic document is an electronic original document, the transmission processing unit 32 may output both the electronic original document and the electronic summary document to the communication interface unit 108. The electronic document (and summary document) 400 output to the communication interface unit 108 is transmitted to a personal computer PC or the like.

  The operation of each block included in the control unit 100 may be realized by executing software stored in the memory unit 102, and at least one of these blocks is realized by hardware. May be.

  Further, in the present embodiment, the summary element correction process is performed by the element correction unit 20, but such a correction process is not necessarily performed. In this case, the summary element information extracted by the summary element extraction unit 16 is output to the partial region extraction unit 22.

  Alternatively, in the present embodiment, the user can select the form of the output data, but either “synthesis” or “separation” may be determined in advance.

<About document summary creation processing>
The document summary creation process according to the embodiment of the present invention will be described below with reference to FIGS.

  FIG. 6 is a flowchart showing document summary creation processing in the embodiment of the present invention. The processing shown in the flowchart of FIG. 6 is stored in advance in the memory unit 102 as a program, and the function of the document summary creation processing is realized by the control unit 100 reading and executing this program. Note that the processing shown below is started when the control unit 100 receives an input from the user to instruct to start the document summary creation processing.

  First, the output form selection unit 12 displays an output form selection screen in the display area 111a of the second operation panel unit 111 (step S102).

  FIG. 7 is a diagram illustrating an example of an output form selection screen. The output form selection screen includes a “Unified” button 51 for receiving a “composite” instruction and a “Separated” button 52 for receiving a “separation” instruction.

  Next, the output form selection unit 12 accepts the selection of the output form by the user (step S104). Of the buttons 51 and 52 displayed on the output form selection screen, the button selected by the user is highlighted with a thick frame.

  When the selection of the output form is completed, the image reading unit 104 reads a document image (S106). The read document image is output to the image buffer unit 14. The summary element extraction unit 16 extracts a plurality of types of document elements including summary elements from the document image in the image buffer unit 14 (step S108). The summary element extraction unit 16 temporarily records management information related to the summary elements of the extracted document elements in the memory unit 102 as “summary element management information”.

FIG. 8 is a diagram for explaining extraction processing of document elements included in a document image.
FIG. 8A is a diagram illustrating an example of a document image generated from the document 300, and FIG. 8B is a state in which document elements are extracted from the document image illustrated in FIG. It is a figure which shows an example. Referring to FIG. 8A, as an example, when a document 300 (5 pages) of “Report on...” Is read by the image reading unit 104, a document corresponding to each page is stored in the image buffer unit 14. Images are stored. As shown in FIG. 8B, the summary element extraction unit 16 extracts a rectangular document element 420 included in such a document image. Note that a known method can be used for extracting the document element 420, but each area is typically extracted based on the character size, the interval between adjacent areas, and the like.

  More specifically, the document element of the predetermined type is extracted by the following method, for example. The “title” can be extracted based on the character size of the character arranged in the document image and the interval between adjacent regions. “Heading” can be extracted based on a relationship with “text” (an area preceding “text”), an offset position or an indent position in a document image, and the like. The “figure” can be extracted by taking a binary pattern of an area other than the text block. The “table” can be extracted based on a combination of ruled lines and text. The “photograph” can be extracted based on the degree of change in shading of the image in the color histogram.

  The “predetermined line at the beginning of a paragraph” can be extracted based on the indentation position in the text block or the interval between the text block and the subsequent text block. The “object of handwritten annotation” can be extracted by detecting the text in the vicinity of the handwritten character detected based on the disorder of the font or line. Alternatively, it can be extracted by following a handwritten line near the handwritten annotation. “Captions” can be extracted as isolated text blocks near the top or bottom of a diagram, table, or photo.

  Note that a plurality of document elements may be extracted redundantly for a common area in the document image. The types extracted as summary elements are not limited to those described above, and various types of document elements can be extracted using a known recognition technique.

Next, the summary element management information described above will be described.
FIG. 9 is a diagram showing a data structure of summary element management information. Referring to FIG. 9, page number, region upper left coordinate, region lower right coordinate, and element type are recorded for each summary element in memory unit 102. The page number is position information for specifying the page on which the corresponding summary element exists. Further, the upper left region coordinates and the lower right region coordinates are position information for specifying the region (rectangle) in the page of the corresponding summary element. The element type is information for specifying the type of the corresponding summary element.

  Referring to FIG. 6 again, when the process of step S108 is completed, the stage detection unit 18 detects the stage, the connection of the stages, and the row direction of each stage from the document image in the image buffer unit 14 (step S110). ). The stage detection unit 18 temporarily records the detected information in the memory unit 102.

  The management information regarding the level detected in step S110 in FIG. 6 is recorded in the memory unit 102 as level management information.

  FIG. 10 is a diagram illustrating a data structure of the column management information. Referring to FIG. 10, in the memory unit 102, a column identification number, a page number, a column upper left coordinate, a column lower right coordinate, a previous paragraph, a subsequent paragraph, and a line direction are recorded for each column. The stage identification number is identification information for uniquely identifying each stage. The page number is position information for specifying the page on which the corresponding stage exists. The upper left coordinates and the lower right coordinates are position information for specifying an area (rectangle) in the page of the corresponding stage. The previous paragraph and the subsequent paragraph are connection information of each stage. The identification number of the stage immediately before the corresponding stage is recorded in the column of the previous paragraph, and the identification number of the stage immediately after the corresponding stage is recorded in the column of the subsequent paragraph. Based on such connection information, the order of the steps in the page can be specified. The row direction is information for specifying the row direction of the corresponding stage, and “horizontal writing” or “vertical writing” is recorded.

  In the flowchart of FIG. 6, the example in which the process of step S108 and the process of step S110 are performed in series is shown, but these processes may be performed in parallel. Moreover, when performed in series, these orders are not ask | required.

  Next, the element correction part 20 performs the correction process of the extracted summary element (step S111). Specifically, the element correction unit 20 first displays a “summary element confirmation screen” on the second operation panel unit 111 (step S302). FIG. 11 is a diagram illustrating an example of the summary element confirmation screen.

  Referring to FIG. 11, extracted summary elements 611 and 612 are displayed in a display area 111 a of second operation panel unit 111 surrounded by a rectangular frame. A delete button 621, an element addition button 622, a range change button 623, an OK button 624, a previous page button 625, a next page button 626, and up / down / left / right cursor buttons 627 are displayed in an area 111b below the display area 111a. By pressing the cursor button 627 and the OK button 624, a user-desired summary element is selected.

  The delete button 621 is a button for accepting an instruction to delete the selected summary element. When detecting that the button 621 has been pressed, the element correction unit 20 deletes the management information for the selected summary element from the summary element management information.

  The element addition button 622 is a button for accepting an instruction to manually add a new summary element. When the button 622 is pressed, the element correction unit 20 displays, for example, a default rectangular frame in the area 111a. Thereafter, when the cursor button 627 is pressed, the rectangular frame is moved according to the instruction. Next, when the OK button 624 is pressed, the element addition button 622 determines the upper left position of the rectangular frame as the upper left corner of the summary element. Next, when the cursor button 627 is pressed, the rectangular frame is enlarged or reduced according to the instruction. When the OK button 624 is further pressed, the lower right position of the rectangular frame is determined as the lower right corner of the summary element. When the region is determined in this way, the element correction unit 20 adds information on the determined region to the memory unit 102 as new summary element information. At that time, the element correction unit 20 records “manual” in the element type column. In the present embodiment, it is predetermined that the type “manual” is a type to be included in the summary image.

  The range change button 623 is a button for accepting an instruction to change the range of the selected summary element. When the cursor button 627 is pressed after the button 623 is pressed, the frame of the selected summary element is enlarged or reduced according to the instruction. When the user presses the OK button 624, the element correction unit 20 overwrites the position information corresponding to the determined range on the area upper left coordinates and area lower right coordinates.

  The previous page button 625 and the next page button 626 are buttons for receiving instructions to change the displayed page to the previous page and the next page, respectively. When these buttons 625 and 626 are pressed, the element correction unit 20 displays a page corresponding to the instruction in the area 111a.

  When the element correction unit 20 receives an instruction regarding correction of the summary element from the user, the element correction unit 20 corrects the summary element according to the instruction. In addition, a display reflecting such correction is performed. Finally, the correction / display processing as described above is repeated until an OK instruction is input from the user. When an OK instruction is finally input from the user, the process proceeds to step S112.

  In step S112, the partial area cutout unit 22 performs a partial area cutout process (step S112). The partial area extraction process will be described with reference to the subroutine of FIG.

(Partial area extraction processing)
The partial area cutting process will be described with reference to FIGS.

  FIG. 12 is a flowchart showing the partial area extraction processing in the embodiment of the present invention. In this flowchart, it is assumed that the processing is based on the assumption that all pages of the document are written horizontally.

  Referring to FIG. 12, partial region cutout unit 22 in the present embodiment first performs the above-described correction processing (steps S202 to S210).

  In step S202, the partial area cutout unit 22 detects a body part region (hereinafter referred to as a “body region”) from each stage extracted in each page. The text region can be detected as a set of rows having the same row height and spacing as disclosed in, for example, Japanese Patent Application Laid-Open No. 2005-71014. Note that the position information of the document element representing the text extracted in step S108 in FIG. 6 may be used.

  When the text area is detected, the partial area cutout unit 22 virtually sets the text reference line (step S204). The text reference line corresponds to one side of the smallest rectangle (referred to as “text frame”) that surrounds each text area, and is a line on the head side that is perpendicular to the row direction. More specifically, in horizontal writing, the upper left corner of the text frame is used as a reference point, and a vertical line passing through the reference point and orthogonal to the row direction is set as the text reference line. In the case of vertical writing, the upper right corner is set as a reference point, and a horizontal line passing through the reference point and orthogonal to the row direction is set as a text reference line.

  The management information related to the text reference line set in step S204 in FIG. 12 is recorded in the memory unit 102 as the text reference line management information.

  FIG. 13 is a diagram illustrating a data structure of the text baseline management information. Referring to FIG. 13, the memory unit 102 stores a column identification number, a text area upper left coordinate, a text area lower right coordinate, and a text reference line position for each text area. The column identification number is information on the column identification number to which the corresponding text area belongs. The text region upper left coordinate and the text region lower right coordinate are position information of the corresponding text region. The text reference line position is information indicating the position of the text reference line in the corresponding text area. If the position of the text reference line can be specified, it is not necessary to include all these pieces of information in the text reference line information.

  FIG. 14 is a diagram for explaining the text reference line. FIG. 14A and FIG. 14B show images of two consecutive pages, respectively. Further, it is assumed that all the levels included in these pages are horizontal writing.

  The area on the left side of the page shown in FIG. 14A includes steps 501a and 502a in order from the top. The area on the right side of the page includes a step 503. The stage 501a includes a heading (summary element area) 561 and a text area 501b. The stage 502a includes a heading (summary element area) 562 and a text area 502b. Since the step 503 includes only the text area, it matches the text area (503).

  A region on the left side of the page shown in FIG. 14B includes a step 504a. Further, the area on the right side of the page includes a stage 505 and a stage 506a. The stage 504a includes a heading (summary element area) 563 and a text area 504b. Since the stage 505 includes only the text area, it matches the text area (505). The stage 506a includes a heading A (summary element region) 564, a heading B (summary element region) 565, and a text region 506b.

  As shown in these drawings, for each of the text areas 501b, 502b, 503, 504b, 505 and 506b, a straight line at the left end of the text area (that is, the left end side of the text frame) 511, 512, 513, 514, 515 , 516 are set as the text reference line.

  Referring again to FIG. 12, partial region cutout unit 22 determines a group of text reference lines in each page (step S206). Specifically, in each page, a text reference line that is close to a predetermined threshold value and in the same direction is determined as one group. Here, the “predetermined threshold value” may be determined in advance corresponding to the page size of the document image.

  In the case of FIG. 14A as an example, the text reference line 511 and the text reference line 512 are determined as one group, and the text reference line 513 is determined as a single group. In the case of FIG. 14B as an example, the text reference line 514 alone is determined as one group. Also, the text reference line 515 and the text reference line 516 are determined as one group.

  When a group is determined on each page, a text group reference line is calculated for each group (step S208). Specifically, the partial area cutout unit 22 first calculates an average position of the text reference line (a vertical position for horizontal writing and a horizontal position for vertical writing). Then, a line parallel to the original reference line group and passing through the average position is calculated as a text group reference line.

  FIG. 15 is a diagram illustrating an example of correcting a layout shift between pages based on a text group reference line in a processing page. The two pages shown in FIGS. 15A and 15B are the same as the two pages shown in FIGS. 14A and 14B, respectively.

  In the page shown in FIG. 15A, a text group reference line 521 based on the text reference line 511 and the text reference line 512 and a text group reference line 522 based on the text reference line 513 are set.

  15B, a text group reference line 523 based on the text reference line 514 and a text group reference line 524 based on the text reference line 515 and the text reference line 516 are set.

  When the text group reference line is calculated, the partial region cutout unit 22 corrects a layout shift between pages based on the text group reference line (step S210). Specifically, in order from the second page, the processing pages are sequentially shifted in the vertical and horizontal directions so that the body group reference line matches the previous page.

  For example, the processing page is shifted so that the text group reference line 523 of the processing page shown in FIG. 15B matches the text group reference line 521 of the previous page shown in FIG. Further, the processing page is shifted so that the text group reference line 524 of the processing page matches the text group reference line 522 of the previous page.

  The shift amount of the processed page is obtained as follows, for example. That is, the partial region cutout unit 22 associates each body group reference line in the processing page with the body group reference line closest in the same direction within a predetermined distance when the body group reference line is superimposed on the previous page. Then, the shift amount is calculated by, for example, the least square method so that the distance between the associated text group reference lines is minimized.

  In this manner, the layout shifted during scanning can be normalized before cutting out the partial area. As a result, hierarchical levels such as headings can be accurately reflected in the summary image.

  In the present embodiment, the text group reference line is obtained for the processing page and then the layout shift between pages is corrected. However, the present invention is not limited to such a method. FIG. 16 is a diagram illustrating an example of correcting a layout shift between pages based on a text reference line in a processing page. According to this example, the processing page is shifted so that the text reference line 514 of the processing page matches the text group reference line 521 of the previous page. Further, the processing page is shifted so that the text reference lines 515 and 516 of the processing page are aligned with the text group reference line 522 of the previous page. Alternatively, the correction process may be performed based only on the text reference line without obtaining the text group reference line.

  Referring to FIG. 12 again, when the layout shift between pages is corrected as described above, the line setting unit 19 sets the position where both end positions in the row direction are within a predetermined threshold in each page. It is determined that they belong to the same step layout frame (step S212). In other words, in the case of horizontal writing, it is determined that the stages whose left and right end positions are close to each other within the predetermined threshold value belong to the same stage layout frame. In the case of vertical writing, it is determined that the steps whose end positions are close to each other within a predetermined threshold value belong to the same step layout frame. Note that the column layout frame is determined in consideration of the row direction of the column.

  The “predetermined threshold value” here is preferably determined based on the character size and line spacing of the document image, and may be determined in advance corresponding to the page size. Alternatively, the line setting unit 19 may calculate a value serving as a determination criterion for the step layout frame for each document image based on the step width in the page.

  The line setting unit 19 sets a virtual rectangle including a step determined to belong to each step layout frame for each step layout frame (step S214). More specifically, based on the position information of the steps belonging to the step layout frame, a virtual rectangle including all document elements (including the summary element region) in the steps belonging to the same step layout frame is set.

  FIG. 17 is a diagram illustrating the virtual rectangle set in step S214 of FIG. The page shown in FIG. 17 is the same as the page shown in FIG. 14B, FIG. 15B, and FIG.

  Referring to FIG. 17, since there is no other stage that satisfies the conditions for the stage layout frame in stage 504 a shown in FIG. 14B, one virtual rectangle 531 is set only by stage 504 a. Therefore, the area of the step 504a matches the area of the virtual rectangle 531. Further, the level 505 and the level 506a illustrated in FIG. 14B are grouped because the conditions for the level layout frame are satisfied, and a virtual rectangle 532 is set for the level 505 and the level 506a.

  When the virtual rectangle is set, the line setting unit 19 sets, as a layout reference line, a line that is one side of the virtual rectangle and is perpendicular to the line direction (step S216). In the case of horizontal writing, the vertical line at the left end of the virtual rectangle is set as the layout reference line. In the case of vertical writing, the horizontal line at the upper end of the virtual rectangle is set as the layout reference line.

  In FIG. 17, the leftmost vertical line 541 of the virtual rectangle 531 is set as the layout reference line. A vertical line 542 at the left end of the virtual rectangle 532 is set as a layout reference line.

  The management information related to the layout reference line set in step S216 in FIG. 12 is recorded in the memory unit 102 as layout reference line management information.

FIG. 18 is a diagram illustrating a data structure of layout reference line management information.
Referring to FIG. 18, in memory unit 102, the stage layout frame identification number, the stage identification number, and the position of the layout reference line are recorded for each stage layout frame. The stage layout frame identification number is identification information for uniquely identifying the stage layout frame. In the column of the column identification number, the column identification number included in the corresponding column layout frame is recorded. The position of the layout reference line is information for specifying the position of the step layout frame. For example, the upper left coordinates and the lower right coordinates are recorded as the position information.

  When the layout reference line is set, the partial area cutout unit 22 cuts out a rectangular area having the layout reference line as an end for each summary element area (step S218). More specifically, in the case of horizontal writing, a rectangular area having a boundary between the layout reference line corresponding to the stage to which the summary element area belongs and the upper and lower right ends of the summary element area is cut out. In the case of vertical writing, a rectangular area whose boundary is the layout reference line and the left and right lower ends of the summary element area is cut out.

  FIG. 19 is a diagram for explaining the rectangular region cut out in step S218 of FIG.

  Referring to FIG. 19, heading 563 is included in rectangular area 571. The heading 564 is included in the rectangular area 572. The heading 565 is included in the rectangular area 573.

  In the headings 563 and 564, the left ends of the headings 563 and 564 coincide with the layout reference lines 541 and 542, respectively. Therefore, the rectangular areas 571 and 572 match the areas of the heading 563 and the heading 564, respectively. On the other hand, the left end of the heading 565 does not coincide with the layout reference line 542. In this case, the left end of the rectangular area 573 is cut out so as to be the layout reference line 542.

When the process of step S218 ends, the process is returned to the main routine.
Referring to FIG. 6 again, when the partial area extraction process described above is completed, summary image generation unit 24 performs a summary image generation process (step S114). This summary image generation processing will be described with reference to the subroutine of FIG.

(Summary image generation processing)
The summary image generation process will be described with reference to FIGS. 9, 10, and 20 to 22.

  FIG. 20 is a flowchart showing a summary image generation process in the embodiment of the present invention.

  Referring to FIG. 20, summary image generation unit 24 determines the order of the summary element areas (step S310). Specifically, the order of the summary element areas is determined based on the position information of each summary document area in the summary element management information in FIG. 9 and the position information and stage connection information in the stage management information in FIG. Is done.

  When the order of the summary element regions is determined, the summary image generation unit 24 sequentially arranges the cut-out rectangular regions along a predetermined element reference line in the image on a predetermined image (step S312). ). In the case of horizontal writing, the summary image generation unit 24 arranges rectangular areas at predetermined intervals in order from the top. On the other hand, in the case of vertical writing, the summary image generating unit 24 arranges the rectangular areas at predetermined intervals in order from the right.

  FIG. 21 is a diagram for explaining the rectangular area (summary element area) arranged in step S312 of FIG.

  Referring to FIG. 21, the cut out rectangular regions 711, 712, 713, 714, 715,... Are arranged at predetermined intervals in order from the top along the element reference line 710 in the output image.

  FIG. 22 is a diagram illustrating an example of a finally generated summary image (one page). In this embodiment, the summary element area is cut out while maintaining the indent from the layout reference line, so that a summary image that directly reflects the hierarchy of the original image document is generated as shown in FIG. Can do. In this figure, the summary image has “Report on ...” as the title and “1. Outline”, “2. Background”, “3 ....”, “A) ...” as the headings. , “B) ...”, “4. Future outlook”. In addition, a graph is arranged as a figure, and a text area “Concluded as a conclusion ...” is arranged as a manual. As described above, according to the present embodiment, it is possible to easily generate a summary image desired by the user without determining the hierarchy of the summary element region.

When the process of step S312 ends, the process returns to the main routine.
(Output data generation processing)
The output data generation process will be described with reference to FIGS. 6 and 23 to 26.

  Referring to FIG. 6 again, when the summary image generation process is completed, output data generation unit 26 determines whether the output form selected in step S104 is “separation” or “combination” (S116). .

  If the selected output form is “separation”, the process proceeds to step S126. On the other hand, if the selected output form is “synthesis”, the process proceeds to step S118.

  In step S 118, the output data generation unit 26 first adds the summary image data input from the summary image generation unit 24 to the head of the document image data input from the image buffer unit 14.

  Further, the bookmark assigning unit 28 generates bookmark data for composite output (step S120). The bookmark assigning unit 28 adds the generated bookmark data to the image data generated in step S118 (referred to as “composite image data”) (step S122). When bookmark data is added to the composite image data, the data storage processing unit 30 stores the composite image data to which the bookmark is added in the storage unit 112 as an electronic composite document 4001 (step S124).

  FIG. 23 is a diagram illustrating an example of a data structure of an electronic document (electronic composite document 4001) when the output form is “composite”.

  Referring to FIG. 23, an electronic composite document 4001 includes a header portion 402, a summary image portion 403, a bookmark data portion 404, a document image portion 406, and a footer portion 408. The header part 402 and the footer part 408 store information about the attributes of the electronic composite document 4001, such as creation date / time / creator / copyright information. The summary image unit 403 stores the summary image generated by the summary image generation unit 24. The bookmark data unit 404 stores bookmark data for associating the summary element area included in the summary image with the position of the summary element area in the original document image. A document image corresponding to each page is stored in the document image unit 406. The document image may be stored in a compressed state as described above. The summary image may also be stored in a compressed state when the number of pages increases.

  FIG. 24 is a diagram illustrating an example of the data structure of bookmark data when the output form is “composite”.

  Referring to FIG. 24, the bookmark data is associated with each summary element area, the page number of the summary image, the upper left area coordinates, the lower right area coordinates, the element type, the page number of the original document image, and the upper left area. Coordinate / region lower right coordinates are included.

  The page number of the summary image is position information for specifying the page where the corresponding summary element area exists in the summary image. Further, the region upper left coordinates and the region lower right coordinates of the summary image are position information for specifying the region (rectangle) in the page of the corresponding summary element region in the summary image.

  The position information may be position information of a rectangular area that is cut out and arranged. Alternatively, since it is sufficient that each summary element area can be specified by the position information, for example, the position information of an area near the rectangular area may be used.

  The element type is information for specifying the type of the document element related to the corresponding summary element area. The element type may not be included in the bookmark data.

  The page number of the original document image is position information for specifying the page in which the corresponding summary element area exists in the document image. Further, the region upper left coordinates and the region lower right coordinates of the summary image are position information for specifying the region (rectangle) in the page of the corresponding summary element region in the document image.

  In this way, a tag indicating a corresponding portion in the original document image is embedded in the summary image in association with each arranged summary element region. As a result, when the summary element area in the summary image is designated (clicked or the like) by the viewer software, it is possible to jump to the corresponding part of the document image and display the corresponding part.

  Referring to FIG. 6 again, in step S126, the output data generation unit 26 receives the summary image data input from the summary image generation unit 24 separately from the document image data input from the image buffer unit 14. Generate. Further, the bookmark assigning unit 28 generates bookmark data for separation output (step S128). The bookmark giving unit 28 adds the generated bookmark data to the summary image data generated by the summary image generating unit 24 (step S130). When bookmark data is added to the summary image data, the data storage processing unit 30 stores the document image data and the summary image data to which the bookmark data is added as an electronic original document 4002A and an electronic summary document 4002B, respectively. 112 (step S132).

  FIG. 25A is a view showing an example of the data structure of the electronic original document 4002A when the output form is “separated”, and FIG. 25B is an electronic summary document when the output form is “separated”. It is a figure which shows an example of the data structure of 4002B.

  Referring to FIG. 25A, an electronic original document 4002A includes a header portion 402, a document image portion 406, and a footer portion 408. Compared with the electronic composite document 4001 in the case of the composite output shown in FIG. 23, the electronic original document 4002A does not include the summary data portion and the bookmark data portion.

  Referring to FIG. 25B, the electronic summary document 4002B includes a header part 412, a summary image part 414, a bookmark data part 416, and a footer part 418. The header part 412 and the footer part 408 store information about the attributes of the electronic summary document 4002B, such as creation date / time / creator / copyright information. The summary image unit 414 stores the summary image generated by the summary image generation unit 24. The bookmark data unit 416 stores bookmark data for associating the summary element area included in the summary image with the position of the summary element area in the original document image.

  FIG. 26 is a diagram illustrating an example of the data structure of bookmark data when the output form is “separated”.

  Referring to FIG. 26, in the bookmark data, the summary image page number, the region upper left coordinate, the region lower right coordinate, the element type, the file name of the original document image, and the page number are associated with each summary element region. -The area upper left coordinates and area lower right coordinates are included. As described above, when compared with the bookmark data in the combined output shown in FIG. 24, the bookmark data in the separated output includes the file name of the original document image. Here, the file name of the original document image is also stored for each summary element area. However, since the file names are all the same, the file name is not limited to this form as long as the file name can be specified.

  In this way, a tag indicating a corresponding portion in the original document image is embedded in the summary image in association with each arranged summary element region. As a result, even in the case of separate output, when the summary element area in the summary image is specified (clicked, etc.) with the viewer software, it is possible to jump to the corresponding part of the document image and display the corresponding part It becomes.

  When the process of step S124 or step S132 ends, the series of document summary creation processes ends.

<Modification>
Next, an MFP according to a modification of the embodiment of the present invention will be described.

  In MFP 1 according to the above-described embodiment, a rectangular area including a summary element area is cut out with the layout reference line as an end portion on the line head side, and the cut out rectangular area is arranged in a predetermined image along the element reference line. Therefore, in the summary image, the position in the row direction (indent) in the original document image is maintained.

  However, it is not limited to the form which cuts out a summary element area | region including an indent part in this way.

  In the modification of the present embodiment, a minimum rectangle surrounding the summary element area is cut out, and a summary image reflecting a hierarchical level such as a heading is generated. Therefore, the process of the partial area cutout unit and the process of the summary image generation unit are different from the above embodiment. The other parts are the same. Therefore, the same reference numerals as those of the MFP 1 described above will be used here.

Only the parts different from the above embodiment will be described below.
The control unit 100 of the MFP 1 according to the modification of the present embodiment replaces the partial region cutout unit 22 and the summary image generation unit 24 of the embodiment with the partial region cutout unit 22 # and the summary image generation unit 24 #. including.

  The partial area cutout unit 22 # detects the shift amount from the minimum rectangular layout reference line surrounding the summary element area. Further, the detected shift amount is recorded in the memory unit 102 in association with the summary element area.

  FIG. 27 is a diagram showing a data structure of summary element management information in a modification of the embodiment of the present invention. Referring to FIG. 27, in addition to the information shown in FIG. 9, the memory unit 102 further records a shift amount for each summary element (summary element region).

  The summary image generation unit 24 # reads the shift amount for each summary element region, shifts the shift amount from the element reference line by the read shift amount, and arranges the summary elements in the output image.

  FIG. 28 is a flowchart showing a partial region extraction process in a modification of the embodiment of the present invention. The same processes as those in the flowchart of FIG. 12 are denoted by the same step numbers. Therefore, the description about them is not repeated.

  Referring to FIG. 28, in the partial region extraction process according to the modification of the present embodiment, processes in steps S220 and S222 are executed instead of step S218 in FIG.

  In step S220, the partial region cutout unit 22 # detects the shift amount from the minimum rectangular layout reference line surrounding the summary element region for each summary element region. That is, the distance between the end position on the line head side of the summary element and the layout reference line is calculated. Specifically, the partial area cutout unit 22 # calculates the shift amount based on the position information of the summary element area (see FIG. 27) and the position information of the layout reference line (see FIG. 18).

  In step S222, the partial region cutout unit 22 # cuts out the minimum rectangle surrounding the summary element region, and records the shift amount in the memory unit 102 in association with each summary element region.

  FIG. 29 is a flowchart showing a summary image generation process in a modification of the embodiment of the present invention. The same processes as those in the flowchart of FIG. 20 are denoted by the same step numbers. Therefore, the description about it (step S310) will not be repeated.

  Referring to FIG. 29, in the summary image generation process according to the modification of the present embodiment, the process of step S320 is executed instead of step S312 of FIG.

  In step S320, the summary image generation unit 24 # reads the shift amount of the summary element area from the memory unit 102, and shifts the summary element area (() by shifting the shift amount from the element reference line of the output image according to the order determined in step S310. Place the minimum rectangle).

  As described above, even if the process as in the modification of the present embodiment is performed, the same result as the arrangement form shown in FIG. 21 is obtained. Therefore, also in the modification, a summary image reflecting the hierarchical structure of the original document can be created without determining the hierarchy of headings and the like.

  In this modification, the value of the detected shift amount itself is recorded, and the summary element area is shifted and arranged by the recorded value. However, the shift amount may not be shifted by the value as it is. For example, the shift amount may be shifted by a constant multiple of the detected shift amount or a quantized value. In this case, such a value may be recorded when recording in the memory unit 102, or the shift amount recorded by storing the detected shift amount and performing a predetermined calculation when arranging it. It may be shifted by a constant multiple of the amount or the quantized value.

  Alternatively, in this modification, the minimum rectangle surrounding the summary element area is cut out, but the minimum rectangle is not necessarily required as long as the hierarchy of the summary element area can be reflected. For example, for all the summary element areas, a rectangular area including a margin for a predetermined distance may be cut out at the beginning of the line. In other words, the side of the smallest rectangle surrounding the summary element may be cut out by extending it by a predetermined distance.

  As described above, in the present embodiment and its modifications, a summary image is generated by cutting out and arranging the summary element region. However, the region other than the summary element region is erased and packed at a predetermined interval. A summary image may be generated.

<Other variations>
In the above-described embodiment and its modification, the case where the processing according to the present invention is executed by the MFP 1 has been described. However, even if the above-described processing is executed in a computer having an image reading function for reading the document 300. Good. In this case, it is also possible to provide a program for executing the processing function shown in FIG. 5 for causing the computer to function as a document processing apparatus. Such a program can be stored in an optical medium such as a CD-ROM (Compact Disk-Read Only Memory) or a computer-readable recording medium such as a memory card and provided as a program product. Alternatively, the program can be provided by being stored in a storage medium such as a hard disk built in the computer. A program can also be provided by downloading via a network.

  Further, the program in the memory unit 102 can be updated by reading, for example, an optical medium in which the program is recorded with a driving device (not shown) of the MFP 1.

  Alternatively, the image reading function may be realized by another device or a computer, the generated document image may be received, and a summary image may be generated according to the above processing. Moreover, although the configuration in which the summary image and the browsing support information (bookmark data) are included in the same document has been illustrated, the browsing support information may not necessarily be added to the same document, and may be output as a separate file. Good.

  The program according to the present invention is a program module that is provided as a part of a computer operating system (OS) and calls necessary modules in a predetermined arrangement at a predetermined timing to execute processing. Also good. In that case, the program itself does not include the module, and the process is executed in cooperation with the OS. A program that does not include such a module can also be included in the program according to the present invention.

  The program according to the present invention may be provided by being incorporated in a part of another program. Even in this case, the program itself does not include the module included in the other program, and the process is executed in cooperation with the other program. Such a program incorporated in another program can also be included in the program according to the present invention.

  The provided program product is installed in a program storage unit such as a hard disk and executed. Note that the program product includes the program itself and a storage medium in which the program is stored.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

1 is a schematic configuration diagram of a system including a document processing apparatus (MFP) according to an embodiment of the present invention and a modification thereof. FIG. 11 is a block diagram showing a schematic functional configuration in an MFP according to an embodiment of the present invention and a modification thereof. (A) is a figure which shows an example of the external appearance of the 1st operation panel part of MFP according to embodiment and its modification of this invention, (b) is according to embodiment and its modification of this invention. 3 is a diagram illustrating an example of an appearance of a second operation panel unit of the MFP. FIG. It is a schematic diagram which shows the schematic hardware constitutions of the personal computer according to embodiment of this invention and its modification. FIG. 5 is a functional block diagram showing a functional configuration of an MFP according to an embodiment of the present invention and a modification thereof. It is a flowchart which shows the document summary creation process in embodiment of this invention and its modification. It is a figure which shows an example of an output form selection screen. 4A and 4B are diagrams for explaining extraction processing of document elements included in a document image, where FIG. 5A is a diagram illustrating an example of a document image generated from a document, and FIG. 5B is a document illustrated in FIG. It is a figure which shows an example of the state by which the document element was extracted with respect to the image. It is a figure which shows the data structure of the summary element management information in embodiment of this invention. It is a figure which shows the data structure of the stage management information in embodiment of this invention and its modification. It is a figure which shows an example of a summary element confirmation screen. It is a flowchart which shows the partial region extraction process in embodiment of this invention. It is a figure which shows the data structure of the text reference line management information in embodiment of this invention and its modification. It is a figure for demonstrating a text reference line. It is a figure which shows the example which correct | amends the shift | offset | difference of the layout between pages based on the text group reference line in a process page. It is a figure which shows the example which correct | amends the shift | offset | difference of the layout between pages based on the text reference line in a process page. It is a figure which shows the virtual rectangle set in step S214 of FIG. It is a figure which shows the data structure of the layout reference line management information in embodiment of this invention and its modification. It is a figure for demonstrating the rectangular area cut out in step S218 of FIG. It is a flowchart which shows the summary image generation process in embodiment of this invention. It is a figure for demonstrating the summary element area | region arrange | positioned in FIG.20 S312. It is a figure which shows an example of the summary image finally produced | generated. It is a figure which shows an example of the data structure of an electronic document in case an output form is "composite". It is a figure which shows an example of the data structure of bookmark data in case an output form is "composite". (A) is a figure which shows an example of the data structure of an electronic manuscript document when an output form is "separation", (b) is an example of the data structure of an electronic summary document when an output form is "separation" FIG. It is a figure which shows an example of the data structure of bookmark data in case an output form is "separation". It is a figure which shows the data structure of the summary element management information in the modification of embodiment of this invention. It is a flowchart which shows the partial area | region extraction process in the modification of embodiment of this invention. It is a flowchart which shows the summary image generation process in the modification of embodiment of this invention.

Explanation of symbols

  1 MFP, 12 output form selection unit, 14 image buffer unit, 16 summary element extraction unit, 18 stage detection unit, 19 line setting unit, 20 element modification unit, 22 partial area extraction unit, 24 summary image generation unit, 26 output Data generation unit, 28 bookmark giving unit, 30 data storage processing unit, 32 transmission processing unit, 100 control unit, 102 memory unit, 104 image reading unit, 106 printing unit, 108 communication interface unit, 110, 111 operation panel unit, 112 Storage unit, 201 CPU, 203 internal bus, 205 display unit, 207 communication interface unit, 209 input unit, 211 hard disk unit, 213 memory unit, 215 CD-ROM drive, 215a CD-ROM, 217 FDD drive, 217a flexible disk, 300 manuscripts, 400 electronic document, 402 header part, 403 summary image part, 404 bookmark data part, 406 document image part, 408 footer part, 412 header part, 414 summary image part, 416 data part, 418 footer part, 420 document element, 501a, 502a, 504a, 506a stage, 501b, 502b, 504b, 506b text area, 503,505 stage (text area), 511, 512, 513, 514, 515, 516 text baseline, 521, 522, 523, 524 text group Reference line, 531 532 Virtual rectangle, 541, 542 Layout reference line, 571, 572, 573 Rectangular area, 611, 612 Summary element, 621 Delete button, 622 Add element button, 623 Change range button, 624 OK button, 625 Previous Page button, 26 Next page button, 627 Cursor button, 710 Element reference line, 711, 712, 713, 714, 715 Rectangular area, 4001 Electronic composite document, 4002A Electronic original document, 4002B Electronic summary document, PC1, PC2, PC3 Personal computer, SRV Server device.

Claims (16)

A document processing apparatus for creating a summary of a document image from a document image,
Extraction means for extracting a predetermined type of document element from the document image;
Detection means for detecting a stage for each page from the document image;
Line setting means for setting at least one layout reference line for each page based on the layout of the stage;
The layout reference line is a line in a direction perpendicular to the row direction of the corresponding stage,
Correction processing means for performing processing for correcting a shift of the layout reference line between pages;
Cutting means for cutting out a partial area including a summary area corresponding to the extracted document element of the predetermined type with reference to the layout reference line;
A document processing apparatus, further comprising: a summary generation unit configured to generate a summary image, which is an image representing the summary, by disposing the partial area in a predetermined image with reference to a predetermined element reference line in the image. . The cutting means cuts out the partial region with the layout reference line as an end,
The document processing apparatus according to claim 1, wherein the summary generation unit sequentially arranges the partial areas along the predetermined element reference line. The cutting means cuts out a minimum rectangle including the summary area as the partial area, and detects a distance between the partial area and the layout reference line for each partial area,
The document processing apparatus according to claim 1, wherein the summary generation unit shifts the partial area from the position of the predetermined element reference line according to the detected distance and arranges the partial area in the image. Wherein the correction processing means, in between pages, including alignment means for aligning the reference position of the body region of the stage, the document processing apparatus according to claim 1. Based on a predetermined condition, for each page, further comprising a rectangle setting means for grouping a plurality of the stages to set at least one virtual rectangle,
It said line setting means sets the layout reference line for each of the imaginary rectangle, the document processing apparatus according to claim 1. The document processing apparatus according to claim 5 , wherein the line setting unit sets a side on a line head side perpendicular to a line direction in the virtual rectangle as the layout reference line. Document generating means for separately generating a first electronic document including the document image and a second electronic document including the summary image;
Further comprising a storage means for storing the first electronic document and the second electronic document, the document processing apparatus according to claim 1. Further comprising: adding means for generating browsing support information for associating with the corresponding portion of the document image for each partial area, and adding the browsing support information to the summary image;
The viewing support information, document identification information for identifying the first electronic document corresponding, and includes location information indicating a correspondence to that location in the document image for each of the partial regions, claim 7 Document processing apparatus described in 1. Document generation means for generating an electronic document including the document image and the summary image;
Further comprising a storage means for storing the electronic document, the document processing apparatus according to claim 1. Further comprising: adding means for generating browsing support information for associating with the corresponding portion of the document image for each partial area, and adding the browsing support information to the summary image;
The document processing apparatus according to claim 9 , wherein the browsing support information includes position information indicating a corresponding location in the document image for each partial area. A correction unit for correcting the document element of the predetermined type extracted by the extraction unit based on an instruction from the user;
Said cutting means cuts out a region of the document element after correction by said correction means as the summary area, the document processing apparatus according to claim 1. The correction means includes designation means for accepting designation of an area desired by the user from the document image,
The document processing apparatus according to claim 11 , wherein the extraction unit extracts an area designated by the designation unit as the summary area. A selection means for accepting a selection from a user to output whether the summary image is output separately from the document image, or the summary image is combined with the document image and output;
First document generation means for generating a first electronic document including the document image and a second electronic document including the summary image when the selection means selects to output separately; ,
A second document generation unit configured to generate a third electronic document by combining the document image and the summary image when the selection unit selects to output the combined image; the document processing apparatus according to claim 1. Said predetermined type is title, headings, drawings, tables, photographs, a predetermined row of the beginning of the paragraph, the handwritten annotations of interest, and at least any one of captions, according to any of claims 1 to 13 Document processing device. In a document processing apparatus including a storage unit and an arithmetic processing unit, a method for creating a summary of the document image from a document image,
The arithmetic processing unit extracting a document element of a predetermined type from the document image;
The arithmetic processing unit detecting a step for each page from the document image;
The arithmetic processing unit sets, for each page, at least one layout reference line in a direction perpendicular to the row direction of the stage based on the layout of the stage;
The arithmetic processing unit performing a process for correcting a shift of the layout reference line between pages;
The arithmetic processing unit cutting out a partial area including a summary area corresponding to the extracted document element of the predetermined type with reference to the layout reference line;
And a step of generating a summary image by arranging the partial area in a predetermined image with reference to a predetermined element reference line in the image. A document summary creation program for causing a computer to execute the document summary creation method according to claim 15 .

Images