JP2008191948A - Layout processor, layout processing method, program and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To facilitate screening by a reader by disassembling many pieces of information composing an input image to areas and arranging the areas by category. <P>SOLUTION: The processor comprises a document attribute determination/disassembling part 2 for extracting an area from the input image; a category division means for dividing each area by category according to the beginning of the area extracted by the document attribute determination/disassembling part 2; and an area arrangement means for arranging areas belonging to different categories and areas belonging to the same category in different directions respectively. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a layout processing apparatus, a layout processing method, a program, and a recording medium.

  Traditionally, for documents such as newspapers and business documents where it is important to understand the background of the entire document and to understand the main points, readers need to be screened (= sieved, selected, selected), for example, "Table of contents, index (keyword), online search, parallel display of opposite document structure" and so on. On the other hand, in large display devices and newspapers, so-called “columns” are divided into small areas and the two-dimensional display is performed by dividing the entire sentence into two-dimensional displays. Methods for facilitating it are also known.

  As such a conventional technique, as a display method on a display device having a limited display area such as a mobile phone, a method of displaying only the title or selecting a sentence and displaying the selected sentence on the layout heading Is disclosed (for example, see Patent Document 1). In addition, a system is disclosed that can easily and quickly grasp the contents of a document by retrieving related images from words in the document data and creating and outputting illustrations of the documents from the images ( For example, see Patent Document 2).

JP 2001-195212 A JP 2006-215681 A

  However, in the conventional technique as described above, in Patent Document 1, if only the title is displayed, the judgment material when the reader performs screening is only the title, and appropriate screening cannot be performed. There was a problem. Further, in Patent Document 2, there is a problem that, with only an illustration, there is a lack of judgment information when a reader performs screening, and appropriate screening cannot be performed.

  The present invention has been made in view of the above, and an object of the present invention is to facilitate screening by a reader by disassembling a large amount of information constituting an input image into regions and arranging them into categories. To do.

  In order to solve the above-described problems and achieve the object, the invention according to claim 1 is directed to an image extracting unit that extracts a region from an input image, and each region according to a sentence head of the region extracted by the image extracting unit. Category dividing means for dividing each category, and area arranging means for arranging areas belonging to the same category as areas belonging to different categories, respectively.

  The invention according to claim 2 is characterized by further comprising area analysis means for analyzing the connection relation of the areas divided by the category dividing means.

The invention according to claim 3 further includes an arrangement unit that arranges a diagram, a table, and a photograph included in the input image next to a related area after the area is analyzed by the area analysis unit. And

  Further, the invention according to claim 4 is characterized in that the arrangement means arranges the area in the form of an enclosed article.

  The invention according to claim 5 is characterized in that the arranging means adjusts the size of the area arranged in accordance with the display size of the display device.

  The invention according to claim 6 further includes receiving means for receiving the input image from an image reading apparatus or a server on a network.

  Further, the invention according to claim 7 is characterized in that the image extracting means decomposes a region using feature information of punctuation marks, paragraph lowering, blank lines, and blank characters.

  The invention according to claim 8 is characterized in that the region analysis means determines a connection relation between the regions by a conjunction.

  The invention according to claim 9 includes an image extraction step of extracting a region from the input image, a category division step of dividing each region into categories according to the beginning of the region extracted in the image extraction step, and a different category. A region arrangement step of arranging regions belonging to the same category as the region to which the regions belong in different directions.

  The invention according to claim 10 further includes a region analyzing step of analyzing a connection relation of the regions divided in the category dividing step.

  The invention according to claim 11 further includes an arrangement step of arranging a diagram, a table, and a photograph included in the input image next to the related region after the region is analyzed in the region analysis step. And

  The invention according to claim 12 is characterized in that the arranging step arranges the area in a form of an enclosed article.

  The invention according to claim 13 is characterized in that in the arranging step, the size of the area arranged is adjusted in accordance with the display size of the display device.

  The invention according to claim 14 includes a receiving step of receiving the input image from an image reading apparatus or a server on a network.

  The invention according to claim 15 is characterized in that in the image extraction step, the region is decomposed using feature information of punctuation marks, paragraph lowering, blank lines, and blank characters.

  The invention according to claim 16 is characterized in that, in the region analysis step, a connection relation between the regions is determined by a conjunction.

  The invention according to claim 17 is characterized by causing a computer to implement the layout processing method according to any one of claims 9 to 16.

  The invention according to claim 18 is a computer-readable recording medium in which the program according to claim 17 is recorded.

  According to the layout processing apparatus of the present invention (Claim 1), an area is extracted from an input image by an image extracting means, and the extracted area is divided into categories by a category dividing means. By arranging regions belonging to the same category as regions belonging to different directions, it is possible to decompose a lot of information constituting the input image into regions and rearrange them into categories, making it easy for readers to screen There is an effect that can be made.

  Further, according to the layout processing apparatus of the present invention (claim 2), in claim 1, the relationship between the regions can be determined by further comprising region analysis means for analyzing the connection relationship of the divided regions. There is an effect that can be done.

  Further, according to the layout processing apparatus of the present invention (Claim 3), after analyzing the area by the area analysis means in Claim 2, the diagram, table, and photograph included in the input image are located near the related area. By arranging, it is possible to easily view diagrams, tables, and photographs related to the area.

  In addition, according to the layout processing apparatus of the present invention (Claim 4), in addition to Claim 2 or 3, the layout processing apparatus further includes an arrangement unit that arranges the area in the form of an enclosed article, thereby facilitating identification for each area. There is an effect.

  According to the layout processing apparatus of the present invention (Claim 5), the display apparatus according to Claim 2, 3 or 4 is adjusted by adjusting the size of the area according to the display size of the display apparatus. Since the display screen is displayed in a large size by making the maximum use of the display screen it has, there is an effect that an easy-to-see image is output.

  Further, according to the layout processing apparatus of the present invention (Claim 6), it is possible to easily input a large number of image sources by capturing an input image from the image reading apparatus or a server on the network. There is an effect.

  According to the layout processing apparatus of the present invention (Claim 7), the area is decomposed by using the feature information of punctuation marks, paragraph lowering, blank lines, and blank characters according to Claim 1, so that a detailed area is obtained. The effect that it can be decomposed | disassembled is produced.

  According to the layout processing apparatus of the present invention (Claim 8), in Claim 2 or 3, there is an effect that the connection relation between the regions can be analyzed by a conjunction.

  In addition, according to the layout processing method of the present invention (claim 9), an area is extracted from an input image by an image extraction process, and the extracted areas are integrated and divided by category by a category division process. In order to realize that it is possible to disassemble a lot of information composing the input image into regions and rearrange them by category by arranging the regions belonging to the same category as the regions belonging to different categories in different directions. There is an effect that screening by can be facilitated.

  Further, according to the layout processing method of the present invention (claim 10), in claim 9, the relationship between the regions can be determined by further including a region analysis step of analyzing the connection relationship of the divided regions. There is an effect that can be done.

According to the layout processing method (claim 11) of the present invention, in claim 10, after analyzing the region in the region analysis step, the figure, table, and photograph included in the input image are located near the related region. By arranging, it is possible to easily view diagrams, tables, and photographs related to the area.

  In addition, according to the layout processing method of the present invention (claim 12), in claim 9 or 10, the layout processing method further includes an arrangement step of arranging the area in the form of an enclosed article, thereby facilitating identification for each area. There is an effect.

  According to the layout processing method of the present invention (claim 13), the display device according to claim 10, 11 or 12 is adjusted by adjusting the size of the area according to the display size of the display device. Since the display screen is displayed in a large size by making the maximum use of the display screen it has, there is an effect that an easy-to-see image is output.

  According to the layout processing method (claim 14) of the present invention, in claim 9, a large number of image sources can be easily input by fetching an input image from an image reading apparatus or a server on a network. There is an effect that can be done.

  Further, according to the layout processing method (claim 15) of the present invention, in the claim 9, the region is decomposed by using the feature information of the punctuation marks, the paragraph lowering, the blank line, and the blank character, thereby subdividing the region. The effect that it can be decomposed | disassembled is produced.

  In addition, according to the layout processing method of the present invention (claim 16), the connection relationship between the areas can be analyzed by a conjunction in claim 9, 10 or 11.

  Moreover, according to the program of this invention (Claim 17), there exists an effect that the layout processing method as described in any one of Claims 9-16 can be implement | achieved by a computer.

  According to the recording medium of the present invention (Claim 18), the layout processing method according to any one of Claims 9 to 16 is executed on a computer by recording the program according to Claim 17. There is an effect that can be done.

  Exemplary embodiments of a layout processing apparatus, a layout processing method, a program, and a recording medium according to the present invention are explained in detail below with reference to the accompanying drawings.

(Embodiment)
The present invention requires “screening (= sieving. Selection, selection)” by the reader for documents such as newspapers and business documents where it is important to understand the background of the entire document and to understand the main points. Therefore, it provides a layout method that is easy to read for “screening”. Also, in large display devices and newspapers, so-called “columns” are divided into small areas and the two-dimensional display is performed to facilitate the two-dimensional grasp of the sentence. In the method, a method for automating the layout of “columns” is provided. This will be specifically described below.

FIG. 1 is a block diagram showing a configuration of a layout automation apparatus according to an embodiment of the present invention. As shown in FIG. 1, a layout automation apparatus 100 as a layout processing apparatus according to the present invention includes a control unit 1 that controls the apparatus according to a program in accordance with a program for the operation of the apparatus as will be described later. A document attribute determination / decomposition unit 2 serving as an image extracting unit that determines a document attribute such as a figure, a table, and a text, and decomposes the document attribute into each document attribute. Paragraph integration as a category division means to integrate about 10 pages on a page
The division unit 3, the paragraph hierarchy analysis unit 4 as an area analysis unit for analyzing the paragraph hierarchy, the block arrangement unit 5 as an arrangement unit to arrange within one page from the analysis result of the paragraph, A column creation unit 6 having a function of region arrangement means for arranging paragraph text in a column (boxed article) configuration is provided. Further, the control unit 1 has a function of inputting an image to be processed from a server or a scanner (see FIG. 2) on a network (see FIG. 2).

  FIG. 2 is a block diagram showing a system configuration including the layout automation apparatus shown in FIG. In FIG. 2, reference numeral 100 denotes a layout automation apparatus configured as shown in FIG. 1, reference numeral 101 denotes a display unit for displaying a layout processing state according to the present invention on, for example, a liquid crystal panel, and reference numeral 102 denotes a layout in the present invention. An operation unit for performing necessary operations, a reference numeral 103 is a scanner for inputting a document to be processed, a reference numeral 104 is a document according to the present invention and processed data (for example, punctuation marks, paragraph lowering, blank lines used for paragraph decomposition) , A large capacity storage device for storing feature information such as blank characters and a disassembled paragraph) as necessary, a reference numeral 105 denotes a network I / F (interface) for performing an interface process with the network 107, Reference numeral 106 denotes a system control unit for overall control of the system, and reference numeral 107 denotes a personal computer. Etc. over data and servers are connected, a network such as the Internet for inputting document information to be processed.

  The layout automation device 100 configured as described above basically executes the following processing. First, as a processing flow, first, an input image is decomposed into each paragraph (≈an area such as a character string), and each paragraph (area) is divided into categories based on the head of each paragraph (area).

An example of this processing is as follows.
1. “Introduction”… “First” etc. “Development”… “Next”, “Mata”, “More”, etc. “But Sentence”… “But”
4). “Conclusion”… “To summarize”, “So”, etc.

  When the input image is decomposed into paragraphs and classified by category as described above, and the layout of each paragraph is changed, the paragraphs belonging to `` different categories '' are arranged vertically and placed in the `` same category '' The paragraphs to which they belong are arranged in the horizontal direction (see Fig. 4-4).

  In addition, when characters such as [FIG. 1] and [Photo 1] are present in a paragraph, the corresponding figure or photo is arranged next to the paragraph (see FIG. 4-2). Further, the display size of one page is changed in accordance with the screen size of the display unit 101 (see FIG. 4-3). Further, a boxed article is created (see Fig. 4-4).

Next, a series of processes of the layout automation device described above will be described with reference to the flowchart shown in FIG. 3 and FIGS. In the flowchart shown in FIG. 3, the document attribute determination / decomposition unit 2 first decomposes the target document into figures, tables, and texts, and the text is further converted into paragraphs (= sentences or paragraphs of sentences, short articles such as newspapers / magazines). (Step S1). Subsequently, the paragraph integration / division unit 3 integrates and divides the paragraphs so that there are, for example, about 10 on one page (step S2), and the paragraph hierarchy analysis unit 4 analyzes the hierarchy of the paragraphs (step S2). Step S3). Subsequently, the block arrangement unit 5 arranges the figures and tables included in the document near the citation unit (step S4), and arranges the blocks representing the paragraph hierarchy on one page (step S5). Subsequently, the column creation unit 6 arranges the text in the paragraph after integration and division in each block in a column (= enclosed article) configuration (step S6). Thereafter, the emphasis process for the first line in each block is executed (step S7).

  Furthermore, each processing content mentioned above is explained in full detail. In step S1, the target document is decomposed into figures, tables, and texts, and the text is further decomposed into paragraphs (= short articles such as sentence sections or paragraphs, newspapers and magazines). The target document is a so-called electronic file (a file created by word processing software, PDF (Portable Document Format: Adobe Corporation), HTML (Hyper Text Markup Language), XML (Extensible Markup Language Text), etc. In the case of a document that can be extracted without requiring a reader process, the document can be decomposed into a figure, a table, or a text by analyzing the document. By importing such a structural document from a server connected on the Internet as shown in FIG. 2, it is possible to easily input many image sources.

  Specifically, what is called a “tag” is often attached, for example, a document format with a tag such as <TEXT> in the text portion and <FIG> in the figure. On the other hand, if the target document is a so-called image file or a document printed on paper, it is decomposed into a figure, a table, and text by OCR processing. A great number of methods for the OCR processing have been proposed so far, and the method is not particularly limited.

  In order to simplify the system configuration, even in the case of a so-called electronic file, it is once converted into an image file format, and as with an image file or a document printed on paper, a chart, table, It may be broken down into text. In the case of electronic files, there are various file formats, and even if one file format is not compatible depending on the version, it is necessary to deal with many cases in order to perform analysis. This is because the system may become large. Conversion to the image file format can be performed using a printer driver.

  Next, break the text into paragraphs. In the case of Japanese, punctuation marks, paragraph lowering, blank lines, blank characters, etc. are decomposed into paragraphs as information. In this step, first, a fine decomposition is performed, and an appropriate number is integrated in a later step. Therefore, the paragraph is decomposed using all feature information that can be decomposed into paragraphs such as punctuation marks, paragraph lowering, blank lines, and blank characters. In this way, by dividing the region using the feature information of punctuation marks, paragraph lowering, blank lines, and blank characters, it is possible to break down into fine regions. It should be noted that in order to integrate paragraphs at a later stage, the feature information used for the paragraph decomposition is stored in a storage device such as the mass storage device 104. An example of processing is shown below.

Paragraph 1 My name is Taro Rikko. Punctuation paragraph 3 This time, it will be changed. Punctuation, blank line paragraph 4 Move to the Ebina office, and read ...

In step S2, the paragraphs are integrated and divided so that there are about 10 per page. The integration / division of paragraphs is performed using the result of disassembling the paragraphs performed in step S1. First, the number of paragraphs decomposed other than the punctuation is counted from the result of disassembling the paragraphs so that there are about 10 per page. In the example of step S1, paragraph 1 and paragraph 2 are calculated as one paragraph. If this number is within the range of about 10 paragraphs specified by the user in advance, the integration / division is not performed. The reason why reading marks are excluded here is that paragraphs are generally composed of sections and paragraphs of sentences as one unit, and from the viewpoint of readability, it is not suitable to lay out in column format for each reading point.

  If the number is less than the range specified by the user in advance, division by reading marks is considered. If the number of characters in the paragraph after division is greater than the number of characters specified by the user in advance, the paragraph is divided. As a result of the division, if the number of paragraphs reaches around 10, no further division is performed.

  On the other hand, if the number is greater than the range specified by the user in advance, consider integration. For the integration, information indicating which information was used for the paragraph decomposition in step S1 is utilized from information stored in the mass storage device 104, for example. More specifically, if the number of paragraphs is about 10, as a result of integration in the order of the paragraphs → paragraphs → blank lines, and the integration of the paragraphs, no further integration is performed. Similarly, when the number of paragraphs reaches about 10 as a result of integration in paragraphs, no further integration is performed.

  Step S3 analyzes the paragraph hierarchy. An example of this processing is shown in FIG. The analysis of the paragraph hierarchy is performed using the result of the integration / division of the paragraph executed in step S2. The morphological analysis is further performed on each paragraph integrated and divided, and the connection relation between the paragraphs is extracted. Specifically, it is determined whether or not the first morpheme of the morpheme analysis result of each paragraph is a predetermined conjunction. In this way, by analyzing the connection relation of the divided areas, the relation of the areas can be determined.

  As a specific example, predetermined conjunctions such as “also” “but” “collectively” “or” “next” “further” and the like are registered in advance. In addition, the relationship between the vertical arrangement and the horizontal arrangement is distinguished by the conjunction.

  In step S4, the figures and tables included in the document are arranged near the citation section. That is, here, as shown in FIG. 4B, “FIG. 1” is arranged next to the quoted part “when combined, as shown in FIG. 1”. In this way, after analyzing the region, it is possible to easily view the diagram, table, and photograph related to the region by arranging the figure, table, and photo included in the input image near the related region. it can.

  In step S5, a block expressing a paragraph hierarchy is arranged on one page. An example of this processing is shown in FIG. Here, as shown in FIG. 4C, the size of the block representing the paragraph hierarchy is adjusted so that the result of the analysis of the paragraph created in step S4 is received within one page. In other words, in order to adjust the size of the area to be arranged in accordance with the display size of the display device, the display area of the display device is used to the maximum, the area after the arrangement is displayed large, and an easy-to-view image is output. It will be.

  In step S6, as shown in the processing example of FIG. 4-4, each block expressing the paragraph hierarchy is arranged in a column (= enclosed article) text in the paragraph after integration / division. In this way, by providing the rearrangement means for rearranging the area into the form of the article, it becomes easy to identify each area. In addition, the connection relationship between each region can be analyzed by a conjunction.

  In step S7, the first line in each block representing the paragraph hierarchy is highlighted. There are various emphasis methods such as bolding, enlarging the font, changing the color, or underlining as shown in FIG. 4-5.

  Therefore, according to this embodiment, after extracting an area from the input image and dividing each extracted area into categories, the areas belonging to the same category as the areas belonging to different categories are arranged in different directions. Therefore, it is possible to decompose a large amount of information constituting the input image into regions and arrange them into categories, thereby facilitating screening by the reader.

  In the embodiments described so far, the input image has been described for horizontal writing or vertical writing. However, the present invention is not limited to this, and the present invention can be applied to a case where horizontal writing and vertical writing are mixed.

  By the way, the layout processing method (operation) in the embodiment described so far can be programmed, recorded on a computer-readable recording medium, and executed on the computer. Also, a part of the layout processing method can be provided on a network and realized through a communication line.

  That is, the layout processing method described in this embodiment is realized by executing a program prepared in advance on a computer (CPU 200) such as a personal computer or a workstation as shown in FIG. This program is read by a computer such as a memory 201, a hard disk 204, a flexible disk 207, a CD-ROM (Compact-Disc Read Only Memory) 206, an MO (Magneto Optical), a DVD (Digital Versatile Disc) by operating the keyboard. It is recorded on a possible recording medium, read from the recording medium by a computer (CPU 200), and displayed on a display device as necessary. Further, the data of this layout processing method can be transmitted and received from the communication device to the external device as necessary.

  The program can be distributed to a device such as a personal computer via the recording medium via a network such as the Internet.

  That is, as shown in FIG. 6, this program can be provided in a state of being installed in advance on a hard disk as a recording medium built in the computer, for example. The program can be temporarily or permanently stored in a recording medium, and can be provided as packaged software by being incorporated in a computer as a unit or being used as a removable recording medium.

  As the recording medium, for example, a flexible disk, a CD-ROM, an MO disk, a DVD, a magnetic disk, a semiconductor memory, and the like can be used.

  The program is transferred from a download site to a computer in a wired or wireless manner via a network 210 such as a LAN (Local Area Network) or the Internet, and is downloaded to a storage device such as a built-in hard disk in the computers 211 to 213. can do.

  As described above, the layout processing device, the layout processing method, the program, and the recording medium according to the present invention are useful for a layout that makes it easy to understand the background and main points of the entire document in newspapers and business documents. In particular, it is suitable for a system or method for automating column layout in newspapers.

It is a block diagram which shows the structure of the layout automation apparatus concerning embodiment of this invention. It is a block diagram which shows the system configuration | structure containing the layout automation apparatus shown in FIG. It is a flowchart which shows the processing operation of the layout automation apparatus concerning embodiment of this invention. It is explanatory drawing which shows the process example of step S3 in FIG. It is explanatory drawing which shows the process example of step S4 in FIG. It is explanatory drawing which shows the process example of step S5 in FIG. It is explanatory drawing which shows the process example of step S6 in FIG. It is explanatory drawing which shows the process example of step S7 in FIG. It is a block diagram which shows the example which makes a computer perform the layout automation method concerning embodiment of this invention. It is a block diagram which shows the example which downloads and performs the layout automation method concerning embodiment of this invention from a network.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Control part 2 Document attribute determination / decomposition | disassembly part 3 Paragraph integration / division | segmentation part 4 Paragraph hierarchy analysis part 5 Block arrangement part 6 Column creation part 100 Layout automation apparatus 101 Display part 102 Operation part 103 Scanner 104 Mass storage device 105 Network I / F
106 System control unit 107 Network

Claims (18)

Image extracting means for extracting a region from the input image;
Category dividing means for dividing each area into categories according to the beginning of the area extracted by the image extracting means,
Area arrangement means for arranging areas belonging to the same category as areas belonging to different categories in different directions;
A layout processing apparatus comprising:   The layout processing apparatus according to claim 1, further comprising a region analysis unit that analyzes a connection relationship between the regions divided by the category division unit.   The layout processing according to claim 2, further comprising an arrangement unit that arranges a diagram, a table, and a photograph included in the input image next to a related region after the region is analyzed by the region analysis unit. apparatus.   The layout processing apparatus according to claim 2, wherein the arrangement unit arranges the area in a form of an enclosed article.   The layout processing apparatus according to claim 2, wherein the arrangement unit adjusts a size of the area in accordance with a display size of the display apparatus.   The layout processing apparatus according to claim 1, further comprising receiving means for receiving the input image from an image reading apparatus or a server on a network.   The layout processing apparatus according to claim 1, wherein the image extraction unit decomposes a region using feature information of punctuation marks, paragraph lowering, blank lines, and blank characters.   The layout processing apparatus according to claim 2, wherein the area analysis unit determines a connection relation between the areas based on a conjunction. An image extraction step of extracting a region from the input image;
A category dividing step of dividing each region into categories according to the beginning of the region extracted in the image extracting step;
A region arrangement step of arranging regions belonging to the same category as regions belonging to different categories in different directions;
A layout processing method comprising:   The layout processing method according to claim 9, further comprising a region analysis step of analyzing a connection relationship between the regions divided in the category division step.   The layout processing according to claim 10, further comprising an arrangement step of arranging a figure, a table, and a photograph included in the input image next to a related region after analyzing the region in the region analysis step. Method.   The layout processing method according to claim 9, wherein the arranging step arranges the area in a form of an enclosed article.   13. The layout processing method according to claim 10, 11 or 12, wherein a size of the area is adjusted in accordance with a display size of the display device.   The layout processing method according to claim 9, further comprising a receiving step of receiving the input image from an image reading device or a server on a network.   The layout processing method according to claim 9, wherein the image extracting step performs region decomposition using feature information of punctuation marks, paragraph lowering, blank lines, and blank characters.   The layout processing method according to claim 10, wherein the region analysis step determines a connection relation between the regions based on a conjunction.   A program for causing a computer to implement the layout processing method according to any one of claims 9 to 16.   A computer-readable recording medium on which the program according to claim 17 is recorded.

Images