JP2013152564A - Document processor and document processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform emphasis expression for recognizing a correspondence between an explanatory text and a diagram in a document, and to perform emphasis display for, when there are two or more corresponding contents, individually identifying the correspondence.SOLUTION: When a link function of an object is achieved so as to easily come and go between an "object" and an "explanatory text of the object", an operation component of the link function is arranged at parts of the "object" and the "explanatory text of the object" as an addition object of the link function, and a function that easily comes and goes is added. In this case, when it is detected that the "explanatory text of the object" is a specific portion in the "object", corresponding expression information to be cooperatively displayed is generated and added to the "explanatory text of the object" and the "object" so that their correspondence can be understood by a reader of a document.

Description

  The present invention relates to a document processing apparatus and a document processing method, and more particularly to a technique suitable for use in processing electronic document data.

  Conventionally, a paper document or an electronic document containing “objects” (for example, photographs, drawings, line drawings, tables, etc.) and “descriptions of objects” (texts in the text for explaining or explaining the objects) in the document. Is widely used. For example, documents including explanatory texts such as academic papers, patents, instruction manuals, and product catalogs.

  "Object description" explains and explains the above-mentioned "object" in the main text, which is the main sentence (in practice, the object is often a graphical representation of the contents of the sentence) It is. The creator of the document combines these “object” and “description of the object” with the intention of being used by the viewer of the document.

  For this reason, an expression such as “FIG. 15” is often used to relate them. As shown in “FIG. 15”, an object for associating each of “object” and “object description” is called “anchor expression”. In addition, there are many explanatory texts explaining the “object” in the vicinity of the “object” itself, and this is called “caption expression”, which often includes the “anchor expression” together.

  An example of this case is shown in FIG. In FIG. 15, 1501 is the first page of the document, 1502 is the second page, 1511 is the body of the document, and 1512 is an anchor expression included in the explanatory text in the document. 1513 is a table that is an object in the document, 1514 is a caption expression of the table object, and 1515 is an anchor expression in the caption expression.

  The document viewer needs to read in consideration of the mutual relationship between the “object” and the “object description”. Therefore, when the viewer sees the text “Table.1 is ...” in the text, the viewer searches for the object corresponding to “Table.1” from the document, and then confirms the original position of the text again. Return to and start reading more.

  On the other hand, when the object having the anchor expression “Table.1” is seen in the text, the text explaining “Table.1” is searched from the text. Then, after reading and confirming the explanatory text, the user returns to the original page again and continues reading. In the case of a multi-page document, when searching for an object corresponding to “Table.1 is ...” in the text or searching for an explanatory text in the text corresponding to the object indicated by “Table.1”, You will need to browse across pages. For this reason, there was a problem that readability was not good. In addition, it is difficult for the viewer to check all the explanations in the text, such as being difficult to find and being written in a plurality of places.

  Therefore, the “document” “diagram” and the “anchor expression” “diagram number” are converted into hypertext to generate an electronic document. Thus, for example, when a “diagram number” in the text is clicked with a mouse or the like, it is possible to have a function of displaying a diagram corresponding to the “diagram number” on the screen. In addition, it is conceivable to add this function by analyzing a document in an electronic document obtained by optically reading a paper document.

  In Patent Document 1, an “object description” having the anchor expression is extracted, and an “anchor expression” of “object” such as a figure is extracted and related. This makes it possible to search for “object” using “object description” as metadata for search.

JP 2000-331056 A

  Therefore, using this, a function (hereinafter referred to as an “object link function”) that enables an operation to easily go back and forth between these is added to an electronic document and an explanatory text of the document. This makes it possible to easily read and check the contents of the text and the figure even in a document using the figure.

  In other words, the information on the relationship between the “description of the object” or “anchor expression” and “object” and the appearance position thereof is extracted, and the link function of this object is created. 1521 and 1522 in FIG. 15 are obtained when this is added. An operation function 1521 is added to “anchor expression” 1512 in the “description in the text”.

  When this is clicked with a mouse or the like, the display part moves to 1502 on the second page, and the figure 1513 is highlighted, indicating that the part is the corresponding part. Similarly, when an operation function 1522 added to “object” 1513 is clicked on with a mouse or the like, the display portion moves to 1501 on the first page. Then, the “anchor expression” of 1512 and the “object description” including the same are highlighted to indicate that the portion is the corresponding portion.

  However, the description content of the “object description” often explains a specific part such as a part rather than the whole “object”. In other words, there are characteristic parts in the figure and some parts in the table that you want to pay attention to, and there are many explanations for these parts.

  Therefore, even when the hypertext is converted into an electronic document as described above, an operation for searching for a portion indicated by “description of object” in the moved “object” occurs. In addition, when the “object description” is a sentence that individually explains a plurality of parts in the “object”, the “object description” is repeatedly displayed between the “object” display and the “object description”. Go to read the document.

At that time, there is a possibility that the operation of searching for the portion indicated by the “object description” will occur many times. In particular, when the “object” is a table or the like, this trend is remarkable because the portion other than the portion of interest is described.
Such a function is generally created by a user who creates a document by using a document editing apparatus or the like to create a highlight display itself.

As in Patent Document 1, when there are clear article formats and rules relating to the contents described in drawings and the like, there are some that automatically add emphasis expression. However, in a normal document, etc., the correspondence between the explanation in the document and the corresponding part in the diagram can be shown, and when there are multiple explanations, the difference between the relevant parts in the diagram can be clarified There was nothing that added such emphasis.
SUMMARY OF THE INVENTION In view of the above-described problems, an object of the present invention is to make it possible to perform an emphasis expression that clearly shows the correspondence between explanatory texts in a document and those in a diagram.

  The document processing apparatus according to the present invention is a document processing apparatus that creates a multi-page electronic document by creating a mutual link between an object and a description of the object for each page of a multi-page electronic document including image data. Area dividing means for dividing the image data to obtain divided areas, attribute information adding means for determining attributes of the divided areas obtained by the area dividing means, and adding character attributes for each area; and Check the correspondence between the character recognition means for recognizing characters in the area to which the character attribute is added by the information addition means, the anchor expression attached to the object, and the anchor expression or the object description in the body of the electronic document. Link information for maintaining the correspondence between the object and the anchor expression in the text or the description of the object. Corresponding to the corresponding part of the text information in the object and the description of the object in the body, which is associated by the link information generation processing means to be formed, the link information generated by the link information generation processing means Correspondence expression adding means for generating and adding correspondence expression information for emphasizing expression, and format conversion means for converting into an electronic document including the link information and correspondence expression information for emphasizing the correspondence. .

  According to the present invention, when a mutual link is used, it is possible to show the corresponding part intuitively and easily, and it is possible to save the trouble of searching for the corresponding part and improve the readability of the document. It becomes.

1 is a block diagram illustrating an example of an image processing system according to the present invention. FIG. 2 is a block diagram illustrating a detailed configuration example of an MFP in FIG. 1. It is a block diagram which shows the structural example of the data processing part of FIG. It is a figure which shows the example of the image data used for description by embodiment of this invention. FIG. 5 is a diagram in which an extension area in FIG. 4 can be designated. It is a figure which shows the example of the electronic document data used by this embodiment. (A) is a detailed diagram of the link processing unit, (b) is a detailed diagram of the corresponding expression adding unit. It is a flowchart which shows the procedure of the whole process performed by embodiment of this invention. It is a flowchart which shows the process which produces | generates and stores the information of a corresponding expression. It is a figure which shows an example of the extracted link information and the extracted corresponding | compatible expression. It is a figure of the state which added the corresponding expression and the link function became usable. It is a figure which shows the content divided into the area | region for description and the screen which is producing the document. It is a figure which shows the example of the screen of the state which is displaying the complement candidate which estimated the input. It is a flowchart which shows the process sequence in 2nd Embodiment. It is a figure which shows the background art of this invention, and shows the document with a link function.

(First embodiment)
DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, the best mode for carrying out the invention will be described with reference to the drawings.
FIG. 1 is a block diagram illustrating a configuration example of a document processing system according to the present embodiment.
In FIG. 1, an MFP (Multi Function Peripheral) 100 that is a multifunction machine that realizes a plurality of types of functions (copying function, printing function, transmission function, etc.) is connected to a LAN 102 constructed in the office A.

  The LAN 102 is also connected to an external network 104 via a proxy server 103. The client PC 101 receives transmission data from the MFP 100 via the LAN 102 and uses functions of the MFP 100. For example, the client PC 101 can also print the printed matter based on the print data by the MFP 100 by transmitting the print data to the MFP 100. The configuration in FIG. 1 is an example, and a plurality of offices having the same components as the office A may be connected on the network 104.

  The network 104 is a communication network typically realized by the Internet, LAN, WAN, telephone line, dedicated digital line, ATM, frame relay line, communication satellite line, cable TV line, data broadcasting radio line, and the like. is there. This may be anything as long as it can transmit and receive data. Each of the various terminals of the client PC 101 and the proxy server 103 has standard components mounted on a general-purpose computer. For example, a CPU, RAM, ROM, hard disk, external storage device, network interface, display, keyboard, mouse, and the like.

  FIG. 2 is a block diagram illustrating a detailed configuration of the MFP 100 which is the image processing apparatus according to the present embodiment. The MFP 100 illustrated in FIG. 2 includes a scanner unit 201 that is an image input device, a printer unit 202 that is an image output device, a control unit 204 that includes a CPU 205 and the like, an operation unit 203 that is a user interface, and the like.

  The control unit 204 is connected to the scanner unit 201, the printer unit 202, and the operation unit 203. On the other hand, the control unit 204 is connected to a LAN 219 or a public line (WAN) 220, which is a general telephone line network, so that image information and device information can be stored. A controller that performs input and output. The CPU 205 controls each unit included in the control unit 204. A RAM 206 is a system work memory for the CPU 205 to operate, and is also an image memory for temporarily storing image data.

  A ROM 210 is a boot ROM, and stores programs such as a system boot program. A storage unit 211 is a hard disk drive and stores system control software and image data. An operation unit I / F 207 is an interface unit with the operation unit (UI) 203 and outputs image data to be displayed on the operation unit 203 to the operation unit 203. An operation unit I / F 207 serves to transmit information input by the user of the image processing apparatus from the operation unit 203 to the CPU 205.

  A network I / F 208 connects the image processing apparatus to the LAN 219 and inputs / outputs packet format information. A modem 209 connects the image processing apparatus to the WAN 220 and inputs / outputs information by demodulating / modulating data. The above devices are arranged on the system bus 221.

  An image bus I / F 212 is a bus bridge that connects the system bus 221 and an image bus 222 that transfers image data at high speed, and converts the data structure. The image bus 222 is composed of, for example, a PCI bus or IEEE1394. The following devices are arranged on the image bus 222. A raster image processor (RIP) 213 realizes a so-called rendering process in which a PDL (page description language) code is analyzed and developed into a bitmap image having a designated resolution.

  When this expansion is performed, attribute information is added in units of pixels or regions. This is called image area determination processing. By the image area determination process, attribute information indicating an object type such as a character (text), a line (line), graphics, an image, or the like is given for each pixel or for each area. For example, an image area signal is output from the RIP 213 according to the object type of the PDL description in the PDL code, and attribute information corresponding to the attribute indicated by the signal value is stored in association with the pixel or area corresponding to the object. Is done. Therefore, associated attribute information is attached to the image data.

  The device I / F 214 connects the scanner unit 201 which is an image input device via a signal line 223 and the printer unit 202 which is an image output device via a signal line 224 to the control unit 204, respectively, and synchronizes the image data. / Perform asynchronous system conversion. A scanner image processing unit 215 corrects, processes, and edits input image data. The printer image processing unit 216 performs correction, resolution conversion, and the like according to the printer unit 202 for print output image data to be output to the printer unit 202. The image rotation unit 217 rotates and outputs the input image data so that it is upright. The data processing unit 218 will be described later.

  Next, details of the data processing unit 218 in FIG. 2 will be described with reference to FIG. The data processing unit 218 includes an area dividing unit 301, an attribute information adding unit 302, a character recognition unit 303, a link processing unit 304, a corresponding expression adding unit 305, and a format conversion unit 306. When the image data 300 scanned by the scanner unit 201 is input, the data processing unit 218 generates and outputs electronic document data 310 by performing processing in each of the processing units 301 to 306.

  Image data scanned by the scanner unit 201 in FIG. 2 or image data (document image) input in the past stored in the storage unit 211 is input to the region dividing unit 301. Then, in order to extract a region (object region) of an object arranged in the page, such as a character, a photo, a chart, an illustration, etc., processing such as extraction / grouping of pixels in the data is performed.

As a region extraction method (region division method) performed by the region dividing unit 301, a known method may be used.
As an example, first, an input image is binarized to generate a binary image, and the resolution of the binary image is reduced to create a thinned image (reduced image). For example, when a 1 / (M × N) thinned image is created, a binary image is divided into M × N pixels, and if there are black pixels in the M × N pixels, the corresponding pixels after reduction are reduced. Is a black pixel, and if it does not exist, it is a white pixel to create a thinned image.

  Next, a portion (connected black pixel) where black pixels are connected in the thinned image is extracted, and a rectangle circumscribing the connected black pixels is created. When the rectangles close to the character image size (one character rectangle) are lined up, or the rectangles that are close to the character image size either vertically or horizontally (connected black pixel rectangles with several characters connected) When the rectangles are arranged, it is highly possible that the image is a character image constituting one character line. In this case, the rectangles are combined to obtain a rectangle representing one character line. A set of rectangles having the same length of the short sides of the rectangle representing one character line and arranged at almost equal intervals in the column direction is likely to be a body part. Extract. In addition, the photograph area, the figure area, and the front area are extracted by the connected black pixels having a size larger than that of the character image.

The attribute information adding unit 302 adds an attribute for each divided region divided by the region dividing unit 301. The processing operation will be described as an example of the input image data in FIG. For ease of explanation, FIG. 5A is a diagram in which the extension area in FIG. 4 can be designated.
The area 506 in FIG. 5A has a certain number of characters and lines in the page, and has a character number, a line number, a paragraph, and the like, and comprehensively determined. An attribute will be added. For the remaining area, first, it is determined whether or not the area includes a rectangle close to the character image size.

  In particular, since the character image rectangle periodically appears in the region including the character image, it can be determined whether or not the character region is included in the divided region. As a result, the region 501 511 to 518 in the table 502, the region 507, the region 504 in the region 503, and the region 505 add the attribute “text region” as a region including characters. However, here, since there is no form such as the number of characters, the number of lines, and paragraphs, the “text” attribute is not added.

  On the other hand, other areas are determined as “noise” if the area size is very small. For connected black pixels with low pixel density, when tracking white pixel contours inside the connected black pixels, if the circumscribed rectangles of the white pixel contours are arranged in an orderly manner, the area is judged to be a “table” and arranged in an orderly manner. If not, it is determined as “line drawing”. The attribute of “photograph” is added because the other pixel density is a picture or a photograph. As a result, it is determined that the area 502 is “table” and the area 503 is “photo”.

  Further, for the table area, the attributes in the table are added to each text area from the positional relationship of the text area and the regularity of the arrangement. Thus, “column label” is added to the text areas 512 and 514, “row label” is added to the text areas 511 and 515, and “value” is added to the text areas 513, 516, 517, and 518.

  Furthermore, if a character area that is determined not to be a text exists in the vicinity (above or below the area) to which “table”, “line drawing”, or “photograph” is added, the “table”, “line drawing” ] And “photograph” areas are determined to be character areas, and a “caption” attribute is added. The area to which “caption” is added is stored in association with the associated area so that the area (“table”, “line drawing”, “photograph”) associated with the “caption” can be specified. Thus, the character area 507 is stored in the table 502 and the character area 505 is stored in association with the photograph area 503 as captions.

  If the text area is larger than the character image of the body part and is at a position different from the column of the body part, the “heading” attribute is added. If the text image is larger than the text image of the text part and exists above the column of the text part, the attribute is set to “subheading”. Further, if the character image is smaller than the size of the character image in the body portion and exists at the lower end portion or the upper end portion of the document, the attribute of “page” (or “page header” or “page footer”) is added. In addition, although it is determined as a text area, if it does not apply to any of “body”, “heading”, “subheading”, “caption”, and “page”, the attribute of “text” remains.

  The character recognizing unit 303 executes a known character recognition process on a region including a character image (“text”, “body”, “heading”, “subheading”, “caption” region), and a character code resulting therefrom The column is stored as character information and associated with the target area.

  As described above, the region position and size, region attribute information, page information, and character recognition result character information (character code information) extracted by the region dividing unit 301, attribute information adding unit 302, and character recognition unit 303. Are stored in the storage unit 211 of FIG.

  FIG. 5B shows an example of information stored in the storage unit 211 of FIG. 2 when the input image data example of FIG. 4 is processed. For 504 in FIG. 5, since this is a region of a character image of a photo or a figure, an attribute “in the photo of 503” is added, as shown by 504 in FIG. 5B. . As for the table 502, which is the table, information stored in the storage unit 211 of FIG. 2 as the internal structure information of the table is shown in FIG.

  The link processing unit 304 creates a link between the caption associated objects (for example, “table”, “line drawing”, “photo”) detected by the attribute information adding unit 302 and “descriptive expression in the text”. Necessary information is generated and stored in the storage unit 211 of FIG. Details of the link processing unit 304 will be described later.

  The correspondence expression adding unit 305, when performing the operation of this part based on the link information created by the link processing unit 304, performs the operation for the text part and the chart part associated with the link. Perform processing to add expressions. The text to be explained using the chart and the expression method that emphasizes the correspondence between the area and part in the chart to be noted in the text are determined, and the link information is displayed as the contents displayed when selecting or operating the link. Correspondence expression information is generated so as to be related to and stored in the storage unit 211 of FIG. Details of the correspondence expression adding unit 305 will be described later.

  The format conversion unit 306 receives information (for example, page information, region position and size) obtained from the input image data 300, the region division unit 301, the attribute information addition unit 302, the character recognition unit 303, and the link processing unit 304. , Attributes, character information, metadata). Then, these are converted into a predetermined electronic document format (for example, PDF, SVG, XPS, OfficeOpenXML, etc.). The electronic document generated by the format conversion includes page display information (such as a display image) using graphics and the like, and content information (metadata and the like) based on a semantic description such as characters.

  There are roughly two processes of the format conversion unit 306. One is a portion corresponding to an area to which image data (for example, a “line drawing” attribute is given by performing filter processing such as flattening, smoothing, edge enhancement, color quantization, and binarization on the image area. Image) can be stored in a predetermined electronic document format.

  Actually, the image data is converted to graphics data (vector data) of vector path description or graphics data (for example, JPEG data) of bitmap description. A known vectorization technique can be used as the technique for converting to vector data. Further, for the “character” area portion, binary image cutout processing, pixel erasure processing from the image data 300, and the like are performed, and character drawing graphics data using a character code as a result of character recognition is obtained. create.

  In particular, for special fonts, character decorations, and “character” area portions of large characters that require the accuracy of character images, graphics data in vector path description or bitmap description is created in the same manner as image data. Also, when searching for an object, a graphics description (vector path description) such as a frame displayed when a link is used / operated or when a search result is specified / emphasized is generated. Then, an electronic document having a predetermined electronic document format is created from the area information (position, size, attribute) stored in the storage unit 211 of FIG. 2 and the character information in the area.

  An example of the electronic document data 310 generated in this way is shown in FIG. 6 shows an example of SVG (Scalable Vector Graphics) based on the data shown in FIGS. 5A and 5B stored in the storage unit 211 of FIG. 2 when the example of the image data 400 of FIG. 4 is processed. This is an example when the description is in the format.

In the electronic document data 600 of FIG. 6, descriptions 601 to 606 are graphics descriptions for the areas 501 to 506 of FIG. Here, reference numerals 601, 604, 605, and 606 are examples of character drawing descriptions using character codes.
Reference numeral 602 corresponds to the table 502 and describes the table. Among them, there are examples of character drawing descriptions by character codes in the text areas from 511 to 518 (actually, those other than 511 to 513 are omitted). Reference numeral 603 denotes an example of a description for pasting a cut-out photographic image. Since these are descriptions within a page, 611 and 612 can be described separately for each page. Reference numeral 613 describes the contents of another page.

  In the electronic document of this embodiment, an operation function or the like can be added in a simple functional programming language, and 621 and 622 are examples of the description. 621 is a part that provides a link function such as the operation function 1521 of FIG. 15, for example, and describes the display / operation when an item whose identification information is “Link1FromSensence1ToTable1” is selected by a user operation or the like. Part.

  Individually, it is shown that a program module (function) named function “DisplayFromSensence1ToTable1” is called and processed. In 622, the processing performed in the function “DisplayFromSentence1ToTable1” is described in the above-described functional programming language (the contents are abbreviated). In this way, the link function is realized by executing a program described in these functional programming languages when using the electronic document.

  Although SVG has been described as an example here, the output format is not limited to SVG, and may be converted to PDF, XPS, Office Open XML, other PDL data formats, and the like.

FIG. 7A is a block diagram illustrating a configuration example of the link processing unit 304 in FIG. 7
The functions 01 to 705 will be described. The processing performed here is the image data 710 created by the character recognition unit 303 from the region division unit 301 in FIG. 3 and stored in the storage unit 211 in FIG. 2, the text contents and charts shown in FIG. This is performed using area information 711 and character information 712.

The link information addition target selection processing unit 701 selects a caption-associated object and its text content as targets for link information generation for the input image data.
The anchor expression extraction processing unit 702 analyzes the character information of the caption area associated with the caption associated object selected by the link information addition target selection processing unit 701, and extracts the anchor expression.

  Here, the character information in the caption area is analyzed, and an anchor expression (for example, “FIG. 1”, “Table. 1”, etc.) is searched from the text information. The part other than is extracted as a caption expression. In addition, by including character code characteristics, a dictionary, and the like, it has a function of eliminating insignificant character strings (insignificant symbol strings, etc.).

  This is to cope with misrecognition of character recognition that interprets decorations, dividing lines, and images that appear at the boundaries of the text portion of the document as characters. In addition, in order to extract anchor expressions, possessing multilingual character string patterns such as figure numbers and misrecognition patterns of character recognition for them, perform anchor expression extraction accuracy and anchor expression character correction It is possible. The same applies to caption expression. In other words, it is possible to perform analysis in natural language processing, correct misrecognition correction of character recognition, etc., and correct and eliminate the boundary with anchor expression, symbols and character decorations that appear at the beginning and end It is also possible to have

  The in-text anchor expression search processing unit 703 searches for the anchor expression (for example, “FIG. 1”, “Table. 1”, etc.) extracted by the anchor expression extraction processing of the anchor expression extraction processing unit 702 from the text of the document. . And the process which detects as an anchor expression in the text corresponding to an object is performed. As a result, the explanation expression in the text including the anchor expression and explaining the object is detected as a candidate explanation expression of the object.

  Here, it is possible to create a search index for speeding up the search (index creation and high-speed search technology using the same can use known index creation / search technology). . In addition, it is possible to increase the speed by performing a batch search with a specific character string of a plurality of anchor expressions, which is more effective. In addition, for explanatory expressions in the text, not only that, but also possesses multilingual character string patterns such as figure numbers, distorted character patterns such as figure numbers, and misrecognition patterns for character recognition for them. . By performing an ambiguous search using these, it is possible to provide a function for improving and correcting the search accuracy.

  The link information generation processing unit 704 includes a caption-associated object selected by the link information addition target selection processing unit 701, an anchor expression in the text searched for and extracted by the anchor expression search processing unit 703 in the text, and a figure or a photo. Create the information needed to generate the link. The link information requires the position of the link itself and the relationship between them. In the present invention, the position information of the character for obtaining the position information of the anchor expression is not stored as it is, but is grouped in units such as paragraphs, that is, the circumscribed rectangle of the character in it is obtained, and only the position information is obtained. Accumulate.

  The link processing control unit 705 creates a description in a programming language of a process for actually generating a link together with other information, and stores it in the electronic document. When the electronic document is used, the process described in the programming language dynamically generates a link function based on the information, and the link function can be used by the user. Details of these will be described later. Here, information for creating a link between a figure or photo and an anchor expression described in the text is generated.

   The link processing control unit 705 performs processing for generating and storing information necessary for link generation by the link information generation processing unit 704. When accumulating here, the processing described in the programming language for link generation and the information such as graphics necessary for link generation are generated according to the description format of the electronic document and the specification of the programming language. Accumulate results. Here, as necessary, it is generated and stored based on a predetermined link trigger or link action setting.

In this way, link information is generated, and the result is stored as link information 713 in the storage unit 211 of FIG. The link processing control unit 705 provides a function for linking and controlling these functions 701 to 704.
Note that the operations of the processing units 701 to 705 in FIG. 7A, which are each part of the link processing unit 304, will be described again in the description of the processing procedure described later.

  FIG. 7B is a block diagram illustrating a configuration example of the correspondence expression adding unit 305 in FIG. Hereinafter, functions of 721 to 725 will be described. The processing performed here is image data 710 created from 301 to 304 in FIG. 3 and stored in the storage unit 211 in FIG. 2, region information 711 and character information relating to the text and chart shown in FIG. 712, which is performed using the link information 713.

  The in-text anchor neighborhood text extraction unit 721 extracts the link information 713 and obtains the location of the anchor expression in the text and the corresponding caption such as a chart from the link between the anchor expressions in the link information 713. From now on, a process of extracting text information in the vicinity of the anchor expression from the text including the anchor expression is performed.

  Here, the text information to be extracted is about sentences or paragraphs that contain anchor expressions in the body, and it is appropriate based on the length of the text, the number of characters, the content of reference expressions in other sentences such as pronouns, etc. Determine the length of text and the number of characters and perform extraction based on that. On the other hand, when the sentence is too long, nearby text may be cut out simply according to the above-mentioned standard. In this way, the function of extracting text information in the vicinity of the anchor expression from the text including the anchor expression is provided. The extracted text will be referred to as anchor text in the body text.

  The text in text / chart text search unit 722 has the same word string as the word or character string in the text in the vicinity of anchor in the text extracted by the text in the vicinity of anchor in text body 721 and the text in the diagram corresponding to the link. Search for items that contain or a string. As a search method, there can be considered a method of performing a search by obtaining a word obtained by comparing words in units or extracting words by morphological analysis in natural language processing.

  Further, since the text in a chart is often an item in the chart, it can be easily predicted that there are more words, short sentences, etc. than sentences and paragraphs. Therefore, when performing search / comparison, it may be more effective to use this as a key for search / comparison. Any method may be used in the embodiment of the present invention. In any case, here, the function of extracting the same text expression appearing in the text in the corresponding chart by the link is provided. As a result, the same text expression is extracted from the word or character string in the text in the vicinity of the anchor in the body and the text in the diagram corresponding to the link. This is called a correspondence expression.

  The correspondence expression content determination unit 723 determines, based on the correspondence expression obtained by the text-in-text / graph-in-text search unit 722, what kind of emphasis expression should be performed on these. I do. There may be a plurality of links for one chart, and there may be a plurality of corresponding expressions for one link information. These correspondence expressions may include text expressions having the same contents. Here, based on this relationship, the user selects the emphasized expression for each link so that the difference between the links and the corresponding expression in the text and chart are clear. .

  Basically, when a link is displayed and used, the same emphasis expression pattern is used for corresponding expressions in the link. In addition, when there are the same corresponding expressions in a plurality of links to the same diagram, the same emphasized expression is used as much as possible. Also, the correspondence expression is regarded as an attribute of the correspondence expression according to the label (name) of the row or column in the table, the proximity of the appearance location in the figure, etc., and the same emphasized expression pattern is used. Like that.

  Here, the emphasis pattern is from underlining and enclosing characters, changing the color and font, and making the characters thicker, blinking, shaking, and changing the color over time. Such a moving expression is also acceptable. Of course, these combinations are also included. And as for each emphasis expression, there are various settings and values even in the same emphasis pattern such as underline, surrounding, character thickness, color scheme, etc. in each of these patterns. It is possible to change. In this way, an emphasized expression for the corresponding expression of each link is selected. This provides a function for determining an emphasized expression that can easily indicate to the user the corresponding expression in the text, the appearance position of the expression in the chart, and the corresponding situation when displaying and using the link.

  Corresponding expression addition processing unit 724 provides a function of performing and storing emphasized expression information for the corresponding expression determined by corresponding expression content determining unit 723 so as to correspond to individual link information. When accumulating here, the processing described in the programming language for link generation and the information such as graphics necessary for link generation are generated according to the description format of the electronic document and the specification of the programming language. Accumulate results. Here, as necessary, it is generated and stored based on a predetermined link trigger or link action setting.

In this way, the correspondence expression information is generated, and the result is stored as the correspondence expression information 714 in the storage unit 211 of FIG. The corresponding expression addition control unit 725 provides a function for linking and controlling these functions 721 to 724.
Note that the operations of the units 721 to 725 in FIG. 7B, which are the components of the correspondence expression adding unit 305, will be described again in the description of the processing procedure described later.
Finally, the data generated here is output to the format conversion unit 306 in FIG. 3 and stored in the electronic document data 310.

  Next, an overview of the entire process executed by the document processing system of the embodiment will be described with reference to the flowchart of FIG. 8 and 9 is executed by the data processing unit 218 in FIG. 2 (each processing unit 301 to 306 in FIG. 3). In the present embodiment, the CPU 205 in FIG. 2 reads and executes a computer program stored in the storage unit 211 (computer-readable storage medium), whereby the data processing unit 218 (each processing unit 301 to 306 in FIG. 3). It shall function. However, the present invention is not limited to this. For example, the data processing unit 218 in FIG. 2 (each processing unit 301 to 306 in FIG. 3) may be realized by hardware such as an electronic circuit.

  FIG. 8 is a flowchart of processing for converting a plurality of pages of image data input by MFP 100 of FIG. 1 into electronic document data consisting of a plurality of pages. For example, the page image of FIG. 4 is input as the image data of a plurality of pages. Hereinafter, each description of the flowchart of FIG. 8 will be given.

  In step S801, the link processing control unit 705 in FIG. 7A performs initialization processing for subsequent processing and preparation processing for enabling the format conversion unit 306 in FIG. 3 to create an electronic document. Do.

  In S802, since the page image of FIG. 4 is stored in the storage unit 211 of FIG. 2, processing is performed in which the pages to be processed are selected in order from the top of the page. By performing this and using the conditional branch in S807 to form a loop process, the processes from S802 to S806 can be performed on each page. Therefore, when this process is first performed, the first page is selected, and each time this process is performed, the next page is selected, and the image data (300 in FIG. 3) in the selected page is selected. 710 in FIG. 7 is a target of subsequent processing.

  In step S803, the region division unit 301 performs region division that extracts a region from the image data for one page selected in step S802. For example, each area shown in FIG. 5A is extracted from the image data 400 of FIG. Each area is an object, which is part of a component in the document, and an object ID is used for identification.

In step S804, the attribute information adding unit 302 adds an attribute to each area divided in step S803. In the example of FIG. 5A, an attribute such as “photo” is added to the area 503, and “caption” is added to the area 505.
Further, information that the accompanying area is 503 is also added to the caption 505. FIG. 5B shows a part of the added result. Among these, for each object ID in FIG. 5A, the values of “attribute” and “object ID with caption” in FIG. 5B are the results of this processing.

  Furthermore, elements such as elements are also regarded as objects and information such as attributes is added to the chart. FIG. 5C shows a part of the result performed on the table 502. In this, information such as attributes from 512 to 518, which are text areas in the table, is shown (however, the information from 514 to 518 is omitted because it is the same). Here, a value of “row position”, a value of “column position” are added as positions in the table, and “row label”, “column label”, and “value” are added as attributes.

  Note that the coordinates X, coordinates Y, width W, and height H indicate the position and size of the object area. These are used when a computer program or a graphics object described in a programming language is created from the link information and the corresponding expression information in S812 and stored in the electronic document data 310. Specifically, it is used when determining the position and size of display parts and graphics objects created by the computer program.

  In step S805, the character recognition unit 303 performs character recognition processing on the region to which the text (text, caption, heading, subheading, etc.) attribute is added in step S804, and stores the result in association with the target region as character information. To do. This result is described in the text information of FIG.

  In S806, the information of FIG. 5B and FIG. 5C created by S803 to S805, that is, the area dividing unit 301, the attribute information adding unit 302, and the character recognizing unit 303 is stored in the storage unit 211 of FIG. Accumulation is performed as information for one page.

  In step S807, it is confirmed whether or not the processing in steps S803 to S806 has been performed on all pages. If the pages remain, the process returns to step S802 to select and process the remaining pages. When all pages are finished, the process proceeds to S808.

  In step S808, in order to extract and accumulate information necessary for link generation, the link processing unit 304 first selects a caption associated object and its text information in the link information addition target selection processing unit 701 in FIG. select. Next, the anchor expression extraction processing unit 702 extracts anchor expressions from them. This is done for all caption-associated objects on all pages. Actually, there is a case where there is no anchor expression, and therefore, the one for which the anchor expression is obtained becomes a processing target after this.

  The anchor expression is character information (character string) for identifying the object accompanied by the caption in the original document, and the caption expression is character information (character string) for explaining the object. In the caption associated with the object, only the anchor expression is described, only the caption expression is described, both are described, and neither is the case.

  For example, the anchor expression is often expressed by a combination of a specific character string such as “figure” or “Fig” and a number or symbol. Therefore, an anchor character string dictionary in which these specific character strings are registered may be prepared in advance, and the anchor expression (anchor character string + number symbol) may be specified by comparing the caption expression with the dictionary. Then, a character string other than the anchor expression among the character strings in the caption area may be determined as the metadata expression.

  For example, in the case of the caption “FIG. 1 AAA”, “FIG. 1” corresponds to the anchor character string, and “AAA” corresponds to the caption character string. There are cases in which the number of characters in the caption expression is extremely small, or it may not be considered a significant character string (for example, in the case of a symbol string "------", etc.). In such a case, there is a possibility that a mark such as a document separator is recognized as a character string, or a character that is not a character may be recognized. In this case, it is determined that it is not a caption. The anchor character string is not extracted.

  Furthermore, since the anchor expression may have a wrong character due to an error in character recognition or the like, a pattern dictionary for correcting the error in character recognition is held, and correction is performed using this. For example, the number “1” and the lowercase letter L “l” (this is easily mistaken for humans).

  In addition, the anchor expression has not only the expression but also a multilingual character string pattern such as a figure number, a character pattern that is notated such as a figure number, and a misrecognition pattern for character recognition. A method for creating an anchor expression dictionary for ambiguous search based on these and performing a search using the dictionary can be considered, but any method may be used in the embodiment of the present invention.

  In step S809, an anchor expression is searched for the body text. Specifically, the link processing unit 304 extracts and accumulates information necessary for link generation. The in-text anchor expression search processing unit 703 performs the anchor expression extraction processing of the anchor expression extraction processing unit 702. The extracted anchor expression is searched from within the text of the document. Then, a search is performed as an anchor expression in the text corresponding to the object. At this time, a plurality of identical anchor expressions may be found in the text. This occurs, for example, when a single chart is referenced many times in the text.

  In step S810, the link processing unit 304 performs link generation processing. Specifically, the link information generation processing unit 704 is necessary for generating a link between the caption-associated object selected by the link information addition target selection processing unit 701, the anchor expression in the text, and the figure or photograph. Create information. Then, the link processing control unit 705 stores information necessary for link generation generated by the link information generation processing unit 704 in the storage unit 211 of FIG.

In step S811, the correspondence expression adding unit 305 generates correspondence expression information and stores the information in the storage unit 211 in FIG. This process will be described in detail later.
In step S812, the format conversion unit 306 performs graphic data generation and the like based on the image data 300 and information stored in the storage unit 211 in FIG. At this time, based on the link information and the corresponding expression information, the link creation process and the corresponding expression process described in the programming language are passed to the format conversion unit 306, and the format conversion unit 306 Will be stored. Above, description of FIG. 8 is complete | finished.

  The above processing is merely processing for a general document image, and the order and details of processing may be anything as long as they are performed in order to realize the object link function of the present invention. For example, character recognition with error correction of character recognition may be performed by application of natural language processing or a dictionary, or information may be extracted and used using a specific expression.

  Further, each analysis method may be optimized based on the document style, the document content classification, and the like. Further, information such as contents of the object may be extracted by using a technique such as image recognition. The input document image may be described in a page description language or the like. In this way, various things can be considered, but what is necessary is that the text of the body and the caption expression or anchor expression in the object are used to realize the object linking function of the embodiment of the present invention. It does n’t matter.

  Next, details of the processing of S811 in FIG. 8 will be described with reference to FIG. The processing here is processing performed in each part in the correspondence expression adding unit 305, and corresponding correspondence information is generated and stored in the storage unit 211 of FIG. For the sake of explanation, the case where the document of FIG. 4 is processed will be described as an example.

  In step S901, the correspondence expression addition control unit 725 in FIG. 7B performs initialization processing for subsequent processing, information stored in the storage unit 211 in FIG. 2, particularly link information in FIG. 7B. 713 is made available.

  In step S <b> 902, the correspondence expression addition control unit 725 performs processing for selecting and extracting the link information that can be used in step S <b> 901 one by one in order. In the present embodiment, the processing from S902 to S911 can be applied to each link information by using the conditional branching in S912 to form a loop processing. Therefore, when this process is performed for the first time, the first link information is selected, and each time this process is performed, the next link information is selected, and the selected link information is the target of the subsequent processes. It becomes.

  Here, first, it is assumed that an object having an object ID of 1001 is selected from the link information in FIG. FIG. 10A is created from FIG. 5B and FIG. 5C created for the document of FIG. 4, and 1001 corresponds to the body text 506 of FIG. 5A. It was created.

  Similarly, 1002 is created for 502 which is the table of FIG. The fact that they have a new object ID means that they are created as new objects that are displayed and functioned in the electronic document in order to realize the display and function of this link. Further, as can be seen from the fact that both are mutually linked by the object ID to be linked, these mean that they are associated via the anchor expression “Table.1” in the processing of FIG. ing.

  However, 1001 is a link from the table 502 to the body text 506, and 1002 is a link from the body text 506 to the table 502. Here, the link is a one-to-one link, but it is not a bidirectional link but two links in one direction. This is because, in practice, a chart may be referenced from multiple parts (sentences, paragraphs, etc.) in the text, so it may be a one-to-many link. This is also to make it possible to have unique functions and expressions.

  In S903, the anchor-in-text anchor text extraction unit 721 extracts corresponding captions such as a chart, a headline, and a label from the link information selected in S902 based on the link information between the anchor expressions in the link information. In the case of the example of FIG. 4, by performing this, it can be seen that in FIG. 10A, 502 that is the basis of 1002 is a table from FIG. 5B, and FIG. ), “Osaka branch”, “2010” and the like, which are row labels and column labels, are obtained.

  In S904, the anchor vicinity text extraction unit 721 in the body extracts text in the vicinity of the anchor expression in the body from the link information selected in S902 based on the link information between the anchor expressions in the body. In the case of the example of FIG. 4, by performing this, in the case of FIG. 10A, it can be seen that 506 that is the basis of 1001 is the text from FIG. The text information in the vicinity of the anchor expression is obtained, and the expression "Annual sales of Company A ..." is obtained.

  In step S <b> 905, the text-in-text / in-table text search unit 722 searches the text information in the vicinity of the anchor expression for text representations such as captions, headings, and labels in the chart obtained by the anchor-inside text extraction unit 721 in the body. I do. In the case of the example of FIG. 4, it is understood that “Osaka branch” and “2010” are included by performing this. This shows that 511, 523, 512, and 522 in FIG. This correspondence is a target for adding a corresponding expression to the link information.

  In step S906, it is determined whether the correspondence is obtained by the text in text / graph in text search unit 722, and a process of branching according to the determination result is performed. If a correspondence relationship is obtained, the process proceeds to S907, and if not, the process proceeds to S910.

  In step S <b> 907, the correspondence expression content determination unit 723 determines what emphasis expression should be performed on the correspondence expression obtained by the text-in-text / graphics-text search unit 722. The emphasis expression method to be determined is determined. In the case of the example of FIG. 4, by doing this, the following is determined.

  This content is shown in FIG. 10 (b). Here, a character string area frame is set for the character string portion of “Osaka branch” and “2010”, which are the corresponding text expressions obtained in S905. However, the character string area frames can be distinguished such that persons corresponding to the same 1001 have different colors, and the corresponding persons have the same color. I will understand. Thus, 1014 is created for “2010” and 1015 is created for “Osaka branch”. Similarly, for 1002, the operation button 1021, the area frame display 1022, and the character string area frame displays 1024 and 1025 are created for the area 502 in FIG.

  In step S <b> 908, the correspondence expression content determination unit 723 adds a target indicated by the relationship such as the arrangement of the charts of the same text. For this, information on the positional relationship and arrangement in the chart is used. In the case of the example of FIG. 4, “Osaka branch” and “2010” are rows and columns from the information in the table of FIG. 5C, respectively, and there is 513 of FIG. Therefore, a character string underline 1023 is created for this “5 million”.

  In S909, the correspondence expression content determination unit 723 creates and adds a corresponding one to the one added in S908. In the case of the example of FIG. 4, it can be seen that by doing this, the one corresponding to 1023 is necessary. However, it is known from the search performed by the text / chart text search unit 722 in S905 that there is no text expression “5 million” in the text. Corresponding to this, text information in the vicinity of the anchor expression is selected. As a result, a character string underline 1013 is created.

  In step S910, the correspondence expression content determination unit 723 determines the entire correspondence expression on the diagram side and the text side. As a highlighting pattern, an operation button 1011 is added to the portion 521 of “Table.1” which is an anchor expression as shown in FIG. This is always performed when a link function is added, and when this is operated, a function for moving the display of a document to the destination corresponding to the link is added.

  Next, an area frame display 1012 is added to the entire vicinity of the anchor expression in the text as shown in FIG. This area frame display has the same emphasized expression as the frame display for the corresponding table 502, and is set to the same color, for example. In addition, it is set so that it can be distinguished from the contents in the document by different colors or movements such as blinking.

  From the above-mentioned conditions, the object IDs 1011 to 1015 correspond to the object IDs 1021 to 1025, respectively, so that different colors and expressions are used for each of the correspondences, but the corresponding objects have the same color and expression. It is like that. Further, the table 502 may be referred to from a plurality of places due to a plurality of anchor expressions in the text. In that case, objects corresponding to the area frame display 1012 and the area frame display 1022 are generated for another anchor expression. These use the same expression pattern as the area frame display 1012 and the area frame display 1022, but are set to have different colors.

  In S911, the correspondence expression addition processing unit 724 collects the correspondence expression information created so far for each link information and stores the information in the storage unit 211 of FIG. This is the correspondence expression information 714 in FIG. Based on this information, an operation, a graphics object, and a computer program for realizing the corresponding expression in S812 of FIG. 8 are created and stored in the electronic document data.

  In step S912, it is determined whether there is link information that has not yet been added so that the corresponding expression information is added to all link information, and a process of branching according to the determination result is performed. If there is link information that has not been added, the process moves to S902 to continue the processing. If there is no link information not added, the process ends.

  FIG. 11 shows the use of the electronic document data created in this way. This is created according to the information of FIG. 10B, and object IDs 1011 to 1015 and 1021 to 1025 correspond to 1111 to 1115 and 1121 to 1125, respectively. 1111 and 1121 can be operated by displaying the link function itself.

  1111 and 1121 are displayed when this page is in a focused state (focused) by a user operation. Further, at that time, when 1111 or 1121 becomes a focused state (focused) by the user's operation, 1112 to 1115 and 1122 to 1125 are also displayed.

  However, if they are not on the same page, only the displayed page is displayed. 1111 and 1121 are changed so that the link destination is in a focused state (focused) when a function is called by being selected by the user. Thereby, the displayed page may be moved.

  For example, if 1111 is selected, 1121 becomes a focused state (focused) state, and the display of 1111 to 1115 and 1121 to 1122 is continued. As a result, the user moves to the table of the portion indicated by the expression of the text of 1111 and is an expression in which 1123 and 1124 indicating the correspondence are linked, and there is also an emphasis expression of 1125. It is possible to easily find the expression “ten thousand”. When the user understands this content, by selecting 1121, it is possible to return to the original body part in the same manner as when 1111 is selected, and the corresponding content part is 1112, 1113, 1114. 1115 can be confirmed again.

  In this way, it is possible to generate a multi-page electronic document by automatically creating a mutual link between an “object” and an “object description” in units of pages by using a multi-page electronic document as an input. It becomes possible. At the same time, the individual description of the “object description” when it is used and the specific part in the corresponding “object” are easily understood.

  Also, the emphasis expression method and its characteristics are determined using the structural relationship of the same word string in a specific part in the object. As a result, the corresponding part can be shown intuitively and easily, so that the correspondence between the specific part in the object and the corresponding part in the description of the object in the text can be easily identified. Can be emphasized. As a result, it is possible to improve the readability of the document by eliminating the trouble of searching for the corresponding part.

(Second Embodiment)
The first embodiment is used when a scanner or the like is used, or when used from an image or data of an electronic document. On the other hand, when creating a document, it is possible to add a similar link function and a corresponding expression to make it easier to understand. By using this, it is possible to create a document with easy-to-understand operations and expressions when the user reads.

  FIG. 12A shows a screen when editing a document. What is created is the diagram of FIG. 4 of the first embodiment, which is a diagram in the middle of creating the document. The document being created is 1200, and 1221 is a cursor, which indicates a place where the user is currently inputting a character or the like.

  Some word processors for document creation / editing in recent years have a function of predicting input contents and indicating input candidates when such character input is performed. Usually, you will learn frequently used expressions such as words and sentences, and show them as input candidates before the user inputs all of them, and if it is appropriate, the user selects it, You can finish entering the word or sentence without entering everything. This is called an input complement function based on input prediction.

  Furthermore, there are some which preferentially use words and sentences frequently used in a document being created as candidates for input prediction, as well as the number and frequency of simple input. This is an effective technique when the frequency of use of words and sentences varies depending on the document.

  When such a method is used, when an input indicating a diagram or part thereof in a document is performed, an expression in a nearby diagram is presented as a prediction candidate, and the case is selected. At this time, not only is it recorded as an input character, but also a diagram having the input portion and the selected expression is provided with link information.

  In addition, correspondence between dialects in the text and charts will be recorded as link information, so corresponding expressions are also proposed as recommended candidates, and the user decides that the contents can be used and selects them Then I will add this. In this way, when the created document is read, it is easy to find and show the corresponding contents in the text content and the chart.

  FIG. 12B shows the contents of the document being input shown in FIG. In the following, the contents of the process at the time of document creation of the document processing apparatus of this embodiment will be described with reference to the flowchart of FIG.

S1401 relates to an initialization process when an instruction to start document editing is given. This makes it possible to refer to, add, modify, and delete information in the document that is necessary for creating the document.
S1402 is the part where the document actually starts to be input, and is received as the contents of the operation instruction including the input contents.
In step S1403, it is determined whether or not the operation instruction instructs to end input. If it is instructed to end input, the process advances to step S1411. Otherwise, it is determined that the operation content is an input instruction or content, and the process proceeds to S1404.

In step S1404, as input prediction, it is checked whether captions, headings, labels, and the like, which are text expressions in nearby charts, can be predicted. In the case of the situation in FIG. 12 (a), 1211 to 1213, etc., 1207 from 1202 in the table and 1204, 1205 from 1203 in the figure are taken out as texts in the neighboring charts and compared with the text of the input contents. Do.
S1405 is a process of conditional branching using the comparison result of S1404. If it is included, the process proceeds to S1406. If not included, the process proceeds to S1410.

  In step S1406, corresponding text such as captions, headings, and labels obtained as a result of the comparison in step S1404 is presented to the user as input completion candidates. FIG. 13 is a diagram actually presented. In FIG. 13, 1301 is a recommended complement candidate, and “2 million” of 1302 and “2010” of 1303 are represented as the candidates. On the other hand, the user can perform complementation by performing an operation of selecting the candidate, and can complete the input word or expression. In addition, it is possible to reject (reject) the selection by performing an operation of explicitly rejecting the selection of the candidate or ignoring the candidate display and continuing the input.

  S1407 is a process of determining and branching the condition by determining how the user has instructed the display performed in S1406, that is, the candidate proposal based on estimation. If the candidate proposed in S1406 is selected, the process advances to S1408. If not selected, the process proceeds to S1410.

  In step S1408, the input completion candidate selected according to the display in step S1406 is added to the input content. In the case of the display in FIG. 13, when “2010” is selected, the last “20” in the text of 1206 is replaced with “2010”. This is an input supplement based on normal prediction.

  In S1409, attention is paid to the fact that the input completion performed in S1408 has been performed, and a process of creating link information between the text 1206 and the table 1202 is performed. Since the processing at this time is performed by the units 701 to 705 in FIG. 7A of the first embodiment, the anchor expression “Table.1” and the like are also used to create link information. As a result, when the image data is input to about 400 in FIG. 4, the same information as in FIGS. 5B and 5C is accumulated.

In step S1410, the link information generated in step S1409 is stored as an electronic document.
S1411 is the same as S811 of FIG. 8 of the first embodiment, and the processing of FIG. 9 is performed using the correspondence expression adding unit of FIG. 7B. Thereby, a correspondence expression is created.

  In S1412, processing is performed to display the corresponding expression so that the user can determine whether or not to add the corresponding expression obtained in S1411 as an expression for making the input document easy to see. When the process of displaying the corresponding expression is performed, the emphasized expression information generating process for generating the emphasized expression information that enables the correspondence relationship to be easily identified is performed. When the input document is created up to the image data 400 of FIG. 4, the corresponding expression candidates shown in FIG. 11 are shown.

  In step S1413, a conditional branching process is performed according to the user's operation with respect to the proposal for adding the corresponding expression in step S1412. When this correspondence expression is selected by the user, the process proceeds to S1414. If not selected by the user, the input is terminated.

  S1414 is processing for adding the correspondence expression proposed in S1413 to the electronic document data. Unlike the first embodiment, since document editing is performed in the second embodiment, when editing is completed, the process is recorded in an electronic document file or the like. At that time, the electronic document as in the first embodiment is used. Processing to convert the file format is performed. Normally, when editing a document, the contents of the document are expanded into the RAM 206 in FIG. 2 or the like for editing and converted into a data structure that is advantageous for editing. When recording in the format, it is returned (converted) to that format.

  By adding the correspondence expression in this way, the correspondence expression as shown in FIG. 11 is added. In particular, in document editing, by creating link information in combination with a character input predictive complement function, corresponding expression candidates are automatically presented, and it is possible to easily add corresponding expressions. ing.

  By collaborating with document editing functions such as this predictive complement function for character input, character input is facilitated, and the user is presented with corresponding expression candidates even though the number of operations has not increased. As a result, the user's burden of document creation can be reduced. In addition to the predictive completion function for character input, in document editing devices, the layout of sentences and diagrams in the document, the addition of decorative parts to make them easy to understand or beautiful, and the management of numbers in anchor expressions Various support functions may be provided. Similarly, by extracting link information at the time of these operations, the present invention can be applied and the effect can be obtained.

  According to the present embodiment, when performing character input by input prediction, it is possible to correspond to the case where the possibility of inputting the same thing increases by comparing with the text content in the object in the same document. The text in the diagram in the same document with the characteristics is presented as an input candidate. When such a method is used, when an input indicating a diagram or part thereof in a document is performed, an expression in a nearby diagram is presented as a prediction candidate, and the case is selected. At this time, it is possible not only to record as input characters but also to provide link information to the chart having the input portion and the selected expression.

  In addition, correspondence between dialects in the text and charts will be recorded as link information, so corresponding expressions are also proposed as recommended candidates, and the user decides that the contents can be used and selects them Then, this is stored as data in the electronic document being edited. In this way, when the created document is read, it is easy to find and show the corresponding contents in the text content and the chart. In addition, such a document can be easily created.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (computer program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various computer-readable storage media. Then, the computer (or CPU, MPU, etc.) of the system or apparatus reads out and executes the program.

218 Data processing unit, 300 image data, 301 area division unit, 302 attribute information addition unit, 303 character recognition unit, 304 link processing unit, 305 corresponding expression addition unit, 306 format conversion unit, 310 electronic document data

Claims (16)

A document processing apparatus for creating a multi-page electronic document by creating a mutual link between an object and a description of an object in a page unit of a multi-page electronic document including image data,
Area dividing means for dividing the image data to obtain divided areas;
Attribute information adding means for determining the attribute of the divided area obtained by the area dividing means and adding a character attribute for each area;
Character recognition means for recognizing characters in the region to which the character attribute is added by the attribute information addition means;
The correspondence between the anchor expression associated with the object and the anchor expression or the description of the object in the text of the electronic document is examined, and the correspondence between the object and the anchor expression or the description of the object in the text is determined. Link information generation processing means for generating link information for holding;
Correspondence expression information that emphasizes the correspondence is generated in the corresponding portion between the text representation in the object and the description of the object in the body, which is associated with the link information generated by the link information generation processing means. A corresponding expression adding means to be added,
A document processing apparatus comprising: format conversion means for converting the link information into an electronic document including correspondence expression information that emphasizes the correspondence.   The link information generation processing unit includes an anchor expression associated with the image data and the object, and a means for extracting a description of the object including the anchor expression and the anchor expression in the text. Document processing device.   The format conversion means includes electronic information including link information that realizes a mutual link between the object and an anchor expression or a description of the object based on the link information, and correspondence expression information that emphasizes the correspondence. The document processing apparatus according to claim 1, wherein the document processing apparatus converts the document into a document.   The correspondence expression adding unit compares the text information in the object with the text information in the description of the object in the body using the link information, and the description of the object in the body explains An emphasis expression that obtains a specific part in the estimated object and makes it possible to easily identify the correspondence between the specific part in the object and the corresponding part in the description of the object in the text. 4. The document processing apparatus according to claim 1, wherein a method and its characteristics are determined, and emphasized expression information is generated based on the determined emphasized expression method and characteristics.   The correspondence expression adding means uses the link information to compare the text information in the object and the text information in the text description of the object in the body in units of word strings to obtain the same word string , Obtaining a specific part in the object that is assumed to be explained by the description of the object in the text, and corresponding to the specific part in the object and the corresponding part in the description of the object in the text In order to easily identify the relationship, the structural expression of the same word string in a specific part in the object is also used to determine the emphasis expression method and its characteristics, and the emphasis expression information is generated based on the determination. The document processing apparatus according to any one of claims 1 to 3.   The document processing apparatus according to claim 1, wherein the object includes a region of a figure, a drawing, a photograph, and an illustration. A document processing apparatus that edits an electronic document using text, diagrams, and photographs as objects,
When inputting characters, input completion means for predicting input contents and displaying them as candidates using the contents input in the past, and allowing the user to select and input characters by predicting input,
When performing character input based on the input prediction, the content of the text in an object in the same document is compared, and if the possibility that the same item is input increases, Recording means for presenting the text in the chart as input candidates, and recording the corresponding relationship as link information when the presented input candidate is selected;
Using the link information, the text information in the object is compared with the text information in the description of the object in the body of the electronic document, and the description of the object in the body of the electronic document explains Emphasis expression information generating means for obtaining a specific portion in the estimated object and generating emphasis expression information that makes it possible to easily identify the correspondence relationship;
A document processing apparatus comprising: storage means for storing data in an electronic document being edited including the link information and emphasized expression information.   The storage means stores as data in an electronic document being edited that includes the link information and emphasis expression information that realizes a mutual link between the object and the anchor expression or description of the object based on the link information. The document processing apparatus according to claim 7, wherein:   The emphasized expression information generation means compares the text information in the object with the text information in the description of the object in the body using the link information, and the description of the object in the body is To obtain a specific part in an object presumed to be explained, and to easily identify the correspondence between the specific part in the object and the corresponding part in the description of the object in the text 9. The document processing apparatus according to claim 7, wherein the emphasis expression method and its characteristics are determined, and emphasis expression information is generated based on the determination method.   The emphasized expression information generating means compares the text information in the object with the text information in the object and the text information in the text description of the object in the body by using the link information, and the same word string is obtained. To obtain a specific part in the object that is supposed to be explained by the description of the object in the text, and to the specific part in the object and the corresponding part in the description of the object in the text. In order to easily identify the corresponding relationship, the emphasis expression method and its characteristics are determined using the structural relationship of the same word string in a specific part in the object, and the emphasis is made based on the determination method. The document processing apparatus according to claim 7, wherein expression information is generated.   The document processing apparatus according to claim 7, wherein the object is a region of a figure, a drawing, a photograph, or an illustration. A document processing method for creating a multi-page electronic document by creating a mutual link between an object and a description of the object for each page of a multi-page electronic document including image data,
A region dividing step of dividing the image data to obtain a divided region;
Attribute information adding step of determining the attribute of the divided region obtained in the region dividing step and adding a character attribute for each region;
A character recognition step for recognizing characters in a region to which a character attribute is added in the attribute information addition step;
The correspondence between the anchor expression associated with the object and the anchor expression or the description of the object in the text of the electronic document is examined, and the correspondence between the object and the anchor expression or the description of the object in the text is determined. A link information generation process for generating link information for holding;
Correspondence expression information that emphasizes the correspondence is generated in a portion corresponding to the text representation in the object and the description of the object in the body, which is associated with the link information generated in the link information generation processing step. A corresponding expression adding step to be added,
A document processing method comprising: a format conversion step of converting the link information and an electronic document including correspondence expression information that emphasizes the correspondence. A document processing method for editing a document using text, charts and photos as objects,
When inputting characters, an input complementing step that predicts the input contents using the contents input in the past, displays them as candidates, and causes the user to select and input characters by predicting input,
When performing character input based on the input prediction, the content of the text in an object in the same document is compared, and if the possibility that the same item is input increases, A step of presenting the text in the chart as an input candidate, and when the presented input candidate is selected, a recording step of recording the correspondence as link information;
Using the link information, the text information in the object is compared with the text information in the text description of the object in the text, and the text in the text is estimated to be explained by the text in the text. Emphasis expression information generation step for generating emphasis expression information that makes it possible to easily identify the corresponding relationship by obtaining a specific part of
A document processing method comprising a storing step of storing the link information and the highlighted expression information as data in an electronic document being edited. A program that causes a computer to execute each step of a document processing method for creating a multi-page electronic document by creating a mutual link between an object and an object description in a page unit of a multi-page electronic document including image data There,
A region dividing step of dividing the image data to obtain a divided region;
Attribute information adding step of determining the attribute of the divided region obtained in the region dividing step and adding a character attribute for each region;
A character recognition step for recognizing characters in a region to which a character attribute is added in the attribute information addition step;
The correspondence between the anchor expression associated with the object and the anchor expression or the description of the object in the text of the electronic document is examined, and the correspondence between the object and the anchor expression or the description of the object in the text is determined. A link information generation process for generating link information for holding;
Correspondence expression information that emphasizes the correspondence is generated in a portion corresponding to the text representation in the object and the description of the object in the body, which is associated with the link information generated in the link information generation processing step. A corresponding expression adding step to be added,
A program causing a computer to execute the format conversion step of converting the link information and the electronic document including correspondence expression information that emphasizes the correspondence. A program that causes a computer to execute each step of a document processing method that edits a document using text, diagrams, and photographs as objects,
When inputting characters, an input complementing step that predicts the input contents using the contents input in the past, displays them as candidates, and causes the user to select and input characters by predicting input,
When performing character input based on the input prediction, the content of the text in an object in the same document is compared, and if the possibility that the same item is input increases, A step of presenting the text in the chart as an input candidate, and when the presented input candidate is selected, a recording step of recording the correspondence as link information;
Using the link information, the text information in the object is compared with the text information in the text description of the object in the text, and the text in the text is estimated to be explained by the text in the text. Emphasis expression information generation step for generating emphasis expression information that makes it possible to easily identify the corresponding relationship by obtaining a specific part of
A program for causing a computer to execute a storing step of storing the link information and the emphasized expression information as data in an electronic document being edited.   A computer-readable storage medium storing the program according to claim 14 or 15.

Images