JP2005167522A - Printing system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology of improving the reduction in a character recognition efficiency caused by an OCR incapable of recognizing semantic connections between words and sentences separated by typesetting, and a technology of attaining simultaneous optical character recognition for a plurality of pieces of printed matter. <P>SOLUTION: Regional elements of a sentence, their connection relations and sequential relations are embedded in the sentence as electronic watermark information so that the sequential relations of characters can accurately be grasped before recognition processing to enhance the character recognition efficiency. Further, a regional designation method adopts a relative coordinate reference region to attain the simultaneous optical character recognition for a plurality of pieces of printed matters while accurately grasping the correspondence to the printed matters. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a printing system including an information input / printing apparatus such as a scanner, a printer, and a FAX.

  As an OCR (Optical Character Reader) technique using a scanner or the like, the following Patent Document 1 is known. In general, in OCR, although local text recognition processing is performed relatively efficiently, a text region is used. It is difficult to automatically identify readings based on the logical structure of the text area, the semantic connection between the text areas, and the logical structure.

  For this reason, in simple OCR for complex text in typesetting, there is a problem that recognition efficiency decreases due to the fact that error correction using a knowledge database cannot function efficiently due to division of words, sentences and paragraphs due to line breaks. there were.

  On the other hand, as disclosed in Non-Patent Document 1 below, it is known that a certain amount of information can be embedded as a digital watermark in binary image data, particularly a printed matter.

JP-A-8-50633

Nikkei Newspaper: October 3, 2003 article

  The problem to be solved by the present invention is to prevent a reduction in character recognition efficiency due to the fact that the semantic connection of words and sentences separated by typesetting, as well as paragraphs, cannot be recognized by OCR. It is to be.

  Another object of the present invention is to reduce the work load such as OCR area designation generated for the worker during OCR due to the above problem.

  The above problem is that the text area element on the print layout and the OCR auxiliary information such as the connection relation and the order relation thereof are automatically generated or generated at the time of printing as the work of the print creator, and used as a digital watermark on the print It is solved by embedding.

  In the printing system of the present invention, the semantic connection of sentences such as words, sentences, and paragraphs divided by typesetting is restored by the OCR auxiliary information embedded as a digital watermark in the sentence area element, and the OCR recognition efficiency is reduced. Can be prevented.

  In addition, when assuming a service that provides a printed material that supports OCR, the shift in the workload for generating the OCR auxiliary information from the OCR time to the printed material generation time is a merit for improving the service.

  As an embodiment of the present invention, an information processing apparatus that generates print image data based on electronic information, an image display apparatus that previews print image data, and an area attribute such as a text area and an image area are indicated on the preview image OCR auxiliary information is extracted from an area instruction means, a procedure on an information processing apparatus that embeds area attributes in image data as digital watermark information, a printing apparatus that prints print image data, and a printed matter printed by the printing apparatus, This is an OCR auxiliary information embedded printing system constructed by an OCR system used as hint information in OCR.

  OCR is generally an optical character recognition process, but here, a process of separating non-text areas such as charts and photographs and reading it as bitmap information is also included as part of the OCR process.

  FIG. 1 is a conceptual diagram of a printed matter that is an object of the present invention assuming A4 (210 mm × 297 mm). As shown in FIG. 1, a normal business document is composed of several text regions (regions 2 to 13) and non-text regions (regions 14) such as charts and photographs.

  In the present embodiment, for the sake of simplicity, it is assumed that each component region is separated by a surrounding rectangular element. Common business documents often meet this assumption. Further, each area indicated by a broken line in FIG. 1 is an example of area division based on a physical layout, not a semantic division.

  The text areas 2 to 13 include a body according to a series of contexts, floating elements that are not directly related to the flow of independent text such as the title area 3, the caption area 7 of the chart, and the footnote area 9. There is.

  Of these, the body according to a series of contexts and the floating elements that can be incorporated into the body flow can be given an order according to the context. For example, in the case of FIG. 1, the date area 2, title area 3, author name area 4, abstract area 5, body areas 6, 8, 10, 12, and 13 can be ordered in this order.

  Among these, the order of the date area 2, the title area 3, the author name area 4, and the abstract area 5 does not necessarily need to be in this order, but the order relation of the texts must be accurately observed. In FIG. 1, an example of ordering of regions is indicated by a circled number in each element region.

  FIG. 2 shows an example of the information table 40 expressing these area information. In FIG. 2, a byte offset 24 to the version information 23 and the additional information 31 is prepared for each byte as the head header 20. Thereafter, each area information entry 21 is sequentially described as a first entry 21a, a second entry 22b,... According to the ordering of each area, with a fixed length of every 5 bytes.

  Also, each area information entry 21 includes a top high 4 bit type identifier 25, a top low 4 bit auxiliary information 26, and a 1 byte left end coordinate value 27, a right end coordinate value 28, a top end coordinate value 29, and a bottom end. Coordinate value 30 consists of a total of 5 bytes. Details of these elements will be described later.

  In the example of FIG. 2, a 4-byte document identification number is given as the additional information 31, but it is also effective to embed date and time, a printer identification code, and the like. The reason why the additional information amount 31 is arranged at the end of the information table 40 is to flexibly cope with the change of the additional information depending on the version or the application.

  For example, by defining a combination type of some information as additional information 31 and switching the implementation of the additional information 31 embedded in the information table 40 according to the value of the version information 23, the document purpose is changed. And flexible support according to the amount of data that can be embedded.

  Here, when the amount of data required for embedding is estimated in the case of FIG. 1, the information amount is header (2 bytes) + (entry 5 bytes) × 13 + (document identification number 4 bytes) = 71 bytes. On the other hand, according to Non-Patent Document 1, the amount of information that can be embedded in printed matter as a digital watermark is approximately 8 bytes per 70 characters. I understand that it is possible. For example, in the patent publication of 25 characters × 50 rows × 2 columns per page, this corresponds to a sentence volume of substantially 1/3 page.

  Accordingly, it is possible to add more information as the additional information 31 depending on the document. If the print quality of the printed matter in which the watermark is embedded does not matter, but rather the amount of embedded information is given priority, a version in which a lot of information is mounted in the additional information 31 may be prepared. Further, for the purpose of minimizing image quality degradation due to an increase in watermark information, the additional information area 31 is not implemented, and the area information entry 21 is limited to a text area premised on character recognition, and information is reduced as much as possible. Such a response is practically effective.

  Next, details of the area information entry 21 will be described. As shown in FIG. 1, each area is defined by a rectangle. Each left end coordinate value 27, right end coordinate value 28, upper end coordinate value 29, and lower end coordinate value 30 of the area information entry 21 are described in 8-bit unsigned integers with 1.2 mm as the minimum unit with reference to the upper left corner 1 of FIG. (However, when the value of the type identifier 25 is a to d in FIG. 3, the reference point is changed as described later).

  Accordingly, when the area 5 in FIG. 1 is taken as an example, the value obtained by dividing the x-direction distance (in mm) between the upper left end 17 of the area 5 and the reference point 1 by 1.2 mm on the printing and rounding down the decimal point. This is the value of the left end coordinate value 27 in the fifth entry 21 of the information table 40.

  Similarly, the value of the right end coordinate value 28 from the x-direction actual size of the lower right end portion 18 and the reference point 1 of the region 5, the value of the right end coordinate value 29 from the y upper end portion 17 of the region 5 and the y-direction actual size of the reference point 1, the region The value of the right end coordinate value 30 is determined from the right lower end 17 of 5 and the actual size in the y direction of the reference point 1.

  In the type identifier 25 of the area information entry 21, the type identifier of the classification shown in FIG. 3 is registered according to the contents of each area. In FIG. 3, the value of the type identifier is represented by a hexadecimal number of 0 to f, and is represented by a binary number in parentheses.

  In the case of the example of FIG. 1, “2 (0010)” representing horizontal text is associated as the value of the type identifier 25 of the area information entry 21 corresponding to the date area 2. Next, in order to register the title area 3 as a text area continuing to the date area 2, “3 (0011)” representing the continuation text is supported as the value of the type identifier 25 of the area information entry 21 corresponding to the title area 3. Let

  The region given the continuation text attribute by the type identifier 25 is interpreted as a text region continuing to the region corresponding to the region information entry 21 having the preceding last text attribute (type identifier value 1 to 4) at the time of OCR. Therefore, whether the area given the continuation text attribute is vertical writing or horizontal writing follows the vertical and horizontal directions of the preceding text attribute area.

  Similarly, the continuation text attribute is given to the area 4 to the area 6, the area 8, and the area 10 to the area 13 in FIG. 1 and registered in the area information entry 21 of the information table 40 in this order.

  As the value of the type identifier of the area information entry 21 corresponding to the chart area 14, “5 (0101)” representing a line drawing is specified in the example of FIG. When the area having the type identifier value of the line drawing attribute 5 is captured by OCR, the resolution is prioritized. On the other hand, if it is considered appropriate to read with emphasis on gradation rather than resolution, “7 (0111)” representing a photograph is designated as the value of the type identifier.

  In addition, in the case of a table region whose matrix structure is clarified by ruled lines, “6 (0110)” representing the table is designated as the value of the type identifier. In this case, during OCR, the matrix structure is analyzed by prioritizing ruled line recognition and used as a hint for recognition processing.

  The caption area 7 and the footnote area 9 attached to the chart are independent sentences that are not inserted in a natural order in the context of the main text. Therefore, as the type identifier of the corresponding area information entry, the horizontal text attribute “1 ( 0001) ".

  The strong continuation text attribute “4 (0100)” in FIG. 3 is used when emphasizing that the context is strongly connected like a text area divided in the layout structure, although it is originally one paragraph. use. In the example of FIG. 1, it is assumed that the last paragraph of the area 10, the area 12, and the area 13 immediately before continuing is divided in the layout.

  In this case, logical continuity can be particularly emphasized by designating “4 (0100)” as the type identifier values of the regions 10, 12, and 13. This is because, in particular, if the original area division is given not by the layout standard but by division along both the paragraph boundary and the layout boundary, it becomes possible to add the structure of the paragraph unit to the recognition result during the subsequent OCR. So it becomes particularly useful information.

  In the 4-bit auxiliary information 26 of the area information entry 21, as the auxiliary information on the layout, the blank statuses of the upper, lower, left and right areas are registered in order from the upper bit.

  For example, when the upper area of the area corresponding to the area information entry 21 is blank as in the area 2 of FIG. 1, the third bit (most significant bit) of the auxiliary information 26 is set to 1, otherwise the bit is set to 0. . Similarly, in the area 9 and the area 3 where the area below is blank, the second bit of the auxiliary information 26 is set to 1. The same applies to the 1st bit (left blank) and 0th bit (right blank) of the auxiliary information 26.

  The auxiliary information 26 is used as a correction hint for movement, rotation, enlargement / reduction, distortion, etc. of the document during OCR, and also has an effect of clarifying an area that does not require scanning and improving processing speed. However, when more reliable positioning information is required, it is also effective to use the positioning reference markers 15 and 16 together. Particularly in the case of a color system, the color of the positioning reference markers 15 and 16 may be inconspicuous if the color is yellow.

  The example of FIG. 1 is an example of embedding OCR auxiliary information in a document that has already been printed. However, the present invention is also effective as a method for embedding a reading instruction in an area that is not printed during printing. An example is shown in FIG.

  In the example of FIG. 4, the text area 51 that is a guide sentence is not an object of OCR, but is a photo area 55 for pasting a photograph 56 and third handwriting entry areas 52, 53, 54 for a third party who received the text. Are subject to OCR.

  In this case, the area information entry 21 of the areas 52 to 55 is registered in addition to the area information entry 21 of the text area 51 in the information table 40 embedded as a digital watermark in the text area 51 of the printed matter 50.

  At this time, the relative coordinate reference attribute “0 (0000)” of FIG. 3 is designated in the area 51 as the type identifier 25 value of each area information entry 21, and the monochrome bitmap relative coordinate reading area is designated in the areas 52 to 54. The attribute “c (1100)” is instructed, and the area 55 is instructed with the color bitmap relative coordinate reading area attribute “d (1101)”.

Furthermore, by registering the area information entry 21 for designating the relative coordinate reading area attribute “b (1011)” by character recognition for the areas 52 to 54, both reading as an image and character recognition are performed. Can be directed.
Here, the relative coordinate reading area in FIG. 3 is the left end coordinate value 27 of the area information entry 21 having the previous relative coordinate reference attribute (the value of the type identifier 25 is “0 (0000)”) in the information table 40. This means that the region is defined by relative coordinate values with the lower end coordinate value 30 as a base point. On the other hand, the absolute coordinate reading area is defined by defining an area with a coordinate value based on the coordinate reference point 1 (paper corner) as in the case of the type identifier values 0 to 7 described above. is there.

  In addition to the region size information obtained from the left end coordinate value 27 to the bottom end coordinate value 30 as the region coordinate information, the relative coordinate reference attribute region is obtained from the combination of the digital watermark information embedded in the region, the coordinate reference point 1 It is used to specify the position of the region in the scanned image without depending on the.

  Further, when printing the printed matter 50 of FIG. 4, the individual printed matter 50 and the read data are set to 1 by changing the document identification number of the additional information 31 of the information table 40 embedded as the electronic watermark for each printed matter. It becomes possible to make it correspond to a pair.

  As a result, even if a plurality of printed materials 50 are laid out on one page and simultaneously scanned as shown in FIG. 5, the region to be OCR can be correctly recognized in association with each printed material.

  Note that the values 8 to d of the type identifier 25 in FIG. 3 indicate an area attribute without print information at the time of printing, and there is a possibility that no data exists in the area or the image quality of the area is not guaranteed. Is treated as a region attribute different from the type identifier values 0 to 7.

  FIG. 6 shows a flow of print generation in which OCR auxiliary information is embedded in the system of the present invention.

  Here, as an embodiment of the present invention, a system including an information processing apparatus 102, an optical reading apparatus 100, an image display apparatus 101, an information input apparatus 103, an area instruction apparatus 104, and a printing apparatus 105 as shown in FIG. Suppose. The optical reading apparatus 100 constructs an OCR system in combination with OCR software installed in the information processing apparatus 101.

  First, in step 60, document data to be printed is created by application software on the information processing apparatus 102.

  In step 61, various printing conditions such as paper size, margin setting, number of prints, and the like are set by a printer driver operating on the information processing apparatus 102.

  In step 62, the printer driver is instructed whether to embed OCR auxiliary information. When the OCR auxiliary information embedding process is performed, the print preview image 106 is displayed on the image display device 101.

  In step 64, the user generates an information table 40 by designating each area and its attribute corresponding to each area in FIG. 1 using the area instruction device 104 for the print preview image 106. .

  Next, when the print command is issued in step 65, the printer driver expands the document data into print image data. In step 66, the previous information table 40 is added to the print image data as a digital watermark. Send the image data to the printer.

  If an instruction not to perform the OCR auxiliary information embedding process is given in the previous step 62, only normal print image data generation in step 69 is performed by the print command in step 68, and the print data generated in step 67 is generated in the printer. Send print image data.

  The example of FIG. 6 is an example in which the user interactively generates the information table 40. However, depending on the form of the document data 70, this can be automatically generated. As a form of such document data 70, a markup language such as TeX in which a logical structure is described and provided independently of a layout structure is suitable. FIG. 7 shows the flow of processing when the information table 40 is automatically generated.

  In FIG. 7, the information table 40 is automatically generated by developing the region information entry 21 in step 72 for the document data 70 and expanding the document data into a bitmap in step 71. In step 73, the information table 40 is added to the bitmap data as digital watermark information, and in step 74, the data is transmitted to the printer engine.

  In this case, since the processing in steps 71 to 73 is automatic processing, the processing can be performed by a controller incorporated in the printing apparatus 105. The printer engine in step 74 means a portion excluding the embedded controller in the printing apparatus 105.

FIG. 8 shows the flow of OCR side processing of the system of the present invention.
In step 80, the information processing apparatus 102 performs scan condition settings such as photo area resolution designation and color / monochrome designation.
In step 81, the original is scanned by the optical reading device 100 and converted into bitmap data.

  In step 82, the presence or absence of the information table 40 as OCR auxiliary information embedded as digital watermark in the scanned bitmap data is detected. If there is no information table 40, the information processing apparatus 102 performs normal character recognition processing 92.

  If there is an information table 40, the information table 40 is extracted in step 83.

  In step 84, based on the extracted information in the information table 40 and simple bitmap data recognition information, translation, rotation, enlargement / reduction, distortion, etc. of the document are detected and corrected.

  In step 85, based on the text area attribute entry of the information table 40, the text area and the continuation relationship between the text areas are extracted. Thereafter, in step 86, character recognition is performed. In character recognition, the use of a database for the context effectively works to increase the recognition rate. Therefore, the recognition processing of the portion that extends over the continuous text area is simultaneously analyzed.

  When the character recognition is completed, in step 87, the first non-character region such as a chart / photo that appears in the entry of the information table 40 is extracted.

  If rescanning is necessary for high resolution reading or color reading as a reading attribute, the scanning condition of the optical reading device 100 is reset in step 89 and the corresponding area is rescanned.

When the non-character string area process remains, the processes of step 87 to step 91 are repeated for the next non-character string area appearing in the information table 40.
When processing of all entries to be processed is completed, data is output or saved in a file in step 93.

  In the above description, the system having the hardware configuration shown in FIG. 9 is assumed. However, since print data generation, printing, and OCR are not performed at the same time, the optical reading device 100, the information processing device 102, and the printing device 106 are The apparatus may be composed of a plurality of devices or integrated in any combination. Moreover, there is no problem even if the system is separated in time and space.

In the printing system of the present invention, the continuation relationship between the text area and the text area can be accurately predicted from the information table embedded as a digital watermark in the text area element during OCR.
For this reason, since it is possible to predict the exact word order even for word / sentence / paragraph division due to text line breaks or text area divisions, it is possible to increase OCR efficiency and reduce the workload of OCR workers. It is reduced.

  Further, since information inserted by a user of a printed matter that does not exist at the time of printing can be relatively designated as an area to be OCRed, it is effective for collective processing of information organization such as a questionnaire.

It is a conceptual diagram of printed matter. It is a block diagram of an information table. It is a correspondence table of type identifier values and content attributes. It is a conceptual diagram of the printed matter which assumed writing. It is an example of simultaneous OCR of a plurality of printed matter. It is a flowchart of printed matter generation. It is a flowchart of printed matter automatic generation. It is a flowchart of a process of OCR. It is a system example.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Coordinate reference point, 2 ... Date area, 3 ... Title area, 4 ... Author name area, 5 ... Area, 6 ... Area, 15, 16 ... Positioning reference marker, 20 ... Header, 21 ... Area information entry, 23 ... version information, 24 ... additional information offset, 25 ... type identifier, 26 ... auxiliary information, 27 ... left end coordinate value, 28 ... right end coordinate value, 29 ... upper end coordinate value, 30 ... lower end coordinate value, 31 ... additional information, 40 ... information table, 50 ... printed matter, 51 ... text area, 52 ... area, 53 ... area, 54 ... area, 55 ... photographic area, 100 ... optical reading apparatus, 101 ... image display apparatus, 102 ... information processing apparatus, 103 ... Information input device 104... Area instruction device 105... Printing device 106.

Claims (8)

An information processing apparatus that generates print image data based on electronic information, a printing apparatus that prints print image data, and a print that includes an OCR system that optically reads information printed on printed matter and performs recognition processing A system,
The information processing apparatus embeds text area information and text area order information as digital watermark information in print image data,
The OCR system performs character recognition processing using digital watermark information reproduced on a printed matter printed by the printing apparatus as auxiliary information. The printing system according to claim 1,
The printing system according to claim 1, wherein the text area order information is given as an order of entries of the text area information. The printing system according to claim 1,
A printing system characterized by embedding area attribute information and area information as electronic watermark information in a text, as well as non-text attribute areas such as charts and photo areas. A printing system according to claim 1 or claim 3, wherein
The information processing apparatus includes an image display device and an image area instruction device,
A printing system comprising a user interface that allows a print preview image to be displayed on the image display apparatus to be instructed in an interactive environment with a user in a text area, a non-text area, and their order relationship. . A printing system according to claim 1 or claim 3, wherein
The information processing apparatus estimates the text region, the non-text region, and their order relationship directly or indirectly from the electronic information. The printing system according to claim 5, wherein
The text area is divided along a logical structure such as a paragraph boundary of a sentence, and connection information is electronically connected to a text area on the same logical structure divided into a plurality of text areas in the document layout. A printing system characterized by being added to watermark information. An information processing device that generates print image data based on electronic information, an image display device, an image area instruction device, a printing device that prints print image data, and optically reads information printed on printed matter A printing system having an OCR system for performing recognition processing,
The information processing apparatus can specify text area information, non-text area information, and order information thereof to be read as OCR for a print preview image displayed on the image display apparatus in an interactive environment with a user. And embedding the text area information, non-text area information, and order information as digital watermark information in the print image data,
The OCR system reads a text area or a non-text area instructed in the digital watermark information for a printed matter in which the digital watermark information is embedded. The printing system according to claim 7,
The OCR system switches a processing method such as color / monochrome, resolution, presence / absence of recognition processing according to attributes of a text region and a non-text region.

Images