US11410444B2 - Information processing apparatus and non-transitory computer readable medium for arranging table image and recognition result - Google Patents
Information processing apparatus and non-transitory computer readable medium for arranging table image and recognition result Download PDFInfo
- Publication number
- US11410444B2 US11410444B2 US17/018,290 US202017018290A US11410444B2 US 11410444 B2 US11410444 B2 US 11410444B2 US 202017018290 A US202017018290 A US 202017018290A US 11410444 B2 US11410444 B2 US 11410444B2
- Authority
- US
- United States
- Prior art keywords
- recognition result
- row
- image
- displayed
- corrected
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active, expires
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
- G06F40/177—Editing, e.g. inserting or deleting of tables; using ruled lines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/103—Formatting, i.e. changing of presentation of documents
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/103—Formatting, i.e. changing of presentation of documents
- G06F40/106—Display of layout of documents; Previewing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/94—Hardware or software architectures specially adapted for image or video understanding
- G06V10/945—User interactive design; Environments; Toolboxes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/40—Document-oriented image-based pattern recognition
- G06V30/41—Analysis of document content
- G06V30/412—Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/40—Document-oriented image-based pattern recognition
- G06V30/41—Analysis of document content
- G06V30/414—Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
Definitions
- the present disclosure relates to an information processing apparatus and a non-transitory computer readable medium.
- An image processing apparatus that recognizes a place filled in by a user in an image obtained by reading a paper document filled in by the user has been known (see, for example, Japanese Unexamined Patent Application Publication No. 2013-012223).
- the image processing apparatus sets in advance a region where a written mark is to be recognized in the place filled in, recognizes the mark in the region in the image as a recognition target, based on the set region, generates an object representing a recognition result for the region where the mark is recognized, and displays the generated object at a position corresponding to the region where the mark is recognized, in a superimposed manner.
- a table recognition apparatus has also been known (see, for example, Japanese Unexamined Patent Application Publication No. 2000-090195).
- the table recognition apparatus analyzes regions and positions of items in a table, based on the arrangement state of ruled lines and character strings contained in an image of the table to acquire a table structure, displays the acquired table structure, issues an instruction to correct a region of an item in the displayed table structure, corrects the region and position of the item in accordance with the correction instruction to acquire a corrected table structure, performs character recognition on each of the items in the corrected table structure, and generates table format data based on a result of the character recognition and the table structure.
- An operator may check and correct a recognition result of recognition of a table image containing a table.
- the operator causes the recognition result having the original table structure to be displayed independent of the table image and compares the table image with the recognition result, the operator needs to check the recognition result while coming and going between the table image and the recognition result because the table image and the recognition result are displayed apart from each other. Thus, such a checking operation is troublesome to the operator.
- Non-limiting embodiments of the present disclosure relate to reducing the labor of an operator for comparing a table image with a recognition result, compared to a case where a recognition result of a table image containing a table is displayed with an original table structure.
- aspects of certain non-limiting embodiments of the present disclosure address the above advantages and/or other advantages not described above. However, aspects of the non-limiting embodiments are not required to address the advantages described above, and aspects of the non-limiting embodiments of the present disclosure may not address advantages described above.
- an information processing apparatus including a processor.
- the processor is configured to acquire a recognition result of a table image containing a table; and perform control such that a screen on which the table image and the recognition result are arranged for each row or column of the table in an alternate manner is displayed.
- FIG. 1 is a diagram illustrating an example of the entire configuration of a form processing system according to an exemplary embodiment of the present disclosure
- FIGS. 2A and 2B are diagrams illustrating how a recognition result correction apparatus configurates a table region based on a form recognition result
- FIGS. 3A and 3B are diagrams illustrating how the recognition result correction apparatus cuts out a table region from a form image and rearranges the table region;
- FIGS. 4A and 4B are diagrams illustrating examples of a correction screen displayed on the recognition result correction apparatus
- FIG. 5 is a diagram illustrating an example of a corrected checking screen displayed on the recognition result correction apparatus
- FIG. 6 is a diagram illustrating an example of the hardware configuration of a recognition result correction apparatus according to an exemplary embodiment of the present disclosure
- FIG. 7 is a block diagram illustrating an example of the functional configuration of a recognition result correction apparatus according to an exemplary embodiment of the present disclosure.
- FIG. 8 is a flowchart illustrating an example of an operation of a recognition result correction apparatus according to an exemplary embodiment of the present disclosure.
- FIG. 1 is a diagram illustrating an example of the entire configuration of a form processing system 1 according to an exemplary embodiment of the present disclosure.
- the form processing system 1 includes an image reading apparatus 10 , a form recognition apparatus 20 , a recognition result correction apparatus 30 , and a form processing apparatus 50 that are connected to one another by a communication line 80 .
- an image reading apparatus 10 , a form recognition apparatus 20 , a recognition result correction apparatus 30 , and a form processing apparatus 50 are illustrated.
- two or more image reading apparatuses 10 , two or more form recognition apparatuses 20 , two or more recognition result correction apparatuses 30 , and two or more form processing apparatuses 50 may be provided.
- the image reading apparatus 10 is an apparatus that reads an image recorded on a recording medium such as paper.
- the image reading apparatus 10 is, for example, a scanner.
- the image reading apparatus 10 of a charge coupled devices (CCD) type that reduces, with a lens, reflected light for light applied to a document from a light source and receives, with a CCD, the reduced reflected light, a contact image sensor (CIS) type that receives, with a CIS, reflected light for light sequentially applied to a document from a light-emitting diode (LED) light source, or the like may be used.
- the image reading apparatus 10 reads a form, in particular, that includes a fill-in field in which a symbol such as a character, number, or the like is handwritten or printed.
- the form recognition apparatus 20 is an apparatus that recognizes fill-in fields and symbols such as characters and numbers filled in the fill-in fields of a form read by the image reading apparatus 10 .
- the form recognition apparatus 20 recognizes the table structure.
- the recognition result correction apparatus 30 is an apparatus that corrects a result of recognition of a form by the form recognition apparatus 20 (hereinafter, referred to as a form recognition result).
- the recognition result correction apparatus 30 displays a correction screen for correcting a form recognition result.
- the recognition result correction apparatus 30 corrects the form recognition result.
- the form processing apparatus 50 is an apparatus that performs predetermined processing on a form recognition result obtained by the form recognition apparatus 20 or a corrected form recognition result obtained by the recognition result correction apparatus 30 , the corrected form recognition result being obtained by correcting the form recognition result obtained by the form recognition apparatus 20 .
- the predetermined processing includes, for example, processing for storing an item associated with a fill-in field of a form and text data obtained by recognizing characters or the like filled in the fill-in field in association with each other into a database.
- the recognition result correction apparatus 30 displays a screen on which a table image containing a table and a recognition result of the table image are arranged for each row or column of the table in an alternate manner.
- a case where a screen on which a table image containing a table and a recognition result of the table image are arranged for each row of the table in an alternate manner is displayed and a case where a screen on which a table image containing a table and a recognition result of the table image are arranged for each column of the table in an alternate manner is displayed are the same with the exception of the difference between rows and columns. Therefore, only the former case will be described below. By replacing rows with columns in the former case, the latter case is obtained. That is, hereinafter, in an exemplary embodiment, “row” will be replaced with “row or column”.
- FIGS. 2A and 2B are diagrams illustrating how the recognition result correction apparatus 30 configurates a table region based on a form recognition result 410 .
- the table region configurated as illustrated in FIG. 2B will be referred to as a “table recognition result 420 ”.
- FIG. 2A illustrates an example of the form recognition result 410 that the recognition result correction apparatus 30 has acquired from the form recognition apparatus 20 .
- the form recognition result 410 is described in, for example, eXtensible Markup language (XML).
- description rows 411 to 414 represent recognition results of the first row of the table. Specifically, four cells are recognized from the first row of the table. Characters “A 1 ”, “M 2 ”, “A 3 ”, and “A 4 ” are recognized from the first cell, the second cell, the third cell, and the fourth cell, respectively.
- description rows 415 to 419 represent recognition results of the second row of the table. Specifically, five cells are recognized from the second row of the table. Characters “B 1 ” and “B 2 ” are recognized from the first cell and the second cell, respectively, no character is recognized from the third cell, and characters “B 3 ” and “B 4 ” are recognized from the fourth cell and the fifth cell, respectively.
- FIG. 2B illustrates an example of the table recognition result 420 configurated based on the form recognition result 410 by the recognition result correction apparatus 30 .
- a row recognition result 421 of the table recognition result 420 includes four cells in accordance with the description rows 411 to 414 in FIG. 2A , and the characters “A 1 ”, “M 2 ”, “A 3 ”, and “A 4 ” are set in the first cell, the second cell, the third cell, and the fourth cell, respectively.
- a row recognition result 422 of the table recognition result 420 includes five cells in accordance with the description rows 415 to 419 in FIG. 2A , and the characters “B 1 ” and “B 2 ” are set in the first cell and the second cell, respectively, no character is set in the third cell, and the characters “B 3 ” and “B 4 ” are set in the fourth cell and the fifth cell, respectively.
- the row recognition result 423 includes four cells, and the characters “C 1 ”, “C 2 ”, “C 3 ”, and “C 4 ” are set in the first cell, the second cell, the third cell, and the fourth cell, respectively.
- description rows corresponding to a row recognition result 424 of the table recognition result 420 are not illustrated.
- the row recognition result 424 includes five cells, and characters “D 1 ”, “D 2 ”, “D 3 ”, “D”, and “ 4 ” are set in the first cell, the second cell, the third cell, the fourth cell, and the fifth cell, respectively.
- the row recognition result 425 includes four cells, and characters “E 1 ”, “E 2 ”, “E 3 ”, and “E 4 ” are set in the first cell, the second cell, the third cell, and the fourth cell, respectively.
- FIGS. 3A and 3B are diagrams illustrating how the recognition result correction apparatus 30 cuts out an image of a table region from a form image 430 and rearranges the image.
- the image of the table region cut out as illustrated in FIG. 3B will be referred to as a “table region image 440 ”.
- FIG. 3A illustrates an example of the form image 430 that the recognition result correction apparatus 30 has acquired from the image reading apparatus 10 .
- a range 431 of the table recognition result 420 illustrated in FIG. 2B is also illustrated. That is, in this example, the range 431 of the table recognition result 420 contains a region in which no cell of the table is present, and the range 431 of the table recognition result 420 does not contain the sixth row of the table. This is because there is a row mistakenly recognized as including five cells and the sixth row is not recognized.
- FIG. 3B illustrates an example of the table region image 440 cut out for individual rows from the form image 430 and rearranged by the recognition result correction apparatus 30 .
- the recognition result correction apparatus 30 cuts out row region images 441 to 445 corresponding to the row recognition results 421 to 425 in FIG. 2B , respectively.
- the recognition result correction apparatus 30 does not delete an image of a region other than the table from the form image 430 . Some parts of the image of the region other than the table are not deleted. Specifically, instead of cutting out only an image of the table, an image of a region around the table as well as the image of the table is cut out. As the image of the region around the table, an image of a region corresponding to a predetermined ratio of the size of the table may be considered. As described above, by cutting out the image of the region around the table as well, a situation in which a part to be compared with the image of the table is lost due to mistakenly recognizing the number of columns at the time of cutting out the image of the table may be avoided. In FIG.
- a region image 441 a , region images 441 b to 445 b , region images 441 c to 445 c , and a region image 445 d are cut out.
- an image of a region corresponding to the predetermined ratio of the size of the table is set as the image of the region around the table.
- the ratio of the region image 441 a and the region image 445 d is higher than the ratio of the region images 441 b to 445 b and the region images 441 c to 445 c .
- the region image 445 d is separated from the row region image 445 .
- FIGS. 4A and 4B are diagrams illustrating examples of a correction screen 450 displayed on the recognition result correction apparatus 30 .
- the correction screen 450 is a screen used by a user to correct the table recognition result 420 .
- FIG. 4A illustrates the correction screen 450 displayed before a user performs correction.
- the recognition result correction apparatus 30 generates the correction screen 450 by inserting the row recognition results 421 to 425 illustrated in FIG. 2B into between the row region images 441 to 445 and the region image 445 d illustrated in FIG. 3B .
- the recognition result correction apparatus 30 highlights a part where the inconsistency is present. After the inconsistency is eliminated, highlighting is removed. In FIGS. 4A and 4B , highlighting is represented by diagonal hatching.
- the recognition result correction apparatus 30 displays a correction candidate display field 452 including correction candidates for the recognition result. Then, the user performs an operation for selecting one of the correction candidates included in the correction candidate display field 452 and correcting the original recognition result of the characters.
- the row recognition result 422 is supposed to include four cells but actually includes five cells, and the entire row corresponding to the row recognition result 422 is highlighted.
- the user performs an operation for deleting the third cell in which no character is set, so that the number of cells is corrected to four.
- the row recognition result 424 is supposed to include four cells but actually includes five cells, and the entire row corresponding to the row recognition result 424 is highlighted.
- the user performs an operation for merging the fourth cell with the fifth cell, characters being set in the fourth cell and the fifth cell in a separated manner, so that the number of cells is corrected to four.
- the recognition result correction apparatus 30 displays a row addition mark 451 above the image of the first row of the row region image 441 . In the case where no row above the first row is recognized, the user operates the row addition mark 451 to add a row not recognized.
- the recognition result correction apparatus 30 also displays a row addition mark 455 below the image of the fifth row of the row recognition result 425 .
- the user operates the row addition mark 455 to add a row not recognized.
- the sixth row is not recognized.
- the user operates the row addition mark 455 to add the sixth row.
- FIG. 4B illustrates the correction screen 450 displayed after the user performs correction as described above.
- the recognition result correction apparatus 30 By arranging row images 461 to 465 cut out from the form image 430 illustrated in FIG. 3A in accordance with corrected row recognition results 471 to 475 obtained by correcting the row recognition results 421 to 425 in FIG. 4A and the corrected row recognition results 471 to 475 in an alternate manner, the recognition result correction apparatus 30 generates the correction screen 450 obtained after correction.
- the recognition result correction apparatus 30 displays a correction confirmation screen 470 for confirming the corrected table recognition result 420 .
- FIG. 5 is a diagram illustrating an example of the correction confirmation screen 470 displayed on the recognition result correction apparatus 30 .
- the correction confirmation screen 470 includes an uncorrected table recognition result 480 and a corrected table recognition result 490 .
- the recognition result correction apparatus 30 highlights parts that are changed by correction in the uncorrected table recognition result 480 and the corrected table recognition result 490 .
- highlighting is represented by diagonal hatching.
- characters “M 2 ” in the second cell of an uncorrected row recognition result 481 of the uncorrected table recognition result 480 are changed to characters “A 2 ” in the second cell of a corrected row recognition result 491 of the corrected table recognition result 490 .
- this cell is highlighted in the uncorrected table recognition result 480 and the corrected table recognition result 490 .
- a cell in which no character is set in an uncorrected row recognition result 482 of the uncorrected table recognition result 480 is deleted in a corrected row recognition result 492 of the corrected table recognition result 490 .
- the deleted cell and a part where the deleted cell was present are highlighted in the uncorrected table recognition result 480 and the corrected table recognition result 490 , respectively.
- a cell in which a character “D” is set and a cell in which a character “ 4 ” is set in an uncorrected row recognition result 484 of the uncorrected table recognition result 480 are merged into a cell in which characters “D 4 ” are set in a corrected row recognition result 494 of the corrected table recognition result 490 .
- the two cells before merging and the single cell into which the two cells are merged are highlighted in the uncorrected table recognition result 480 and the corrected table recognition result 490 , respectively.
- a corrected row recognition result 496 is added to below a corrected row recognition result 495 of the corrected table recognition result 490 .
- the corrected row recognition result 496 is highlighted in the corrected table recognition result 490 .
- the uncorrected table recognition result 480 is displayed along with the corrected table recognition result 490 on the correction confirmation screen 470 , the present disclosure is not limited to this arrangement.
- the uncorrected table recognition result 480 is not necessarily displayed along with the corrected table recognition result 490 , and only the corrected table recognition result 490 may be displayed.
- FIG. 6 is a diagram illustrating an example of the hardware configuration of the recognition result correction apparatus 30 according to an exemplary embodiment.
- the recognition result correction apparatus 30 includes a processor 31 as operation means and a main memory 32 and a hard disk drive (HDD) 33 as storing means.
- the processor 31 executes various types of software such as an operating system (OS) and applications and implements functions described later.
- the main memory 32 is a memory region in which the various types of software and data and the like used for execution of the various types of software are stored.
- the HDD 33 is a memory region in which input data to be input to the various types of software, output data output from the various types of software, and the like are stored.
- the recognition result correction apparatus 30 also includes a communication interface (I/F) 34 for performing communication with the outside of the recognition result correction apparatus 30 , a display device 35 such as a display, and an input device 36 such as a keyboard and a mouse.
- I/F communication interface
- FIG. 7 is a block diagram illustrating an example of the functional configuration of the recognition result correction apparatus 30 according to an exemplary embodiment.
- the recognition result correction apparatus 30 according to an exemplary embodiment is an example of an information processing apparatus.
- the recognition result correction apparatus 30 includes a reception unit 41 , a form recognition result storing unit 42 , a form image storing unit 43 , a table recognition result generation unit 44 , a table region image generation unit 45 , a screen generation unit 46 , a display control unit 47 , an operation receiving unit 48 , and a transmission unit 49 .
- the reception unit 41 receives the form recognition result 410 from the form recognition apparatus 20 .
- the reception unit 41 also receives the form image 430 from the image reading apparatus 10 .
- the form recognition result storing unit 42 stores the form recognition result 410 received by the reception unit 41 .
- the form image storing unit 43 stores the form image 430 received by the reception unit 41 .
- the table recognition result generation unit 44 generates the table recognition result 420 based on the form recognition result 410 stored in the form recognition result storing unit 42 .
- the table recognition result 420 is used as a recognition result of a table image containing a table, and the table recognition result generation unit 44 is provided as an example of means for acquiring the recognition result of the table image.
- the table recognition result generation unit 44 corrects the form recognition result 410 stored in the form recognition result storing unit 42 on the basis of contents of the corrected row recognition result.
- the table region image generation unit 45 cuts out the table region image 440 , based on coordinates of a table region stored in the form recognition result storing unit 42 , from the form image 430 stored in the form image storing unit 43 . In this processing, the table region image generation unit 45 also cuts out a row region image for each row from the table region image 440 .
- the table region image generation unit 45 cuts out a row image, based on coordinates of the corrected row in the form recognition result 410 .
- the screen generation unit 46 generates the correction screen 450 for correcting the table recognition result 420 , on the basis of the table region image 440 generated by the table region image generation unit 45 and the table recognition result 420 generated by the table recognition result generation unit 44 .
- the correction screen 450 on which row region images cut out from the table region image 440 and row recognition results cut out from the table recognition result 420 are arranged in an alternate manner is generated.
- the correction screen 450 is used as an example of a screen on which a table image and a recognition result are arranged for each row of the table in an alternate manner
- the screen generation unit 46 is provided as an example of means for performing control such that this screen is displayed.
- the screen generation unit 46 identifies a part of the table recognition result 420 in which an inconsistency is present, and highlights the part including the inconsistency on the correction screen 450 .
- the screen generation unit 46 is provided as an example of means for performing control such that a part including an inconsistency between the row structure in a table image and the row structure in a recognition result is displayed in a manner different from the other parts.
- the part including the inconsistency may be identified using, for example, methods described below.
- the row is identified as a part including an inconsistency.
- the number of columns of the table may be specified by a user operation received by the operation receiving unit 48 .
- the screen generation unit 46 is an example of means for identifying a part including an inconsistency on the basis of the number of columns of a table input by an operator.
- the number of columns of a table may be acquired by analyzing the table recognition result 420 .
- the number of columns may be defined as the number of columns of the table.
- the screen generation unit 46 is an example of means for identifying a part including an inconsistency on the basis of the number of columns of a table acquired by analyzing a recognition result.
- the cell is identified as a part including an inconsistency.
- the screen generation unit 46 when the operation receiving unit 48 receives a user operation for correcting the table recognition result 420 on the correction screen 450 , the screen generation unit 46 generates the correction screen 450 on which the table recognition result 420 is corrected. Specifically, as illustrated in FIG. 4B , the screen generation unit 46 generates the correction screen 450 on which row images cut out by the table region image generation unit 45 and corrected row recognition results are arranged alternately.
- a corrected row recognition result is used as an example of a corrected recognition result obtained by correcting a recognition result
- the screen generation unit 46 is provided as an example of means for performing control such that a corrected recognition result is displayed in accordance with an operation performed by an operator for correcting a recognition result. Then, the screen generation unit 46 notifies the table recognition result generation unit 44 of the corrected row recognition result.
- the screen generation unit 46 may generate the correction confirmation screen 470 on which no row images are arranged and only corrected row recognition results are arranged.
- the screen generation unit 46 is an example of means for performing control such that an image cut out from a table image is not displayed when a corrected recognition result is displayed.
- the screen generation unit 46 may generate the correction confirmation screen 470 on which the uncorrected table recognition result 480 is arranged along with the corrected table recognition result 490 .
- the screen generation unit 46 is an example of means for performing control such that an uncorrected recognition result is displayed along with a corrected recognition result when the corrected recognition result is displayed.
- the screen generation unit 46 may highlight changed parts in the uncorrected table recognition result 480 and the corrected table recognition result 490 on the correction confirmation screen 470 .
- the screen generation unit 46 is an example of means for performing control such that an uncorrected recognition result is displayed along with a corrected recognition result when the corrected recognition result is displayed.
- the display control unit 47 controls the display device 35 to display the correction screen 450 and the correction confirmation screen 470 generated by the screen generation unit 46 .
- the operation receiving unit 48 receives a user operation for correcting a table recognition result on the correction screen 450 . Furthermore, the operation receiving unit 48 may receive a user operation for specifying the number of columns of a table in a form.
- the transmission unit 49 transmits the form recognition result 410 to the form processing apparatus 50 .
- the reception unit 41 first receives the form recognition result 410 from the form recognition apparatus 20 and stores the form recognition result 410 into the form recognition result storing unit 42 , and the reception unit 41 receives the form image 430 from the image reading apparatus 10 and stores the form image 430 into the form image storing unit 43 . Then, the recognition result correction apparatus 30 generates and displays the correction screen 450 for correcting the form recognition result 410 stored in the form recognition result storing unit 42 .
- FIG. 8 is a flowchart illustrating an example of a process performed by the recognition result correction apparatus 30 at this time.
- attention is paid to, out of various corrections on the form recognition result 410 , an operation on the correction screen 450 for correcting the table recognition result 420 .
- the table recognition result generation unit 44 first determines whether or not the form recognition result 410 stored in the form recognition result storing unit 42 contains a table (step S 301 ).
- the table recognition result generation unit 44 ends the process.
- the table recognition result generation unit 44 In the case where it is determined that the form recognition result 410 contains a table, the table recognition result generation unit 44 generates the table recognition result 420 based on the form recognition result 410 (step S 302 ).
- the table region image generation unit 45 generates the table region image 440 based on the form image 430 stored in the form image storing unit 43 (step S 303 ). Specifically, by cutting out, based on coordinates of a table region stored in the form recognition result storing unit 42 , from the form image 430 stored in the form image storing unit 43 , the table region image generation unit 45 generates the table region image 440 .
- the table region image generation unit 45 generates row region images based on the table region image 440 generated in step S 303 (step S 304 ). Specifically, by cutting out from the table region image 440 for individual rows, the table region image generation unit 45 generates the row region images.
- the table region image generation unit 45 rearranges the row region images generated in step S 304 with spaces therebetween (step S 305 ).
- the screen generation unit 46 generates the correction screen 450 (step S 306 ).
- the row recognition results are obtained by dividing the table recognition result 420 generated in step S 302 in units of rows.
- the screen generation unit 46 highlights a part having a problem on the correction screen 450 generated in step S 306 (step S 307 ). Specifically, in the case where the number of columns of the table is different from the number of columns in a row of the table recognition result 420 , the screen generation unit 46 highlights the row as a part having a problem. Alternatively, in the case where the reliability of a recognition result of characters in a cell in the table recognition result 420 is equal to or less than a predetermined threshold, the cell is identified as a part having a problem.
- the display control unit 47 controls the display device 35 to display the correction screen 450 , which is generated in step S 306 , on which the part having a problem is highlighted in step S 307 (step S 308 ).
- the table recognition result generation unit 44 determines whether or not there is an unprocessed table in the form recognition result 410 (S 309 ).
- the table recognition result generation unit 44 causes the process to return to step S 302 and performs the processing of steps S 302 to S 308 for the table.
- the table recognition result generation unit 44 causes the process to end.
- a user operation for correcting a row recognition result is performed on the correction screen 450 displayed as described above.
- the operation receiving unit 48 first receives the user operation. Then, the screen generation unit 46 notifies the table recognition result generation unit 44 of a corrected row recognition result obtained by correcting the row recognition result. Then, the table recognition result generation unit 44 corrects the form recognition result 410 stored in the form recognition result storing unit 42 on the basis of contents of the corrected row recognition result.
- the table region image generation unit 45 cuts out a row image based on coordinates of a row after the correction in the form recognition result 410 from the form image 430 stored in the form image storing unit 43 .
- the screen generation unit 46 generates the correction screen 450 on which the row image cut out by the table region image generation unit 45 and the corrected row recognition result are arranged alternately.
- the display control unit 47 controls the display device 35 to display the correction screen 450 .
- the transmission unit 49 transmits the form recognition result 410 stored in the form recognition result storing unit 42 and corrected to the form processing apparatus 50 .
- the transmission unit 49 directly transmits the form recognition result 410 stored in the form recognition result storing unit 42 to the form processing apparatus 50 .
- the process performed by the recognition result correction apparatus 30 is configured as a program such as application software or the like.
- a program implementing an exemplary embodiment is regarded as a program for causing a computer to implement a function for acquiring a recognition result of a table image containing a table and a function for performing control such that a screen on which a table image and a recognition result are arranged for each row or column of the table in an alternate manner.
- a program implementing an exemplary embodiment is not only able to be provided by communication means but also able to be stored in a recording medium such as a compact disc-read only memory (CD-ROM) and provided.
- a recording medium such as a compact disc-read only memory (CD-ROM) and provided.
- processor refers to hardware in a broad sense.
- the processor includes general processors (e.g., CPU: Central Processing Unit), dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Integrated Circuit, FPGA: Field Programmable Gate Array, and programmable logic device).
- general processors e.g., CPU: Central Processing Unit
- dedicated processors e.g., GPU: Graphics Processing Unit
- ASIC Application Integrated Circuit
- FPGA Field Programmable Gate Array
- programmable logic device e.g., programmable logic device
- processor is broad enough to encompass one processor or plural processors in collaboration which are located physically apart from each other but may work cooperatively.
- the order of operations of the processor is not limited to one described in the embodiments above, and may be changed.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Character Discrimination (AREA)
- Character Input (AREA)
Abstract
Description
Claims (11)
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JPJP2020-007235 | 2020-01-21 | ||
JP2020-007235 | 2020-01-21 | ||
JP2020007235A JP2021114211A (en) | 2020-01-21 | 2020-01-21 | Information processing device and program |
Publications (2)
Publication Number | Publication Date |
---|---|
US20210224529A1 US20210224529A1 (en) | 2021-07-22 |
US11410444B2 true US11410444B2 (en) | 2022-08-09 |
Family
ID=76857118
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/018,290 Active 2040-09-26 US11410444B2 (en) | 2020-01-21 | 2020-09-11 | Information processing apparatus and non-transitory computer readable medium for arranging table image and recognition result |
Country Status (3)
Country | Link |
---|---|
US (1) | US11410444B2 (en) |
JP (1) | JP2021114211A (en) |
CN (1) | CN113221608A (en) |
Citations (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5420695A (en) * | 1992-03-05 | 1995-05-30 | Ricoh Company, Ltd. | Image reading and forming apparatus for identifying and correcting the gridlines of a table |
JP2000090195A (en) | 1998-09-11 | 2000-03-31 | Canon Inc | Method and device for table recognition |
US20080040655A1 (en) * | 2006-08-14 | 2008-02-14 | Fujitsu Limited | Table data processing method and apparatus |
US20090110282A1 (en) * | 2007-10-31 | 2009-04-30 | Fujitsu Limited | Image recognition apparatus, image recognition method, and storage medium recording image recognition program |
US20100008578A1 (en) * | 2008-06-20 | 2010-01-14 | Fujitsu Frontech Limited | Form recognition apparatus, method, database generation apparatus, method, and storage medium |
JP2013012223A (en) | 2012-08-29 | 2013-01-17 | Canon Software Inc | Image processing device and method, and program |
US20170116179A1 (en) * | 2014-06-06 | 2017-04-27 | Maud GAGNÉ-LANGEVIN | System and method for generating task-embedded documents |
US20170262722A1 (en) * | 2016-03-09 | 2017-09-14 | Canon Kabushiki Kaisha | Information processing apparatus, program, and information processing method |
US20190171704A1 (en) * | 2017-12-01 | 2019-06-06 | International Business Machines Corporation | Heuristic Domain Targeted Table Detection and Extraction Technique |
US10607381B2 (en) * | 2014-07-07 | 2020-03-31 | Canon Kabushiki Kaisha | Information processing apparatus |
US20210042555A1 (en) * | 2019-08-09 | 2021-02-11 | Hitachi, Ltd. | Information Processing Apparatus and Table Recognition Method |
US20210209297A1 (en) * | 2018-06-29 | 2021-07-08 | Microsoft Technology Licensing, Llc | Table detection in spreadsheet |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH05120472A (en) * | 1991-10-28 | 1993-05-18 | Sharp Corp | Character recognizing device |
JPH06274680A (en) * | 1993-03-17 | 1994-09-30 | Hitachi Ltd | Method and system recognizing document |
JPH08329187A (en) * | 1995-06-06 | 1996-12-13 | Oki Electric Ind Co Ltd | Document reader |
-
2020
- 2020-01-21 JP JP2020007235A patent/JP2021114211A/en active Pending
- 2020-09-07 CN CN202010926832.1A patent/CN113221608A/en active Pending
- 2020-09-11 US US17/018,290 patent/US11410444B2/en active Active
Patent Citations (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5420695A (en) * | 1992-03-05 | 1995-05-30 | Ricoh Company, Ltd. | Image reading and forming apparatus for identifying and correcting the gridlines of a table |
JP2000090195A (en) | 1998-09-11 | 2000-03-31 | Canon Inc | Method and device for table recognition |
US20030123727A1 (en) * | 1998-09-11 | 2003-07-03 | Tomotoshi Kanatsu | Table recognition method and apparatus, and storage medium |
US6628832B2 (en) | 1998-09-11 | 2003-09-30 | Canon Kabushiki Kaisha | Table recognition method and apparatus, and storage medium |
US20080040655A1 (en) * | 2006-08-14 | 2008-02-14 | Fujitsu Limited | Table data processing method and apparatus |
US20090110282A1 (en) * | 2007-10-31 | 2009-04-30 | Fujitsu Limited | Image recognition apparatus, image recognition method, and storage medium recording image recognition program |
US20100008578A1 (en) * | 2008-06-20 | 2010-01-14 | Fujitsu Frontech Limited | Form recognition apparatus, method, database generation apparatus, method, and storage medium |
JP2013012223A (en) | 2012-08-29 | 2013-01-17 | Canon Software Inc | Image processing device and method, and program |
US20170116179A1 (en) * | 2014-06-06 | 2017-04-27 | Maud GAGNÉ-LANGEVIN | System and method for generating task-embedded documents |
US10607381B2 (en) * | 2014-07-07 | 2020-03-31 | Canon Kabushiki Kaisha | Information processing apparatus |
US20170262722A1 (en) * | 2016-03-09 | 2017-09-14 | Canon Kabushiki Kaisha | Information processing apparatus, program, and information processing method |
US20190171704A1 (en) * | 2017-12-01 | 2019-06-06 | International Business Machines Corporation | Heuristic Domain Targeted Table Detection and Extraction Technique |
US20210209297A1 (en) * | 2018-06-29 | 2021-07-08 | Microsoft Technology Licensing, Llc | Table detection in spreadsheet |
US20210042555A1 (en) * | 2019-08-09 | 2021-02-11 | Hitachi, Ltd. | Information Processing Apparatus and Table Recognition Method |
Also Published As
Publication number | Publication date |
---|---|
JP2021114211A (en) | 2021-08-05 |
CN113221608A (en) | 2021-08-06 |
US20210224529A1 (en) | 2021-07-22 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
KR20190033451A (en) | Image processing apparatus, image processing method, and storage medium | |
US8675260B2 (en) | Image processing method and apparatus, and document management server, performing character recognition on a difference image | |
US10417516B2 (en) | System and method for preprocessing images to improve OCR efficacy | |
US11348331B2 (en) | Information processing apparatus and non-transitory computer readable medium | |
US8682075B2 (en) | Removing character from text in non-image form where location of character in image of text falls outside of valid content boundary | |
JP5771108B2 (en) | System, method, and program for supporting proofreading of text data generated by optical character recognition | |
US11080472B2 (en) | Input processing method and input processing device | |
US11240399B2 (en) | Image processing apparatus | |
US11410444B2 (en) | Information processing apparatus and non-transitory computer readable medium for arranging table image and recognition result | |
JP2010061471A (en) | Character recognition device and program | |
US11055551B2 (en) | Correction support device and correction support program for optical character recognition result | |
JP2011237905A (en) | Data entry system and data entry method | |
US20200021711A1 (en) | Image processing apparatus | |
JP2019169182A (en) | Information processing device, control method, and program | |
US20180032244A1 (en) | Input control device, input control method, character correction device, and character correction method | |
JP2010039783A (en) | Device, system, method and program of document processing | |
US11170211B2 (en) | Information processing apparatus for extracting portions filled with characters from completed document without user intervention and non-transitory computer readable medium | |
US20210042555A1 (en) | Information Processing Apparatus and Table Recognition Method | |
JP7019963B2 (en) | Character string area / character rectangle extraction device, character string area / character rectangle extraction method, and program | |
JP4357226B2 (en) | Form definition device, form definition method, and form definition program | |
US20240106938A1 (en) | Information processing system, method, and non-transitory computer readable medium | |
US20220383023A1 (en) | Information processing apparatus, non-transitory computer readable medium storing program, and information processing method | |
US11659106B2 (en) | Information processing apparatus, non-transitory computer readable medium, and character recognition system | |
US11574490B2 (en) | Information processing apparatus and non-transitory computer readable medium storing information processing program | |
JP6869394B1 (en) | Verification device, verification method, and verification program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: FUJI XEROX CO., LTD., JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:TANAKA, TAKUMI;REEL/FRAME:053747/0153 Effective date: 20200730 |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: APPLICATION DISPATCHED FROM PREEXAM, NOT YET DOCKETED |
|
AS | Assignment |
Owner name: FUJIFILM BUSINESS INNOVATION CORP., JAPAN Free format text: CHANGE OF NAME;ASSIGNOR:FUJI XEROX CO., LTD.;REEL/FRAME:056078/0098 Effective date: 20210401 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |