JP6536542B2 - Information processing apparatus, control method, program - Google Patents

Description

  BACKGROUND OF THE INVENTION Field of the Invention The present invention relates to an image processing technique capable of easily acquiring a value of a desired part of a document image.

  Conventionally, various image processing techniques are disclosed. Among them, a method has been proposed in which a form such as a report or slip is read by a scanner and the value of the necessary item is recognized and acquired from the read-in data using an OCR (Optical Character Recognition) technology. .

  Patent Document 1 describes a method of improving the recognition accuracy by setting the display position, dictionary, and importance of each item to be recognized, and determining the priority order of reading of the item according to the importance and whether to skip reading or not. ing.

Unexamined-Japanese-Patent No. 2004-164217

  However, the method described in Patent Document 1 is a method that can be applied only to fixed form slips in which the display position for each item is defined. For example, in the case of financial reports etc. having different forms for each company, each form of form It is necessary to create configuration information, which is very time-consuming.

  In addition, when you look at the display method on the form for items for which values are to be acquired, the display format is various, such as that values are displayed immediately next to the item name or values are displayed across reference information. It is.

  It is desirable to identify a portion where a desired item is displayed from a form (document) having various formats as described above and obtain a value of the item.

Therefore, an object of the present invention is to provide a mechanism capable of setting item value acquisition for documents of a plurality of formats, and easily acquiring values of desired items from documents of a plurality of formats .

According to the present invention, in a document in which a plurality of key character strings and character strings related to the key character strings are arranged, positions of an area for acquiring the related character strings from the document and positions of the key character strings in the document An information processing apparatus including an acquisition condition storage unit that stores an acquisition condition including a relationship, which is arranged in a predetermined direction from the excluded character string in the document and an excluded character string storage unit that stores an excluded character string An exclusion area specifying unit that specifies an exclusion area that is an area including the character string; an identification unit that specifies an area excluding the exclusion area among the acquisition areas that conform to the acquisition condition ; And acquiring means for acquiring a character string meeting the acquisition condition from the identified area.

According to the present invention, it is possible to set item value acquisition for documents of a plurality of formats, and it is possible to provide a mechanism capable of easily acquiring the value of a desired item from a document of a plurality of formats .

It is a figure showing an example of composition of an image processing system. It is a block diagram which shows schematic structure of PC201. It is a flowchart which shows the outline of the image processing performed by an image processing system. It is a flowchart of value acquisition processing. It is a data figure showing an example of block information. It is a data figure showing an example of character recognition field information. It is a data figure showing an example of a setting file. It is a data figure showing an example of a setting file. It is an image figure showing an example of a document picture to read. It is an image figure at the time of displaying block information on a document image. It is an image figure at the time of displaying on a document image as character recognition area | region information as a selection area. It is a figure which shows an example of the screen which displays a recognition result. It is a figure which shows an example of the screen which displays a recognition result. It is an image figure explaining an example of value acquisition processing.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a view showing a configuration example of an image processing system to which an image processing method according to an embodiment of the present invention is applied.

  In FIG. 1, the image processing system includes, for example, a personal computer (PC) 201 as an information processing apparatus, a scanner 202 as an image reading apparatus, and a printer 203 as a printing apparatus. Connected to each other.

  The scanner 202 can optically read and digitize a paper document, and send the image data to the PC 201. The PC 201 executes predetermined image processing on the received image data. At this time, the operator can operate a keyboard, a mouse or the like to confirm and correct the processing result. The printer 203 receives, from the PC 201, image data on which predetermined image processing has been performed, and performs printing.

  The network 204 is realized by the Internet, LAN or WAN, a telephone line, a dedicated digital line, an ATM or frame relay line, a communication satellite line, a cable television line, a wireless line for data broadcasting, or a combination thereof. It is a so-called communication network, as long as it can transmit and receive data.

  Although the image processing method according to the embodiment of the present invention will be described as applied to the illustrated image processing system, the present invention is not limited to this, and is applied to a multifunction device in which a scanner and a printer are integrally configured. It may be. Further, the PC 201 may execute the image processing method of the present invention not only on image data input from the scanner 202 but also on document image data captured by a digital camera or the like. It does not limit the method.

  FIG. 2 is a block diagram showing a schematic configuration of the PC 201 of FIG.

  In the PC 201, the CPU 101 controls the entire apparatus in accordance with a control program stored in the ROM 102. The ROM 102 stores various programs including control programs such as processing to be described later which the CPU 101 executes, and various parameter data. The RAM 103 temporarily stores a program loaded from the storage device 104, and stores an area image and various data. The RAM 103 also functions as a work area or a temporary save area for data.

  The storage device 104 is configured by, for example, a hard disk, a CD-ROM, etc., and stores various data including a database for managing image data. The display 105 is configured of, for example, an LCD or a CRT. The input device 106 is configured of, for example, a mouse, a keyboard, a pen tablet, and the like.

  A network interface (I / F) 109 communicates with an external device (not limited to the scanner 202 or the printer 203, and not shown server or external storage device, etc.) connected on the network 204 to read programs and data. , Write.

  FIG. 3 is a flowchart showing an outline of image processing executed by the image processing system of FIG. The present process is a process executed by the CPU 101 based on an image processing program in the PC 201. A flowchart showing the detailed processing of FIG. 3 will be described with reference to FIG.

  First, in step S301, the PC 201 controls the scanner 202 to read an image of a paper document, and acquires the image data. Next, the PC 201 performs block selection processing on the image to extract each area of a table, characters, pictures, figures, frames, and lines from the image.

  In block selection processing, the image data of one page read as shown in FIG. 9 is recognized as a block for each object, each block is judged as an attribute such as character / drawing / photograph / line / table, etc. It is processing to divide into the area to have.

  Specifically, first, the input image is binarized into black and white, outline tracking is performed, and a block of pixels surrounded by black pixel outlines is extracted. With regard to a block of large black pixels, outline tracking is performed on the white pixels inside to extract blocks of white pixels, and furthermore, from inside the block of white pixels having a certain area or more, the black pixels are recursively extracted. Extract the mass.

  A block of black pixels obtained in this manner is classified by size and shape, and is classified into regions having different attributes. For example, if the aspect ratio is close to 1 and the size is within a certain range as a pixel block corresponding to characters, further adjacent characters can be aligned well grouped part character area, flat pixel block line area constant The area occupied by black pixel blocks larger than the size and including square white pixel blocks in an orderly arrangement is obtained by the table area, the area in which irregular pixel blocks are scattered is obtained by the photograph area, and other arbitrary block selection processing The block information for each block is shown in FIG. An example of an image corresponding to the block information shown in FIG. 5 is FIG. Although each block information is not displayed on the screen, each block is shown in FIG. 10 for the sake of explanation.

  Among the blocks obtained by the block selection process, a block having a text attribute (type) is obtained, and character recognition processing is performed on this block. The character recognition technology is a known technology and therefore the description thereof is omitted.

  From the character string obtained as a result of character recognition, it is determined whether it is an area extending over a plurality of lines, and in the case of extending over a plurality of lines, it is divided into each line and registered as an area. In the case of one line, it registers as an area as it is. Since the area (coordinates) obtained when characters are recognized can also be acquired, the coordinates are registered. An example of data registered as a region is FIG. Block 4 of FIG. 5 is divided into 13 areas and registered.

  Next, according to the acquisition condition 701 set in the setting file (FIG. 7), the area for acquiring the value is specified. In this specification, character recognition processing is performed on a block obtained by block selection, line division is performed, and an area is registered. From this area, the key area is searched based on the search character string (for example, HDL cholesterol) of the key area of the acquisition condition 701, and from this key area, the search direction (for example, horizontal) is used to Identify the horizontal area as the value acquisition area. If the condition of the time value format (for example, numeral, three digits) is not met, the area next to the search direction is further specified as the value acquisition area.

  At this time, if a plurality of search strings in the configuration file are registered separated by commas as in 702, the second search string (for example, a triglyceride) is found if there is no string matching the first search string. Search for).

  In addition, if a plurality of search directions in the setting file are registered with comma separation as shown in 703, if there is no value matching the first search direction, the second search direction (for example, lower) is searched.

  Furthermore, in addition to the search direction described above, the exclusion column setting condition 801 and the priority column setting condition 803 shown in FIG. 8 determine the column to be excluded and the column to be prioritized, and specify an area for acquiring a value. In the present embodiment, the exclusion column and the priority column are described as a series of vertically arranged items, but depending on the form such as a form, they are described as a series of horizontally arranged items (may be referred to as excluded lines or priority lines). It is also good.

  In the excluded column setting condition 801, it is determined that information different from the current inspection result, such as the inspection reference value or the previous inspection value, is displayed in the column in which the set character string 802 is displayed. It is a condition to set an object.

  The priority column setting condition 803 is a condition for setting an object for which it is determined that the present inspection result is displayed in the column in which the set character string 804 is displayed.

  Then, the character string (number) registered in the area specified as the value acquisition area is acquired as the present inspection result.

  In step S302, the PC 201 displays the character string (number) acquired in step S301 on the display 105 as the inspection result of this time. 12 and 13 show display examples on the display, and the respective screens will be described later. In addition, the data group of the acquired values is exported and output in a CSV file or the like according to an output instruction from the user.

  The value acquisition process of step S301 will be described with reference to FIG.

  In step S401, the PC 201 reads the acquisition condition 701, the exclusion column setting condition 801, and the priority column setting condition 803 from the setting file. The respective conditions are as described in step S301. The setting file to be used may be arbitrarily selected and read by the user.

  In step S402, the PC 201 reads an image acquired from a scanner or an image stored in a predetermined folder.

  In step S403, the PC 201 executes block selection processing using the read image. This block selection is performed using a block selection library. The block selection process is as described in step S301. Also, the block information is the same as in FIG. Furthermore, character recognition processing may be performed by a block selection library.

  In step S404, the PC 201 acquires block information having a text type and a table type from the block selection library.

  In step S405, the PC 201 performs character recognition processing on the acquired block. The character recognition process is a known technique and the description thereof will be omitted, but for example, there is a technique of collating an entry character with a character template using pattern matching to extract a character candidate.

  The character recognition process can acquire a character string (including numbers) and area information (coordinates) of the character string. The area information is an area indicating a frame surrounding a character string. The result of character recognition shall be managed in memory for each block.

  In step S405, character recognition is first performed on all blocks, and the process proceeds to step S406. However, character recognition may be performed for each block and determination in S406 may be performed. That is, the present invention is not limited to the procedure of performing the determination in step S406 after character recognition is performed on all blocks.

  In step S406, the PC 201 determines whether the process has been performed on all blocks. If the process has not been performed on all the blocks, the process moves to step S 407 to obtain a character string of the block to be processed next. If the process has been performed for all blocks, the process moves to step S411.

  In step S407, the PC 201 acquires the character recognition result (character string) of the block to be processed from the memory.

  In step S408, the PC 201 determines whether the acquired character string is a character string of a plurality of lines. If a plurality of character strings have been acquired, the process proceeds to step S410. If a single-line character string has been acquired, the process proceeds to step S409.

  In step S409, the PC 201 stores the character recognition result in the memory. The character recognition result is registered in the character recognition area information of FIG. In FIG. 6, an area ID, a character string obtained from the character recognition result, and area information (coordinates) indicating the position where the character string is present are registered.

  In step S410, the PC 201 divides the area for each line to generate and register character recognition area information. As shown in 601 of FIG. 6, the character string of the block 4 is divided into 13 areas and registered. In area information (coordinates), coordinates of positions where character strings of respective lines are registered.

  In step S411, the PC 201 acquires a search character string (see FIG. 7) of the key area of the acquired acquisition condition 701, and searches for character recognition area information. In accordance with the search string in the key area, the matching area is identified by the character string in the character recognition area information. Thereby, the matched area becomes a key search area.

  In step S412, a search character string (see FIG. 8) of the read exclusion string setting condition 801 is acquired, character recognition area information is searched, and an area having a matching value is specified. At this time, when the area in which the values match is a table area, the column in which the self exists in the table is set as the "excluded column".

  When a plurality of search character strings of the exclusion string setting condition 801 are registered, the above is performed on all the search character strings.

  If the location matched with the search string in the exclusion string is not in tabular form, it is possible to judge an area in the same row as the X coordinate of its own character string as the “deletion string”. In addition, when the excluded column is in the horizontal alignment, it may be determined by the Y coordinate.

  In step S413, a search character string (see FIG. 8) of the read priority column setting condition 803 is acquired, character recognition area information is searched, and an area having a matching value is specified. At this time, when the area in which the values match is a table area, the column in which the self exists in the table is set as the "preferential column".

  When a plurality of search character strings of the priority column setting condition 803 are registered, the above is performed on all the search character strings.

  If the location that matches the search string in the priority column is not in tabular form, it may be determined as a "priority column" an area aligned with the X coordinate of its own character string. In addition, when the priority column is horizontally arranged, it may be determined by the Y coordinate.

  In step S414, the PC 201 acquires the key area and the search direction (for example, right) of the acquisition condition 701, and identifies the area one right from the key area. Based on the coordinates of the key area, a character recognition area information (for example, FIG. 6) is referenced to search for and specify an area having coordinates in the right direction. This specified area is the value acquisition area. Note that among the areas having coordinates in the right direction, the area having the closest coordinates is taken as the first area and the second area in order. When there are a plurality of key areas, the value acquisition areas are specified in accordance with the respective conditions.

  When acquiring a value acquiring area in the above, if there is a “priority column” in the value search direction, the column (for example, 1101 in FIG. 11) is used as a candidate for an acquiring area to preferentially acquire a value. Do. In addition, when there are a plurality of "preceding columns", they are considered as candidate areas for acquiring values in order of priority.

  If there is an “excluded column” in the value search direction, the column (eg, 1102 in FIG. 11) is excluded from the value acquisition area.

  For example, in the case of the example report 1401 shown in FIG. 14, when acquiring the value of the key item “HDL cholesterol”, the column 1402 in which the reference value is displayed does not contain a character string representing an exclusion column and Although it is not determined, the column 1403 including the character string “current” is determined to be the priority column, and the column 1402 is skipped to set the area of the character string “51” of the column 1403 as a value acquisition area.

  Further, in the case of the example report 1411 shown in FIG. 14, the column 1412 including the character string “reference value” is determined to be an exclusion column, and the character strings “40 to 86” in the exclusion column in the search direction are excluded from the target An area of the character string “51” of the column 1413 in which the exclusion column is skipped is used as a value acquisition area. The column 1413 is not determined to be a priority column because it does not include a character string representing the priority column.

  In step S415, the PC 201 acquires a value from the acquisition region of the value identified in step S414. When the character string acquired at this time is different from the value format (see FIG. 7) of the acquired acquisition condition 701, the next character string of the search direction (see FIG. 7) of the acquired acquisition condition 701 is acquired. Repeat this until you get a string that matches the value format, or until the end of the same table.

  In step S416, the PC 201 determines whether a value can be acquired. If the value can be acquired, step S418 is performed. If the value can not be acquired, step S417 is performed.

  In step S417, the PC 201 determines whether values have been acquired for all the search directions set in the search direction of the read acquisition condition 701 (see FIG. 7), and if it is performed, the step is performed. If S418 is not performed, the search direction is set to the next direction, and step S414 is performed.

  In step S418, the PC 201 executes step S419 when acquiring values for all the recognized key items, and processing for acquiring values when there is a key item for which acquisition of values has not been performed (step S414). ~ Execute step S417).

  In step S419, it is determined whether it is an area | region which should acquire a value about what was not able to acquire a value by step S415 among the area | regions specified as a value acquisition area. The determined result is reflected when displaying the result in step S302.

  A specific process will be described in the case where a report etc. captured as an image is in tabular form. If an error occurs for a key item in step S 415 and a value can not be obtained for the key item, another value for which the value can be normally obtained for the same row for the area for which a value was to be obtained If an acquisition area exists, it is determined as a "priority area". Conversely, if there is no other value acquisition area in which values can be acquired normally in the same column, it is determined as a "non-priority area". That is, since there is a high possibility that the column in which the value acquisition area where the value could be acquired normally exists is a column in which items to be acquired are arranged, it is necessary to acquire the value also in the area where an error occurs in the same column. It is judged as the area.

  FIG. 13 shows an example of the determination result. In FIG. 13, with respect to the key item “ALT (GPT)”, two places 1302 and 1303 are specified as value acquisition areas, and in both cases, the value acquisition is an error. In the value acquisition area 1302, since values can be acquired normally in the value acquisition area of the same row for other key items “AST (GOT)” etc., it is determined to be a “priority area”, that is, a key area for which values should be acquired. Ru. On the other hand, in the value acquisition area 1303, since there is no value acquisition area which can acquire values normally in the same row for other key items, it is determined to be a "non-priority area", that is, a key area for which values need not be acquired. Ru.

Also, if the report etc. captured as an image is not in tabular form, “Priority area” if there is another key item whose value could be acquired in an area having an X coordinate close to the X coordinate of the key area that resulted in an error. If there is none, it may be determined as a "non-priority area". Further, in the case of the table format, the area may be specified not by the column but by the row, and when not in the table format, it may be specified by the Y coordinate.

  Next, the screen displayed by the value output process of step S302 will be described.

  FIG. 12 is a screen image of a recognition result screen 1201 which is an example of the screen displayed in step S302.

  The recognition result screen 1201 displays an image 1202 read on the left side and a recognition result list 1203 for each examination item on the right side.

  If there is an item whose value can not be acquired, such as when the key area is blank or there is no character string that conforms to the value format in step S 415, the item is highlighted as an error item (1204, 1205). If there is an error in the recognition result, it is possible for the user to input a correction value into the correction input area (1206). When the correction input area is input, the input value is a registered value.

  Further, the priority column and the exclusion column specified by the priority column setting condition 803 and the exclusion column setting condition 801 may be displayed in a distinguishable manner.

  FIG. 13 is a screen image of a recognition result screen 1301 which is an example of a screen on which the determination result in step S419 is reflected.

  Since the value acquisition area 1302 can not acquire a value in step S415 and is determined to be the "priority area" in step S419, the value acquisition area 1302 and the recognition result 1304 of the value acquisition area 1302 are highlighted.

  On the other hand, since the value acquisition area 1303 can not acquire a value in step S415 and is determined to be a "non-priority area" in step S419, the value acquisition area 1303 and the recognition result 1305 of the value acquisition area 1303 indicate "priority area". "Is displayed in a different format.

  Also, the recognition result screens 1201 and 1301 have output buttons, and output the recognition result and the correction result to the CSV file if there is a correction of the value by the user. When multiple images are read, value acquisition and confirmation are performed for all the images, and when the output button is pressed for the last image, the values are output collectively to the CSV file. . The format of the output file is an example and is not limited.

  By the above, it becomes possible to easily obtain the value of a desired item from a document having a plurality of formats.

  As mentioned above, although one Embodiment was shown, this invention can take the embodiment as a system, an apparatus, a method, a program, a recording medium etc., for example, and, specifically, it is comprised from a some apparatus The present invention may be applied to a single system or to an apparatus comprising a single device. For example, the configuration may be realized in a cloud environment. In that case, the configuration file creation tool is executed on the server on the cloud environment.

  Further, the program in the present invention is a program that can execute the processing method of the flowcharts shown in FIG. 3 to FIG. Note that the storage medium may be configured to store a program that allows a computer to execute the processing method of FIGS. 3 to 4. The program in the present invention may be a program for each processing method of each device in FIGS. 3 to 4.

  As described above, the recording medium recording the program for realizing the functions of the above-described embodiments is supplied to the system or apparatus, and the computer (or CPU or MPU) of the system or apparatus stores the program stored in the recording medium. It goes without saying that the object of the present invention can be achieved also by reading and executing.

  In this case, the program itself read out from the recording medium realizes the novel function of the present invention, and the recording medium storing the program constitutes the present invention.

  As a recording medium for supplying the program, for example, a flexible disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, DVD-ROM, magnetic tape, non-volatile memory card, ROM, EEPROM, silicon Disks, solid state drives, etc. can be used.

  Further, by executing the program read by the computer, not only the functions of the above-described embodiment are realized, but also an operating system (OS) or the like running on the computer is actually executed based on the instructions of the program. It goes without saying that the processing is partially or entirely performed, and the processing realizes the functions of the above-described embodiments.

  Furthermore, after the program read from the recording medium is written to the memory provided to the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function expansion board is read based on the instruction of the program code. It goes without saying that the case where the CPU or the like provided in the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiment are realized by the processing.

  Further, the present invention may be applied to a system constituted by a plurality of devices or to an apparatus comprising a single device. It goes without saying that the present invention can also be applied to the case where it is achieved by supplying a program to a system or apparatus. In this case, by reading a recording medium storing a program for achieving the present invention into the system or apparatus, the system or apparatus can receive the effects of the present invention.

  Further, by downloading and reading out a program for achieving the present invention from a server on a network, a database or the like by a communication program, the system or apparatus can receive the effects of the present invention.

  In addition, the structure which combined each embodiment mentioned above and its modification is also contained in this invention altogether.

101 CPU
102 RAM
104 storage device 201 PC
202 Scanner 203 Printer

Claims (8)

In a document in which a plurality of key character strings and character strings related to the key character string are arranged, acquisition including a positional relationship between an area for acquiring the related character string from the document and the key character string in the document An information processing apparatus comprising acquisition condition storage means for storing conditions, comprising:
Exclusion string storage means for storing exclusion strings;
An exclusion area specifying unit for specifying an exclusion area which is an area including a character string arranged in a predetermined direction from the exclusion character string in the document;
An identification unit that identifies an area excluding the excluded area among the acquired areas that conform to the acquisition condition;
An acquisition unit configured to acquire a character string that conforms to the acquisition condition from the area identified by the identification unit. Priority string storage means for storing a priority string;
Priority area specifying means for specifying a priority area which is an area including character strings arranged in a predetermined direction from the priority character string in the document;
Equipped with
The information processing apparatus according to claim 1, wherein the specifying unit specifies an area included in the priority area among the areas to be acquired that conform to the acquisition condition.   The information processing apparatus according to claim 1, wherein the exclusion area is a column or a row of a table in the document.   The information processing apparatus according to any one of claims 1 to 3, wherein the predetermined direction of the exclusion area is orthogonal to the direction indicated by the positional relationship included in the acquisition condition. In a document in which a plurality of key character strings and character strings related to the key character string are arranged, acquisition including a positional relationship between an area for acquiring the related character string from the document and the key character string in the document An information processing apparatus comprising acquisition condition storage means for storing conditions, comprising:
Priority string storage means for storing a priority string;
Priority area specifying means for specifying a priority area which is an area including character strings arranged in a predetermined direction from the priority character string in the document;
An identification unit that identifies an area included in the priority area among the acquisition areas that conform to the acquisition condition;
An acquisition unit configured to acquire a character string that conforms to the acquisition condition from the area identified by the identification unit. In a document in which a plurality of key character strings and character strings related to the key character string are arranged, acquisition including a positional relationship between an area for acquiring the related character string from the document and the key character string in the document A control method of an information processing apparatus, comprising: acquisition condition storage means for storing conditions; and exclusion character string storage means for storing exclusion character strings,
An exclusion area identification step of identifying an exclusion area which is an area including a character string arranged in a predetermined direction from the exclusion character string in the document;
A specifying step of specifying a region excluding the excluded region among the regions to be acquired that conform to the acquisition condition;
A control method of an information processing apparatus, comprising: an acquisition step of acquiring a character string that meets the acquisition condition from the area specified by the specification unit. In a document in which a plurality of key character strings and character strings related to the key character string are arranged, a position indicating a positional relationship between an area for acquiring the related character string from the document and the key character string in the document A control method of an information processing apparatus, comprising: acquisition condition storage means for storing conditions; and priority character string storage means for storing priority character string information,
An identification step of identifying an area capable of acquiring a character string based on the key character string and the position condition, prioritizing an area specified by the priority character string information;
A control method of an information processing apparatus, comprising: an acquisition step of acquiring the related character string from the specified area in the document. A program for causing an information processing apparatus to function as each unit according to any one of claims 1 to 5.

Images