JP6931168B2 - Information processing device, control method, program - Google Patents

Description

The present invention relates to an image processing technique that makes it possible to easily obtain a value of a desired portion of a document image.

Conventionally, various image processing techniques have been disclosed. Among them, a method has been proposed in which a form such as a report or a slip is captured by a scanner, and the value of a necessary item is recognized and acquired from the captured data by using OCR (Optical Character Recognition) technology. ..

Patent Document 1 describes a method of improving recognition accuracy by setting a display position, a dictionary, and importance for each item to be recognized, and determining the priority order of reading the item and whether or not the item can be skipped according to the importance. ing.

Japanese Unexamined Patent Publication No. 2004-164217

However, the method described in Patent Document 1 is a method that can be applied only to a standard form in which a display position for each item is defined. For example, in the case of a financial report having a different format for each company, each form is different. It is very time-consuming because it is necessary to create the setting information in.

Also, looking at the display method on the form for the item for which the value should be acquired, there are various display forms, such as the value being displayed immediately next to the item name or the value being displayed with the reference information in between. Is.

A method of identifying a place where a desired item is displayed from a form (document) having such various formats and acquiring the value of the item is desired.

Therefore, an object of the present invention is to provide a mechanism capable of appropriately obtaining a desired character string from a document in which a plurality of key character strings and character strings related to the key character string are arranged.

The present invention is an information processing apparatus that acquires a character string in a document in which a plurality of key character strings and character strings related to the key character string are arranged, and acquires the key character string and the character string in the document. character area excluding the area specified by the Jo Tokoro exclusion strings from the area determined based on the acquisition condition for specifying the region, and determining means for determining as a character string acquisition region, which is the decision to It is characterized by including an acquisition means for acquiring a character string from a column acquisition area.

According to the present invention, a desired character string can be appropriately obtained from a document in which a plurality of key character strings and character strings related to the key character strings are arranged.

It is a figure which shows the configuration example of an image processing system. It is a block diagram which shows the schematic structure of PC201. It is a flowchart which shows the outline of the image processing executed by the image processing system. It is a flowchart of a value acquisition process. It is a data figure which shows an example of block information. It is a data figure which shows an example of the character recognition area information. It is a data figure which shows an example of a setting file. It is a data figure which shows an example of a setting file. It is an image diagram which shows an example of the document image to read. It is an image diagram when the block information is displayed on the document image. It is an image diagram when the character recognition area information is displayed on a document image as a selection area. It is a figure which shows an example of the screen which displays the recognition result. It is a figure which shows an example of the screen which displays the recognition result. It is an image diagram explaining an example of a value acquisition process.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a diagram showing a configuration example of an image processing system to which the image processing method according to the embodiment of the present invention is applied.

In FIG. 1, the image processing system includes, for example, a personal computer (PC) 201 as an information processing device, a scanner 202 as an image reading device, and a printer 203 as a printing device, which are connected via a network 204. Connected to each other.

The scanner 202 can optically read a paper document, digitize it, and send the image data to the PC 201. The PC 201 executes a predetermined image processing on the received image data. At that time, the operator can operate the keyboard, mouse, or the like to check and correct the processing result. The printer 203 receives the image data subjected to the predetermined image processing from the PC 201 and prints the image data.

The network 204 is realized by any one of the Internet, LAN and WAN, telephone line, dedicated digital line, ATM and frame relay line, communication satellite line, cable TV line, wireless line for data broadcasting, etc., or a combination thereof. It is a so-called communication network, as long as it can send and receive data.

The form in which the image processing method according to the embodiment of the present invention is applied to the illustrated image processing system will be described, but the present invention is not limited to this, and is applied to a multifunction device in which a scanner and a printer are integrally configured. There may be. Further, the PC 201 is not limited to the image data input from the scanner 202, and the image processing method of the present invention may be executed on the document image data taken by a digital camera or the like, and the input destination or input of the image data may be executed. It does not limit the method.

FIG. 2 is a block diagram showing a schematic configuration of PC 201 of FIG.

In the PC 201, the CPU 101 controls the entire device according to the control program stored in the ROM 102. The ROM 102 stores various programs including control programs such as processes executed by the CPU 101, which will be described later, and various parameter data. The RAM 103 temporarily stores the program loaded from the storage device 104, and stores the area image and various data. Further, the RAM 103 functions as a data work area or a temporary save area.

The storage device 104 is composed of, for example, a hard disk, a CD-ROM, or the like, and stores various data including a database for managing image data. The display 105 is composed of, for example, an LCD or a CRT. The input device 106 is composed of, for example, a mouse, a keyboard, a pen tablet, or the like.

The network interface (I / F) 109 communicates with an external device (not limited to the scanner 202 and the printer 203, but also a server and an external storage device (not shown)) connected on the network 204 to read programs and data. , Write.

FIG. 3 is a flowchart showing an outline of image processing executed by the image processing system of FIG. This process is a process executed by the CPU 101 based on the image processing program in the PC 201. A flowchart showing the detailed processing of FIG. 3 will be described with reference to FIG.

First, in step S301, the PC 201 controls the scanner 202 to read an image of a paper document and acquire the image data. Next, the PC 201 performs a block selection process on the image to extract each area of a table, characters, a picture or a figure, a frame, and a line from the image.

The block selection process recognizes the image data of one page read as shown in FIG. 9 as a block for each object, determines each block as an attribute such as characters / drawings / photographs / lines / tables, and sets different attributes. It is a process to divide into the area to have.

Specifically, first, the input image is binarized to black and white, and contour line tracking is performed to extract a pixel block surrounded by a black pixel contour. For a block of black pixels with a large area, the outline is traced to the white pixel inside to extract the block of white pixels, and the block of black pixels recursively from the inside of the block of white pixels over a certain area. Extract the mass.

The black pixel mass thus obtained is classified by size and shape, and classified into regions having different attributes. For example, a pixel block having an aspect ratio close to 1 and a constant size is defined as a pixel block equivalent to a character, a part where adjacent characters can be grouped in a well-aligned manner is a character area, and a flat pixel block is a line area. The area occupied by the black pixel clusters that are larger than the size and contain the square white pixel clusters in a well-aligned manner is obtained by the table area, the area where the irregular pixel clusters are scattered is the photographic area, and other arbitrary block selection processing is performed. The block information for each block is shown in FIG. FIG. 10 is an example of an image corresponding to the block information shown in FIG. Although each block information is not displayed on the screen, each block is shown in FIG. 10 for the sake of explanation.

Of the blocks obtained by this block selection process, a block having a text attribute (type) is acquired, and character recognition processing is performed on this block. Since the character recognition technology is a known technology, the description thereof will be omitted.

From the character string obtained as a result of character recognition, it is determined whether the area extends over a plurality of lines, and if it extends over a plurality of lines, it is divided into each line and registered as an area. In the case of one line, it is registered as the area as it is. Since the area (coordinates) obtained when the character is recognized can also be acquired, the coordinates are registered. FIG. 6 shows an example of data registered as an area. The block 4 of FIG. 5 is divided into 13 areas and registered.

Next, the area for acquiring the value is specified according to the acquisition condition 701 set in the setting file (FIG. 7). In this identification, the block obtained by the block selection is subjected to character recognition processing, the line is divided, and the area is registered. From this area, the key area is searched based on the search character string (for example, HDL cholesterol) of the key area of the acquisition condition 701, and from this key area, the search direction (for example, horizontal) is used to obtain the key area. Specify the horizontal area as the value acquisition area. If the conditions of the current value format (for example, numbers and three digits) are not met, the area next to the search direction is further specified as the value acquisition area.

At this time, if multiple search character strings in the configuration file are registered, such as 702, separated by commas, if there is no character string that matches the first search character string, the second search character string (for example, triglyceride) ) Is searched.

Further, if a plurality of search directions of the setting file are registered separated by commas as in 703, the second search direction (for example, bottom) is searched when there is no value matching the first search direction.

Further, in addition to the above search direction, the exclusion column setting condition 801 and the priority column setting condition 803 shown in FIG. 8 determine the columns to be excluded and the columns to be prioritized, and specify the area for which the value should be acquired. In this embodiment, the exclusion column and the priority column are described as a series of vertically arranged items, but depending on the format of the form or the like, they may be referred to as a series of horizontally arranged items (may be called excluded rows and priority rows). May be good.

In the exclusion column setting condition 801, it is determined that information different from the current inspection result, such as the inspection reference value and the previous inspection value, is displayed in the column in which the set character string 802 is displayed. It is a condition to set the target.

The priority column setting condition 803 is a condition for setting a target for which it is determined that the current inspection result is displayed in the column in which the set character string 804 is displayed.

Then, the character string (number) registered in the area specified as the value acquisition area is acquired as the inspection result this time.

In step S302, the PC 201 displays the character string (number) acquired in step S301 on the display 105 as the current inspection result. Examples of display on the display are FIGS. 12 and 13, and the respective screens will be described later. In addition, the data group of the values acquired in the CSV file or the like is exported and output according to the output instruction from the user.

The value acquisition process in step S301 will be described with reference to FIG.

In step S401, the PC 201 reads the acquisition condition 701, the exclusion column setting condition 801 and the priority column setting condition 803 from the setting file. Each condition is as described in step S301. The setting file to be used may be arbitrarily selected and read by the user.

In step S402, the PC 201 reads an image captured from the scanner or an image stored in a predetermined folder.

In step S403, the PC 201 executes the block selection process using the read image. This block selection is executed using the block selection library. The block selection process is as described in step S301. The block information is the same as in FIG. Further, the character recognition process may be executed by the block selection library.

In step S404, the PC 201 acquires block information having a text type and a table type from the block selection library.

In step S405, the PC 201 performs character recognition processing on the acquired block. The character recognition process is a known technique and the description thereof will be omitted. However, for example, there is a technique of collating the entered character with the character template and extracting the character candidate by using pattern matching.

By the character recognition process, a character string (including numbers) and area information (coordinates) of the character string can be acquired. The area information is an area indicating a frame surrounding the character string. The result of character recognition shall be managed in memory for each block.

In step S405, character recognition is first executed for all blocks, and the process proceeds to step S406. However, character recognition may be performed for each block to determine S406. That is, the procedure is not limited to the procedure of executing the determination in step S406 after executing the character recognition for all the blocks.

In step S406, the PC 201 determines whether or not the process has been executed for all the blocks. If the processing has not been performed for all the blocks, the processing is moved to step S407 in order to acquire the character string of the block to be processed next. When the processing is executed for all the blocks, the processing is moved to step S411.

In step S407, the PC 201 acquires the character recognition result (character string) of the block to be processed from the memory.

In step S408, the PC 201 determines whether the acquired character string is a character string having a plurality of lines. When the character strings of a plurality of lines are acquired, the process is moved to step S410. Further, when the character string of one line is acquired, the process is moved to step S409.

In step S409, the PC 201 stores the character recognition result in the memory. The character recognition result is registered in the character recognition area information of FIG. In FIG. 6, the area ID, the character string obtained from the character recognition result, and the area information (coordinates) indicating the position where the character string is located are registered.

In step S410, the PC 201 divides the area for each line to generate and register the character recognition area information. As shown in 601 of FIG. 6, the character string of the block 4 is registered by dividing it into 13 areas. As the area information (coordinates), the coordinates of the position where the character string of each line is located are registered.

In step S411, the PC 201 acquires the read search character string (see FIG. 7) of the key area of the acquisition condition 701 and searches for the character recognition area information. According to the search character string of the key area, the area that matches in the character string of the character recognition area information is specified. As a result, the matched area becomes the key search area.

In step S412, the read search character string (see FIG. 8) of the exclusion column setting condition 801 is acquired, the character recognition area information is searched, and the area where the values match is specified. At this time, if the area where the values match is the table area, the column in the table in which it exists is defined as the "excluded column".

When a plurality of search character strings of the exclusion string setting condition 801 are registered, the above is performed for all the search character strings.

If the part that matches the search character string of the exclusion string is not in tabular format, the area of the same arrangement as the X coordinate of the own character string may be determined as the "exclusion column". Further, when the exclusion columns are arranged side by side, it may be determined by the Y coordinate.

In step S413, the read search character string (see FIG. 8) of the priority column setting condition 803 is acquired, the character recognition area information is searched, and the area where the values match is specified. At this time, if the area where the values match is the table area, the column in the table in which the user exists is regarded as the "priority column".

When a plurality of search character strings of the priority column setting condition 803 are registered, the above is performed for all the search character strings.

If the part that matches the search character string of the priority column is not in the tabular format, the area of the same sequence as the X coordinate of the own character string may be determined as the "priority column". Further, when the priority columns are arranged side by side, it may be determined by the Y coordinate.

In step S414, the PC 201 acquires the key area and the search direction (for example, right) of the acquisition condition 701, and identifies an area one right from the key area. Based on the coordinates of the key area, the area having the coordinates in the right direction is searched and specified with reference to the character recognition area information (for example, FIG. 6). This specified area becomes the value acquisition area. In addition, among the regions having coordinates in the right direction, the first region and the second region are set in order from the region having the closest coordinates. If there are multiple key areas, specify the value acquisition area according to each condition in the same way.

When acquiring the value acquisition area in the above, if there is a "priority column" in the value search direction, that column (for example, 1101 in FIG. 11) is set as a candidate for the acquisition area for preferentially acquiring the value. do. If there are multiple "priority columns", they are candidates for the value acquisition area in order of priority.

If there is an "excluded column" in the value search direction, that column (for example, 1102 in FIG. 11) is excluded from the target of the value acquisition area.

For example, in the case of the report example 1401 shown in FIG. 14, when the value of the key item “HDL cholesterol” is acquired, the column 1402 in which the reference value is displayed does not include the character string representing the exclusion column, so that the exclusion column is Although it is not determined, the column 1403 including the character string "this time" is determined to be the priority column, the column 1402 is skipped, and the area of the character string "51" of the column 1403 is used as the value acquisition area.

Further, in the case of the report example 1411 shown in FIG. 14, the column 1412 including the character string "reference value" is determined to be an exclusion column, and the character strings "40 to 86" in the exclusion column in the search direction are excluded from the target. The area of the character string "51" of the column 1413 in which the excluded column is skipped is set as the value acquisition area. Note that column 1413 is not determined to be a priority column because it does not include a character string representing the priority column.

In step S415, the PC 201 acquires the value from the value acquisition area specified in step S414. If the character string acquired at this time is different from the value format of the read acquisition condition 701 (see FIG. 7), the next character string in the search direction (see FIG. 7) of the read acquisition condition 701 is acquired. Repeat this until the end of the same table can be obtained as a character string that matches the value format.

In step S416, the PC 201 determines whether or not the value can be acquired, and if the value can be acquired, the PC 201 executes step S418. If the value cannot be obtained, step S417 is performed.

In step S417, the PC 201 determines whether or not the values have been acquired for all the search directions set in the search direction (see FIG. 7) of the read acquisition condition 701, and if so, the step. S418 is carried out, and if it is not carried out, step S414 is carried out with the search direction set to the next direction.

In step S418, the PC 201 executes step S419 when the value acquisition is performed for all the recognized key items, and the value acquisition process (step S414) when there is a key item for which the value acquisition has not been performed. -Step S417) is carried out.

In step S419, it is determined whether or not the area specified as the value acquisition area for which the value could not be acquired in step S415 was the area for which the value should be acquired. The determined result is reflected when displaying the result in step S302.

A case where a report or the like in which a specific process is captured as an image is in a tabular format will be described. In step S415, if an error occurs for a certain key item and the value cannot be acquired for the key item, another value for which the value can be normally acquired in the same column for the area for which the value is to be acquired is obtained. If the acquisition area exists, it is determined as the "priority area". On the contrary, if there is no other value acquisition area in the same column for which the value can be acquired normally, it is determined as "non-priority area". In other words, for a column that has a value acquisition area where the value could be acquired normally, there is a high possibility that the items to be acquired are lined up, so the value should be acquired even for the area where an error occurred in the same column. It is judged as an area.

FIG. 13 shows an example of the determination result. In FIG. 13, for the key item “ALT (GPT)”, two places, 1302 and 1303, are specified as value acquisition areas, and the value acquisition is an error in each of them. In the value acquisition area 1302, since the values can be normally acquired in the value acquisition area in the same column for other key items such as "AST (GOT)", it is determined as the "priority area", that is, the key area for which the value should be acquired. NS. On the other hand, in the value acquisition area 1303, since there is no value acquisition area in which the value can be normally acquired in the same column for other key items, it is determined as a "non-priority area", that is, a key area in which the value does not need to be acquired. NS.

In addition, if the report etc. captured as an image is not in tabular format, if there is another key item for which the value can be obtained in the area with the X coordinate close to the X coordinate of the key area in which the error occurred, the "priority area" , If it does not exist, it may be determined as a "non-priority area". Further, in the case of the tabular format, the area may be specified by the row instead of the column, and when it is not in the tabular format, the area may be specified by the Y coordinate.

Next, the screen displayed by the value output process in step S302 will be described.

FIG. 12 is a screen image of the recognition result screen 1201 which is an example of the screen displayed in step S302.

The recognition result screen 1201 displays the read image image 1202 on the left side and the recognition result list 1203 for each inspection item on the right side.

In step S415, if there is an item for which the value cannot be acquired, such as when the key area is blank or when there is no character string matching the value format, it is highlighted as an error item (1204, 1205). If there is an error in the recognition result, the user can have the correction value input in the correction input area (1206). When input is made in the correction input area, the input value becomes the registered value.

Further, the priority column and the exclusion column specified by the priority column setting condition 803 and the exclusion column setting condition 801 may be displayed in an identifiable manner.

FIG. 13 is a screen image of the recognition result screen 1301 which is an example of the screen reflecting the determination result in step S419.

Since the value acquisition area 1302 could not acquire the value in step S415 and was determined to be the "priority area" in step S419, the value acquisition area 1302 and the recognition result 1304 of the value acquisition area 1302 are highlighted.

On the other hand, since the value acquisition area 1303 could not acquire the value in step S415 and was determined to be the "non-priority area" in step S419, the value acquisition area 1303 and the recognition result 1305 of the value acquisition area 1303 are "priority areas". Is displayed in a different format.

Further, the recognition result screens 1201 and 1301 are provided with an output button, and output the recognition result and the correction result if the value is corrected by the user to a CSV file. When multiple images are loaded, value acquisition and confirmation are executed for all images, and when the output button is pressed for the last image, the values are collectively output to the CSV file. .. The format of the output file is an example and is not limited.

The above makes it possible to easily obtain the value of a desired item from a document having a plurality of formats.

Although one embodiment has been described above, the present invention can take an embodiment as, for example, a system, an apparatus, a method, a program, a recording medium, or the like, and specifically, is composed of a plurality of devices. It may be applied to a system or a device consisting of one device. For example, it may be a configuration realized in a cloud environment. In that case, the configuration file creation tool is executed on the server in the cloud environment.

Further, the program in the present invention is a program in which a computer can execute the processing method of the flowchart shown in FIGS. 3 to 4. The storage medium may be configured to store a program in which a computer can execute the processing methods of FIGS. 3 to 4. The program in the present invention may be a program for each processing method of each device of FIGS. 3 to 4.

As described above, a recording medium on which a program that realizes the functions of the above-described embodiment is recorded is supplied to the system or device, and the computer (or CPU or MPU) of the system or device stores the program in the recording medium. Needless to say, the object of the present invention can be achieved by reading and executing.

In this case, the program itself read from the recording medium realizes the novel function of the present invention, and the recording medium storing the program constitutes the present invention.

Recording media for supplying programs include, for example, flexible disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, DVD-ROMs, magnetic tapes, non-volatile memory cards, ROMs, EEPROMs, and silicon. Disks, solid state drives, etc. can be used.

Further, by executing the program read by the computer, not only the function of the above-described embodiment is realized, but also the OS (operating system) or the like running on the computer is actually operated based on the instruction of the program. Needless to say, there are cases where a part or all of the processing is performed and the processing realizes the functions of the above-described embodiment.

Further, after the program read from the recording medium is written in the memory provided in the function expansion board inserted in the computer or the function expansion unit connected to the computer, the function expansion board is based on the instruction of the program code. It goes without saying that there is a case where the CPU or the like provided in the function expansion unit performs a part or all of the actual processing, and the function of the above-described embodiment is realized by the processing.

Further, the present invention may be applied to a system composed of a plurality of devices or a device composed of one device. It goes without saying that the present invention can also be applied when it is achieved by supplying a program to a system or an apparatus. In this case, by reading the recording medium in which the program for achieving the present invention is stored into the system or the device, the system or the device can enjoy the effect of the present invention.

Further, by downloading and reading a program for achieving the present invention from a server, database, or the like on the network by a communication program, the system or device can enjoy the effect of the present invention.

It should be noted that all the configurations in which each of the above-described embodiments and modifications thereof are combined are also included in the present invention.

101 CPU
102 RAM
104 Storage device 201 PC
202 Scanner 203 Printer

Claims (10)

An information processing device that acquires a character string in a document in which a plurality of key character strings and character strings related to the key character string are arranged.
In the document, the region excluding the region specified by Jo Tokoro exclusion strings from the area determined based on the acquisition condition for specifying a region to obtain the key character string and the character string, the character string acquisition region And the decision-making means to decide as
An information processing apparatus including an acquisition means for acquiring a character string from the determined character string acquisition area. Said determining means, in the document, the area specified by the priority string among plants constant region which is determined based on the acquisition condition for specifying a region to obtain the key character string and the character string, the character string acquisition region determining as the information processing apparatus according to claim 1, wherein the. The information processing apparatus according to claim 1 or 2, wherein the area specified by the exclusion character string is a series of areas in the document. The information processing apparatus according to any one of claims 1 to 3, wherein the area specified by the exclusion character string is a column or row of a table in the document. The determination means according to claim 1 to 4, wherein the character string acquisition area is determined by skipping the area specified by the predetermined exclusion character string among the areas specified by the key character string and the acquisition condition. The information processing device according to any one of the items. The information processing apparatus according to any one of claims 1 to 5, further comprising a display control means for displaying information obtained from the acquired character string. The acquisition condition includes a direction indicating the positional relationship between the key character string and the character string to be acquired.
The information processing apparatus according to any one of claims 1 to 6, wherein the determination means determines an area in the direction from the key character string as a character string acquisition area. An information processing device that acquires a character string in a document in which a plurality of key character strings and character strings related to the key character string are arranged.
In said document, a key character string and the region specified by the priority string among plants constant region which is determined based on the acquisition condition for specifying a region to get a string, determining means for determining as a string acquisition area When,
An information processing apparatus including an acquisition means for acquiring a character string from the determined character string acquisition area. It is a control method of an information processing device that acquires a character string in a document in which a plurality of key character strings and character strings related to the key character string are arranged.
Determining means, in the document, the region excluding the region specified by Jo Tokoro exclusion strings from the area determined based on the acquisition condition for specifying a region to obtain the key character string and the character string, The decision step to decide as the character string acquisition area and
A control method of an information processing device, wherein the acquisition means includes an acquisition step of acquiring a character string from the determined character string acquisition area. A program that can be executed by an information processing device that acquires a character string in a document in which a plurality of key character strings and character strings related to the key character string are arranged.
The information processing device
In the document, the region excluding the region specified by Jo Tokoro exclusion strings from the area determined based on the acquisition condition for specifying a region to obtain the key character string and the character string, the character string acquisition region And the decision-making means to decide as
A program for functioning as an acquisition means for acquiring a character string from the determined character string acquisition area.

Images