JP2018092459A - Image analyzing device, image analyzing method, and image analyzing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve accuracy in extraction of character information from an image as an analysis subject such as an insurance policy.SOLUTION: An image analyzing device 10 for analyzing an object image, comprises an image analysis part 23 and a structure analysis part 25. The image analysis part 23 extracts character information from the object image. The structure analysis part 25 associates the character information with character information pairing with the character information. The association of the character information is performed on the basis of a structure distance obtained as a function of an image of a peripheral region in which the distance in the object image of two pieces of character information and the character information of the object image are extracted. The image analyzing device may further construct a mapping part that displays a combination of the character information associated by the image analysis part 23 in accordance with the object image, and receives an input of correction by a user in a case of including an error in the combination of the character information.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image analysis apparatus, an image analysis method, and an image analysis program for analyzing an analysis target image such as an insurance policy.

  There are multiple insurance companies that provide life insurance and non-life insurance, and each insurance company sells multiple insurance products. Each insurance product has parameters such as the amount of insurance and the insurance period. In addition, special agreements are often signed with the main insurance contract. For this reason, it is difficult for policyholders who are not so used to handling insurance to determine whether or not the insurance he / she has contracted is appropriate for him / her.

  An insurance agent that handles insurance products of a plurality of insurance companies may grasp the contents of insurance that the customer has already contracted based on the insurance policy and propose appropriate insurance to the customer.

JP 2015-46027 A

  Insurers are often dependent on the know-how of the person in charge to offer appropriate insurance to customers. However, depending on the company that provides insurance products and the type of insurance, the format of insurance policies and terms used are diverse, and it is difficult to memorize all of them. Therefore, for insurance analysis and proposal creation, it is desirable to use a computer to equalize and improve quality.

  However, the contents of life insurance and non-life insurance contracts are often written on about one insurance policy. Insurance policy is generally printed on paper media, and even if the contract contents are stored as electronic data by the insurance company, the contractor of the other party of the insurance company has the insurance contents as electronic data. rare. For this reason, in order to input the contract contents to the computer, the person in charge or the like has to input it manually, and the time required for contract analysis and proposal creation may be reduced.

  Accordingly, an object of the present invention is to improve the accuracy of extracting character information from an analysis target image such as an insurance policy.

  In order to achieve the above-described object, the present invention provides an image analysis apparatus that analyzes a target image, an image analysis unit that extracts character information from the target image, and a position of the character information in the target image. And a structure analysis unit that associates character information with a pair of character information.

  According to another aspect of the present invention, there is provided an image analysis program for causing a computer to analyze a target image. The computer analyzes an image analysis function for extracting character information from the target image, and the character information based on the position of the character information in the target image. And a structure analysis function for associating the character information paired with the information.

  According to another aspect of the present invention, there is provided an image analysis method for analyzing a target image, wherein an image analysis step for extracting character information from the target image is paired with the character information based on a position of the character information in the target image. And a structural analysis step for linking the character information.

  ADVANTAGE OF THE INVENTION According to this invention, the precision which extracts character information from analysis object images, such as insurance policy, can be improved.

It is a block diagram in one embodiment of an image analysis device concerning the present invention. It is a top view of the insurance policy image of the analysis object of one embodiment of the image analysis device concerning the present invention. It is a flowchart which shows the flow of the analysis of one Embodiment of the image analyzer which concerns on this invention. It is an example of a part of insurance policy image in one embodiment of an image analysis device concerning the present invention. It is an example of a part of a preprocessed image in an embodiment of an image analysis apparatus according to the present invention. It is an example of a part of insurance policy image in one embodiment of an image analysis device concerning the present invention. It is an example of a part of insurance policy image in one embodiment of an image analysis device concerning the present invention. It is a mapping screen in one embodiment of the image analysis apparatus according to the present invention.

  An image analysis apparatus according to an embodiment of the present invention will be described with reference to the drawings. This embodiment is merely an example, and the present invention is not limited to this. The same or similar components are denoted by the same reference numerals, and redundant description is omitted.

  FIG. 1 is a block diagram of an image analysis apparatus according to an embodiment of the present invention.

  The image analysis apparatus 10 according to the present embodiment includes an image storage unit 20, an image analysis preprocessing unit 21, a preprocessed image storage unit 22, an image analysis unit 23, an image analysis post-processing unit 24, a structure analysis unit 25, and a mapping unit. 26, a pattern classification unit 27, a pattern storage unit 28, an extracted character string storage unit 29, a dictionary 30, a scanner 12, a display 13, a keyboard 14, and a mouse 15. The image analysis apparatus 10 is configured on a single computer, for example. A part of the image analysis device 10, for example, an image storage unit 20, an image analysis pre-processing unit 21, a pre-processed image storage unit 22, an image analysis unit 23, an image analysis post-processing unit 24, a structure analysis unit 25, a mapping unit 26, Part or all of the pattern classification unit 27, the pattern storage unit 28, the extracted character string storage unit 29, and the dictionary 30 may be distributed on a plurality of computers connected to each other via a network.

  FIG. 2 is a plan view of an insurance policy image to be analyzed by the image analysis apparatus of the present embodiment.

  The image analysis apparatus 10 according to the present embodiment analyzes, for example, an insurance policy image (insurance policy image 90) printed on paper. Moreover, not only what was printed on paper but the content displayed on the display of a smart phone or a computer can also be analyzed.

  Next, a flow of analysis processing of the insurance policy image 90 using the image analysis apparatus 10 of the present embodiment will be described.

  FIG. 3 is a flowchart showing an analysis flow of the image analysis apparatus according to the present embodiment.

  The insurance policy image 90 is scanned by, for example, the scanner 12 and stored in the image storage unit 20 of the image analysis apparatus 10 (S1). You may image with a camera, such as a smart phone, instead of the scanner 12. FIG. When imaging with a camera, trapezoidal correction or horizontal correction may be performed on the image.

  Next, the image analysis preprocessing unit 21 performs preprocessing on the insurance policy image 90 stored in the image storage unit 20 (S2).

  Specifically, first, the image analysis preprocessing unit 21 detects a table-set area 91 from the insurance policy image 90. The table area can be detected by searching for a straight line having a predetermined length or more and a curve connecting the straight lines, that is, an area surrounded by the ruled line 92.

  FIG. 4 is an example of a part of an insurance policy image in the present embodiment. FIG. 5 is an example of a part of the preprocessed image in the present embodiment.

  Next, the image analysis preprocessing unit 21 removes the ruled line 92 from the table region 91 to generate a preprocessed image 93. The preprocessed image 93 is output from the image analysis preprocessing unit 21 and stored in the preprocessed image storage unit 22.

  FIG. 6 is another example of a part of the preprocessed image in the present embodiment.

  Further, the image analysis preprocessing unit 21 may rearrange the images included in the preprocessed image 93 as shown in FIG. In rearranging images, an image including a character string is divided into chunks. For example, image blocks including character strings are arranged vertically.

  The rearrangement of the images may be such that the contents of the items are arranged side by side with respect to the item names in the cells 94 of the table region 91. Alternatively, the cells 94 may be arranged vertically. When rearrangement is performed, the position of each cell 94 before the rearrangement is stored together with the image of the cell 94.

  When the image analysis preprocessing unit 21 rearranges the images included in the preprocessed image 93, a delimiter (delimiter) such as “======” is inserted between the cells 94 as an image. May be.

  Next, the image analysis unit 23 performs image processing on the preprocessed image 93 and extracts character information (S3). The image analysis unit 23 can use a general OCR (Optical Character Recognition) engine. As the OCR engine, an OCR engine published on the Internet may be used.

  Since the image analysis preprocessing unit 21 removes the ruled lines 92, the accuracy of character recognition is improved. In addition, since the image analysis preprocessing unit 21 rearranges the images in units of cells 94, for example, it becomes easier to recognize characters that should be grouped, so that the accuracy of character recognition is improved as a whole.

  The character information extracted by the image analysis unit 23 is stored in the extracted character string storage unit 29 together with the position of the character information on the insurance policy image 90. When the image analysis preprocessing unit 21 has rearranged the images included in the preprocessed image 93, the position of the cell 94 stored based on the position of the character information on the preprocessed image 93 , The position of the character information on the insurance policy image 90 is known.

  When a delimiter (delimiter) is inserted as an image in the preprocessed image 93, the delimiter is also extracted as character information. As a result, the character string included in each cell 94 can be easily extracted by dividing the character information by the delimiter characters.

  Next, the image analysis post-processing unit 24 performs post-processing on the extracted character information (S4). In the post-processing, for example, the dictionary 30 is used to check whether the extracted character information is recognized correctly.

  FIG. 7 is an example of a part of an insurance policy image in the present embodiment.

  For example, consider the case where the image shown in FIG. 7 is recognized as “daily amount for hospitalization” as a result of OCR. The result character string “daily amount for hospitalization pick-up” is divided into a plurality of sub character strings so that the first character advances to the right one character at a time from the beginning of the result character string, for example, two consecutive characters. That is, it is divided into six sub-character strings of “hospitalization”, “hospital pick-up”, “pick-up”, “payment”, “gold day”, and “daily amount”. Therefore, the sub character string in which the first character is adjacent in the result character string has, for example, one overlapping character.

  Next, with respect to each divided character string “hospital”, “hospital pick-up”, “pick-up”, “payment”, “gold day” and “daily amount”, the appearance frequency in the dictionary stored in the dictionary 30 is determined. Investigate. As a result, for example, the appearance frequency is 200 for “hospitalization”, 0 for “hospital pick-up”, 5 for “pickup”, 100 for “payment”, 50 for “Friday”, “day “Forehead” is 100. As described above, the appearance frequency of the character string including the error in the OCR, that is, “pick-up” and “pick-up” becomes extremely small. Therefore, a portion (duplicate character) common to character strings whose appearance frequency is lower than a predetermined threshold, that is, “pickup” that is the third character is determined to be an OCR error.

  Furthermore, the appearance frequency of the character string added with one character after “in” and the character string added with one character before “attached” is checked in the dictionary, and the character string to be added is a common character string. Extract the string with the highest frequency. It is assumed that a character common to the extracted character string (a character to be added) is a correct character.

  In this way, the character analysis recognition accuracy is improved by the image analysis post-processing unit 24 dividing the character string obtained by the image analysis by the N-gram method and matching it with the dictionary. In particular, it is effective for analyzing images of documents in which languages such as Japanese, Korean, Chinese, and the like that do not have word breaks are described. A recursive neural network (RNN) may be used to learn the correctness of character recognition and determine the correctness of character recognition.

  In post-processing, morphological analysis of the extracted character information may be performed. By morphological analysis, the part of speech of the character string included in the extracted character information can be analyzed. The analysis result is stored together with the character string. You may analyze not only a part of speech but whether it corresponds to a proper noun, whether it corresponds to a date, whether it corresponds to a quantity, etc.

  Next, the pattern classification unit 27 determines the pattern of the insurance policy image 90 (S5). Specifically, it is determined which of the document patterns stored in the pattern storage unit 28 the insurance policy image 90 corresponds to.

  Here, the pattern is a document format commonly used for each type of a specific insurance company or insurance product. The features of the insurance policy image 90 include, for example, the arrangement of ruled lines and frame lines, and whether or not the company name / insurance product name is described in the insurance policy. For example, the horizontal pixel density spectrum in which values obtained by adding the pixel values of the insurance policy image 90 in the horizontal direction are arranged for each position in the vertical direction, and the pixel values of the insurance policy image 90 are set in the vertical direction. The format can be specified by using the vertical pixel density spectrum in which the sum of the values is arranged for each position in the horizontal direction. The pattern can be specified by comparing the horizontal pixel density spectrum and the vertical pixel density spectrum with the reference pattern stored in advance in the pattern storage unit 28 and calculating the similarity. A cascade type classifier (Haar classifier) may be used for pattern identification or ruled line / frame line recognition. For pattern determination, it may be used that a specific character string such as a company name or a product name is described at a specific position.

  Next, the structure analysis unit 25 extracts the item name key and the value of the item name item for each piece of character information extracted by the image analysis unit 23 and corrected by the image analysis post-processing unit 24 as necessary. The value is combined (linked) as a set (S6).

  A key can be specified by searching whether or not a target character string is included in a database storing character strings that can be item names. Or it is good also considering the character string described in the cell 94 of the left side or the upper side of the area | region 91 of a table | surface group as a key.

  A structured distance is used to associate a key and a value. The structured distance is a virtual distance obtained by adding a restriction on a description position of a word on a document to a distance between actual description positions. What is the restriction on the position of words in a document? In Japanese horizontal writing, characters are written from left to right and from top to bottom, and when there are ruled lines, ruled lines And the characters existing in the area surrounded by the frame form a group of meanings.

  The structured distance is a function of the physical distance (distance between character strings), the position of the character string, and the image around the character string. For example, the structured distance is the physical distance multiplied by the structuring factor. The structured distance may be a physical distance plus a structured bias. The physical distance is the distance between the end of the first character string, that is, the rightmost character in the case of horizontal writing, and the first character of the second character string, that is, the leftmost character in the case of horizontal writing.

  The structuring coefficient is increased when there is a horizontal ruled line 92 between the first character string and the second character string. In addition, when the first character strings considered as items are arranged one above the other, the structuring coefficient increases when there are a plurality of vertical ruled lines 92 between the first character string and the second character string. You may do it. When the first character string considered as an item is arranged on the left and right, the structuring coefficient is increased when there are a plurality of horizontal ruled lines 92 between the first character string and the second character string. May be. In addition, when going to the right side, the structuring coefficient may be positive, and when going to the left side, the structuring coefficient may be negative.

  For the combination of key and value, a vector from the first character string to the second character string may be used.

  In addition, a value candidate list in which no item name is described may be stored in advance together with an item name not described such as “product name”. When the extracted character information corresponds to one of the candidate lists, the extracted character information may be stored as a value and the corresponding item name may be stored as a key.

  When morphological analysis or the like is performed in post-processing (S4), character string features such as part of speech may be used for association. For example, it can be linked using a rule that only a quantity corresponds to a value for a specific key.

  Keys and values do not have to be a simple one-to-one relationship. For example, a plurality of values may be combined for one key. The key may have a hierarchical structure (tree structure) in which a sub key is subordinate to a main key. In this case, the number of hierarchies may be three or more.

  Next, the mapping unit 26 maps the extracted character string on the original insurance policy image 90 (S7).

  FIG. 8 is a mapping screen in the image analysis apparatus of the present embodiment.

  The mapping screen 40 is displayed on the display 13. The mapping screen 40 includes an image display unit 41 and a correspondence display unit 42. An insurance policy image 90 is displayed on the image display unit 41. The correspondence display unit 42 displays combinations of keys and values.

  The extracted character string is displayed as a set of key and value. The mapping unit 26 further prompts the user to confirm the correspondence relationship between the key and the value combined by the structure analysis unit 25 so that it can be corrected if there is an error.

  Specifically, first, for example, the insurance policy image 90 and the extracted character information are displayed on the image display unit 41 of the display 13. The extracted character information is displayed, for example, as a key and value pair on the correspondence display section 42 on the left side of the insurance policy image 90. Here, the key and value pairs displayed first are created by the structure analysis unit 25.

  In addition, the portion where the character information is extracted is colored. When the pointer is moved to the colored part by mouse operation etc. and clicked, the character information of the key or value corresponding to that part can be distinguished from the part where the key and value pair is displayed. For example, it is displayed by changing the color. At this time, on the insurance policy image 90, for example, it is displayed by changing the color so that the portion from which the character information associated with the key or value corresponding to the selected portion is read can be distinguished from the others. The If there is no mistake in this correspondence, the user leaves it or inputs that there is no mistake. If there is an error in the correspondence, the key or value value is corrected in the portion where the key and value pair is displayed.

  When correcting, if there is an error in character recognition, input from the keyboard or the like. If there is an error in the association (association), the correspondence relationship may be corrected by clicking on the portion of the insurance policy image 90 where the corresponding key or value is written.

  Thus, in the present embodiment, errors can be reduced by the user correcting the result of character recognition.

  Further, by learning the correction by the user, the accuracy of character recognition and the accuracy of mapping (linking) can be improved. For example, when a specific character is misrecognized as another character and the number of times the user corrects it increases, the specific character can be recognized correctly. Also, if the mapping is incorrect, the user can correctly combine the key and value by correcting the association.

  If the pattern classification unit 27 can recognize that the insurance policy image 90 matches the format already stored, the key position and the value position on the insurance policy image 90 have been grasped. . Therefore, in this case, the structure analysis unit 25 can easily associate the key and the value.

  If the pattern classification unit 27 cannot recognize that the insurance policy image 90 matches the format already stored, it is necessary to grasp the key position and the value position on the insurance policy image 90. However, the user corrects the mapping by the mapping unit 26 and the combination of the mapped key and value as necessary, so that errors in the combination of key and value can be extremely reduced.

  As described above, the image analysis apparatus 10 according to the present embodiment has the structure analysis unit 25 that associates character information with a pair of character information based on the position of the character information in the insurance policy image 90. . Therefore, it is possible to grasp the structure of the document represented by the insurance policy image 90 that is the analysis target image, that is, the correspondence between the item name and the corresponding value. As a result, the recognition accuracy of character information can be improved based on the correspondence between the item name and the corresponding value.

  Further, the structure analysis unit 25 associates the two character information based on the structured distance obtained as a function of the distance in the insurance policy image 90 and the image of the surrounding area from which the character information in the image is extracted. It is carried out. Therefore, based on information such as not only the relationship between character strings but also the positional relationship between two character strings and the presence of ruled lines between them, the relationship between item names and corresponding values can be more accurately determined. I can grasp it.

  Further, in the present embodiment, the mapping unit 26 displays the character information set linked by the structure analysis unit 25 in association with the insurance policy image 90, and when there is an error in the combination of character information Accepts correction input by. For this reason, the correspondence between the item name and the value corresponding thereto can be grasped more accurately. Furthermore, by learning the correction history by the user, it is possible to improve the accuracy of grasping the correspondence relationship between the item name and the value corresponding thereto.

  In learning, it is possible to improve the accuracy of grasping the correspondence relationship by learning the position of each of the key and value on the image itself or the vector from the key to the value.

  The pattern classification unit 27 determines which of the patterns stored in the pattern storage unit 28 the insurance policy image 90 corresponds to. For this reason, when the insurance policy image 90 matches the pattern stored in the pattern storage unit 28, it is possible to easily grasp the correspondence between the item name and the value corresponding thereto. Therefore, the structure analysis unit 25 can easily associate the correspondence between the item name and the corresponding value.

  In addition, although the present embodiment has been described with respect to an example of a system that analyzes insurance policies, it is also applicable to a system that analyzes contents described in other printed materials, etc., in which terms that are described in a certain form are limited to some extent. Applicable. For example, you can analyze the contents of a driver's license, analyze a pamphlet of an apartment, analyze the flow of deposits from a bank passbook, analyze the contents of a registered copy, and analyze the results of a health checkup. Can be.

DESCRIPTION OF SYMBOLS 10 ... Image analyzer, 12 ... Scanner, 13 ... Display, 14 ... Keyboard, 15 ... Mouse, 20 ... Image storage part, 21 ... Image analysis pre-processing part, 22 ... Pre-processed image storage part, 23 ... Image analysis part , 24 ... Image analysis post-processing unit, 25 ... Structure analysis unit, 26 ... Mapping unit, 27 ... Pattern classification unit, 28 ... Pattern storage unit, 29 ... Extracted character string storage unit, 30 ... Dictionary, 40 ... Mapping screen, 41 ... Image display section, 42 ... Corresponding display section, 90 ... Insurance policy image, 91 ... Table layout area, 92 ... Ruled line, 93 ... Pre-processed image, 94 ... Cell

Claims (9)

In an image analysis apparatus for analyzing a target image,
An image analysis unit for extracting character information from the target image;
A structure analysis unit for associating the character information paired with the character information based on the position of the character information in the target image;
An image analyzing apparatus comprising:   The structural analysis unit performs association based on a structured distance obtained as a function of a distance between the two pieces of character information in the target image and an image of a peripheral region from which the character information of the target image is extracted. The image analysis apparatus according to claim 1.   A mapping unit that displays the set of character information linked by the structural analysis unit in association with the target image, and that receives a correction input by a user when the combination of the set of character information includes an error; The image analysis apparatus according to claim 2.   The image analysis apparatus according to claim 3, wherein the structure analysis unit learns the correction input history received by the mapping unit and reflects it in the association. A pattern storage unit that stores a plurality of patterns of the target image and positions where the pairs of character information are described in the respective patterns;
A pattern classification unit that determines which of the patterns stored in the pattern storage unit the target image corresponds to,
When it is determined that the target image corresponds to any one of the patterns stored in the pattern storage unit, the structure analysis unit performs the character information based on a position where the pair of character information is described in the pattern. The image analysis apparatus according to claim 2, wherein the linking is performed. The target image includes characters and graphics other than characters,
The image analysis apparatus according to claim 1, further comprising an image analysis preprocessing unit that removes at least a part of a figure other than the characters from the target image.   A sub-character string having two or more divided characters that partially overlap each other is generated from at least a part of the target character string of the character information, an appearance frequency of the sub-character string in the dictionary is obtained, and the appearance frequency is predetermined The image analysis post-processing unit further determines that a duplicate character common to the sub-character string smaller than the value of the character string is an error in the character information by the image analysis unit. The image analysis apparatus according to any one of the above. In an image analysis program for causing a computer to analyze a target image,
An image analysis function for extracting character information from the target image;
A structure analysis function for associating the character information paired with the character information based on the position of the character information in the target image;
An image analysis program that realizes In an image analysis method for analyzing a target image,
An image analysis step of extracting character information from the target image;
A structural analysis step for associating the character information paired with the character information based on the position of the character information in the target image;
An image analysis method characterized by comprising: