JP2021099688A - Image processor, image processing method, and program - Google Patents

Abstract

To provide an image processor that selects a target with an item name and an item value for which a combination is to be regenerated.SOLUTION: An image processor 100 includes: an area analysis unit 202 that identifies a character string area included in a scan image; a character recognition unit 203 that recognizes a character string in the character string area; an item name extraction unit 205 and an item value extraction unit 207 that extract an item value and an item name from the recognized character string; a combination generation unit 208 that determines the extracted item value associated with the extracted item name based on at least a positional relationship between the item value and the item name, and generates a combination of one item name and one or more item values so that the determined item value is not associated with a plurality of item names; and a target selection unit 209 that selects, based on the positional relationship, a target including an item name and an item value for which the combination is to be regenerated.SELECTED DRAWING: Figure 2

Description

The present disclosure relates to a technique for generating a combination of an item name and an item value included in a scanned image.

There is a method of extracting specific information contained in a scanned image from a scanned image obtained by scanning a form or the like.

In Patent Document 1, among the character strings recognized from the form image, the data character string (item value) and the item name are associated with each other based on the positional relationship between the character string determined to be the item name and the data character string. The method for determining the attribute of the data character string is described.

Japanese Unexamined Patent Publication No. 2008-204226

When character recognition processing is performed from a scanned image obtained by scanning a form, erroneous recognition of characters may occur due to deterioration of the scanned image due to dirt or blurring of the form. When the result of the character recognition process including the erroneously recognized character string is processed to generate the combination of the item name and the item value corresponding to the item name based on the positional relationship, the erroneous combination is generated. There are times. Therefore, it is required to regenerate the combinations for the candidates of the erroneously generated combinations so that the erroneous combinations are not output. However, it is troublesome to select a candidate for an erroneously generated combination.

An object of the present disclosure is to select an object for regenerating a combination of an item name and an item value.

The image processing device of the present disclosure is an image processing device that extracts an item name and an item value included in a scanned image and generates a combination of one item name and one or a plurality of item values. A specific means for specifying a character string area included in an image, a character recognition means for recognizing a character string in the character string area, an extraction means for extracting an item value and an item name from the recognized character string, and the extraction. A determination means for determining the extracted item value associated with the item name based on at least the positional relationship between the item value and the item name, and the determined item value are associated with a plurality of item names. It is characterized by having a generation means for generating the combination so as not to be present, and a selection means for selecting an object including an item name and an item value for which the generation is redone by using the positional relationship.

According to the technique of the present disclosure, it is possible to select a target for regenerating the combination of the item name and the item value.

The figure which shows the hardware configuration of an image processing apparatus. The figure which shows the functional structure of an image processing apparatus. The figure for demonstrating the extraction process of item name and item value. The figure for demonstrating the process of associating an item value with an item name. The figure for demonstrating the process of narrowing down the item values associated with a plurality of item names. The figure for demonstrating the process of narrowing down the item values associated with a plurality of item names. The figure which shows the combination of the generated item name and the item value. The figure for demonstrating the extraction process of item name and item value. The figure for demonstrating the process of associating an item value with an item name. The figure which shows the combination of the generated item name and the item value. A flowchart showing a process of generating a combination from a scanned image. The figure for demonstrating the process of selecting the target of re-judgment. The figure for demonstrating the extraction process of item name and item value. The figure which shows the combination of the generated item name and the item value. The figure for demonstrating the process of selecting the target of re-judgment. The figure for demonstrating the process of selecting the target of re-judgment. The figure for demonstrating the process of selecting the target of re-judgment.

Hereinafter, details will be described based on the embodiments with reference to the accompanying drawings. The configuration shown in the following embodiments is only an example, and the present invention is not limited to the illustrated configuration.

<Embodiment 1>
[Hardware configuration]
FIG. 1 is a diagram showing a hardware configuration of the image processing device 100 of the present embodiment. The image processing device 100 includes a CPU 101, a ROM 102, a RAM 103, an auxiliary storage device 104, a display unit 105, an operation unit 106, a communication I / F 107, a scanner 109, and a bus 108.

The CPU 101 realizes each function of the image processing device 100, which will be described later, by controlling the entire image processing device 100 by using computer programs and data stored in the ROM 102 and the RAM 103. The image processing device 100 may have one or more dedicated hardware different from the CPU 101, and the dedicated hardware may execute at least a part of the processing by the CPU 101. Examples of dedicated hardware include ASICs (application specific integrated circuits), FPGAs (field programmable gate arrays), and DSPs (digital signal processors). The ROM 102 stores a program or the like that does not need to be changed. The RAM 103 temporarily stores programs and data supplied from the auxiliary storage device 104, data supplied from the outside via the communication I / F 107, and the like. The auxiliary storage device 104 is composed of, for example, a hard disk drive or the like, and stores various data such as image data of a scanned image.

The scanner 109 is an image scanning unit that reads a document and generates image data of a scanned image, and is, for example, an overhead scanner. The scanner 109 may be configured as a device separate from the image processing device 100, in which case the scanner 109 may be connected via the communication function of the communication I / F 107.

The display unit 105 is composed of, for example, a liquid crystal display, an LED, or the like, and displays a GUI (Graphical User Interface) for the user to operate the image processing device 100. The operation unit 106 is composed of, for example, a keyboard, a mouse, a touch panel, or the like, and inputs various instructions to the CPU 101 in response to an operation by the user. The CPU 101 operates as a display control unit that controls the display unit 105 and an operation control unit that controls the operation unit 106.

The communication I / F 107 is used for communication with an external device of the image processing device 100. For example, when the image processing device 100 is connected to an external device by wire, a communication cable is connected to the communication I / F 107. When the image processing device 100 has a function of wirelessly communicating with an external device, the communication I / F 107 includes an antenna. The bus 108 connects each part of the image processing device 100 to transmit information.

In the present embodiment, it is assumed that the display unit 105 and the operation unit 106 exist inside the image processing device 100, but at least one of the display unit 105 and the operation unit 106 is used as another device outside the image processing device 100. It may exist.

The image processing device 100 of the present embodiment is realized as, for example, an image processing device included in a device such as an MFP having a scanner function. Alternatively, the image processing device of the present embodiment may be realized by a server or the like that acquires a scanned image from a device having a scanner unit and performs image processing.

[Functional configuration]
The image processing device 100 of the present embodiment is a device that recognizes a character string from a scanned image obtained by scanning a document by a scanner 109. The image processing device 100 also has a function of extracting information such as a total amount of money from the result of character recognition of a scanned image obtained by scanning a document such as a receipt.

FIG. 2 is a block diagram showing a functional configuration of the image processing device 100 of the present embodiment. The image processing device 100 includes an image processing unit 201, an area analysis unit 202, a character recognition unit 203, an item name extraction unit 205, an item value extraction unit 207, a combination generation unit 208, a target selection unit 209, a regeneration unit 210, and an output unit. It has 211.

The image processing unit 201 performs image processing such as binarization, noise removal, and ruled line removal on the scanned image obtained by scanning the original by the scanner. The area analysis unit 202 analyzes the scanned image processed by the image processing unit 201 to specify an area such as a character string area or an image area included in the scanned image.

The character recognition unit 203 performs character recognition processing (OCR processing) on the character string area of the scanned image using the information of the image-processed scanned image and the specified character string area. The character recognition result obtained as a result of processing by the character recognition unit 203 includes the recognized character string, the position of the entire character string for each recognized character string, and the size indicating the width and height of the entire character string. And are included. The character string may be one character or may be composed of a plurality of characters.

The character recognition result also includes the position of the characters that make up the character string, the size indicating the width and height of the character, the character code of the first to lower candidates, and the likelihood (matching degree). ..
For the coordinates in this scanned image, for example, a coordinate system is used in which the origin is at the upper left, the vertical direction is downward, and the horizontal direction is right. As for the position of the character string or the character constituting the character string, for example, the upper left coordinate value is held as each position.

The item name extraction unit 205 extracts a character string which is an item name from the character recognition result by using the item name dictionary 204. Words to be extracted as item names are registered in the item name dictionary 204, and the item name extraction unit 205 can use only the first candidate having a high degree of matching with the words registered in the item name dictionary 204 from the character recognition result. The item name is extracted by searching the character string including the subordinate candidates.

The item value extraction unit 207 extracts a character string which is an item value by using the regular expression rule 206. In the regular expression rule 206, a regular expression for extracting a character string described by an amount or a identifiable pattern is registered. For example, a regular expression is registered so that a character pattern in which one or more consecutive numbers follow "\" is extracted as an item value meaning an amount of money, that is, an item value whose attribute is an amount of money.

The combination generation unit 208 generates a combination of the item name and the item value based on the positional relationship between the item name and the item value and the attribute of the item value. First, the combination generation unit 208 determines the item value associated with the item name based on a predetermined definition. For example, in the combination generation unit 208, among the item values whose attribute is the amount of money, the positional relationship with the item name "total" is defined in advance such as "left-right", "top-bottom", "upper left-lower right". Determine the item value associated with the item name "total" based on the positional relationship. Further, the predetermined definition is stored in the ROM 102 in advance, for example, and the combination generation unit 208 reads the predetermined definition and determines the item value associated with the item name.

Then, when there is an item value associated with a plurality of item names, the combination generation unit 208 performs a narrowing process so that the item value is associated with one item name, thereby causing the item value and the item name. Generate a combination of. In other words, the combination of the item value and the item name is generated so that the item value is not associated with a plurality of item names. Details will be described later.

The item value associated with a certain item name is not limited to one (also referred to as "1: 1"), and a plurality of item values are associated with one item name (also referred to as "1: n"). Combinations may be generated.

The target selection unit 209 selects an item value and the item name that are erroneously combined with another item name as a result of generating the combination as a target for redoing the generation of the combination. Details will be described later.

If the association between the item value and the item name selected by the target selection unit 209 is incorrect, the regeneration unit 210 cancels the association and regenerates the combination of the item name and the item value. The output unit 211 outputs a combination of the item name and the item value in which the result of the re-determination by the regeneration unit 210 is reflected.

The functions of each part of FIG. 2 are realized by the CPU of the image processing device 100 expanding the program code stored in the ROM into the RAM and executing it. Alternatively, some or all the functions of each part of FIG. 2 may be realized by hardware such as an ASIC or an electronic circuit.

[Regarding the process of determining the combination of item name and item value]
Here, a process of extracting an item name and an item value included in the scanned image from the result of character recognition processing of the scanned image and generating a combination of the item name and the item value will be described with reference to the figure. The image processing apparatus of the present embodiment generates a combination of an item value and an item name so that a certain item value is not associated with a plurality of item names. In the present embodiment, in order to extract the value of the total amount of money included in the scanned image and the value of each amount of the subtotal amount included in the scanned image from the scanned image, a combination of the item name and the item value is used. Will be described as generating.

FIG. 3A is a diagram showing an example of a character recognition result obtained as a result of character recognition of a scanned image. In FIG. 3A, the recognized character string is arranged at the position indicated by the position coordinates of the character string to represent the character recognition result. The character recognition result of FIG. 3A is an example in which the character string included in the scanned image is correctly recognized. In order to extract predetermined information from this character recognition result, the item value associated with each item name is determined.

FIG. 3B is a diagram showing the result of extracting the "total" included in the item name dictionary as the item name from the character recognition result. The rectangle drawn on the outside of the character string is a rectangle indicating the character string area, and the numerical value displayed next to the character string area indicates the degree of agreement with the word dictionary. Since all the characters of the word "total" in the item name dictionary match in the "total" of the character recognition result, the degree of matching with the word in the item name dictionary is "100". Further, since only the "total" of the word "total" in the item name dictionary matches the character recognition result "subtotal", the degree of matching with the word in the item name dictionary is expressed as "50". Even if the word does not completely match the word in the item name dictionary, if the degree of match is equal to or higher than a predetermined value, the word is extracted as the character string of the word in the item name dictionary. For example, in the present embodiment, if the degree of matching is 50 or more, it is extracted as a character string of a word in the item name dictionary.

FIG. 3C is a diagram showing the result of extracting the word “subtotal” of the item name dictionary as the item name. FIG. 3D is a diagram showing the result of extracting the word “consumption tax” in the item name dictionary as the item name.

FIG. 3E is a diagram showing a result of extracting an item value whose attribute is an amount of money from the character recognition result of FIG. 3A based on a regular expression pattern indicating an amount of money. When extracting an item value, the degree of match is not displayed because the item value is extracted only when it matches the regular expression pattern.

FIG. 4 is a diagram for explaining a process of determining an item value associated with an item name based on a predetermined definition.

FIG. 4A is a diagram illustrating a process of determining an item value associated with an item name for extracting information on the total amount of money included in the scanned image. The attribute of the item value that should be associated with the item name "total" for extracting the information of the total amount is defined as the amount. The positional relationship between the item name and the item value is "item name (left) -item value (right)", "item name (top) -item value (bottom)", "item name (upper left) -item value. (Lower right) ”is defined as one of. Further, the item value associated with the item name is defined as one (in a "1: 1" relationship). The item value corresponding to the "total" extracted in FIG. 3B is determined according to this predetermined definition. As a result, as shown in FIG. 4A, "¥ 1,000", "¥ 6,000", and "¥ 6,600" of the character string area 401 are determined as item values associated with "total". Will be done.

FIG. 4B is a diagram for explaining a process of determining an item value associated with an item name in order to output a line item amount for extracting a breakdown amount of the subtotal amount. In order to extract the subtotal breakdown amount, the item value corresponding to the item name extracted by the word in the item name dictionary as "total" or "amount" is determined. However, unlike FIG. 4A, the positional relationship between the item name and the item value to be associated is defined as only "item name (top) -item value (bottom)", and the item name The associated item values are defined as being plural (having a "1: n" relationship). According to this definition, the item value associated with the item name of "total" is determined. As a result, as shown in FIG. 4B, the character string area 401 is "¥ 1,000", the character string area 402 is "¥ 2,000", and the character string area 403 is "¥ 3,000", "¥ 3,000". "6,000", "¥ 600", and "¥ 6,600" are determined as associated item values.

FIG. 4C is a diagram illustrating a process of determining an item value associated with an item name in which a word in the item name dictionary is extracted as “subtotal” or “consumption tax”. Specifically, the positional relationship of the item value to be associated with the item name is defined as only "item name (left) -item value (right)". Further, the item value associated with the item name is defined as one (in a "1: 1" relationship). When the item value associated with the item name is determined according to this definition, "¥ 6,000", "¥ 600", and "¥ 6,600" are the items of "subtotal" or "consumption tax" as shown in Fig. 4 (c). Determined as the item value associated with the name.

In the present embodiment, the item values associated with the item names of the subtotal and the consumption tax are used as the item values of the excluded items. That is, the information to be extracted that is output in this embodiment is only the value of the total amount and the value of each amount in the subtotal breakdown. Therefore, since the values of the subtotal and consumption tax amounts are not the information to be extracted, the item values associated with the item names of the subtotal and consumption tax are excluded items so that they are not extracted as the information to be extracted. Is used as the value of.

Next, comparing FIGS. 4A to 4C, it is determined that "¥ 6,000", "¥ 6,600", and "600" can be associated with a plurality of item names. .. Further, "¥ 1,000" in the character string area 401 is associated with "total", which is an item name for extracting the total amount in FIG. 4A, and is a subtotal in FIG. 4B. It is also associated with "total", which is the item name for extracting the breakdown of the amount. Further, in FIG. 4A, a plurality of item values for extracting the total amount of money are determined.

Therefore, as the next process, a narrowing process is performed so that only one of these item values corresponds to the item name. Further, as shown in FIG. 4A, when a plurality of item names are extracted as item names for extracting information, a process for narrowing down to one item name for extracting the information is performed. Will be done. By performing the narrowing process, a combination of the item name for extracting the information and the item value associated with the item name is generated.

The narrowing process is performed according to predetermined rules such as the degree of matching with words in the item name dictionary, the priority of the extraction target (total amount> subtotal breakdown amount), and the priority of the positional relationship (left / right> top / bottom). The predetermined rule is stored in the ROM 102 in advance in the same manner as the predetermined definition.

5 to 6 are diagrams for explaining an example of the narrowing down process. FIG. 5 is a diagram showing a state after one item name is narrowed down as an item name for extracting the total amount. FIG. 6 is a diagram showing a state before the process of determining whether to exclude the item value of the exclusion item.

First, one item value associated with the item name "total" for extracting the total amount having the highest extraction priority is determined. As shown in FIG. 4A, the degree of matching of the associated item names in "¥ 6,000" is smaller than that of other item names. Therefore, "¥ 6000" is canceled from the item value associated with "total". Next, since the positional relationship priority is left / right> up / down, as shown in FIG. 4A, "¥ 1,000" located under the item name is canceled from the item value associated with "total". As a result, as shown in FIG. 5A, "¥ 6,600" is determined as one item value associated with the item name "total" for extracting the total amount.

At this time, as shown in FIGS. 5 (b) and 5 (c), the item values associated with the item names for extracting the subtotal amount breakdown and the item values associated with the item names of the excluded items are There is no change from the states shown in FIGS. 4 (b) and 4 (c).

Next, the item values to be excluded are determined from the item names for extracting the subtotal breakdown amount. Since the priority is total amount> subtotal breakdown amount, among the item values of the subtotal breakdown amount in FIG. 5 (b), "¥ 6,600" determined as the item value indicating the total amount in FIG. 5 (a). Is canceled from the associated item value. The result of removing the item value having a low priority is as shown in FIG. 6 (b).

Finally, the exclusion item candidates in FIG. 6 (c) are compared with the item values associated with the item names in FIGS. 6 (a) and 6 (b), and the same item values as the item values of the exclusion items are compared. , A process of determining whether to cancel the association with the item name is performed.

Of the item values associated with the item names extracted to extract the information in FIGS. 6 (a) and 6 (b), the item values having the same values as the item values of the excluded items in FIG. 6 (c) , Compare the degree of matching of the item names associated with each item value. If the matching degree is the same or the matching degree of the item name extracted as the excluded item is larger, the association with the item name extracted for information extraction is canceled for the item value.

Explaining a specific example using a figure, first, when comparing FIG. 6 (a) and FIG. 6 (c), since there is "¥ 6,600" as an exclusion item candidate in FIG. 6 (c), " It is determined whether to exclude "¥ 6,600" from the item value of the item name indicating the total amount in FIG. 6 (a). Comparing the degree of matching of the item names associated with "¥ 6,600" between FIGS. 6 (a) and 6 (c), the degree of matching of FIG. 6 (a) is large. Therefore, it is determined that "¥ 6,600" in FIG. 6A is not excluded (cannot be canceled) from the item value associated with the item name extracted as the total amount.

Next, when FIG. 6 (b) and FIG. 6 (c) are compared, since there are "6,000 yen" and "600 yen" as exclusion item candidates in FIG. 6 (c), FIG. 6 (b) ) "¥ 6,000" and "¥ 600" are determined to be excluded from the item values associated with the item names of the subtotal amount breakdown amount. The degree of matching of the item names associated with "¥ 6,000" and "¥ 600" is the same when compared with FIGS. 6 (b) and 6 (c). If the item names have the same degree of matching, they are excluded. Therefore, among the item values associated with the item names of the subtotal amount breakdown amounts in FIG. 6B, the association between "¥ 6,000" and "¥ 600" is canceled.

7 (a) to 7 (c) are the results of the narrowing down process. In this way, a combination of the item name "total" for extracting the total amount and the item value "¥ 6,600" is generated. In addition, a combination of the item name "total" for extracting the breakdown of the subtotal breakdown amount and the item values "¥ 1,000", "¥ 2,000", and "¥ 3,000" is generated.

Of the "total" item values whose positional relationship is associated based on the definition of "item name (top) -item value (bottom)", the character string area 401 in FIGS. 4 (a) and 4 (b). There may be an item value associated with both "totals", such as "¥ 1,000" in. In this case, a rule that gives priority to the item value corresponding to the item name "total" for extracting the subtotal breakdown amount may be used. That is, in the states of FIGS. 4 (a) and 4 (b), first, the association with the item name "total" in the item value "¥ 1,000" of FIG. 4 (a) is canceled, and then thereafter. May be processed.

[When there is a misrecognition of a character string]
When character recognition processing is performed from a scanned image obtained by scanning a form, erroneous recognition of the character recognition processing may occur due to deterioration of the scanned image due to dirt or blurring of the form. When the process of generating the combination of the item name and the item value is performed from the result of the erroneously recognized character recognition process, the item value associated with the item name may not be determined correctly. For example, an item value that should be combined with a certain item name may be generated so as to be a combination with another item name.

The case where the character recognition result includes the erroneously recognized character string will be described with reference to FIGS. 8 to 10. FIG. 8A is a diagram showing an example of the character recognition result obtained as a result of character recognition of the scanned image. The character recognition result of FIG. 8 (a) should be the same as the character recognition result of FIG. 3 (a), but as shown by "■■■" in FIG. 8 (a), "consumption tax" There is a misrecognition for the character string area that should be recognized as. Even in such a case, as shown in FIGS. 8 (b) and 8 (c), the item names “total” and “subtotal” are correctly selected as in FIGS. 3 (b) and 3 (c). Is extracted. Further, the item values are also correctly extracted as in FIG. 3 (e) as shown in FIG. 8 (e). However, as shown in FIG. 8D, even if the character candidates in the lower candidates are used, even one character does not match the word "consumption tax" in the item name dictionary, so the item name of "consumption tax" is not extracted. become.

FIG. 9 is a diagram showing the result of determining the item value associated with the item name from the item name and the item value extracted in FIG. 8 according to a predetermined definition. As shown in FIG. 9 (c), since the item name "consumption tax" is not extracted, unlike FIG. 4 (c), the item value associated with "consumption tax" in which "¥ 600" is an excluded item. Not determined as.

FIG. 10 is a diagram showing the result of performing the narrowing down process from the result of the process of determining the item value associated with the item name shown in FIG. The method of narrowing down processing is the same as the method described with reference to FIGS. 5 to 6. As shown in FIG. 10 (b), as a result of not being able to recognize the "consumption tax", "¥ 600" is erroneously included as an item value to be combined with the item name "total" for extracting the subtotal breakdown amount. It remains as it is. In this way, when a character is erroneously recognized, a combination of an item name and an item value may be erroneously generated. Therefore, the information on the subtotal breakdown amount is output as it is erroneously extracted.

Therefore, in the present embodiment, a method of selecting an item value determined to be associated with a plurality of item names and an item value around the item value as a candidate for re-combination generation will be described. ..

[About the generation process of the combination of the item name and the item value]
FIG. 11 is a flowchart showing the image processing of the present embodiment. The series of processes shown in the flowchart of FIG. 11 is performed by the CPU of the image processing device 100 expanding the program code stored in the ROM into the RAM and executing it. Further, some or all the functions of the steps in FIG. 11 may be realized by hardware such as an ASIC or an electronic circuit. The symbol "S" in the description of each process means a step in the flowchart.

In S1101, the image processing unit 201 acquires a scanned image obtained by scanning the document with the scanner 109. The scanned image is stored in a storage unit such as RAM or ROM.

In S1102, the image processing unit 201 reads out the scanned image stored in the storage unit and performs image processing on the scanned image. Examples of image processing include noise removal, ruled line removal, and binarization. When the scanned image includes a small page area of a plurality of receipts, the image processing unit 201 performs a process of separating them, and performs image processing for each separated image.

In S1103, the area analysis unit 202 performs area analysis of the scanned image image-processed in S1102. The area analysis unit 202 divides the scanned image into areas having attributes such as characters, tables, images, line drawings, and lines. The area of characters and images existing inside the table is also divided, and the divided area is output as a rectangular area having attribute information. Then, the area including the character string is specified as the character string area and output.

In S1104, the character recognition unit 203 recognizes the characters included in the scanned image using the image-processed scanned image and the area analysis result of S1103, and outputs the result as the character recognition result. In the character recognition result, the combination of the character rectangle (upper left coordinate, width and height in the image) of each character, the character code of the candidate character, and the likelihood is ranked from the first candidate to a plurality of lower candidates. There is. The image used in the character recognition process may be different from the image for area analysis used in S1103. For example, the image processing unit 201 converts the scanned image into a binarized image with a threshold value for character recognition different from that for area analysis, and the character recognition unit 203 performs character recognition processing on the binarized image for character recognition. May be executed.

In S1105, the item name extraction unit 205 extracts the item name by searching the word in the item name dictionary 204 from the character recognition result. Even if the word does not completely match the word in the item name dictionary 204, if the degree of matching is equal to or higher than a predetermined value, the character string of the character recognition result is extracted as the item name. The item name extraction unit 205 generates a character string by, for example, replacing the character string generated from the first candidate of the character recognition result based on the lower candidate, the misrecognition pattern of character recognition, and the similar character pattern. To do. When the character string matches the character string indicating the item name and the degree of coincidence is equal to or more than a predetermined value, the item name extraction unit 205 sets the character string area corresponding to the character string to a character string including the item name to be searched. Extract the item name as an area.

The extracted result includes the character string extracted as the item name, the degree of matching with the word in the item name dictionary, and the information indicating the character string area (upper left coordinate in the image, width and height). In addition, the extracted result includes a reference to the character recognition result of each character.

In S1106, the item value extraction unit 207 extracts the item value from the character recognition result by searching the regular expression using the regular expression stored in the regular expression rule 206. The item value extraction unit 207 generates a character string by, for example, replacing the character string generated from the first candidate of the character recognition result based on the lower candidate, the misrecognition pattern of character recognition, and the similar character pattern. To do. Then, when the character string matches the regular expression, the item value extraction unit 207 extracts the item value assuming that the character string area corresponding to the character string is the character string area including the item value to be searched.

The extracted result includes the character string extracted as the item value and the information indicating the character string area (upper left coordinate, width and height in the image). In addition, the extracted result includes a reference to the character recognition result of each character.

In S1107, the combination generation unit 208 determines the item value associated with each item name based on a predetermined definition such as the positional relationship between the extracted item name and the extracted item value. The process of this step is the process described with reference to FIG. Further, the result of processing up to this step on the character recognition result of FIG. 8A is as shown in FIG.

In S1108, the combination generation unit 208 narrows down the item values associated with the item names generated in S1107 according to a predetermined rule so that the item values are not associated with a plurality of item names. Generate a combination of item name and item value. The details of the narrowing-down process are as described with reference to FIGS. 5 to 6. The result of processing up to this step with respect to the character recognition result of FIG. 8A is in the state shown in FIG.

The combination of the item name and the item value generated as a result of the processing of S1108 may be generated erroneously. For example, in FIG. 10B, the combination is generated so that "¥ 600" becomes a part of the item value that is the combination with the "total". Therefore, in the present embodiment, in S1109, the target selection unit 209 specifies an item value that may be associated with the item name as a combination with another item name by mistake, and regenerates the combination. Performs the process of selecting the item value and item name to be performed.

FIG. 12 (a) is a diagram in which FIGS. 9 (a) to 9 (c), which are the results of the determination process of S1107, are superimposed for explaining the process in S1109. In this step, first, the target selection unit 209 specifies an item value determined to be associated with a plurality of item names as a result of the processing of S1107.

For example, as shown in FIG. 12A, "¥ 6,000" is associated with both the item name of "total" and the item name of "subtotal". Therefore, "6,000" is specified as an item value associated with a plurality of item names. Similarly, "¥ 6,600" is also specified as an item value associated with a plurality of item names. The target selection unit 209 selects the item value associated with the specified plurality of item names and the item name associated with the item value as a target (re-judgment target) for redoing the generation of the combination. ..

Next, the target selection unit 209 also selects the item values around (neighboring) the item values associated with the specified plurality of item names as the item values to be redetermined. For example, the item value associated with the item name is often to the right of the item name or below the item name. Therefore, as shown in FIG. 12B, the target selection unit 209 has the item values of "¥ 6,000" and "¥ 6,600" associated with the specified plurality of item names, respectively. The item value in the downward or right direction from the position of is selected as the item value to be re-judged.

As a result, as shown in FIG. 12 (c), "¥ 600" and its item name are also selected as re-judgment targets. As a result of this step processing, the target selection unit 209 selects the item values of "¥ 6,000", "¥ 600", and "¥ 6,600" and the item names associated with the item values as re-judgment targets. be able to.

In addition, the target selection unit 209 selects only the item values in the downward direction or the right direction from the respective positions of the item values associated with the specified plurality of item names as the item values to be re-determined. May be good.

Alternatively, the re-judgment target may be selected based on the positional relationship included in the predetermined definition used when determining the item value associated with the item name in S1107. For example, the positional relationship between the item name and the item value is "item name (left) -item value (right)", "item name (top) -item value (bottom)", "item name (upper left) -item value (right). In the case of "below)", it is assumed that the item value is determined to be associated with the item name. In this case, even if the item value in the downward, right, or lower right direction from the position of the item value associated with the specified plurality of item names is selected as the item value included in the re-judgment target. Good.

In S1110, the regeneration unit 210 determines whether the re-determination target has been selected as a result of the processing of S1108. If the re-judgment target is selected, the process proceeds to S1111.

In S1111, the regeneration unit 210 regenerates (regenerates) the combination of the item name and the item value for the selected redetermination target. The method of regenerating the combination of the item name and the item value is not limited.

For example, the regeneration unit 210 determines whether the item value selected as the re-judgment target is actually associated with the item name by mistake, and the item value determined to be incorrect is associated with the item value. Performs the process of canceling the association with the item name. For example, the item value associated with the specified plurality of item names is used as a reference for determination.

For example, if there is an item value associated with another item name between the item name and the item value associated with that item name, either item value is incorrectly associated with the item name. Judge that the combination is generated as it is. In this method, one of the item values is deleted from the re-judgment target based on the degree of matching of the item names associated with the item values.

Alternatively, focusing on the item value "¥ 6,000" associated with a plurality of item names as shown in FIG. 12 (c), the re-judgment target "¥ 600" and the associated "total" are , It is associated so as to straddle "6,000 yen". When the matching of the item names is the same, there is a method of determining that the correspondence between the item names and the item names, which is determined so as to straddle the item values associated with the plurality of item names in this way, is incorrect. In the example of FIG. 12C, it is determined that the association between "total" and "¥ 600" is incorrect. Therefore, the regeneration unit 210 cancels the association with the "total" of "¥ 600" and regenerates the combination.

Alternatively, as a method of redoing the generation of the combination of the item name and the item value, the user may be instructed to instruct the item value that is erroneously associated with another item name. For example, among the generated combinations, the combination including the re-judgment target and the re-judgment target are displayed on the display unit 105, and the user is instructed to instruct the item value that is erroneously associated with another item name. .. Then, the regenerating unit 210 may acquire the instruction and regenerate the combination so that the item value to be redetermined is associated with the correct item name based on the user's instruction. In this case, the user does not have to select the combination to be regenerated from all the combinations, which reduces the user's trouble.

As described above, in the present embodiment, the re-judgment target is selected in S1109 after the combination is generated in S1108. Therefore, the regeneration process can be performed only for the combination including the re-determination target.

As a result of the processing of this step, the combination for extracting the total amount is as shown in FIG. 12 (d). Further, the combination for extracting the subtotal amount breakdown is as shown in FIG. 12 (e). This is the same state as in FIGS. 7A and 7B described above as the correct state. As a result of selecting the re-judgment target in this way, it is possible to prevent the erroneously generated combination from being output as it is.

When the process of S1111 is completed, or when the re-judgment target is not selected in S1110, the process proceeds to S1112. In S1112, the output unit 211 outputs the combination of the generated item name and the item value to a storage unit such as a ROM, a display unit, or another device.

As described above, according to the present embodiment, it is possible to select a target for regenerating the combination of the item name and the item value. Therefore, it is possible to suppress the output of an incorrect combination of the item name and the item value.

<Embodiment 2>
In the first embodiment, a method of selecting a re-judgment target based on the item values associated with a plurality of item names has been described. However, in the case of a scanned image that includes item values of multiple attributes, such as a scanned image that includes the amount and date, even if the scanned image is erroneously recognized, the item values will be in multiple item names. It may not be determined to be associated. Therefore, in the present embodiment, a method of enabling the re-judgment target to be selected even in such a case will be described.

The present embodiment will be described mainly on the differences from the first embodiment. The parts not specified in particular have the same configuration and processing as in the first embodiment.

FIG. 13A is a diagram showing the character strings of the character recognition result of the scanned image, which is different from that of FIG. 3A, on the coordinates. The character recognition result of FIG. 13A shows an example in which the character string included in the scanned image is correctly recognized.

FIG. 13B is a diagram showing the result of extracting the word “charge” of the item name dictionary as the item name from the character recognition result. Since all the characters of the character string "charge" of the character recognition result match the word "charge" in the item name dictionary, the degree of matching is "100". Further, since only the words "charge" and "gold" in the item name dictionary match in the character string "Kanazawa" of the character recognition result, the degree of matching is expressed as "50". Even if the word does not exactly match the word in the item name dictionary, it is extracted as the item name if the degree of match is equal to or greater than a predetermined value.

FIG. 13C is a diagram showing the result of extracting the word “date” of the item name dictionary as the item name. FIG. 13D is a diagram showing the result of extracting the item value of the amount of money as the attribute from the character recognition result of FIG. 13A based on the regular expression pattern indicating the amount of money. FIG. 13 (e) is a diagram showing a result of extracting an item value whose attribute is a date from the character recognition result of FIG. 13 (a) based on a regular expression pattern indicating a date.

14 (a) and 14 (b) are diagrams for explaining the generation process of the combination of the item name and the item value. FIG. 14A is a diagram illustrating a process of generating a combination of the item name “charge” and the item value for extracting the item value of the charge included in the scanned image.

The attribute of the item value that should be associated with the item name of "charge" is defined as the attribute of the amount. In addition, the positional relationship of the item value to be associated with the item name of "charge" is either "item name (left) -item value (right)" or "item name (top) -item value (bottom)". Is defined as. Further, the item value associated with the item name is defined as one (in a "1: 1" relationship). According to this definition, the item value associated with the item name of "charge" is determined. Then, as shown in FIG. 14A, a combination of the item name “charge” and the item value “¥ 14,320” indicating the amount of money is generated. Since the item name "charge" has a higher degree of matching than the item name "Kanazawa", as a result of the narrowing down process, a combination is generated so that "¥ 14,320" can be associated with the "charge".

FIG. 14B is a diagram illustrating a process of generating a combination of the item name “date” and the item value for extracting the item value of the date included in the scanned image. The item value attribute that should be associated with the "date" item name is defined as the date attribute. In addition, the positional relationship of the item value to be associated with the item name of "date" is either "item name (left) -item value (right)" or "item name (top) -item value (bottom)". Is defined as. Further, the item value associated with the item name is defined as one (in a "1: 1" relationship). According to this definition, the item value associated with the item name of "date" is determined. As a result, a combination of the date and the item value "January 1, 2019" is generated in the item name "date".

In this way, even if the scanned image contains item values with different attributes such as amount and date, the item values corresponding to the item names are associated with each other, and the combination of the item names and the item values can be obtained. Can be generated.

Next, a case where the character recognition result includes a result of erroneous recognition will be described. FIG. 15A is a diagram showing an example of the character recognition result obtained as a result of character recognition of the scanned image. The character recognition result of FIG. 15 (a) should be the same as the character recognition result of FIG. 13 (a), but it is recognized as a "charge" as shown by "■■" in FIG. 15 (a). There is a misrecognition of the character string area to be done. In addition, "Kanazawa" is misrecognized as "Kin ■". Even in such a case, since "gold" is included in the item name dictionary "charge", it is extracted as a character string of the item name "charge". On the other hand, for "■■", even if the character candidates in the lower candidates are used, even one character does not match with "charge", so that it is not extracted as the item name of "charge".

FIG. 15B is a diagram showing a combination of the item name and the item value generated by associating the item name with the item name extracted in the character recognition result of FIG. 15A. In this way, the item value "¥ 14,320" is erroneously associated with "gold ■" to generate a combination.

Therefore, in the present embodiment, in order to select the re-judgment target from FIG. 15B, a line segment connecting the item name and the item value is set, and a method of selecting the re-judgment target based on the line segment will be described.

Specifically, the line segment connecting the point in the character string area containing the item name and the point in the character string area including the item value associated with the item name is the item value and the item value. Set for the item name associated with. Then, if there are line segments that intersect each other among the set line segments, the combination of the item name and the item value in the character string area located at the end point of each intersecting line segment is selected as the re-judgment target. Will be done.

Explaining with reference to FIG. 15 (c), a line segment 1501 connecting the character string areas of "gold ■" extracted as the item name and "¥ 14,320" associated with "gold ■" is Set. Next, a line segment 1502 connecting the "date" extracted as the item name and the "January 1, 2019" associated with the "date" is set. In FIG. 15C, since the line segment 1501 and the line segment 1502 intersect, it is determined that there is an intersecting line segment. Then, "¥ 14,320" at the end of one intersecting line segment 1501 and its item name are selected as re-judgment targets, and similarly, at the end of the other intersecting line segment 1502. A certain "date" and its item name are selected as re-judgment targets.

The line segment is, for example, a line segment connecting any corner of the character string area of the item name and any corner of the character string area of the item value, and is the length of the line segment as shown in FIG. 15 (c). The line segment is set by determining the point where is the longest. For example, if the positional relationship between the character string area of the item name and the character string area of the item value is up and down, the upper corner of the upper character string area and the lower corner of the lower character string area The line connecting the lines is set. Also, when the positional relationship between the character string area of the item name and the character string area of the item value is left and right, the corner on the left side of the character string area on the left and the corner on the right side of the character string area on the right are The connecting line is set.

The method of selecting the re-judgment target according to the present embodiment is also applicable to the character recognition result including only the item value of a single attribute.

FIG. 16A is a diagram in which the processing results of FIGS. 10A to 10B showing the results of the combinations generated by the combination generation unit 208 in S1108 are superimposed. FIG. 16B is a diagram showing a line segment connecting the item value of each combination and the item name to which the item value is associated. The item name "total" for extracting the subtotal breakdown amount is a line segment connected to each of the associated item values "¥ 1,000", "¥ 2,000", "¥ 3,000", and "¥ 600". 1601a to d are set.

Similarly, a line segment 1602 connecting the “subtotal” extracted as the item name and the “¥ 6,000” associated with the “subtotal” is set. Further, a line segment 1603 connecting the extracted "total" and the "¥ 6,600" associated with the "total" is set as the item name for extracting the total amount. Since the line segment 1601d and the line segment 1602 intersect, it is determined that there is an intersecting line segment.

Therefore, as shown in FIG. 16C, "¥ 600" at the end of the line segment 1601d and its item name are selected as re-judgment targets. Further, "¥ 6,000" at the end of the line segment 1602 and its item name are selected as re-judgment targets. As described above, also by the method of the present embodiment, it is possible to select the item value to be re-determined including "¥ 600".

As described above, according to the present embodiment, it is possible to select the re-judgment target even when the character recognition result includes item values of a plurality of different attributes.

<Embodiment 3>
In the second embodiment, a method of selecting a re-judgment target by setting a line segment between the item name and the item value has been described. However, if the character string area is not specified correctly, the line segments that should normally intersect may not intersect. In the present embodiment, a method of enabling the re-judgment target to be selected even when the character string area is not correctly specified will be described.

The present embodiment will be described mainly on the differences from the second embodiment. The parts not specified in particular have the same configuration and processing as in the second embodiment.

FIG. 17A is a diagram based on the character recognition result which should be the character recognition result shown in FIG. 3A. FIG. 17A shows a state after the line segment is set in order to select the re-judgment target by the method of the second embodiment. However, in FIG. 17A, “¥ 6,0” associated with the “subtotal” is a character string that is erroneously recognized because the character string area is not correctly specified, and is originally shown in FIG. As shown in (a), it is assumed that the character string should be recognized as "¥ 6,000". The line segment 1701 connects the character string areas of the item name "subtotal" and "¥ 6,0" associated with the item name "subtotal". In FIG. 17A, unlike FIG. 16C, the line segment 1701 and the line segment 1601d do not intersect. Therefore, the item value "¥ 600" or "¥ 6,0" may not be selected as the re-judgment target.

Therefore, in the present embodiment, the line segments are made to intersect even when the erroneous recognition of the character occurs because the character string area is not correctly specified. Specifically, the target selection unit 209 determines whether the lateral component values of the coordinate values at the right ends of the character string areas existing above and below the character string area to be processed, which is the end of the line segment, are the same. judge. If it can be determined that they are the same, the target selection unit 209 sets a line segment for the right end position of the character string area to be processed based on the lateral component value of the right end coordinate value of any of the upper and lower character strings. For example, the target selection unit 209 modifies the size of the character string area to be processed so that the right end of the character string area to be processed is the same as the upper and lower character string areas, and sets the line segment. It should be noted that the same does not have to be exactly the same, and for example, it may be the same as long as the difference is within a predetermined range.

FIG. 17B is a diagram showing a line segment 1702 set after correcting the right end position of the character string area including the item value of “6,0” which is the character string area to be processed. By correcting the right end position of the character string area to be processed in this way, the line segment 1702 and the line segment 1601d intersect. Therefore, the item values "¥ 6.0" and "¥ 600" can be selected as re-judgment targets.

The target selection unit 209 determines whether the lateral component values of the leftmost coordinate values of the character string areas existing above and below the character string area to be processed are the same. Then, when it can be determined that they are the same, the target selection unit 209 may set a line segment for the left end position of the character string area to be processed according to the left end of any of the upper and lower character strings.

Also, in the explanation so far, the line segment set to be drawn from the upper left to the lower right has been illustrated and explained, but the line segment is set to be drawn from the upper right to the lower left. May be good.

In FIG. 17C, when the positional relationship between the item name and the item value is vertically related, the line segment 1703 connecting the upper right of the character string area on the upper side and the lower left of the character string area on the lower side is It is set. As shown in FIG. 17C, by setting the line segment 1703, the line segment 1703 and the line segment 1701 intersect even when the character string area is not correctly specified. Therefore, the item values "¥ 6.0" and "¥ 600" can be selected as re-judgment targets. In this way, when the re-judgment target is not selected, it may be determined whether there is an intersecting line segment when a line segment drawn from the upper right to the lower left is set, and the re-judgment target may be selected. ..

Alternatively, the target selection unit 209 sets a line segment in the direction from the upper left to the lower right and a line segment from the upper right to the lower left for the item name and all the item values associated with the item name. Set multiple patterns of line segments as if they were set. Then, the target selection unit 209 determines whether there is a pattern in which the line segments intersect among the plurality of patterns, and if there is a pattern in which the line segments intersect, in the character string area at the end of each intersecting line segment. Based on this, the re-judgment target may be selected.

As described above, according to the present embodiment, the re-judgment target can be selected even when the character string area is not correctly specified.

<Other Embodiments>
In the above-described embodiment, the consumption tax amount, which is the tax amount, and the subtotal, which is the item obtained by subtracting the consumption tax amount from the total amount, are described as excluded items. In addition, the item values of the subtotal and the consumption tax amount may be used to obtain the certainty of the numerical value of the amount extracted as the total amount.

As a result of the regeneration process of S1111 in the flowchart of FIG. 11, in FIG. 12 (f), "¥ 6,000" is determined as the item value associated with the "subtotal", but the "consumption tax amount" The item value has not been determined. In FIG. 12 (d), since the total amount is determined to be "¥ 6,600", the item value "¥ 600" is the consumption tax amount by acquiring that the consumption tax rate is 10% as external information. Can be determined to be equal to. Similarly, from the subtotal "6,000" and the consumption tax rate, it can be determined that the item value "¥ 600" is equal to the consumption tax amount.

Therefore, the item value of the consumption tax amount may be derived based on "¥ 6,600" extracted as the total amount or "¥ 6,000" which is the item value of the subtotal. Alternatively, when any of the consumption tax, the subtotal amount, and the total amount is not extracted, the unextracted item value may be derived from the consumption tax rate and the extracted item value.

That is, it may not be possible to generate a combination with the item value among the item names of the total amount, the consumption tax amount, and the subtotal amount. In this case, the item value of the item name for which the combination was not generated may be derived based on the combination of the item names of the total amount, the consumption tax amount, and the subtotal amount generated by the combination generation unit 208.

If the consumption tax amount can be determined, it can be verified that the subtotal "¥ 6,000" + the consumption tax amount "¥ 600" = the total amount "¥ 6,600". Therefore, it can be determined that the total "¥ 6,600" can be accurately extracted. In this way, it may be important to extract something other than the extraction target in order to show that the output target is correct. Therefore, the item value extracted as the exclusion item may be used to confirm the accuracy of the generated combination. That is, the image processing device may have a checking unit that checks the item values indicating the total amount, the consumption tax amount, and the subtotal amount based on the item values indicating the total amount, the consumption tax amount, and the subtotal amount.

The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100 Image processing device 203 Character recognition unit 205 Item name extraction unit 207 Item value extraction unit 208 Combination generation unit 209 Target selection unit

Claims (17)

An image processing device that extracts item names and item values included in a scanned image and generates a combination of one item name and one or more item values.
Specific means for identifying the character string area included in the scanned image, and
A character recognition means for recognizing a character string in the character string area and
An extraction means for extracting an item value and an item name from the recognized character string, and
A determination means for determining the extracted item value associated with the extracted item name based on at least the positional relationship between the item value and the item name.
A generation means for generating the combination so that the determined item value is not associated with a plurality of item names.
A selection means for selecting an object including an item name and an item value for which the generation is redone using the positional relationship, and
An image processing device characterized by having. The selection means
The item value determined by the determination means is specified so as to be associated with a plurality of item names, and the item value existing at the position of the scanned image in the downward direction or the right direction of the specified item value is at least said. The image processing apparatus according to claim 1, wherein the image processing apparatus is selected as an item value included in the target. The selection means
A line segment connecting a point in the character string area in which the item name is recognized and a point in the character string area in which the item value associated with the item name is recognized is set, and the set line segment is set. The image processing apparatus according to claim 1, wherein when there is a line segment that intersects with another line segment, the target is selected based on the character string area at the end of the intersecting line segment. The selection means
Of the intersecting line segments, from the item value recognized from the character string area at the end of the first line segment and the character string area at the end of the second line segment intersecting with the first line segment. The image processing apparatus according to claim 3, wherein the recognized item value and the recognized item value are selected as the item value included in the target. The selection means
The image processing apparatus according to claim 3 or 4, wherein as a point in the character string region, a point at which the line segment is longer than a line segment connecting other points is set. The image processing apparatus according to claim 3 or 4, wherein the character string area is rectangular, and points in the character string area are located at any corner of the character string area. The selection means
When the positions of the character string area of the extracted item name and the character string area of the item value associated with the item name are up and down, the upper corner and the lower corner of the upper character string area are located. A line segment connecting the lower corners of the character string area is set, and when the positions are left and right, the corners on the left side of the character string area on the left and the right side of the character string area on the right are set. The image processing apparatus according to claim 6, wherein a line segment connecting a certain corner is set. The selection means
A line segment of a plurality of patterns connecting the character string area of the extracted item name and the character string area of the item value associated with the item name is set, and the line segments of the plurality of patterns intersect. The image processing apparatus according to any one of claims 3 to 7, wherein when there is a pattern, the target is selected based on the character string area at the end of each intersecting line segment. The selection means
Any one of claims 3 to 8, wherein the line segment is set for the item name included in the combination generated by the generation means and the item value associated with the item name. The image processing apparatus according to. The character to be processed for setting the line segment based on the character string area located above the character string area to be processed and the character string area located below the character string area to be processed. The image processing apparatus according to any one of claims 3 to 9, further comprising a correction means for correcting the size of the column region. The correction means
When the lateral component values of the rightmost coordinates in the upper and lower character string areas of the character string area to be processed are the same, the size is modified so as to match the right end position of the upper and lower character string areas. The image processing apparatus according to claim 10. The correction means
When the lateral component values of the leftmost coordinates in the upper and lower character string areas of the character string area to be processed are the same, the size is modified so as to match the left end position of the upper and lower character string areas. The image processing apparatus according to claim 10 or 11. The scanned image is an image obtained by scanning a manuscript containing the total amount, the tax amount included in the total amount, and the subtotal amount which is the amount obtained by subtracting the tax amount from the total amount.
Among the item names for extracting the total amount, the tax amount, and the subtotal amount, if there is an item name for which a combination with the item value is not generated, the item name indicating the total amount, the tax amount, or the subtotal amount is used. The image processing apparatus according to any one of claims 1 to 12, further comprising a derivation means for deriving the item value of the item name for which the combination was not generated based on the associated item value. .. The extraction means
Any of claims 1 to 13, characterized in that the recognized character string is extracted as the item name if the degree of matching with the word indicating the item name among the recognized character strings is equal to or more than a predetermined value. The image processing apparatus according to item 1. By determining whether the association between the item value of the target selected by the selection means and the item name is incorrect, and canceling the association between the item value of the target and the item name determined to be incorrect, the above. The image processing apparatus according to any one of claims 1 to 14, further comprising a second generation means for regenerating the combination. An image processing method that extracts item names and item values included in a scanned image and generates a combination of one item name and one or more item values.
A specific step for identifying the character string area included in the scanned image, and
A character recognition step for recognizing a character string in the character string area, and
An extraction step for extracting an item value and an item name from the recognized character string, and
A determination step of determining the extracted item value associated with the extracted item name based on at least the positional relationship between the item value and the item name.
A generation step of generating the combination so that the determined item value is not associated with a plurality of item names, and
A selection step of selecting an object including an item name and an item value for which the generation is redone using the positional relationship, and
An image processing method characterized by having. A program for causing a computer to function as each means of the image processing apparatus according to any one of claims 1 to 15.

Images