JP7445364B2 - Image processing program, image processing method, and image processing device - Google Patents

Description

The present invention relates to an image processing program, an image processing method, and an image processing apparatus.

2. Description of the Related Art As one of the form processing systems used in banks and the like, a system that handles forms with a notification seal affixed thereto is known. This type of form processing system includes an image processing device. The image processing device obtains an optically read image of the form, extracts the impression of the notification seal from the obtained image of the form, and electronically converts the extracted impression of the notification seal and the impression of the notification seal registered in advance. Processes such as verification (so-called seal verification) are performed.

The following methods are known as methods for extracting the impression of a notification seal from an image of a form. For example, a method (method 1) of extracting the impression of a notification seal based on the position of a predefined notification seal field is known (for example, see Patent Document 1). In addition, for example, the position of the notification seal column in the image of the form is detected using a character string indicating the item name of the notification seal (for example, "notice seal"), and the notification seal is determined based on the detected position of the notification seal column. A method (Method 2) of extracting a seal impression is also known. Furthermore, for example, a method is known in which customer seal impressions are extracted by detecting seal impressions from an image of a form and excluding previously registered excluded seal impressions (for example, an inspection seal impression) from among the detected seal impressions. (For example, see Patent Document 2).

In addition, as a method for extracting the seal imprint of a notification seal from an image of a form without using a character string indicating the item name of the notification seal as in method 2, for example, an object detection AI (artificial A method (Method 3) of extracting the seal impression of the notification seal using the date stamp area specifying technology described in the patent application No. 2018-091876 by the applicant of the present application is also considered.

Japanese Patent Application Publication No. 2011-197787 JP2013-167928A

However, in method 1, it is necessary to define the position of the notification seal column in advance, so when handling various forms with different positions of the notification seal column, it takes a lot of effort to define it. Become. For example, a bank account transfer request form, which is one of the forms, is created by the receiving institution (payee), so the bank transfer request form has a different layout (i.e., a bank transfer request form with a different position of the notification seal column). There are many types, and it is not easy to individually define the position of the notification seal field.

In method 2, when the design of the notification seal field is high, it may not be possible to detect the position of the notification seal field using the character string indicating the item name of the notification seal. In general, the notification seal field tends to have a high design quality, unlike the account number field, amount field, etc. FIG. 13 is a diagram illustrating a notification seal column with a high design quality. The notification seal column illustrated in FIG. 13 is a notification seal column in which a character string indicating the item name of the notification seal is displayed in outline. In the case of such a notification seal column, it may not be possible to detect the position of the notification seal column using the character string indicating the item name of the notification seal.

In method 3, if there are other seal impressions such as a correction seal, inspection seal, sample seal, etc. in addition to the seal impression of the notification seal, it is not possible to uniquely identify the seal impression of the notification seal. FIG. 14 is a diagram illustrating a seal impression other than the notification seal. The seal imprint illustrated in (a) of FIG. 14 is an imprint of an inspection seal, and the seal imprint illustrated in (b) of FIG. 14 is an imprint of a sample seal. In this way, if there are seal impressions other than the notification seal, it is not possible to uniquely identify the seal imprint of the notification seal.

In view of the above-mentioned circumstances, it is an object of the present invention to provide a technology that makes it unnecessary to predefine the position of the notification seal column and allows extraction of the impression of the notification seal from the image of the form.

One aspect of the program is an image processing program that detects a keyword area indicating the item name of a notification seal in an image generated by scanning a form created by a receiving institution that uses a form-based service. and a second detection process for detecting a seal imprint area in the image of the form, and based on the execution results of the first detection process and the second detection process, the image of the form is The method is characterized by causing a computer to execute a process of specifying the area of the seal impression of the notification seal, and the specifying process means the notification seal by executing the first detection process and the second detection process. When a plurality of keyword areas indicating item names are detected and a plurality of seal imprint areas are detected, based on the positional relationship between the plurality of keyword areas and the plurality of seal imprint areas, the notification seal is It is characterized by specifying the region of the seal impression to be verified .

One aspect of the method is an image processing method in which a first method detects a keyword area indicating the item name of a notification seal in an image generated by scanning a form created by a receiving institution that uses a form-based service. and a second detection process for detecting a seal imprint area in the image of the form, and based on the execution results of the first detection process and the second detection process, the image of the form is The area of the seal imprint of the notification seal is specified , and the identification includes a plurality of areas of keywords indicating item names meaning the notification seal by executing the first detection process and the second detection process . and when a plurality of areas of the seal impression are detected, specifying an area of the seal impression of the notification seal to be verified based on the positional relationship between the area of the plurality of keywords and the area of the plurality of seal impressions. It is characterized by

One aspect of the device is that it is an image processing device that detects a keyword area indicating the item name of a notification seal in an image generated by scanning a form created by a receiving institution that uses a form-based service. a notification seal item name detection unit that executes a detection process, a seal imprint detection unit that executes a second detection process that detects a seal imprint area in the image of the form, and the first detection process and the second detection process. and a notification seal imprint identification unit that specifies a seal imprint area of a notification seal in the image of the form based on the execution result of the process, and the identification is performed by the first detection process and the second detection process. When a plurality of keyword areas indicating the item name meaning the notification seal are detected by executing the detection process, and a plurality of areas of the seal impression are detected, the area of the plurality of keywords and the area of the plurality of seal impressions are detected. The method is characterized in that an area of the seal imprint of the notification seal to be verified is specified based on the positional relationship of the notification seal.

According to the present invention, there is no need to predefine the position of the notification seal column, and it is possible to extract the impression of the notification seal from the image of the form.

1 is a diagram illustrating the configuration of a form processing system. FIG. 1 is a diagram illustrating a functional configuration of an image processing device according to an embodiment. 3 is a flowchart illustrating image processing according to an embodiment. FIG. 3 is a diagram schematically showing a specific example of image processing according to an embodiment. 12 is a flowchart illustrating details of the process in S5. It is a figure which shows typically the specific example of the process of S5. It is a flowchart illustrating the details of the process of S51. FIG. 3 is a diagram (part 1) schematically showing a specific example of the process of S51. FIG. 7 is a diagram (part 2) schematically showing a specific example of the process of S51. 12 is a flowchart illustrating details of the process of S53. It is a figure which shows typically the specific example of the process of S53. FIG. 2 is a diagram illustrating the hardware configuration of a computer. It is a diagram illustrating a notification seal field with a high design quality. It is a figure which illustrates the imprint of a seal other than a notification seal.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a diagram illustrating the configuration of a form processing system.

A form processing system 1 illustrated in FIG. 1 is a system that handles forms stamped with a notification seal. The form processing system 1 includes an image processing device 2, a scanner device 3, an input device 4, a display device 5, and a server device 6.

The image processing device 2 extracts the impression of the notification seal and other necessary information from the image of the form, and, for example, electronically compares the extracted impression of the notification seal with the impression of a previously registered notification seal. The image processing device 2 obtains an image of a form from the scanner device 3, for example. The scanner device 3 is a device that converts paper media such as forms into digital data (images).

The input device 4 is a device that inputs various information to the image processing device 2. The input device 4 is, for example, a keyboard device. The display device 5 is a device that displays various information such as the contents and results of processing performed by the image processing device 2, information input through the input device 4, and images of forms. The display device 5 is, for example, a liquid crystal display.

The server device 6 holds a character recognition dictionary that the image processing device 2 refers to when recognizing characters in an image, and a keyword table that the image processing device 2 refers to when extracting keywords from the character recognition results. It is a device. The server device 6 is communicably connected to the image processing device 2 via the network 7 .

FIG. 2 is a diagram illustrating a functional configuration of an image processing apparatus according to an embodiment.
As illustrated in FIG. 2, the image processing device 2 includes a scanner control unit 210, a character recognition processing unit 220, a character string extraction unit 230, a seal imprint detection unit 240, a notification seal imprint identification unit 250, and a display control unit 210. 260 and a main control section 270. Further, the image processing device 2 includes a storage unit (not shown).

The scanner control unit 210 controls the process of acquiring an image of a form from the scanner device 3.
The character recognition processing unit 220 refers to the character recognition dictionary 610 held by the server device 6 and recognizes characters included in the image of the form.

The character string extraction unit 230 refers to the keyword table 620 held by the server device 6 and extracts a keyword (character string) indicating the item name from the character recognition result by the character recognition processing unit 220. In the keyword table 620, character strings that can be keywords indicating item names are registered in advance. For example, "account number," "account," "bank transfer number," etc. are registered as character strings that can be keywords indicating the item name of an account number, and "notification seal" is registered as a character string that can be a keyword indicating the item name of a notification seal. ”, “delivery seal”, “registration seal”, etc. are registered.

Further, the character string extraction unit 230 extracts a character string (for example, “1234567”) corresponding to a keyword indicating an item name (for example, “account number”) from the character recognition result by the character recognition processing unit 220.

Further, the character string extraction unit 230 includes a notification seal item name detection unit 231.
The notification seal item name detection unit 231 executes a first detection process of detecting a keyword area (for example, a rectangular area including the keyword) indicating the item name of the notification seal in the image of the form. The first detection process is performed using, for example, existing technology.

The seal imprint detection unit 240 executes a second detection process of detecting a seal imprint area (for example, a rectangular area including a seal imprint) in the image of the form. The second detection process is performed using, for example, an object detection AI technology such as deep learning or a date stamp area identification technology described in the patent application No. 2018-091876 filed by the applicant of the present application.

The notification seal imprint identification unit 250 determines the seal imprint area of the notification seal in the image of the form based on the execution results of the first detection process by the notification seal item name detection unit 231 and the second detection process by the seal imprint detection unit 240. Identify. For example, by executing the first detection process by the notification seal item name detection unit 231 and the second detection process by the seal imprint detection unit 240, one or more keyword areas indicating the item name of the notification seal are detected, and the area of the seal impression is detected. If one or more are detected, the area of the seal impression of the notification seal is specified based on the positional relationship between the area of the keyword indicating the item name of the notification seal and the area of the seal impression. In this case, for example, a seal imprint area that satisfies a predetermined condition is specified as a seal imprint area of the notification seal. The predetermined conditions include, for example, the condition that the keyword exists on the right side or below the area of the keyword indicating the item name of the notification seal, and the distance from the keyword area is less than or equal to a predetermined value, or the condition that the keyword indicating the item name of the notification seal The condition is that the keyword exists on the right side or below the area, the distance from the keyword area is less than or equal to a predetermined value, and the area size is larger than the predetermined size. Further, for example, if only one seal imprint area is detected by executing the first detection process by the notification seal item name detection unit 231 and the second detection process by the seal imprint detection unit 240, the seal imprint area is , specified as the area of the seal impression of the notification seal.

The display control unit 260 controls the display on the display device 5. For example, the display control unit 260 generates screen data including the results of processing an image of a form, and causes the display device 5 to display (output) the screen data.

The main control unit 270 controls a series of processes performed by the scanner control unit 210, character recognition processing unit 220, character string extraction unit 230, seal imprint detection unit 240, notification seal imprint identification unit 250, and display control unit 260.

FIG. 3 is a flowchart illustrating image processing according to one embodiment. FIG. 4 is a diagram schematically showing a specific example of image processing according to an embodiment.

When the image processing device 2 starts image processing for extracting necessary information from a form, it first acquires an image of the form, as illustrated in FIG. 3 (S1). The process of S1 is performed by the scanner control unit 210.

Next, the image processing device 2 refers to the character recognition dictionary 610 held by the server device 6 and recognizes the characters included in the image of the form (S2). The process of S2 is performed by the character recognition processing unit 220.

Next, the image processing device 2 refers to the keyword table 620 held by the server device 6 and extracts keywords indicating the item name from the character recognition results, and also indicates the item name of the notification seal in the image of the form. A first detection process for detecting a keyword area is executed (S3). The process of S3 is performed by the character string extraction unit 230, and in particular, the first detection process of the process of S3 is performed by the notification seal item name detection unit 231 included in the character string extraction unit 230.

Next, the image processing device 2 extracts a character string corresponding to the keyword indicating the item name from the character recognition result (S4). The process of S4 is performed by the character string extraction unit 230.

Next, the image processing device 2 executes a second detection process for detecting a seal imprint area in the image of the form, and also performs a second detection process on the image of the form based on the execution results of the first detection process and the second detection process. The region of the seal imprint of the notification seal is specified (S5). The process of S5 is performed by the seal imprint detection section 240 and the notification seal imprint identification section 250.

Next, the image processing device 2 causes the display device 5 to display the processing results for each item name (S6). For example, as a result of processing the item name of a notification seal, the coordinates indicating the area of the seal imprint of the specified notification seal (for example, the coordinates of two diagonal points of a rectangular area containing the seal imprint) are displayed, and items other than notification seals are displayed. As a result of name processing, a character string representing the content of the keyword representing the item name and coordinates representing the area of the character string (for example, coordinates of two diagonal points of a rectangular area containing the character string) are displayed. The process of S6 is performed by the display control unit 260.

According to such image processing, for example, in the process of S1, an image 8 of a form as schematically shown in FIG. 4(a) is obtained. The image 8 of the form includes an area 801 for a keyword (``account number'') indicating the item name of the account number, an area 802 for the character string (``1234567'') corresponding to the keyword indicating the item name of the account number, and a notification stamp. areas 803, 804, and 805 for keywords (``notification stamps'') that indicate the item name, an area 806 for seal impressions with notification seals, and areas for seal impressions with seals other than notification seals (for example, correction stamps). 807. In the keyword areas 803, 804, and 805 that indicate the item name of the notification seal, area 803 indicates the keyword area that indicates the item name of the original notification seal, and areas 804 and 805 indicate the area of the keyword that indicates the item name of the notification seal included in the explanatory text, etc. Indicates an area for keywords indicating item names.

Thereafter, in the process of S3, as schematically shown in FIG. The character string (``notification seal'') contained in areas 803, 804, and 805 is extracted as a keyword indicating the item name of the notification seal, and the areas 803, 804, and 805 indicate the item name of the notification seal. Detected as a keyword area.

Thereafter, in the processes of S4 and S5, as schematically shown in FIG. The character string corresponding to the keyword is extracted, and the area 806 is specified as the area of the seal impression of the notification seal.

Thereafter, in the process of S6, the processing results (contents, coordinates) for each item name are displayed on the display device 5, as shown in FIG. 4(d). For example, as a result of processing the item name of account number, the character string ("1234567") corresponding to the keyword indicating the item name of account number and the coordinates of the area of the character string ("(x1, y1, x2, y2) ") is displayed, and the coordinates ("(x3, y3, x4, y4)") of the area of the seal impression of the notification seal are displayed as the processing result of the item name of the notification seal. Note that since the character string corresponding to the keyword indicating the item name of the notification seal does not originally exist, it is not included in the processing result of the item name of the notification seal.

FIG. 5 is a flowchart illustrating details of the process in S5. FIG. 6 is a diagram schematically showing a specific example of the process of S5.

As illustrated in FIG. 5, when the process of S5 is started, first, the seal imprint detection unit 240 executes a second detection process of detecting a seal imprint area in the image of the form (S51). Note that the seal imprint area detected in the process of S51 is also a candidate for the seal imprint area of the notification seal in the image of the form.

Next, the notification seal imprint identifying unit 250 determines whether a plurality of seal imprint areas have been detected by the execution of the second detection process by the seal imprint detecting unit 240 (S52).

If the determination result in S52 is YES, the notification seal imprint identification unit 250 identifies the form on the basis of the execution results of the first detection process by the notification seal item name detection unit 231 and the second detection process by the seal imprint detection unit 240. The area of the seal imprint of the notification seal in the image is specified (S53), and the coordinates indicating the identified area of the seal imprint of the notification seal are output to the main control unit 270 (S54). However, if the region of the seal impression of the notification seal cannot be specified in the process of S53, the coordinates are not output.

On the other hand, if the determination result in S52 is NO, the notification seal imprint identification unit 250 determines whether the number of seal imprint areas detected by the execution of the second detection process by the seal imprint detection unit 240 is 1 or 0. The next process is performed depending on the situation (S54). If the number of detected seal imprint areas is one, the area of the seal imprint is specified as the area of the seal imprint of the notification seal in the image of the form, and the coordinates indicating the area of the specified seal imprint of the notification seal are controlled as the main control. 270. On the other hand, if the number of detected seal imprint areas is 0, it is assumed that there is no seal imprint area of the notification seal in the form image, and the coordinates are not output.

According to the process illustrated in FIG. 5, for example, in the process of S51, areas 806 and 807 in the image 8 of the form are detected as a seal imprint area, as schematically shown in FIG. be done.

Thereafter, in the process of S53, as schematically shown in FIG. 6(b), an area 806 in the image 8 of the form is specified as the area of the seal impression of the notification seal.

Thereafter, in the process of S54, as shown in FIG. be done.

FIG. 7 is a flowchart illustrating details of the process in S51. FIGS. 8 and 9 are diagrams schematically showing a specific example of the process of S51.

As described above, the seal imprint detection unit 240 that performs the process of S51 performs the second detection process using object detection AI technology such as deep learning, or as described in the patent application No. 2018-091876 by the applicant of the present application. This is done using the date stamp area identification technology. Here, an example will be explained in which the object detection AI technology such as deep learning is used.

As illustrated in FIG. 7, when the process of S51 is started, the seal imprint detection unit 240 first converts the size of the form image to fit the learned neural network model (S511). In response to input of an image of a predetermined size, the trained neural network model outputs coordinates indicating the area of the object in the image and feature amounts indicating the likelihood of the object being a seal impression. The object area is an area determined as a group of objects.

Next, the seal imprint detection unit 240 reads the trained neural network model (S512). The trained neural network model can be managed by a storage unit (not shown) included in the image processing device 2 or by the server device 6.

Next, the seal imprint detection unit 240 inputs the size-converted image of the form into the loaded neural network model, and obtains the coordinates indicating the area of the object and the feature amount representing the seal imprint-likeness (S513). ).

Next, the seal imprint detection unit 240 determines the coordinates indicating the area of the object where the feature amount representing the seal impression-likeness exceeds a predetermined value from the obtained coordinates indicating the area of the object and the feature amount representing the seal imprint-likeness. It is determined that the coordinates shown are the coordinates shown (S514).

Next, the seal imprint detection unit 240 converts the size-converted image of the form to the original size, and converts the coordinates indicating the area of the object determined as the coordinates indicating the area of the seal imprint to the original size. (S515).

Next, the seal imprint detection unit 240 outputs the converted coordinates to the main control unit 270 as coordinates indicating the area of the seal imprint (area candidate for the seal imprint of the notification seal) in the image of the form (S516).

In this way, the seal imprint detection unit 240 detects the area of the seal imprint in the image of the form, and outputs the coordinates indicating the area.

According to the process illustrated in FIG. 7, for example, in the process of S511, as schematically shown in FIG. Here, the size is converted to a size of 512 pixels x 512 pixels).

Thereafter, in the process of S513, as schematically shown in FIG. The indicated coordinates and the feature amount representing the seal imprint-likeness are acquired. In FIG. 8B, the coordinates of regions 808, 809, 810, 811, 812, 813, and 814 are obtained as the coordinates of the object region. Although omitted in FIG. 8(b), by performing the process of S513, the feature amount representing the seal imprint-likeness for the object in each area of areas 808, 809, 810, 811, 812, 813, and 814 is added. is obtained.

Thereafter, in the process of S514, as schematically shown in FIG. It is determined as the coordinates indicating the area of the seal impression. Note that the value of the feature amount approaches 1 as the object becomes more similar to the seal impression.

Thereafter, in the process of S515, as schematically shown in FIG. 9(b), the size-converted form image 8a is converted to the original size, and the coordinates of the object determined to indicate the area of the seal imprint are Coordinates indicating areas 811 and 812 are converted to coordinates in the image 8 of the form whose size has been converted to the original size. Thereby, the coordinates indicating the areas 806 and 807 in the image 8 of the form are obtained as the coordinates after the conversion.

Thereafter, in the process of S516, as shown in FIG. "(x5, y5, x6, y6)") is output to the main control unit 270 as coordinates indicating the seal imprint area (region candidate for the seal imprint of the notification seal) in the image 8 of the form.

FIG. 10 is a flowchart illustrating details of the process of S53. FIG. 11 is a diagram schematically showing a specific example of the process of S53.

As illustrated in FIG. 10, when the process of S53 is started, the notification seal imprint identification unit 250 first indicates a keyword area indicating the item name of the notification seal detected by the notification seal item name detection unit 231. The coordinates and the coordinates indicating the seal imprint area detected by the seal imprint detection unit 240 are mapped onto a coordinate system corresponding to the image of the form (S531). That is, the area of the keyword indicating the item name of the notification seal and the area of the seal imprint are mapped.

Next, the notification seal imprint specifying unit 250 specifies the seal imprint area of the notification seal based on the positional relationship between the area of the keyword indicating the mapped item name of the notification seal and the seal imprint area (S532). In this identification, for example, a seal imprint area that satisfies a predetermined condition is specified as a seal imprint area of the notification seal. The predetermined conditions include, for example, the condition that the keyword exists on the right side or below the area of the keyword indicating the item name of the notification seal, and the distance from the keyword area is less than or equal to a predetermined value, or the condition that the keyword indicating the item name of the notification seal The condition is that the keyword exists on the right side or below the area, the distance from the keyword area is less than or equal to a predetermined value, and the area size is larger than the predetermined size.

Next, the notification seal imprint specifying unit 250 outputs the coordinates indicating the area of the identified notification seal imprint in the image of the form to the main control unit 270 (S533).

According to the process illustrated in FIG. 10, for example, in the process of S531, as schematically shown in FIG. Keyword areas 101, 102, and 103 indicating names and seal impression areas 104 and 105 are mapped.

Thereafter, in the process of S532, among the seal imprint areas 104 and 105, the seal imprint area 104 that satisfies a predetermined condition as shown in FIG. 11(b) is specified as the seal imprint area of the notification seal. The predetermined condition shown in FIG. 11(b) is that the area exists on the right side or below the keyword area indicating the item name of the notification seal, and the distance from the keyword area indicating the item name of the notification seal is less than or equal to a predetermined value. (in this case, 1 cm or less), and the area size is larger than a predetermined size (in this case, a size of 2.4 cm x 2.4 cm). Note that the distance from the keyword area indicating the item name of the notification seal being less than or equal to a predetermined value means, for example, that the distance between the reference point of the keyword area indicating the item name of the notification seal and the reference point of the seal imprint area is a predetermined value. This means the following. The reference point is, for example, the center point of the area.

Thereafter, in the process of S533, as shown in FIG. 11(c), the coordinates ("(x3, y3, x4 , y4)'') are output to the main control section 270.

As described above, according to the image processing performed by the image processing device 2, the process of detecting the keyword area indicating the item name of the notification seal and the seal imprint area in the image of the form is executed, and based on the processing result, Since the region of the seal imprint of the notification seal in the image of the form is specified, it is no longer necessary to predefine the position of the notification seal column, and the imprint of the notification seal can be extracted. In addition, it is possible to identify the region of the notification seal imprint on a document image that includes not only the seal impression of the notification seal but also the impression of a correction seal, inspection seal, sample seal, etc., and extract the seal imprint of the notification seal. become. Furthermore, for an image of a form that includes a well-designed notification seal field, even if the keyword area indicating the item name of the notification seal is not detected, if only one seal imprint area is detected, the seal imprint is detected. Since the area is specified as the region of the seal impression of the notification seal, it becomes possible to extract the seal impression of the notification seal.

Note that the functional configuration of the image processing device 2 is not limited to the configuration illustrated in FIG. 2, and can be modified as appropriate. For example, it is also possible to manage one or both of the character recognition dictionary 610 referred to by the character recognition processing unit 220 and the keyword table 620 referred to by the character string extraction unit 230 in a storage unit (not shown) included in the image processing device 2. It is. Further, the image processing device 2 may be configured to perform the above-described image processing by acquiring an image of a form provided via a portable recording medium such as a memory card or an optical disk, or a network, for example.

In addition, the image processing device 2 can be realized by a computer and a program executed by the computer. The image processing device 2 implemented by a computer and a program will be described below with reference to FIG. 12.

FIG. 12 is a diagram illustrating the hardware configuration of a computer.
As illustrated in FIG. 12, the computer 11 includes a CPU (Central Processing Unit) 1101, a memory 1102, an auxiliary storage device 1103, an input device 1104, and a display device 1105. The computer 11 also includes an input/output interface 1106, a communication control device 1107, and a medium drive device 1108. These elements 1101 to 1108 in the computer 11 are interconnected by a bus 1109, allowing data to be exchanged between the elements.

The CPU 1101 controls the overall operation of the computer 11 by executing various programs including an operating system. For example, the CPU 1101 executes an image processing program including each of the processes described using FIGS. 3 to 11.

The memory 1102 includes a ROM (Read Only Memory) and a RAM (Random Access Memory), which are not shown. For example, a predetermined basic control program read out by the CPU 1101 when the computer 11 is started is recorded in the ROM of the memory 1102 in advance. Further, the RAM of the memory 1102 is used by the CPU 1101 as a working storage area as necessary when executing various programs. The RAM of the memory 1102 can be used to temporarily store, for example, an image of a form, coordinates indicating a keyword area indicating an item name of a notification seal, coordinates indicating an area of a seal impression, and the like.

The auxiliary storage device 1103 is, for example, a magnetic disk such as an HDD (Hard Disk Drive), or a nonvolatile memory such as a flash memory. The auxiliary storage device 1103 can store various programs executed by the CPU 1101, various data, and the like. The auxiliary storage device 1103 can be used to store, for example, the above-mentioned image processing program. Further, the auxiliary storage device 1103 can be used to store, for example, images of forms, processing results for each item name, and the like.

The input device 1104 is, for example, a keyboard device, a mouse device, a touch panel device, or the like. When the operator of the computer 11 performs a predetermined operation on the input device 1104, the input device 1104 transmits input information associated with the operation content to the CPU 1101. The input device 1104 corresponds to the input device 4 in the form processing system 1 in FIG.

The display device 1105 is, for example, a liquid crystal display device. The display device 1105 can be used, for example, to display the operating status of the computer 11, images of forms, processing results for each item name, and the like. The display device 1105 corresponds to the display device 5 in the form processing system 1 in FIG.

The input/output interface 1106 connects the computer 11 to electronic components, other electronic devices (for example, the scanner device 3), and the like. The input/output interface 1106 can also be used, for example, to connect a flash memory equipped with a USB (Universal Serial Bus) standard connector.

The communication control device 1107 is a device that connects the computer 11 to a network and controls various communications between the computer 11 and other electronic devices via the network. The communication control device 1107 can be used, for example, to refer to the character recognition dictionary and keyword table held by the server device 6 in the form processing system 1 of FIG. 1.

The medium drive device 1108 reads programs and data recorded on the portable recording medium 12 and writes data stored in the auxiliary storage device 1103 to the portable recording medium 12. As the portable recording medium 12, for example, there is a memory card (flash memory) of the Secure Digital (SD) standard. The portable recording medium 12 can be used to store the above-mentioned image processing program, images of forms, processing results for each item name, and the like. Furthermore, if the computer 11 is equipped with an optical disk drive that can be used as the medium drive device 1108, various optical disks that can be recognized by the optical disk drive can be used as the portable recording medium 12. Examples of optical discs that can be used as the portable recording medium 12 include Compact Discs (CDs), Digital Versatile Discs (DVDs), and Blu-ray Discs (Blu-ray is a registered trademark).

In the computer 11, when an operator operates the input device 1104 and inputs an instruction to start an image processing program, the CPU 1101 reads the above-described image processing program from the auxiliary storage device 1103 and the like and executes it. While executing the image processing program, the CPU 1101 processes the character recognition processing unit 220, the character string extraction unit 230 including the notification seal item name detection unit 231, the seal imprint detection unit 240, and the notification processing unit 220 in the image processing apparatus 2 illustrated in FIG. It functions (operates) as a seal imprint identification section 250 and a display control section 260. Further, when acquiring an image of a form from the scanner device 3, the CPU 1101 and the input/output interface 1106 function as the scanner control unit 210 in the image processing device 2 illustrated in FIG. Further, when recognizing characters by referring to a character recognition dictionary 610 held by a server device 6 or the like other than the computer 11, the communication control device 1107 takes on a part of the functions of the character recognition processing section 220. Further, when extracting keywords by referring to a keyword table 620 held by a server device 6 or the like other than the computer 11, the communication control device 1107 takes on a part of the functions of the character string extraction section 230.

Note that the computer 11 that operates as the image processing device 2 does not need to include all of the elements 1101 to 1108 illustrated in FIG. 12, and some of the elements can be omitted depending on the purpose and conditions. For example, the computer 11 may be one in which the medium drive device 1108 and the communication control device 1107 are omitted.

Although the embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above, and various improvements and changes can be made without departing from the gist of the present invention.

1 Form processing system 2 Image processing device 3 Scanner device 4 Input device 5 Display device 6 Server device 7 Network 8, 8a Image 9 Neural network model 10 Coordinate system 11 Computer 12 Portable recording medium 101, 102, 103, 104, 105 Area 210 Scanner control unit 220 Character recognition processing unit 230 Character string extraction unit 231 Notification seal item name detection unit 240 Seal imprint detection unit 250 Notification seal imprint identification unit 260 Display control unit 270 Main control unit 610 Character recognition dictionary 620 Keyword table 801, 802, 803, 804, 805 area 806, 807, 808, 809, 810 area 811, 812, 813, 814 area 1101 CPU
1102 Memory 1103 Auxiliary storage device 1104 Input device 1105 Display device 1106 Input/output interface 1107 Communication control device 1108 Media drive device 1109 Bus

Claims (7)

A first detection process for detecting a keyword area indicating an item name of a notification seal in an image generated by scanning a form created by a receiving institution that uses a form-based service ; and a seal imprint area in the image of the form. a second detection process for detecting the
An image processing program that causes a computer to execute a process of specifying an area of a seal imprint of a notification seal in an image of the form based on the execution results of the first detection process and the second detection process, the process of specifying. is, when a plurality of keyword areas indicating the item name meaning the notification seal are detected by executing the first detection process and the second detection process, and a plurality of areas of the seal imprint are detected, the plurality of areas of the seal impression are detected . An image processing program that specifies a region of a seal impression of the notification seal to be verified based on a positional relationship between a keyword region and the plurality of seal impression regions. The image processing program according to claim 1, wherein the identifying process identifies an area of the seal impression that satisfies a predetermined condition as an area of the seal impression of the notification seal. 3. The image processing program according to claim 2, wherein the predetermined condition is a condition that the keyword exists on the right side or below the keyword area and that a distance from the keyword area is equal to or less than a predetermined value. The predetermined condition is that the seal imprint exists on the right side or below the keyword area, the distance from the keyword area is less than or equal to a predetermined value, and the area size of the seal imprint is larger than the predetermined size. The image processing program according to claim 2. The identifying process includes, when only one seal imprint area is detected by executing the first detection process and the second detection process, determining the seal imprint area as the seal imprint area of the notification seal. The image processing program according to claim 1, characterized in that: A first detection process for detecting a keyword area indicating an item name of a notification seal in an image generated by scanning a form created by a receiving institution that uses a form-based service ; and a seal imprint area in the image of the form. a second detection process for detecting the
An image processing method that specifies an area of a seal imprint of a notification seal in an image of the form based on the execution results of the first detection process and the second detection process, wherein the identification is performed in the first detection process. and when a plurality of keyword areas indicating the item name meaning the notification seal are detected by executing the second detection process, and a plurality of areas of the seal imprint are detected, the plurality of keyword areas and the plurality of keyword areas are detected. An image processing method characterized by specifying an area of a seal imprint of the notification seal to be verified based on a positional relationship with an area of the seal imprint . a notification seal item name detection unit that executes a first detection process of detecting a keyword area indicating the item name of the notification seal in an image generated by scanning a form created by a receiving institution that uses the form-based service; ,
a seal imprint detection unit that performs a second detection process for detecting a seal imprint area in the image of the form;
an image processing apparatus, comprising: a notification seal imprint identifying section that specifies a seal imprint area of a notification seal in the image of the form based on the execution results of the first detection process and the second detection process; is, when a plurality of keyword areas indicating the item name meaning the notification seal are detected by executing the first detection process and the second detection process, and a plurality of areas of the seal imprint are detected, the plurality of areas of the seal impression are detected . An image processing apparatus characterized in that a region of a seal impression of the notification seal to be verified is specified based on a positional relationship between a keyword region and the plurality of seal impression regions.

Images