JP2025015911A - Image processing device, image processing method, and program - Google Patents

Abstract

To properly check a masked image.SOLUTION: An image processing device has acquisition means for acquiring a document image, image processing means for generating a masked image where a target area including a predetermined character string in the document image is masked so that the predetermined character string included in the document image cannot be recognized, and control means for controlling the presentation of the predetermined character string so that it is recognizable by a user on the basis of the user's instruction.SELECTED DRAWING: Figure 12

Description

This disclosure relates to processing of document images.

Document data, such as scanned images obtained by scanning documents such as invoices, may be sent to related departments or other companies.

Patent document 1 discloses a method for masking information such as personal information and confidential information in document data by blacking out character strings indicating the information.

JP 2021-68023 A

Before outputting a masked image obtained by masking document data using the method of Patent Document 1, it may be necessary to check the masked image. When a masked image is previewed to check the masked image, the masked character string in the masked image is displayed in such a way that the user cannot recognize it. For this reason, the user is unable to check whether the masked character string has been masked in error.

The image processing device disclosed herein is characterized by having an acquisition means for acquiring a document image, an image processing means for generating a masking image in which a target area in the document image containing a predetermined character string is masked so that the user cannot recognize the predetermined character string, and a control means for controlling the presentation of the predetermined character string in a manner that the user can recognize it, based on an instruction from the user.

According to the present disclosure, masking images can be properly checked.

FIG. 1 is a diagram showing an example of the configuration of an image processing system. FIG. 2 is a block diagram showing the hardware configuration of an MFP. FIG. 2 is a block diagram showing a hardware configuration of an external storage device. FIG. 2 is a block diagram showing the functional configuration of the MFP. 11 is a flowchart for explaining a flow from generating a masking image to transmitting the same. FIG. 13 is a diagram showing an example of a setting screen. 10 is a flowchart for explaining details of a masking region specifying process. 10 is a flowchart for explaining details of acquisition of an information type to be masked. 11 is a flowchart for explaining a masking image generating process. FIG. 4 is a diagram for explaining a preview display screen. 10 is a flowchart illustrating details of a printing process for confirmation. FIG. 11 is a diagram showing an example of a printed matter for confirming a masking image. FIG. 13 is a diagram showing an example of a list print screen. 10 is a flowchart for explaining details of a preset registration process. FIG. 13 is a diagram showing an example of a preset registration screen.

Below, the embodiments for implementing the technology of this disclosure are described with reference to the drawings. Note that the following embodiments do not limit the technology related to the claims, and not all of the combinations of features described in the following embodiments are necessarily essential to the solution of the technology of this disclosure.

First Embodiment
[System configuration]
1 is a diagram showing an example of the configuration of an image processing system according to the present embodiment. The image processing system 1 includes an MFP (Multifunction Peripheral) 110 and an external storage 120. The MFP 110 is communicably connected to a server that provides various services on the Internet via a LAN (Local Area Network).

MFP 110 has multiple functions such as a scanner function and a printer function, but is an image forming device, and also includes the functions of an information processing device and an image processing device. MFP 110 also has a function of transferring an image file of a scanned image obtained by scanning a paper document to external storage 120 where the file can be saved.

The external storage 120 is a device that provides a service that allows files received via the Internet to be stored and files to be obtained from an external device via a web browser. The service provided by the external storage 120 is, for example, a cloud service. The device that provides the storage service is not limited to the external storage 120, and multiple devices may exist in the image processing system 1.

The image processing system 1 of this embodiment is configured to include an MFP 110 and an external storage 120, but, for example, some of the functions of the MFP 110 may be realized by another server device located on the Internet or a LAN. Also, the external storage 120 may be located on the LAN rather than on the Internet. In addition to the external storage 120, a mail server may be included in the image processing system 1. The mail server may send the scanned image generated by the MFP 110 by attaching the scanned image to an email. The MFP 110 may include a storage function that the external storage 120 has.

[MFP Hardware Configuration]
2 is a block diagram showing the hardware configuration of the MFP 110. The MFP 110 has a control unit 210, an operation unit 220, a printer 221, a scanner 222, and a modem 223.

The control unit 210 is composed of the various units described below, and controls the overall operation of the MFP 110. The CPU 211 reads out various control programs stored in the ROM 212 or HDD 214, and executes processes for implementing various functions of the MFP 110, such as scanning, printing, and communication. The RAM 213 is used as a temporary storage area such as the main memory and work area of the CPU 211. Note that in this embodiment, one CPU 211 executes each process shown in the flowcharts described below using one memory (RAM 213 or HDD 214), but this is not limited to this. For example, each process may be executed by multiple CPUs or multiple RAMs or HDDs working together. The HDD 214 is a large-capacity storage unit that stores image data and various programs.

The operation unit I/F 215 is an interface that connects the operation unit 220 and the control unit 210. The operation unit 220 is equipped with a touch panel that also functions as a display unit, a keyboard, and the like, and accepts operations, inputs, and instructions from the user. Note that touch operations on the touch panel include operations using a person's finger and operations using a touch pen.

The printer I/F 216 is an interface that connects the printer 221 and the control unit 210. Image data to be printed is transferred from the control unit 210 to the printer 221 via the printer I/F 216, and the image is printed on a recording medium such as paper.

The scanner I/F 217 is an interface that connects the scanner 222 and the control unit 210. The scanner 222 optically reads a paper document (original) set on a platen or ADF (Auto Document Feeder) (not shown) to generate a scanned image, which is then input to the control unit 210 via the scanner I/F 217. The MFP 110 can print (copy output) the scanned image read and generated by the scanner 222 on the printer 221, store it in the HDD 214, or send it as a file or email to an external device such as the external storage 120 via the LAN.

The modem I/F 218 is an interface that connects the modem 223 to the control unit 210. The modem 223 transmits image data by facsimile between the modem 223 and a facsimile device (not shown) on the PSTN.

The network I/F 219 is an interface that connects the control unit 210 (MFP 110) to a LAN. The MFP 110 uses the network I/F 219 to send scanned images to devices that provide various services on the Internet and to receive various information.

[Hardware configuration of external storage]
3 is a block diagram showing the hardware configuration of the external storage 120. The external storage 120 includes a CPU 311, a ROM 312, a RAM 313, a HDD 314, and a network I/F 315. The CPU 311 reads out a control program stored in the ROM 312 and executes various processes to control the overall operation. The RAM 313 is used as a temporary storage area such as a main memory and a work area for the CPU 311. The HDD 314 is a large-capacity storage unit that stores image data and various programs. The network I/F 315 is an interface that connects the control unit 310 to the Internet. The external storage 120 receives various processing requests from other devices (such as the MFP 110) via the network I/F 315 and transmits and receives various information.

[Functional configuration of MFP]
4 is a block diagram showing the functional configuration of the MFP 110 of this embodiment. Of the various functions of the MFP 110, the following description focuses on functions related to the process of scanning a document, digitizing it (making it into a file), and storing it in the external storage 120.

The functions of the MFP 110 include two functional modules: a native function unit 410 and an additional function unit 420. The native function unit 410 is an application that is provided as standard in the MFP 110, whereas the additional function unit 420 is an application that is additionally installed in the MFP 110. The additional function unit 420 is an application based on Java (registered trademark), and functions can be easily added to the MFP 110. Note that other additional applications (not shown) may be installed in the MFP 110. The functional units of the MFP 110 are realized by the CPU 211 reading into the RAM 213 and executing a program stored in the ROM 212 or HDD 214 of the MFP 110.

The native function unit 410 has a scan execution unit 411, an internal data storage unit 412, a print execution unit 413, and a UI display control unit 414.

The additional function unit 420 has a main processing unit 421, a scan instruction unit 422, an image processing unit 423, a data management unit 424, a print instruction unit 425, an Internet access unit 426, a display control unit 427, an information detection unit 428, and a document type determination unit 429. It also has a masking target identification unit 430.

The main processing unit 421 performs processing related to all of the functional units included in the additional functional unit 420. Specifically, the main processing unit 421 controls the overall processing executed by the additional functional unit 420, and requests processing from each functional unit included in the additional functional unit 420.

The scan instruction unit 422 requests the scan execution unit 411 to perform a scan process according to the scan settings input via the setting screen 600 (see FIG. 6). The scan instruction unit 422 requests a scan process in response to pressing the scan execution button 601 (see FIG. 6) on the setting screen 600.

The scan execution unit 411 acquires a scan request including scan settings from the scan instruction unit 422. The scan execution unit 411 causes the scanner 222 to execute a reading operation of a paper document placed on a platen via the scanner I/F 217 in accordance with the acquired scan request, and generates a scanned image. An image representing the contents of a document, such as the scanned image generated in this way, is called a document image.

The internal data storage unit 412 stores the document image data in the HDD 214. Information on an image identifier that uniquely identifies the stored document image is notified to the scan instruction unit 422. The image identifier is a number, symbol, alphabet, or the like that uniquely identifies an image stored in the MFP 110.

The image processing unit 423 is a functional unit that performs analysis and processing on the document image. The image processing unit 423 acquires an image identifier from the scan instruction unit 422, and acquires document image data corresponding to the acquired image identifier from the internal data storage unit 412. The image processing unit 423 performs image rotation and tilt correction processing, character string area detection processing, and character recognition processing (Optical Character Recognition processing or OCR processing) on the acquired document image.

The image processing unit 423 of this embodiment synthesizes a mask so that the mask is superimposed on the target area in the document image, and instructs the internal data storage unit 412 to save data of the masked image, which is the image obtained as a result. The process of synthesizing a mask on a document image is called masking (processing) or blackout processing. Alternatively, masking processing may also be called masking. The target area that is the subject of masking processing is called a masking area. In other words, a masking image is an image obtained by masking a masking area in a document image. The masked area in a masking image that has been subjected to masking processing may also be called a mask. Details of the processing of the image processing unit 423 and other functions will be described later.

In the description of this embodiment, the masking area is assumed to be a rectangular area, but the shape of the masking area may be any shape, such as an ellipse or a triangle. The position and size of a rectangular masking area are represented by the coordinates of the starting point, which is the upper left position, and the coordinates of the end point, which is the lower right position. For example, if the coordinates of the starting point are (441, 957) and the coordinates of the end point are (1369, 1
7), the position and size of the masking region are expressed as "(441, 957), (1369, 1057)."

The image processing unit 423 sends the image identifier to the print instruction unit 425 and the Internet access unit 426 according to the output settings set for the masking image.

The data management unit 424 stores information obtained by processing the document image, such as the document type of the document image, a character string corresponding to the information type detected from the document image, masking information, and masking display information, in the storage unit in association with an image identifier. The data management unit 424 also associates the information type set when generating the masking image with the document type and saves it as a preset in the HDD 214. In this embodiment, a preset is a template in which the information type to be masked is saved in association with the document type. A file name, a save destination, and the like may also be associated with the preset.

The print instruction unit 425 acquires an image identifier, which is an identifier of the image to be printed, from the image processing unit 423. The print instruction unit 425 sends a print processing request including the print settings input via the UI screen and the acquired image identifier to the print execution unit 413.

The print execution unit 413 acquires a print request and an image identifier from the print instruction unit 425. The print execution unit 413 acquires an image corresponding to the image identifier from the internal data storage unit 412. The print execution unit 413 performs processing to convert the acquired image data into image data for printing according to the print settings included in the print request. The print execution unit 413 causes the printer 221 to execute printing based on the image data for printing via the printer I/F 216, and generates a printed matter. If the image to be printed is a masking image, the masking image is printed on the recording medium.

The Internet access unit 426 transmits a processing request to the external storage 120 that provides a cloud service such as a storage function (storage service). In the cloud service, a process of saving a file in the cloud storage and retrieving the saved file from an external device is generally performed using a protocol such as REST or SOAP. The cloud service publishes various interfaces for executing such a process. The Internet access unit 426 performs operations on the cloud service using the published interface of the cloud service. The Internet access unit 426 acquires an image file and transmission information corresponding to the image identifier acquired from the image processing unit 423 from the data management unit 424. The Internet access unit 426 uses the transmission information acquired from the data management unit 424 to transmit the image file acquired from the data management unit 424 to the external storage 120 via the network I/F.

The display control unit 427 controls where necessary components such as buttons and images included in a UI (User interface) screen for accepting user operations are displayed. The display control unit 427 controls the display of a UI screen via the UI display control unit 414 on the liquid crystal display unit of the touch panel included in the operation unit 220 of the MFP 110. The UI display control unit 414 displays a UI screen for accepting various user operations on the touch panel of the operation unit 220. For example, the UI display control unit 414 controls the display of a setting screen 600 (see FIG. 6) on which scan settings and scan start operations can be performed. The UI display control unit 414 also displays a preview display screen 1000 (see FIG. 10) on which a preview of a masking image is displayed and on which the user can correct the masking area and instruct the start of output.

The information detection unit 428 performs information detection processing to obtain character strings recognized from a document image from the image processing unit 423 and detect character strings corresponding to predetermined items (called information types) such as personal information or confidential information from the obtained character strings. The information detection unit 428 requests the data management unit 424 to associate the detected character strings with the information types corresponding to the detected character strings and store them in the HDD 214.

The information type is the type of concept that a character string represents, and is used to specify the character string to be masked. Information types that the information detection unit 428 can detect include, but are not limited to, types of personal information or confidential information, such as "company name," "name," "telephone number," and "address."

As a result of the information detection process by the information detection unit 428, for example, the character string "Yamada Taro" is detected as a character string corresponding to the information type "Name". In this embodiment, the information detection unit 428 is assumed to hold items of predetermined detectable information types, but the location where the detectable information types are held is not limited to the information detection unit 428.

The document type determination unit 429 determines the document type of the document image based on the character string recognized from the document image. The document type may be, for example, a type that indicates the purpose of the document, such as "contract," "delivery note," or "invoice." In this embodiment, a trained model generated by machine learning the features of the document type using samples of multiple document types as training data is used to determine the document type. In this embodiment, a trained model obtained by machine learning based on Transformer is used. Other examples that may be used include a bidirectional Long Short-Term Memory (LSTM) neural network, a Sequence to Sequence model, and a recurrent neural network (RNN).

The masking target identification unit 430 determines the information type to be masked based on the preset, the processing result of the information detection unit 428, and the processing result of the document type determination unit 429. Details will be described later.

[Overall processing flow]
Fig. 5 is a flowchart for explaining a flow of generating a masking image from a document image obtained by scanning a document and transmitting the masking image. The series of processes shown in the flowchart in Fig. 5 are performed by the CPU 211 of the MFP 110 expanding program code stored in the ROM 212 or HDD 214 into the RAM 213 and executing it, thereby realizing each functional unit and executing the process of each step. Alternatively, some or all of the functions of the steps in Fig. 5 may be realized by hardware such as an ASIC or an electronic circuit.

In S501, the display control unit generates a setting screen that enables the user to select settings for the output method, scanning, and masking, and displays the generated setting screen on the operation unit 220 via the UI display control unit 414.

Figure 6 is a diagram showing an example of the setting screen displayed in S501. The setting screen 600 includes an output method setting area 610, a scan setting area 620, a masking setting area 630, a scan execution button 601, and a back button 602.

The output method setting area 610 includes an output method selection button 611, an output format selection pull-down 612, and an output destination designation button 613. The output method selection button 611 is a button that allows the user to specify the output method for the masking image. The user can select the output method from "print only," "save only," or "print and save." In the following explanation, it is assumed that "save only" has been selected.

The output format selection pull-down 612 is a pull-down for selecting the file format of the masking image when "Print and save" or "Save only" is selected from the output method selection buttons 611.

The output destination designation button 613 is a button for selecting a destination where the masking image is saved when "Print and save" or "Save only" is selected from the output method selection buttons 611. From the output destination designation button 613, the user can select one of the destinations: "New input," "Address book," or "Send to self."

The "New Input" button included in the output destination designation button 613 is a button for the user to arbitrarily designate the destination of the masking image. In addition to the HDD 214 or the external storage 120, the user can designate a destination by IP address via the network I/F 219 as the destination. The "Address Book" button included in the output destination designation button 613 is a button for sending the masking image by email to an email address entered by the user. The "Send to Me" button included in the output destination designation button 613 is a button for sending the image to an authentication server (not shown) that identifies the user based on the user information entered by the user. The authentication server sets the registered email address based on the authenticated user information. The user can input any destination, email address, and user information via the operation unit 220. The output destination designation button 613 may be configured to further include other save destination buttons. In this embodiment, the description will be given assuming that the destination of the file of the masking image is the external storage 120.

The scan settings area 620 includes items that allow the user to select scan settings. For example, it includes items such as "original size," "gray/color," "single-sided/double-sided," and "advanced settings," which are configurable items. When "advanced settings" is pressed, a screen not shown in the figure is displayed, and on this screen, changes to detailed settings such as resolution, image quality, mixed original sizes, and density are accepted. The scan execution button 601 is a button for executing a scan using the scan settings selected in the scan settings area 620.

The masking setting area 630 includes a masking selection button 631, a masking method selection button 632, and a masking mode selection pull-down 633.

The masking selection button 631 is a button that accepts the selection of whether to mask or not mask the document image generated when the scan execution button 601 is pressed. The flowchart in FIG. 5 is a flowchart for explaining the processing when "Masking" is selected in the masking selection button 631.

The masking method selection button 632 is a button that allows the user to select whether to automatically or manually set a masking area for the scanned image when "Masking" is selected with the masking selection button 631.

The masking mode selection pull-down 633 is a pull-down for manually selecting a mode for setting a masking area when "Manual masking setting selection" is selected with the masking method selection button 632. When "Automatic masking setting" is pressed with the masking method selection button 632 to select automatically setting a masking area, "Automatic" is set in the masking mode selection pull-down 633. In the following explanation, it is assumed that "Automatic masking setting" is pressed with the masking method selection button 632 to select automatically setting a masking area.

An application (called a masking app) that masks a partial area of a document image and transmits it to a cloud service can be used by installing it on the MFP 110. When the masking app is installed on the MFP 110, a masking setting area 630 including a button that enables the masking function is displayed on the setting screen 600. When the scan execution button 601 is pressed, job information including the contents of the settings selected on the setting screen 600 is generated. Then the process proceeds to S502.

In S502, the main processing unit 421 requests the scan execution unit 411 via the scan instruction unit 422 to scan the paper document placed on the platen. The scanned image data obtained as a result of the scan is stored in the RAM 213.

In the following description, the document image is assumed to be the scanned image obtained as a result of S502, and the masking process is performed on the scanned image, but the document image to be masked is not limited to a scanned image. The main processing unit 421 may obtain data of the document image stored in the HDD 214 via the internal data storage unit 412, and subject the document image to masking process. Alternatively, the main processing unit 421 may obtain data of the document image to be masked from the external storage 120 via the Internet access unit 426. Data of the document image to be masked may be obtained using any other method.

In S503, the main processing unit 421 requests the image processing unit 423 to perform processing for identifying a masking area based on the job information generated in S501 and the document image data generated in S502, and the image processing unit 423 performs processing based on the request. Details of the processing of the image processing unit 423 in S503 will be described later.

In S504, the main processing unit 421 requests the image processing unit 423 to create a masking image, and the image processing unit 423 performs processing based on the request. Details of the processing of the image processing unit 423 in S504 will be described later.

In S505, the main processing unit 421 outputs the masking image created in S504 to the external storage 120 via the Internet access unit 426.

As described above, the generated masking image is described as being output to the external storage 120, but the method of outputting the masking image is not limited to sending it to the outside. The main processing unit 421 may perform output processing of the masking image so that it is emailed to an arbitrary destination. Alternatively, the main processing unit 421 may store the masking image in the HDD 214 via the internal data storage unit 412. Alternatively, the main processing unit 421 may perform processing to execute printing via the print instruction unit 425 so that the masking image is printed and output. Alternatively, the main processing unit 421 may store the masking image using other methods.

In S506, the main processing unit 421 generates a preset registration screen 1500 (see FIG. 15) for the display control unit 427. Then, it requests the UI display control unit 414 to display the preset registration screen. The UI display control unit 414 displays the preset registration screen 1500 on the touch panel of the operation unit 220, and registers a preset based on the selection received from the user. Details will be described later.

In this embodiment, an example is described in which the display control unit 427 displays a UI screen on the touch panel of the operation unit 220, but the display control unit 427 may be configured to provide each screen in this embodiment to another device, and the display unit of the other device may display each screen.

[Masking area identification process]
FIG. 7 is a flowchart for explaining details of the process of S503 in this embodiment.

In S701, the image processing unit 423 acquires a document image from the HDD 214. The image processing unit 423 performs corrections such as skew correction and rotation correction on the acquired document image. The document image in the following steps refers to the document image corrected in S701. The data of the document image after correction is stored in the RAM 213.

In S702, the image processing unit 423 converts the document image into a one-channel image to improve the accuracy of OCR processing. Then, the image processing unit 423 extracts a character string area contained in the converted document image. Then, OCR processing (character recognition processing) is performed on the extracted character string area to obtain data of the character string (character string data) contained in the document image. The recognized character string and the character string area corresponding to the recognized character string are stored in the RAM 213.

Character string regions are extracted by performing block selection (BS) processing. In block selection (BS) processing, a document image is divided into object units and block regions within the document image are extracted. Then, the attributes of each extracted block region are determined. Specifically, for example, attributes such as text, photographs, and diagrams are determined, and the document image is divided into block regions with different attributes. Block selection processing can be achieved using known region determination techniques.

In S703, the information detection unit 428 performs information detection processing on each character string recognized in S702. As a result, character strings corresponding to predetermined information types are detected from the character strings recognized in S702. The information types in this embodiment are described as "company name," "name," "telephone number," and "address," but the information types to be detected are not limited.

Table 1 is an example of an "information detection processing result list" showing an example of the results of the information detection processing executed in S703. In this embodiment, the information detection processing result list stores information types and character strings corresponding to the information types detected from the document image in association with each other. The information detection processing result list is stored in RAM 213.

When the information detection unit 428 detects "Iroha Co., Ltd." as a character string corresponding to the information type "company name," as shown in Table 1, "Iroha Co., Ltd." is stored in the same row as "company name" as a character string corresponding to the information type of "company name." Similarly, "Koyano" and "Tanaka" are detected as character strings corresponding to the information type of "name," and "090-0000-2111" is detected as a character string corresponding to the information type of "telephone number," so the character strings corresponding to the information types are stored in the corresponding rows as shown in Table 1. It is assumed that a character string corresponding to the information type of "address" was not detected. The information detection process result list in this embodiment is generated so as not to include information types for which no character strings were detected.

Ｓ７０４において文書種別決定部４２９は、Ｓ７０２で抽出された文字列から文書画像の文書種別を決定する。本実施形態では、文書種別決定部４２９は、文字列を入力すると文書種別を出力するように予め学習した学習済みモデルを用いて、文書画像の文書種別を決定するものとして説明するが、その他の方法で文書種別が決定されてもよい。例えば、予め文書種別に対応するキーワードを設定することで文書種別が決定されてもよい。

In S704, the document type determination unit 429 determines the document type of the document image from the character string extracted in S702. In this embodiment, the document type determination unit 429 is described as determining the document type of the document image using a trained model that has been trained in advance to output a document type when a character string is input, but the document type may be determined by other methods. For example, the document type may be determined by setting a keyword corresponding to the document type in advance.

In S705, the masking target identification unit 430 determines whether the document type was determined in S704. If the masking target identification unit 430 determines that the document type was determined (YES in S705), the process proceeds to S706, and if the masking target identification unit 430 determines that the document type could not be determined (NO in S705), the process ends this flowchart.

In S706, the masking target identification unit 430 obtains the information type to be masked that corresponds to the document type.

Figure 8 is a flowchart explaining the details of the processing of S706.

In S801, the masking target identification unit 430 requests the data management unit 424 to acquire a preset corresponding to the document type. The data management unit 424 acquires the preset corresponding to the document type determined in S704 from the external storage 120 via the Internet access unit 426. The acquired preset is stored in the RAM 213. The preset may be held in the HDD 214, in which case the preset may be acquired from the HDD 214.

A preset is information that is generated in advance for each document type, and includes the information type to be masked for each document type. For example, a user may register an information type and a document type in advance, and the preset is stored in the external storage 120 or the HDD 214. Alternatively, the preset may be registered from the preset registration screen 1500 (see FIG. 15).

For example, if the document type is determined to be an invoice in S704, a preset associated with the invoice is obtained in S801. Table 2 shows an example of information about a preset associated with an invoice.

Ｓ８０２においてマスキング対象特定部４３０は、Ｓ８０１で文書画像の文書種別に対応付けられたプリセットを取得できたか否かを判定する。プリセットが取得できた場合（Ｓ８０２がＹＥＳ）、Ｓ８０３に進み、プリセットが取得できなかった場合（Ｓ８０２がＮＯ）、マスキング対象の情報種別はないとして本フローチャートを終了してＳ７０７へ処理を進める。

In S802, the masking target identification unit 430 determines whether or not a preset associated with the document type of the document image in S801 has been acquired. If a preset has been acquired (YES in S802), the process proceeds to S803. If a preset has not been acquired (NO in S802), the process ends this flowchart as there is no information type to be masked, and the process proceeds to S707.

In S803, the masking target identification unit 430 stores in the RAM 213 the information type to be masked, which is included in the preset acquired in S801. Then, this flow ends and the process proceeds to S707 in FIG. 7. If the document type determined in S704 is an invoice, and a preset associated with the invoice is acquired, as shown in Table 2, "Name", "Telephone number", and "Address" are stored in the RAM 213 as the information types to be masked.

In S707, the image processing unit 423 determines an information type that matches the information type to be masked obtained in S706, from among the information types corresponding to the character strings detected in S703. The image processing unit 423 determines an area in the document image that includes a character string corresponding to the information type determined to match as an area where masking is required. A character string area that includes a character string indicating that masking is required becomes a masking area. The image processing unit 423 determines a character string area that includes a character string corresponding to an unmatched information type as an area where masking is not required.

For example, suppose that character strings corresponding to the information types of "company name", "name", and "phone number" are detected in S703 as shown in Table 1. Also suppose that "name", "phone number", and "address" are acquired as the information types to be masked in S706. In this case, the matching information types are "name" and "phone number". For this reason, it is determined that the masking of the character strings corresponding to the information types of "name" and "phone number" among the character strings detected in S703 is "necessary". On the other hand, it is determined that the masking of the character string corresponding to the information type of "company name" is "not required".

The image processing unit 423 obtains the character string area of the character string indicating whether masking is required from the character string area extracted in S702. The image processing unit 423 generates masking information as the processing result of S707. Table 3 is a masking information list showing an example of masking information.

マスキング情報は、Ｓ７０３で情報検出処理の検出対象となる情報種別、Ｓ７０３の情報検出処理の結果検出された情報種別に対応する文字列、マスキングの要否、文字列領域が対応付けられている情報である。表３のマスキング情報リストは、表１の情報検出処理結果情報リスト、表２のプリセットの情報に基づき画像処理部４２３がＳ７０７の処理を行った結果生成されたマスキング情報の一例を表す。

The masking information is information in which the information type to be detected in the information detection process in S703, a character string corresponding to the information type detected as a result of the information detection process in S703, whether masking is required, and a character string area are associated with each other. The masking information list in Table 3 shows an example of masking information generated as a result of the image processing unit 423 performing the process in S707 based on the information detection process result information list in Table 1 and the preset information in Table 2.

The character string area of the masking information holds coordinate information in the document image of the character string area that contains a character string corresponding to the information type. For "company name", "name", and "phone number", character strings corresponding to the information type were detected, so character string area information is held. On the other hand, since a character string corresponding to the information type of "address" was not detected, "- (none)" is held in the character string area corresponding to the information type of "address", indicating that no character string was detected. The image processing unit 423 saves the masking information, document type, and information detection results in RAM 213.

If it is determined that the document type cannot be determined (NO in S705) or if a preset corresponding to the document type is not found, masking information is not generated in the process of S503.

[Masking image generation]
FIG. 9 is a flowchart for explaining the masking image generation process in S504 of FIG.

In S901, the image processing unit 423 acquires the document image corrected in S701, the masking information generated in S707, and the masking display information stored in advance from the RAM 213. If the masking information is changed in S905 (described later), the changed masking information is acquired. If the masking display information is changed in S908 or S909, the changed masking display information is acquired from the RAM 213. The image processing unit 423 performs masking processing on the document image based on the acquired masking information and masking display information.

Table 4 is a table showing an example of masking display information. The masking display information includes information on the display type, display color, and transparency. In this embodiment, the display type is rectangular and the display color is black. The image processing unit 423 performs masking on the document image in accordance with the masking display information. Transparency is a value in the range of 0.0 to 1.0, and the lower the value, the lower the transparency. A transparency of 0.0 indicates a state in which the masked character string is not visible to the user.

The method of masking with a transparency of 0.0 may be any method that makes it impossible for the user to recognize the text contained in the masking area. For example, if the display color is black, masking may be performed by filling the masking area with black. If the display color is the background color, masking may be performed by hiding the masking area with the background color. Masking may be performed in any other manner.

画像処理部４２３は、取得したマスキング情報におけるマスキング要否が「要」の文字列領域の座標情報を取得する。画像処理部４２３は、文書画像上の同座標が示す矩形領域であるマスキング領域に対して、マスキング表示情報に基づくマスクが合成されるようにマスキングを行い、マスキング画像を生成する。画像処理部４２３は、生成されたマスキング画像をＲＡＭ２１３に記憶させる。マスキング情報が生成されていない場合は、文書画像に対してマスキング処理は行われない。

The image processing unit 423 acquires coordinate information of a character string area where the masking necessity in the acquired masking information is "Needed". The image processing unit 423 performs masking on the masking area, which is a rectangular area on the document image indicated by the coordinates, so that a mask based on the masking display information is synthesized, and generates a masked image. The image processing unit 423 stores the generated masked image in the RAM 213. If masking information has not been generated, no masking process is performed on the document image.

In S902, the main processing unit 421 requests the display control unit 427 to display a preview of the masked image generated in S901. Specifically, the display control unit 427 retrieves the masked image generated in S901 from the RAM 213, generates a preview display screen 1000 (see FIG. 10), and displays it on the touch panel of the operation unit 220 via the UI display control unit 414. Note that if masking processing was not performed on the document image as a result of S901, a preview of the document image that has not been masked is displayed. Even in this case, for ease of explanation, the image previewed on the preview display screen 1000 will be referred to as a masked image.

Figure 10 is a diagram for explaining the preview display screen. Figure 10 (a) is a diagram showing an example of the preview display screen generated in S902. The preview display screen 1000 includes an image display area 1010, a document type display area 1020, a masking information display area 1030, a send button 1041, a save settings button 1042, a confirmation print button 1043, and a list display button 1044.

The components included in the image display area 1010 will now be described. The preview display area 1001 is an area in which the masking image generated in S901 is displayed as a preview. If the masking image generated does not fit within the preview display area 1001, a scroll bar is automatically displayed.

The enlarge display area button 1002 is a button for increasing the display magnification of the image displayed in the preview display area 1001 by a fixed amount, thereby enlarging and previewing the generated masking image. The fit display area button 1003 is a button for changing the magnification to the maximum magnification so that the entire generated masking image fits within the preview display area 1001. The reduce display area button 1004 is a button for decreasing the display magnification of the image displayed in the preview display area 1001 by a fixed amount, thereby reducing and previewing the generated masking image.

The previous page button 1007 is a button for displaying the masking image of the previous page when there are multiple pages of masking images. The next page button 1009 is a button for displaying the masking image of the next page when there are multiple pages of masking images. The page number display area 1008 is an area for displaying the page of the masking image currently being displayed and the total number of pages.

The masking area selection button 1013 is a button for selecting a new masking area from the preview display area 1001. The mask deletion button 1011 is a button for changing the masking necessity of the masking area selected in the preview display area 1001 to "not required" and releasing the mask. The masking image previewed in the preview display area 1001 includes a mask 1005 superimposed on the character string to be masked. When the masking area selection button 1013 is pressed, a mask such as the mask 1005 becomes selectable.

The masking instruction button 1012 is a button for instructing that masking processing be performed on unmasked areas in the current masking image.

The mask display checkbox 1014 is a checkbox for generating and previewing a confirmation masking image that makes the mask on the masking image transparent so that the user can see the masked character string. Each time the mask display checkbox 1014 is pressed, the state transitions between a checked state and an unchecked state. Details will be described later.

The document type display area 1020 includes a document type selection dropdown 1021. The document type selection dropdown 1021 is displayed with the document type determined in S704 selected as the default. If the document type cannot be determined, a blank not shown in the figure indicating that the document type is "not set" is displayed as the default in the document type display area 1020. The user can select a document type from the document type selection dropdown 1021.

The masking information display area 1030 is an area that displays the contents of the masking information corresponding to the document type selected in the document type selection dropdown 1021. If masking information is generated in S503, the contents of the generated masking information are displayed by default.

If blank is selected in the document type selection dropdown 1021, indicating that the document type is not set, no masking information exists, so no information is displayed in the masking information display area 1030.

When a document type is selected from the document type selection dropdown 1021, the masking target identification unit 430 performs processing similar to S706 and S707 to obtain the information type to be masked that corresponds to the selected document type, and generates masking information. The display control unit 427 displays the contents of the newly generated masking information in the masking information display area 1030.

The masking information display area 1030 displays information types and character strings corresponding to the information types in a tree structure, with the parent being the information type and the child being the character string corresponding to the information type. In other words, one or more character strings corresponding to each of the multiple information types are displayed in association with each other.

The masking information display area 1030 has a check box 1036 corresponding to each information type. It also has a check box 1037 corresponding to the character string corresponding to each information type. When the check box 1036, 1037 is checked, the information type or character string corresponding to the check box indicates that masking is "required." When the check box 1036, 1037 is not checked, the information type or character string corresponding to the check box indicates that masking is "not required." To improve visibility, information types that require masking are displayed at the top, but the display order is not important.

Check boxes 1037 for character strings corresponding to information types for which the masking information indicates whether masking is required and coordinates are held in the masking area are displayed in a pre-checked state. Check boxes 1036, 1037 for information types or character strings for which the masking information indicates whether masking is not required are displayed in a pre-unchecked state.

Information types that are not subject to masking and for which the masking necessity is "not required" are not included in the presets. However, the information types that are subject to detection by the information detection process are items that indicate personal information or confidential information, and are items for which the user is likely to instruct masking. For this reason, information types that are not subject to masking and for which the masking necessity is "not required" are also displayed for the purpose of presenting them to the user.

The warning mark 1031 is a mark for warning the user about an information type for which masking is "necessary" but no character string has been detected. The masking area edit buttons 1032 to 1035 are buttons for editing or adding masking areas. Details will be described later.

In S903, the display control unit 427 determines whether or not the user has pressed the Send button 1041 on the preview display screen 1000 via the UI display control unit 414. If the display control unit 427 determines that the Send button 1041 has been pressed (YES in S903), the process proceeds to S911. If the display control unit 427 determines that the Send button 1041 has not been pressed (NO in S903), the process proceeds to S904.

In S904, the display control unit 427 determines via the UI display control unit 414 whether an operation that requires a change to the masking information has been performed on the preview display screen 1000. Operations that require a change to the masking information include operations such as pressing the Delete Mask button 1011, the Masking Instruction button 1012, or the Edit Masking Area buttons 1032 to 1035, or pressing the check boxes 1036 and 1037. Or, an operation of selecting a different document type from the document type selection dropdown 1021.

If the display control unit 427 determines that an operation requiring a change to the masking information has been performed (YES in S904), the process proceeds to S905. If the display control unit 427 determines that an operation requiring a change to the masking information has not been performed (NO in S904), the process proceeds to S906.

In S905, the image processing unit 423 acquires the current masking information and changes the masking information based on the user's operation on the preview display screen 1000. The masking information generated in S707, or the updated masking information if S905 has been executed previously, is acquired, and a process of changing the acquired masking information is performed.

For example, suppose that the user presses one of the masking area edit buttons 1032 to 1035, and then drags the touch panel on which the preview display area 1001 is displayed to input a start point and an end point. The display control unit 427 acquires the positions of the start point and end point selected on the touch panel of the operation unit 220 via the UI display control unit 414, and converts them into a rectangular area on the masking image. Specifically, when it detects that the masking image in the preview display area 1001 has been touched with a finger, it detects the point touched with the finger as the start point of the rectangular area. The display control unit 427 detects the point where the finger is released after dragging from the start point as the end point of the rectangular area. This operation corresponds to an operation that requires a change of the masking information. In this case, in S905, the image processing unit 423 updates the masking information so that the detected rectangular area is included as information on the masking area of the information type corresponding to the masking area edit buttons 1032 to 1035.

In the masking information list of Table 3, the information type "Address" has the masking requirement set to "Required", but the character string area (masking area) is not specified. To add masking area information for the information type "Address", the user presses the masking area edit button 1034 associated with "Address" in the masking information display area 1030. Then, as shown in the dotted rectangular area 1006 in FIG. 10(b) , the user drags his/her finger over the preview display area 1001 to specify a rectangular area. The image processing unit 423 stores the coordinate information of the rectangular area specified by the user in the character string area corresponding to the information type "Address" of the masking information, and updates the masking information as shown in Table 5.

他にも例えば、マスキング指示ボタン１０１２が選択されている場合、表示制御部４２７は、プレビュー表示領域１００１のマスキング画像上が指でタッチされたことを検知すると指でタッチされた点を始点として検知する。表示制御部４２７は、始点からドラッグして指が離れた点を終点として検知する。この操作は、マスキング情報の変更が必要な操作に該当する。この場合、Ｓ９０５において画像処理部４２３は、検知された始点と終点で特定される矩形領域を新たなマスキング領域の情報として保持されるようにマスキング情報を更新する。具体的には、画像処理部４２３は、情報種別「矩形領域」を、取得したマスキング情報に追加する。情報種別「矩形領域」に対応する、情報種別に対応する文字列には「－（なし）」、マスキング要否に「要」を保持させる。そして、情報種別「矩形領域」に対応する文字列領域に、ユーザが指定した矩形領域の始点の座標および終点の座標を保持させる。更新されたマスキング情報はＲＡＭ２１３に記憶される。

For example, when the masking instruction button 1012 is selected, the display control unit 427 detects that the masking image in the preview display area 1001 is touched with a finger and detects the point touched with the finger as the start point. The display control unit 427 detects the point where the finger is released after dragging from the start point as the end point. This operation corresponds to an operation that requires a change of the masking information. In this case, in S905, the image processing unit 423 updates the masking information so that the rectangular area specified by the detected start point and end point is held as information of a new masking area. Specifically, the image processing unit 423 adds the information type "rectangular area" to the acquired masking information. The character string corresponding to the information type "rectangular area" holds "- (none)" and the masking necessity "necessary". Then, the character string area corresponding to the information type "rectangular area" holds the coordinates of the start point and the coordinates of the end point of the rectangular area designated by the user. The updated masking information is stored in the RAM 213.

Table 6 shows the masking information after a character string area in which the masking requirement of the new masking area "rectangle area" is "required" has been added. Masking areas may also be added by other methods.

他にも例えば、マスキング領域選択ボタン１０１３が選択されている場合、表示制御部４２７は、プレビュー表示領域１００１のマスキング画像上のマスクが指でタッチされたことを検知すると、タッチされたマスクが選択されたと認識する。そして、当該選択に続けてマスク削除ボタン１０１１を押下する操作は、マスキング情報の変更が必要な操作に該当する。この場合、Ｓ９０５において画像処理部４２３は、マスキング情報における選択されたマスクに対応するマスキング領域に対応付けられたマスキング要否を「不要」に変更する。

For another example, when the masking area selection button 1013 is selected, the display control unit 427 recognizes that the touched mask has been selected when it detects that the mask on the masking image in the preview display area 1001 has been touched with a finger. Then, the operation of pressing the mask deletion button 1011 following the selection corresponds to an operation that requires a change of the masking information. In this case, in S905, the image processing unit 423 changes the masking necessity associated with the masking area corresponding to the selected mask in the masking information to "not required."

Alternatively, suppose that the display control unit 427 receives a press on a check box 1036, 1037, and the checked state of the check box is unchecked. In this case, the image processing unit 423 changes the corresponding masking necessity in the masking information from "required" to "not required." Alternatively, suppose that the display control unit 427 receives a press on a check box 1036, 1037, and the unchecked check box is changed to a checked state. In this case, the image processing unit 423 changes the corresponding masking necessity in the masking information from "not required" to "required."

When the masking information is updated in S905, the process returns to S901. In S901, the image processing unit 423 creates a masking image based on the changed masking information. Then, in S902, the created masking image is displayed in the preview display area 1001. For example, as shown in FIG. 10(c), assume that a masking area for the information type "Address" has been newly set by the user. In this case, a masking image in which the address character string "Ota Ward, Tokyo" is masked so that it cannot be recognized by the user is displayed in the preview display area 1001.

Meanwhile, in S906, the display control unit 427 determines whether or not the mask display check box 1014 has been pressed via the UI display control unit 414. If the display control unit 427 determines that the mask display check box 1014 has been pressed (YES in S906), the process proceeds to S907. If the display control unit 427 determines that the mask display check box 1014 has not been pressed (NO in S906), the process proceeds to S910.

In S907, the display control unit 427 determines whether the mask display check box 1014 is checked. If the display control unit 427 determines that the mask display check box 1014 is checked (YES in S907), the process proceeds to S908. If the display control unit 427 determines that the mask display check box 1014 is not checked (NO in S907), the process proceeds to S909.

In S909, the display control unit 427 changes the transparency of the masking display information to 0.5. Then, the process returns to S901. In the next step S901, a masking image is generated based on the masking display information with a transparency of 0.5, and in S902, a preview of the masking image is displayed in which the masking area, which is the character string area in which the masking requirement is "required", is masked with a transparency of 0.5. The masking image masked with a transparency of 0.5 is an image for the user to confirm, and is controlled so that it is not transmitted to the outside.

FIG. 10D is a diagram showing an example of an image display area 1010 of a preview display screen 1000 in which a masked image generated based on masking display information with a transparency of 0.5 is previewed. A transparency of 0.5 means that the transparency of the mask is increased, so the masking image is generated so that a transparent mask is synthesized. Therefore, the character string included in the masking area in the masking image is displayed in a state where it can be recognized through the mask. In this manner, in this embodiment, by unchecking the mask display checkbox 1014, the masked image can be previewed in an unmasked state so that the masked character string can be seen. Therefore, the user can confirm the location of the masking area, as well as which character string is masked, and can check whether the masking area has been set erroneously.

In this embodiment, the transparency of the mask when the mask display checkbox 1014 is not checked is set to 0.5, but the transparency may be a value other than 0.5 as long as it is visible to the user.

On the other hand, in S908, the display control unit 427 sets the transparency of the masking display information to 0.0. Then, the process returns to S901. By setting the transparency to 0.0, the character string included in the masked area of the masking image becomes unrecognizable in the preview display area 1001.

In this manner, in this embodiment, when the mask display check box 1014 is checked, the mask 1005 is displayed in the masked image displayed as a preview. When the mask display check box 1014 is not checked, the mask 1005 is displayed in an unmasked state, allowing the display to be switched.

On the other hand, in S910, the display control unit 427 performs processing to check the masking process before outputting to the external storage 120 in S505.

Figure 11 is a flowchart explaining the details of the processing of S910.

In S1101, the display control unit 427 determines whether the Confirm Print button 1043 on the preview display screen 1000 has been pressed. If the display control unit 427 determines that the Confirm Print button 1043 has been pressed (YES in S1101), the process proceeds to S1102. If the display control unit 427 determines that the Confirm Print button 1043 has not been pressed (NO in S1101), the process proceeds to S1103.

In S1102, the display control unit 427 performs a confirmation print of the masking image generated in S901. First, the display control unit 427 acquires the masking image generated in S901 from the RAM 213. The display control unit 427 requests the print execution unit 413 to print the masking image via the print instruction unit 425. The print execution unit 413 executes printing (confirmation printing) of the acquired masking image.

Figure 12 shows an example of a printout obtained by checking and printing the masking image generated in S901. Printout 1201 in Figure 12(a) is a printout obtained by checking and printing the masking image generated in S901 based on masking display information with a transparency of 0.0. Because the transparency is set to 0.0, the masking image is printed in such a way that "Koyano", "090-0000-2111", and "Ota-ku, Tokyo", which correspond to the information types "Name", "Telephone Number", and "Address" that are the targets of masking, are not visible to the user.

Printed material 1202 in FIG. 12(b) is a printed material obtained by checking and printing the masked image generated in S901 based on the masking display information with a transparency of 0.5. Because the transparency is set to 0.5, unlike printed material 1201, it is a printed material of the masked image in which the character strings "Koyano", "090-0000-2111", and "Ota-ku, Tokyo" that are the targets of masking can be seen.

In this embodiment, a method has been described for generating a masking image in which the masked character string is visible when the mask display check box 1014 is not checked (mask display released state). There are no limitations on the method for generating a masking image in which the masked character string is visible when the mask display is released. Alternatively, when the mask display is released, the display type of the masking display information in Table 4 may be changed to "underline". In this case, in S901, the image processing unit 423 generates a masking image by combining an underline with the masked area in the document image.

Printout 1203 in FIG. 12(c) is an example of a printout obtained by checking and printing a masking image generated based on masking display information with a display type of "underline." By underlining the masking target in this way, the area to be masked and the contents of the text to be masked can be visually confirmed. Alternatively, when the mask display is released, a confirmation masking image may be generated in which the area other than the area to be masked is filled with a background color so that only the area to be masked is displayed.

In addition, to ensure that the printout of the masking image for confirming the mask display release state is promptly discarded, the system may be controlled so that a character string or image urging the user to discard the masking image is superimposed on the masking image before printing the masking image for confirmation.

In S1103, the display control unit 427 determines whether the list display button 1044 on the preview display screen 1000 in FIG. 10A has been pressed. If the display control unit 427 determines that the list display button 1044 has been pressed (YES in S1103), the process proceeds to S1104. If the display control unit 427 determines that the list display button 1044 has not been pressed (NO in S1103), the process ends this flowchart and returns to S901 in FIG. 9.

In S1104, the display control unit 427 acquires the document image data and masking information from the RAM 213 via the image processing unit 423. The display control unit 427 cuts out a masking area, which is a character string area associated with "needed" as the masking information's masking necessity, from the document image to generate a partial image. The partial image is stored in the RAM 213.

In S1105, the display control unit 427 acquires the partial image and masking information generated in S1104 from the RAM 213, and generates a list print screen for the user to check the masking area. The display control unit 427 displays the generated list print screen on the touch panel of the operation unit 220 via the UI display control unit 414.

Figure 13 is a diagram showing an example of a list print screen. The list print screen 1300 includes a masking information list 1301, a back button 1302, and a list print button 1303. The back button 1302 is a button for closing the list print screen 1300. The list print button 1303 is a button for instructing printing of the masking information list 1301.

Masking information list 1301 is a confirmation list display image in which the partial images generated in S1104, the information types corresponding to the partial images, and character strings corresponding to the information types are associated with each row. The information types are the information types associated with the masking areas referenced when cutting out the partial images in the masking information. The character strings corresponding to the information types are the character strings associated with the masking areas referenced when cutting out the partial images in the masking information.

In S1106, the display control unit 427 determines whether or not the back button 1302 has been pressed. If the display control unit 427 determines that the back button 1302 has been pressed (YES in S1106), this flowchart ends and the process returns to S901 in FIG. 9. If the display control unit 427 determines that the back button 1302 has not been pressed (NO in S1106), the process proceeds to S1107.

In S1107, the display control unit 427 determines whether the list print button 1303 has been pressed. If the display control unit 427 determines that the list print button 1303 has been pressed (YES in S1107), the process proceeds to S1108. If the display control unit 427 determines that the list print button 1303 has not been pressed (NO in S1107), the process returns to S1105.

In S1108, the image processing unit 423 generates a list display image representing the masking information list 1301 and stores it in the RAM 213. The display control unit 427 acquires the generated list display image from the RAM 213. The display control unit 427 requests the print execution unit 413 to print the acquired list display image via the print instruction unit 425. The print execution unit 413 prints the list display image and returns to S1105.

In addition, to ensure that the printed matter obtained by printing the list display image is promptly discarded, the list display image may be controlled to be printed after a character string or image encouraging the user to discard the printed matter is superimposed on the list display image.

In S1108, in addition to requesting printing of the list display image, the display control unit 427 may also obtain the masking image generated in S901 from the RAM 213 and request the print execution unit 413 to print the obtained masking image via the print instruction unit 425.

In this way, a confirmation print or a list display image can be printed before outputting the masked image. For example, if the touch panel of the MFP 110 is small and the screen size is small, the masked parts of the preview image may be displayed small, resulting in low visibility. In such a case, there is a risk that the masked parts may not be confirmed. In this embodiment, when confirmation print or printing of the list display image is selected, the user can check the printed matter. Therefore, the user can perform the confirmation work in a state where the visibility of the masked parts is improved. Therefore, the user can accurately check whether the masked parts are excessive or insufficient, or if there are errors, and it is possible to appropriately mask and output character strings related to confidential information or the range of disclosure. Returning to FIG. 9, the rest of the flowchart will be explained.

In S911, the display control unit 427 acquires the document image corrected in S701, the masking information generated in S707, and the masking display information from the RAM 213 via the image processing unit 423. If the masking information is changed in S905, the changed masking information is acquired. The image processing unit 423 acquires the start and end coordinates of the character string area in which the masking necessity of the acquired masking information is "necessary", and performs masking on the masking area, which is the area corresponding to the coordinate position on the document image, to generate a masking image. The image processing unit 423 performs masking using the display type and display color among the information included in the acquired masking display information. In S911, the image processing unit 423 performs masking with the transparency set to 0.0 without using the transparency included in the acquired masking display information. That is, in S911, a masking image is generated based on the masking display information in Table 4. The image processing unit 423 holds the generated masking image for transmission in the RAM 213. In this way, the masking image for output is masked with a transparency of 0.0 so that the text in the masking area is not visible. After S911, the masking display information is changed to the state shown in Table 2.

[Registering presets]
FIG. 14 is a flowchart for explaining details of the preset registration process in S506 in this embodiment.

In S1401, the display control unit 427 displays a screen (not shown) for prompting the user whether to register a preset on the touch panel of the operation unit 220 via the UI display control unit 414.

In S1402, the display control unit 427 determines whether or not a user operation has been performed to register a preset. If the display control unit 427 determines that a user operation has been performed to register a preset (YES in S1402), the process proceeds to S1403. If the display control unit 427 determines that a user operation has not been performed to register a preset (NO in S1402), the process ends this flowchart.

In S1403, the display control unit 427 requests the UI display control unit 414 to display a preset registration screen on the touch panel of the operation unit 220. If the information type of the generated masking information holds items other than "rectangular area", the display control unit 427 generates a preset registration screen 1500 (see FIG. 15) for selecting an information type. If the information type of the masking information holds only "rectangular area", the display control unit 427 generates a preset registration screen (not shown) for area selection.

Figure 15 is a diagram showing an example of a preset registration screen that is displayed when the information type of the masking information holds an item other than "rectangular area." The preset registration screen 1500 in Figure 15 has a preset name display area 1510, a document type selection area 1520, a registration information display area 1530, a save button 1503, and a cancel button 1504. The cancel button 1504 is a button for ending this flowchart without performing the preset registration process.

The preset name display area 1510 is an area that displays the preset name, which is a name used to identify the preset to be registered. Pressing the preset name display area 1510 transitions to a screen for modifying the preset name, and the modification of the preset name is accepted. The preset name is assumed to be a character string that indicates the document type.

The document type selection area 1520 includes a pull-down menu that allows the user to select the document type to be registered as a preset. The drop-down menu included in the document type selection area 1520 displays the document type selected in the document type selection drop-down 1021 of the document type display area 1020 as selected by default.

The registration information display area 1530 includes display areas 1531-1533 generated for each information type. The display areas 1531-1533 include toggle buttons 1535-1537 for selecting whether or not to register the corresponding information type in the preset as an information type to be masked. When toggle buttons 1535-1537 are selected to ON, the corresponding information type is registered in the preset as an information type to be masked. When toggle buttons 1535-1537 are selected to OFF, the corresponding information type is not registered in the preset. The initial display of toggle buttons 1535-1537 on the preset registration screen 1500 is set according to whether or not masking of masking information is required.

The save button 1503 is a button for registering the registration type selected in the registration information display area 1530 as a preset for the document type selected in the document type selection area 1520, as a masking target. As a result, preset information such as that shown in Table 2 is generated.

In S1404, the display control unit 427 requests the data management unit 424 to save the preset information generated in S1403. The data management unit 424 saves the preset information in the external storage 120 via the Internet access unit 426. Alternatively, the preset information may be saved in the HDD 214 via the internal data saving unit 412.

When a preset of the same document type is saved in the HDD 214 or the external storage 120, the display control unit 427 may request the touch panel of the operation unit 220 to display a warning message asking the user whether or not to overwrite the saved preset. In addition, when the preset registration is completed normally, the display control unit 427 may request the operation unit 220 to display a message indicating that the preset registration is completed.

As described above, according to this embodiment, since a masked image can be displayed in a mask-display-off state, it is possible to check the contents of the masked parts of the generated masked image before outputting the masked image. This makes it possible to prevent excesses, shortages, or errors in the masked parts. In addition, it is possible to mask and transmit text related to confidential information or the scope of disclosure so that it cannot be viewed.

Second Embodiment
In this embodiment, a method for more appropriately checking the masking process on the list print screen 1300 will be described. The present embodiment will be described focusing on the differences from the first embodiment. The configuration and process are the same as those of the first embodiment unless otherwise specified.

In this embodiment, in S901 of the flowchart in FIG. 9, the image processing unit 423 acquires the document image, masking information, and masking display information from the RAM 213, as in the first embodiment. Furthermore, in this embodiment, in S901, the image processing unit 423 acquires masking display correction information from the RAM 213.

Table 7 shows an example of masking display correction information. The masking display correction information includes an area correction width and an area correction height. For example, the display correction width is 5 and the area correction height is 2.

Ｓ９０１において画像処理部４２３は、マスキング表示補正情報の数値に従って、取得したマスキング情報に含まれる文字列領域の始点の座標および終点の座標を変更して変更後の座標情報を、変更後の文字列領域の情報として保持する。始点座標のｘ座標の値を領域補正幅で減算し、ｙ座標の値を領域補正高さで減算する。また、終点座標のｘ座標の値を領域補正幅で加算し、ｙ座標の値を領域補正高さで減算する。例えば文字列領域が「（３４１，４６１）、（９９７，５４５）」の場合、変更後の文字列領域は「（３３６，４５９）、（１００２，５４７）」となる。なお、Ｓ９０１では、マスキング要否が「要」の文字列については、変更後の座標に基づきマスキングされてもよいし、変更前の座標に基づきマスキングされてもよい。

In S901, the image processing unit 423 changes the coordinates of the start point and the end point of the character string area included in the acquired masking information according to the numerical value of the masking display correction information, and holds the changed coordinate information as information of the changed character string area. The x-coordinate value of the start point coordinate is subtracted by the area correction width, and the y-coordinate value is subtracted by the area correction height. In addition, the x-coordinate value of the end point coordinate is added by the area correction width, and the y-coordinate value is subtracted by the area correction height. For example, if the character string area is "(341, 461), (997, 545)", the changed character string area becomes "(336, 459), (1002, 547)". In S901, the character string whose masking necessity is "necessary" may be masked based on the changed coordinates, or may be masked based on the coordinates before the change.

In S1104 in the flowchart of FIG. 11, the display control unit 427 extracts from the document image a masking area, which is a character string area associated with the masking information's masking necessity field set to "required", to generate a partial image. In this embodiment, the masking area is extracted based on the coordinates of the character string area (masking area) changed in S901 to generate a partial image. Therefore, a partial image is generated in which a larger area is extracted than in the first embodiment.

In S1105, the display control unit 427 generates a list printing screen based on the partial image and masking information generated in S1104, and displays the list printing screen on the touch panel of the operation unit 220 via the UI display control unit 414.

In this manner, in this embodiment, a partial image of an area obtained by expanding the area to be masked can be displayed as a list print image. Therefore, the user can confirm whether a character string detected as a character string corresponding to a certain information type is actually a character string corresponding to that information type. For example, as shown in Table 1, assume that the character string "Kiyano" is detected as a character string corresponding to the information type "name". However, there is a possibility that "Kiyano" included in the character string "Kiyano Co., Ltd." of the company name is detected as a character string corresponding to the information type "name". Therefore, in this embodiment, a partial image including a surrounding area of the character string area where the masking necessity is "necessary" is displayed as the list print screen of FIG. 13. Therefore, the user can confirm as an image whether "Kiyano" detected as a character string corresponding to a certain information type is a part of a character string, such as "Kiyano Co., Ltd.". Therefore, it is possible to confirm whether an appropriate character string has been detected as a masking target.

<Other embodiments>
The present disclosure can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present disclosure can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

The disclosure of the above-mentioned embodiment includes the following configurations:

(Configuration 1)
An acquisition means for acquiring a document image;
an image processing means for generating a masking image in which a target area including a predetermined character string in the document image is masked so that the predetermined character string included in the document image cannot be recognized by a user;
a control means for controlling the presentation of the predetermined character string in a manner recognizable by the user based on an instruction from the user;
13. An image processing device comprising:

(Configuration 2)
The image processing device according to configuration 1, wherein the control means presents a confirmation image that enables the user to recognize a position of the target area in the document image and the predetermined character string including the target area.

(Configuration 3)
the masking image is an image in which a mask is combined with the target area of the document image,
The control means
The image processing device according to configuration 2, characterized in that the confirmation image in which a mask having a higher transparency than the transparency of the mask in the output masking image is combined with the target area of the document image is presented to the user.

(Configuration 4)
The control means
3. The image processing apparatus according to claim 2, wherein the confirmation image is presented in which an underline is synthesized in the target area of the document image.

(Configuration 5)
The control means
5. The image processing device according to claim 2, wherein the confirmation image is displayed on a display unit to present the confirmation image.

(Configuration 6)
The control means
6. The image processing device according to any one of configurations 2 to 5, wherein the confirmation image is presented by processing the confirmation image to be printed.

(Configuration 7)
The control means
the masking image in which the target area is masked so that the user cannot recognize the predetermined character string included in the document image; and the confirmation image in which the user can recognize the predetermined character string included in the document image.
and switching between the displayed images based on an operation by the user.

(Configuration 8)
The image processing means includes:
further generating a screen showing a list of the predetermined character strings included in the target area of the document image;
The control means
2. The image processing device according to configuration 1, wherein the screen is presented to the user, so that the predetermined character string included in the target area is presented so as to be recognizable by the user.

(Configuration 9)
The image processing means includes:
9. The image processing device according to configuration 8, further comprising: generating a partial image corresponding to the target region in the document image; and generating the screen including the partial image.

(Configuration 10)
The image processing means includes:
10. The image processing apparatus according to claim 9, further comprising: generating the partial image of an area including the target area in the document image and larger than the target area.

(Configuration 11)
11. The image processing device according to configuration 9 or 10, wherein on the screen, the partial image, a character string represented by the partial image, and an information type of the character string are associated with each other.

(Configuration 12)
The control means
12. The image processing device according to any one of configurations 8 to 11, wherein the screen is displayed on a display unit.

(Configuration 13)
The control means
13. The image processing device according to any one of configurations 8 to 12, further comprising: a processor for processing the screen so that the screen can be printed.

(Configuration 14)
14. The image processing device according to any one of configurations 1 to 13, further comprising a receiving means for receiving an instruction to change the target area from the user, wherein the image processing means generates the masking image in which the target area changed by the user is masked.

(Configuration 15)
15. The image processing device according to any one of configurations 1 to 14, wherein the masking image is an image in which the target area is filled with a predetermined color.

(Configuration 16)
16. The image processing device according to any one of configurations 1 to 15, further comprising a detection unit for detecting the predetermined character string from the document image.

(Configuration 17)
17. The image processing apparatus according to any one of configurations 1 to 16, further comprising an output unit that outputs the generated masking image based on an instruction from the user.

(Configuration 18)
An acquisition step of acquiring a document image;
an image processing step of generating a masking image in which a target area including a predetermined character string in the document image is masked so that the predetermined character string included in the document image cannot be recognized by a user;
a control step of performing control for presenting the predetermined character string in a manner recognizable by the user based on an instruction from the user;
13. An image processing method comprising:

(Configuration 19)
18. A program for causing a computer to execute each of the means of the image processing device according to any one of configurations 1 to 17.

110 MFP
427 Display control unit 423 Image processing unit

Claims (19)

An acquisition means for acquiring a document image;
an image processing means for generating a masking image in which a target area including a predetermined character string in the document image is masked so that the predetermined character string included in the document image cannot be recognized by a user;
a control means for controlling the presentation of the predetermined character string in a manner recognizable by the user based on an instruction from the user;
13. An image processing device comprising: The image processing device according to claim 1 , wherein the control means presents a confirmation image that enables the user to recognize the position of the target area in the document image and the predetermined character string that includes the target area. the masking image is an image in which a mask is combined with the target area of the document image,
The control means
3 . The image processing apparatus according to claim 2 , further comprising: a confirmation image in which a mask having a higher transparency than a mask in the output masking image is combined with the target area of the document image, the confirmation image being presented to the user. The control means
The image processing apparatus according to claim 2 , wherein the confirmation image is presented in which an underline is synthesized in the target area of the document image. The control means
The image processing device according to claim 2 , wherein the confirmation image is presented by displaying the confirmation image on a display unit. The control means
The image processing apparatus according to claim 2 , wherein the confirmation image is presented by performing processing so that the confirmation image is printed. The control means
6. The image processing device according to claim 5, characterized in that the masking image in which the target area is masked so that the user cannot recognize the specified character string included in the document image, and the confirmation image in which the user can recognize the specified character string included in the document image are switched and displayed based on the user's operation. The image processing means includes:
further generating a screen showing a list of the predetermined character strings included in the target area of the document image;
The control means
The image processing device according to claim 1 , wherein the screen is presented to the user so that the predetermined character string included in the target area is presented in a manner that the user can recognize the predetermined character string. The image processing means includes:
The image processing apparatus according to claim 8 , further comprising: generating a partial image corresponding to the target region in the document image; and generating the screen including the partial image. The image processing means includes:
The image processing apparatus according to claim 9 , further comprising: a step of generating the partial image of an area including the target area in the document image and larger than the target area. The image processing apparatus according to claim 9 , wherein on the screen, the partial image, a character string represented by the partial image, and an information type of the character string are associated with each other. The control means
The image processing device according to claim 8 , wherein the screen is displayed on a display unit. The control means
The image processing apparatus according to claim 8, further comprising: a processor for processing the screen so that the screen can be printed. The image processing device according to claim 1 , further comprising a receiving unit configured to receive an instruction to change the target area from the user, wherein the image processing unit generates the masking image in which the target area changed by the user is masked. The image processing apparatus according to claim 1 , wherein the masking image is an image in which the target area is filled with a predetermined color. The image processing apparatus according to claim 1 , further comprising a detection unit for detecting the predetermined character string from the document image. The image processing apparatus according to claim 1 , further comprising an output unit that outputs the generated masking image based on an instruction from the user. An acquisition step of acquiring a document image;
an image processing step of generating a masking image in which a target area including a predetermined character string in the document image is masked so that the predetermined character string included in the document image cannot be recognized by a user;
a control step of performing control for presenting the predetermined character string in a manner recognizable by the user based on an instruction from the user;
13. An image processing method comprising: A program for causing a computer to execute each of the means of an image processing device according to any one of claims 1 to 17.

Images