JP2023038551A - Image processing program, image processing method, and image processing apparatus - Google Patents

Abstract

To provide an information processing program, an image processing method, and an image processing apparatus that prevent leakage of personal information from an image including a two-dimensional code.SOLUTION: An image processing program causes a computer to perform processing of decoding a two-dimensional code included in an image, and masking a first image area in the two-dimensional code when predetermined information satisfying a rule that defines personal information is included in a decode result.SELECTED DRAWING: Figure 7

Description

The present application relates to an image processing program, an image processing method, and an image processing apparatus.

There is known a technique of transferring image data from which a QR code (Quick Response code (registered trademark)) is attached to an image represented by image data, with the QR code removed from the image (see, for example, Patent Document 1). . Also known is a technique of capturing an image displayed on a terminal, determining a mask position of the image based on a two-dimensional code added to the captured image, and applying mask processing to the mask position (for example, patent Reference 2).

JP 2009-278579 A JP 2011-238107 A

By the way, a two-dimensional code including a QR code may contain personal information such as my number and name. If the personal information is converted into a two-dimensional code, it will be difficult to distinguish the personal information visually. However, if an image containing a two-dimensional code is generated and distributed, and the two-dimensional code is decoded during the distribution process, personal information may be leaked.

Accordingly, it is an object of one aspect to provide an image processing program, an image processing method, and an image processing apparatus that suppress leakage of personal information from an image containing a two-dimensional code.

In one embodiment, the image processing program decodes a two-dimensional code included in an image, and if the decoding result includes predetermined information that satisfies a rule defining personal information, the first image region of the two-dimensional code and let the computer do the work.

Personal information leakage from an image containing a two-dimensional code can be suppressed.

FIG. 1 is a diagram illustrating an example of an image processing system. FIG. 2 shows an example of the hardware configuration of the image processing server. FIG. 3 is an example of the functional configuration of the image processing server. FIG. 4A is a first example of rule information. FIG. 4B is a second example of rule information. FIG. 5 is a third example of rule information. FIG. 6 is a flow chart showing an example of processing executed by the image processing server. FIG. 7A is an example of image data before masking. FIG. 7B is a diagram for explaining an example of decoding a two-dimensional code. FIG. 7(c) is a diagram for explaining an example of identifying personal information. FIG. 7D is an example of image data after masking.

Hereinafter, the form for carrying out this case will be described with reference to the drawings.

As shown in FIG. 1, the image processing system ST includes a user terminal 100 and an image processing server 200. As shown in FIG. The image processing server 200 is an example of an image processing device and executes an image processing method. A user terminal 100 is operated by the user 10 . User terminal 100 includes input device 110 , display device 120 and control device 130 . The input device 110 corresponds to, for example, a keyboard and a mouse. A liquid crystal display, for example, corresponds to the display device 120 . The control device 130 receives instructions input through the input device 110, and controls the display of the display device 120 according to the received instructions. Further, the control device 130 controls communication with the image processing server 200 according to the received instruction. Note that FIG. 1 shows a PC (Personal Computer) as an example of the user terminal 100, but the user terminal 100 may be a smart terminal such as a smart phone or a tablet terminal.

Control device 130 and image processing server 200 are connected to each other via communication network NW. The communication network NW includes one or both of a LAN (Local Area Network) and the Internet. The communication form of the communication network NW may be wired communication or wireless communication.

The control device 130 can transmit image data to the image processing server 200 based on the above instructions. For example, controller 130 can transmit image data that includes a two-dimensional code. The two-dimensional code may be a matrix format two-dimensional code such as a QR code, or a stack format two-dimensional code such as PDF (Portable Data File) 417 .

Upon receiving the image data transmitted from the control device 130, the image processing server 200 executes various types of processing, including image processing, on the image data. For example, the image processing server 200 masks (hides) the image area of the two-dimensional code included in the image data. After completing the processing, the image processing server 200 transmits image data after image processing to the control device 130 . Thereby, the control device 130 receives the image data after the image processing. Therefore, if the image area of the two-dimensional code is masked, even if the two-dimensional code contains personal information, the two-dimensional code cannot be decoded (decoded or restored). Personal information leakage can be suppressed.

Next, a hardware configuration of the image processing server 200 will be described with reference to FIG. Since the hardware configuration of the user terminal 100 is basically the same as that of the image processing server 200, detailed description thereof will be omitted.

The image processing server 200 includes a CPU (Central Processing Unit) 200A as a processor, RAM (Random Access Memory) 200B and ROM (Read Only Memory) 200C as memories, and a network I/F 200D. The image processing server 200 may include at least one of a HDD (Hard Disk Drive) 200E, an input I/F 200F, an output I/F 200G, an input/output I/F 200H, and a drive device 200I, if necessary. The CPU 200A to the drive device 200I are interconnected by an internal bus 200J. That is, the image processing server 200 can be implemented by a computer.

An input device 710 is connected to the input I/F 200F. The input device 710 includes a keyboard and a mouse. A display device 720 is connected to the output I/F 200G. As the display device 720, there is a liquid crystal display. A semiconductor memory 730 is connected to the input/output I/F 200H. Examples of the semiconductor memory 730 include USB (Universal Serial Bus) memory and flash memory. The input/output I/F 200H reads the image processing program stored in the semiconductor memory 730 . The input I/F 200F and the input/output I/F 200H are provided with USB ports, for example. The output I/F 200G has, for example, a display port.

A portable recording medium 740 is inserted into the drive device 200I. Examples of the portable recording medium 740 include removable discs such as CD (Compact Disc)-ROM and DVD (Digital Versatile Disc). The drive device 200I reads the image processing program recorded on the portable recording medium 740 . The network I/F 200D has, for example, a LAN port. Network I/F 200D is connected to communication network NW.

The CPU 200A temporarily stores image processing programs stored in the ROM 200C and HDD 200E in the above-described RAM 200B. The RAM 200B temporarily stores the image processing program recorded on the portable recording medium 740 by the CPU 200A. As the CPU 200A executes the stored image processing program, the CPU 200A realizes various functions described later and also executes various processes described later. Incidentally, the image processing program may correspond to a flow chart described later.

Next, the functional configuration of the image processing server 200 will be described with reference to FIGS. 3 to 5. FIG. Note that FIG. 3 shows the essential functions of the image processing server 200 .

As shown in FIG. 3, the image processing server 200 includes a storage unit 210, a processing unit 220, and a communication unit 230. The storage unit 210 can be realized by one or both of the RAM 200B and HDD 200E described above. The processing unit 220 can be realized by the CPU 200A described above. The communication unit 230 can be implemented by the network I/F 200D described above. Therefore, the storage unit 210, the processing unit 220, and the communication unit 230 are connected to each other.

Here, the storage section 210 includes an image storage section 211 and a rule storage section 212 . On the other hand, the processing unit 220 includes a transmission/reception unit 221 , an OCR (Optical character recognition) unit 222 and a decoding unit 223 . The processing unit 220 also includes a first determination unit 224 , a second determination unit 225 and a mask unit 226 .

The image storage unit 211 stores image data. More specifically, the image storage unit 211 stores image data transmitted from the control device 130 and received by the transmission/reception unit 221 . That is, when receiving the image data, the transmission/reception unit 221 stores the received image data in the image storage unit 211 .

The rule storage unit 212 stores rule information. The rule storage unit 212 may store one piece of rule information, or may store a plurality of pieces of rule information. Rule information is information that defines personal information. The rule information includes item names and rules, as shown in FIGS. The type of personal information is registered in the item name. For example, as shown in FIG. 4A, a character string of My Number, which is one piece of personal information, is registered as the item name. As shown in FIG. 4B, a character string of a credit card number, which is one piece of personal information, may be registered as the item name. Alternatively, as shown in FIG. A character string of a certain name may be registered.

In addition, the personal information registered in the item name is not particularly limited to each character string of my number, credit card number, and name. For example, the personal information may be a character string such as a postal code, a character string such as an address, or a character string including a bank account number. In other words, personal information is not limited to a character string of information that uniquely identifies an individual, but may be a character string of information that allows an individual to be estimated.

Rules included in rule information include regular expressions and calculation formulas. A regular expression can also identify personal information, and a formula can also identify personal information. A regular expression is an identifier that determines whether or not it is personal information by pattern matching. In the case of regular expressions, for example, a glob expression format can be adopted. For example, in the regular expression for the item name "My Number", the combination of characters and symbols "\s*" following the character string "My Number" has zero or more spaces (spaces) between the symbol ": (colon)" ). The same applies to the character-symbol combination "\s*" following the symbol ":". A combination of characters, symbols, and numerical values "\d{12}" represents a 12-digit numerical value.

That is, since My Number is represented by a 12-digit numerical value, if the character string "My Number", the 12-digit numerical value, and the symbol ":" located between them are included in the two-dimensional data as predetermined information , can be determined to be personal information by a regular expression. The specific explanation of My Number is defined and disclosed in the My Number System of the Ministry of Internal Affairs and Communications.

On the other hand, the calculation formula is an identifier for determining whether or not personal information is personal information based on a numerical value that defines personal information. If it is a calculation formula with the item name "My Number", substitute the 11-digit number other than the last (first digit) of the 12-digit number in the formula, and the calculation result will be at the end of the 12-digit number. If it matches the check digits, it can be determined that the 12-digit number is My Number. For example, if it is a 12-digit number "123456789107", the formula is first "1 x (11-5) + 2 x (10-5) + ... + 5 x (7-5) + 6 x (6 + 1) + . . . 0×(1+1)” is calculated and “213” is calculated. Next, "213/11" is calculated, the remainder "4" is calculated, "11-4" is calculated, and the calculation result "7" is calculated. Since the calculation result "7" matches the check digit "7" located at the end of the 12-digit number, it can be determined that the 12-digit number is my number.

Although the details are omitted, it is basically possible to determine whether or not the item name "credit card number" shown in FIG. can. On the other hand, in the case of the item name "name" shown in FIG. 5, since the name does not contain a numerical value, it is sufficient to use a regular expression alone to determine whether or not it is the name. The name may be written in kanji, hiragana, or a mixture of kanji and hiragana. Moreover, in the case of a name, it may be written in katakana or in alphabet.

Returning to FIG. 3, the transmission/reception unit 221 receives image data and stores it in the image storage unit 211 as described above. Send to 130. The OCR unit 222 acquires image data stored in the image storage unit 211 and performs optical character recognition processing to generate OCR data. That is, the OCR unit 222 converts image data from which characters cannot be extracted into OCR data from which characters can be extracted. A decoding unit 223 determines whether or not the OCR data contains a two-dimensional code, and if so, decodes the two-dimensional code. Thereby, the information contained in the two-dimensional code can be specified.

The first judging section 224 judges whether or not the decoding result includes the predetermined information that satisfies the above rules. The second determination unit 225 determines whether or not the OCR data includes character information that satisfies the rule when predetermined information that satisfies the rule is included in the decoding result. If the decoded result includes predetermined information that satisfies the rule, it can be determined that the predetermined information included in the decoded result is personal information. In this case, since there is a high possibility that the character information included in the OCR data is also personal information, the second determination unit 225 further determines whether or not the character information is personal information using rules. A masking unit 226 masks the image area of the two-dimensional code in the image data when predetermined information that satisfies the rules is included in the decoding result. If the OCR data contains text information that satisfies the rule, the masking unit 226 further masks the image area of the text information in the image data.

Next, details of the processing executed by the image processing server 200 will be described with reference to FIGS. 6 and 7. FIG.

First, as shown in FIG. 6, the transmitter/receiver 221 receives image data (step S1). For example, when part of the screen displayed on the display device 120 is transmitted as image data from the control device 130 , the transmission/reception unit 221 receives the image data via the communication unit 230 . As shown in FIG. 7( a ), the two-dimensional code 11 may be included depending on the image data received by the transmitter/receiver 221 . Control device 130 can generate image data corresponding to a portion of the screen by using a so-called screen shot technique in which a portion of the screen is specified.

When the transmission/reception unit 221 receives the image data, the OCR unit 222 generates OCR data (step S2). More specifically, when image data is received, the transmission/reception unit 221 stores the image data in the image storage unit 211 , so the OCR unit 222 first acquires the image data from the image storage unit 211 . Upon acquiring the image data, the OCR section 222 performs optical character recognition processing to generate OCR data.

When the OCR section 222 generates OCR data, the decoding section 223 determines whether or not there is a two-dimensional code in the OCR data (step S3). For example, in the case of a QR code, the decoding unit 223 scans the OCR data and determines whether or not there is a predetermined area in which a square pattern image is arranged in any three of the four corners of the OCR data. . This pattern image is sometimes called a cutout symbol or a position detection pattern. If the OCR data does not contain this predetermined area, the decoding unit 223 determines that there is no two-dimensional code (step S3: NO). If the OCR data contains this predetermined area, the decoding unit 223 determines that there is a two-dimensional code (step S3: YES).

If there is a two-dimensional code, the decoding section 223 decodes the two-dimensional code (step S4). By decoding the two-dimensional code, the conversion source information representing the conversion source of the two-dimensional code can be specified. After decoding the two-dimensional code, the first determination unit 224 determines whether or not the decoding result includes predetermined information that satisfies the rules (step S5).

For example, as shown in FIG. 7(b), depending on the decoding result of the two-dimensional code 11, there are character strings, symbols, and numerical values that match the regular expression (see FIG. 4(a)) of the item name "my number". may be included. Furthermore, there are cases where the numerical value satisfies the calculation formula (see FIG. 4(a)). In this way, when both the regular expression and the calculation formula are satisfied, the first determination unit 224 determines that the decoding result includes predetermined information that satisfies the rule (step S5: YES). On the other hand, if both the regular expression and the calculation formula are not satisfied, or if either the regular expression or the calculation formula is satisfied, the first determination unit 224 determines that the predetermined information that satisfies the rule is not included in the decoding result ( Step S5: NO).

If the decoding result contains predetermined information that satisfies the rule, the masking unit 226 masks the image area of the two-dimensional code in the image data (step S6). The mask may be any process that renders the image area of the two-dimensional code unreadable. For example, the mask may be a process of blurring the image area of the two-dimensional code by mosaic or the like, or a process of filling it with a color (black, white, red, etc.). In the case of the process of blurring the image area of the two-dimensional code, as shown in FIG. 7(d), the image area of the two-dimensional code is blurred so that the image area of the two-dimensional code cannot be deciphered.

When the masking unit 226 masks the image area of the two-dimensional code, the second judging unit 225 judges whether or not the OCR data contains character information that satisfies the rules (step S7). More specifically, the second determination unit 225 determines whether or not the OCR data contains character information that satisfies the rule on a line-by-line basis. If personal information is included in the two-dimensional code, there is a high possibility that character information representing personal information is also included in the OCR data. Therefore, the second determination unit 225 executes the process of step S7 only when personal information is included in the two-dimensional code.

For example, as shown in FIG. 7(c), depending on the character information contained in the OCR data, there are cases where character strings and symbols that match the regular expression (see FIG. 5) of the item name "name" are included. be. Thus, if the regular expression is satisfied, the second determination unit 225 determines that the OCR data contains character information that satisfies the rule (step S7: YES). On the other hand, if the regular expression is not satisfied, the second determination unit 225 determines that the OCR data does not contain character information that satisfies the rule (step S7: NO).

For example, the second determination unit 225 determines whether or not the OCR data includes character information and numerical values that satisfy the regular expression of the item name “credit card number” and the calculation formula (see FIG. 4(b)). can also In this embodiment, the member number is shown in FIGS. 7(a) and 7(c), but the OCR data contains character information and numerical values that satisfy the regular expression of the item name "credit card number" and the calculation formula. not Therefore, the second determination unit 225 determines that the OCR data does not contain character information and numerical values that satisfy the rule.

If the OCR data contains character information that satisfies the rule, the masking unit 226 masks the image area of the character information in the image data (step S8). For example, the masking unit 226 can mask an image area of character information including a specific name, except for the character string "name" and the symbol ":". The masking unit 226 may mask the image area of the character information including the character string “name”, the symbol “:”, and the specific name.

Also, the mask may be any process that renders the image area of the character information unreadable. For example, a process of blurring an image area of text information may be used as a mask, or a process of filling in with color may be used as a mask. In the case of the process of filling in the image area of the character information excluding the character string "name" and the symbol ":" with black, as shown in FIG. Browsing of the character information becomes impossible. By making the mask for the image area of the two-dimensional code and the mask for the image area of the character information different, it is possible to determine whether the masked image area is the image area of the two-dimensional code or the image area of the character information. can.

After masking the image area of the character information, the transmitting/receiving section 221 transmits the image data with the masked image area to the control device 130 (step S9), and ends the process. When both the image area of the two-dimensional code and the image area of the character information are masked, the control device 130 receives the image data shown in FIG. 7(d). If the control device 130 displays the received image data on the display device 120 , the user 10 can confirm the image data through the display device 120 .

Here, if there is no two-dimensional code in the OCR data in the process of step S3, the transmitting/receiving section 221 transmits the image data in which the image area is not masked, and terminates. The same applies when the predetermined information that satisfies the rule is not included in the decoding result in the process of step S5. On the other hand, if the OCR data does not contain character information that satisfies the rule in the process of step S7, the transmitting/receiving unit 221 transmits the image data in which the image area of the two-dimensional code is masked, and ends the process. That is, the transmitting/receiving unit 221 transmits image data in which the image area of the two-dimensional code is masked and the image area of the character information is not masked, and ends.

In the above-described embodiment, the case where the image data contains one two-dimensional code was explained, but when the image data contains a plurality of two-dimensional codes, the image processing server 200 performs You can do the process like For example, the first determination unit 224 determines whether or not each of the plurality of two-dimensional codes contains predetermined information that satisfies the rule. The masking unit 226 masks the image area of the two-dimensional code containing the predetermined information, and excludes the image area of the two-dimensional code not containing the predetermined information from being masked. That is, the image data includes a masked two-dimensional code image area and an unmasked two-dimensional code image area.

On the other hand, when at least one image area of the two-dimensional code is masked, the second determination unit 225 determines whether or not the OCR data includes character information that satisfies the rule. If character information that satisfies the rule is included, the masking section 226 masks the image area of the character information. Such processing can also prevent personal information from being leaked from an image containing a two-dimensional code.

Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to the specific embodiments, and various modifications can be made within the spirit and scope of the present invention described in the claims.・Changes are possible. For example, the image processing server 200 may be implemented as a physical server or a virtual server.

100 user terminal 200 image processing server 210 storage unit 211 image storage unit 212 rule storage unit 220 processing unit 221 transmission/reception unit 222 OCR unit 223 decoding unit 224 first determination unit 225 second determination unit 226 masking unit

Claims (9)

Decode the two-dimensional code contained in the image,
masking a first image region of the two-dimensional code when predetermined information that satisfies a rule defining personal information is included in the decoding result;
An image processing program that causes a computer to execute processing. The rule defines the personal information with a regular expression,
masking the first image region if the decoding result includes the predetermined information that satisfies the regular expression;
2. The image processing program according to claim 1, characterized by: The rule defines the personal information by a regular expression and a calculation formula,
masking the first image region when the decoding result includes the predetermined information that satisfies both the regular expression and the calculation formula;
2. The image processing program according to claim 1, characterized by: if the image contains text information that satisfies the rule, further masking a second image region of the text information;
4. The image processing program according to any one of claims 1 to 3, characterized by: The rule defines the personal information with a regular expression,
further masking the second image region if the image contains the textual information that satisfies the regular expression;
5. The image processing program according to claim 4, characterized by: The rule defines the personal information by a regular expression and a calculation formula,
further masking the second image region when the image contains the character information that satisfies both the regular expression and the calculation formula;
5. The image processing program according to claim 4, characterized by: masking the first image region and the second image region by different image processing for the first image region and the second image region;
7. The image processing program according to any one of claims 4 to 6, characterized by: Decode the two-dimensional code contained in the image,
masking the image area of the two-dimensional code when the decoding result contains predetermined information that satisfies a rule defining personal information;
An image processing method in which processing is performed by a computer. a decoding unit that decodes the two-dimensional code included in the image;
a masking unit for masking an image area of the two-dimensional code when predetermined information that satisfies a rule defining personal information is included in the decoding result;
An image processing device comprising:

Images