KR20230106931A - Method and apparatus for image deidentification and restoration based on histogram data - Google Patents

Abstract

In order to achieve the above object, a method for restoring a non-identified image based on histogram information according to an embodiment of the present invention includes receiving a QR code including an input image in which a non-identified region is masked and histogram information, the QR code Obtaining the histogram information based on , detecting a masked area in the input image, and performing unmasking on the masked area using the histogram information.

Description

Method and apparatus for image de-identification and restoration based on histogram information

The present invention relates to a technique capable of improving de-identified image restoration performance using histogram information of an original image.

Specifically, the present invention relates to a technique of extracting histogram information of a non-identification area in an image and storing it in a QR code or the like.

In addition, the present invention relates to a technique of restoring an image using histogram information stored in a QR code or the like.

In line with the recent trend in which the subject of personal privacy information protection was limited to online (text, image, video, etc.), it is expanding to various objects in the physical world, images captured by CCTV or offline documents (prints, resident registration cards, driver's licenses, The need to protect personal information included in passports, etc.) is emerging. In a similar context, unlike applications in the online environment where de-identified data is output in the form of a digital file and can be acquired and restored through a PC or a client device owned by the user, even offline where only de-identified information can be checked with the naked eye. There is a need for a technology that allows authorized users to safely restore de-identified data.

After the de-identification of important information in a document is applied, the user can restore the de-identified content through various methods, and there is a difference in the accuracy of the restoration information depending on the restoration performance. For example, when a non-identified image or document is received and restored through a camera, the restoration information cannot be clearly identified due to the effects of the camera's resolution and color characteristics, brightness of light, lighting, and shadow. do. Therefore, a restoration performance improvement method capable of maintaining the accuracy of restoration information at a certain level during character restoration is also absolutely necessary.

Through technology that can protect personal information and important information in offline documents, information leakage of identification cards and important confidential documents that are left exposed on websites is prevented, and personal information marked on mail is not exposed to third parties. Safety and security of various offline application services in which personal information is used, such as preventing cases of being leaked to individuals and being abused for crimes, as well as converting and distributing videos taken out from the CCTV integrated control center into non-identified safe documents. Security can be improved.

Korean Registered Patent Publication No. 10-2192235 (Title of Invention: VSTO-based electronic document de-identification service providing device)

An object of the present invention is to improve de-identified image restoration performance using character histogram information in an original image.

In addition, an object of the present invention is to improve a phenomenon in which restoration performance is deteriorated due to environmental factors when performing a function of restoring an unidentified document or image, and to provide restored text with improved visibility to a user.

To achieve the above object, a histogram information-based image de-identification method according to an embodiment of the present invention includes setting a de-identification area including character information in an input image, and histogram information for the de-identification area. Extracting a, generating a QR code in which the histogram information is stored, masking the non-identification area, and generating a non-identification image including the QR code.

In this case, the histogram information may include a starting point of the text data, a duration of the data value, and a text data value in the image.

At this time, the step of performing the masking may be performed using the user's private key.

In this case, extracting the histogram information may include performing image inversion of the non-identification region and generating a horizontal histogram based on the inverted image.

In addition, an apparatus for image de-identification based on histogram information according to an embodiment of the present invention for achieving the above object includes a memory in which at least one program is recorded, and a processor executing the program, wherein the program is an input Setting a non-identification area including character information in an image, extracting histogram information for the non-identification area, generating a QR code storing the histogram information, masking the non-identification area It includes instructions for performing the step of performing, and the step of generating the non-identification image including the QR code.

In addition, a histogram information-based non-identification image restoration method according to an embodiment of the present invention for achieving the above object includes the steps of receiving a QR code including an input image in which a non-identification area is masked and histogram information, the Obtaining the histogram information based on a QR code, detecting a masked area in the input image, and performing unmasking on the masked area using the histogram information.

In this case, the histogram information may include a starting point of the text data, a duration of the data value, and a text data value in the image.

In this case, the method may further include calculating a histogram value for the unmasked image and displaying a restored character based on a comparison result between the histogram value of the unmasked image and a threshold value.

In this case, the threshold may correspond to a value obtained by multiplying a histogram value of the original image by a preset value.

In this case, in the step of expressing the restored character, if the histogram value of the unmasked image is smaller than the threshold value, the step of detecting the masked region in the input image may be performed again.

At this time, the step of expressing the restored character is performed only when both the threshold probability condition for the histogram true value and the threshold probability condition for the histogram false value are satisfied based on the histogram graph in the horizontal direction of the unmasked image. Restored characters can be expressed.

In addition, an apparatus for reconstructing a non-identified image based on histogram information according to an embodiment of the present invention for achieving the above object includes a memory in which at least one program is recorded, and a processor executing the program, the program comprising: Receiving a QR code including an input image in which a non-identification region is masked and histogram information, obtaining the histogram information based on the QR code, detecting a masked region in the input image, and the and instructions for performing the step of unmasking the masked region using histogram information.

According to the present invention, the performance of restoring a de-identified image can be improved by using character histogram information in an original image.

In addition, the present invention can improve the phenomenon of deterioration of restoration performance due to environmental factors when performing a function of restoring an unidentified document or image, and provide restored text with improved visibility to the user.

1 is a flowchart illustrating a method for de-identifying an image based on histogram information according to an embodiment of the present invention.
2 is a flowchart illustrating a method for restoring a de-identified image based on histogram information according to an embodiment of the present invention.
3 is a diagram showing the overall flow of a de-identification image generation and restoration process according to an embodiment of the present invention.
4 is a block diagram showing the configuration of an image de-identification server according to an embodiment of the present invention.
5 is a block diagram showing the configuration of a de-identification image restoration apparatus according to an embodiment of the present invention.
6 shows the overall structure of a de-identification image generation and restoration process using original image histogram information according to an embodiment of the present invention.
7 is a diagram illustrating a process of analyzing a character histogram of an original image, extracting parameters, and storing a QR code.
8 is a diagram conceptually illustrating a process of analyzing and restoring a character histogram of a restored image.
9 is a diagram exemplarily illustrating a threshold value comparison process for increasing character restoration accuracy.
10 is a diagram showing the configuration of a computer system according to an embodiment.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various different forms, only these embodiments make the disclosure of the present invention complete, and common knowledge in the art to which the present invention belongs. It is provided to fully inform the holder of the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numbers designate like elements throughout the specification.

Although "first" or "second" is used to describe various elements, these elements are not limited by the above terms. Such terms may only be used to distinguish one component from another. Therefore, the first component mentioned below may also be the second component within the technical spirit of the present invention.

Terms used in this specification are for describing embodiments and are not intended to limit the present invention. In this specification, singular forms also include plural forms unless specifically stated otherwise in a phrase. As used herein, "comprises" or "comprising" implies that a stated component or step does not preclude the presence or addition of one or more other components or steps.

Unless otherwise defined, all terms used herein may be interpreted as meanings commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless explicitly specifically defined.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, and when describing with reference to the drawings, the same or corresponding components are given the same reference numerals, and overlapping descriptions thereof will be omitted. .

1 is a flowchart illustrating a method for de-identifying an image based on histogram information according to an embodiment of the present invention.

The image de-identification method according to an embodiment of the present invention may be performed in an image de-identification server, but the scope of the present invention is not limited thereto.

Referring to FIG. 1 , in the histogram information-based image de-identification method according to an embodiment of the present invention, a de-identification area including text information in an input image is set (S110).

Next, histogram information is extracted for the non-identified area (S120).

In this case, the histogram information may include a starting point of the text data, a duration of the data value, and a text data value in the image.

At this time, the step of extracting the histogram information (S120) may include performing image inversion of the non-identification area and generating a histogram in a horizontal direction based on the inverted image. there is.

Next, a QR code in which the histogram information is stored is generated (S130).

Next, masking is performed on the non-identification area (S140), and a non-identification image including the QR code is generated (S150).

At this time, the masking step (S140) may be performed using the user's private key.

2 is a flowchart illustrating a method for restoring a de-identified image based on histogram information according to an embodiment of the present invention.

The non-identification image restoration method according to an embodiment of the present invention may be performed in a user device such as a smart phone, but the scope of the present invention is not limited thereto.

Referring to FIG. 2 , the histogram information-based de-identification image restoration method according to an embodiment of the present invention receives an input image in which a de-identification area is masked and a QR code including histogram information (S210).

In this case, the histogram information may include a starting point of the text data, a duration of the data value, and a text data value in the image.

Next, the histogram information is obtained based on the QR code (S220).

Next, a masked area in the input image is detected (S230), and unmasking of the masked area is performed using the histogram information (S240).

At this time, although not shown in FIG. 2, the method according to the embodiment includes calculating a histogram value for the unmasked image, and a character restored based on a result of comparing the histogram value for the unmasked image with a threshold value. A step of expressing the may be further included.

In this case, the threshold may correspond to a value obtained by multiplying a histogram value of the original image by a preset value.

In this case, in the step of expressing the restored character, if the histogram value of the unmasked image is smaller than the threshold value, the step of detecting the masked region in the input image may be performed again.

At this time, the step of expressing the restored character is performed only when both the threshold probability condition for the histogram true value and the threshold probability condition for the histogram false value are satisfied based on the histogram graph in the horizontal direction of the unmasked image. Restored characters can be expressed.

3 is a diagram showing the overall flow of a de-identification image generation and restoration process according to an embodiment of the present invention.

As shown in FIG. 3 , the server 100 is responsible for generating a de-identified image 600 from a selected area 200 in an original image. More specifically, the server 100 sets a region 200 to be de-identified in the original image, inverts the set region image, and extracts histogram information in a horizontal direction related to characters in the image (300).

Histogram information in the horizontal direction means the starting point of text data, the duration of data values, and the total text data values in an image. The starting point of the character and the duration of the character increase according to the number of lines in which the character is represented in the image. For example, in the case of a character composed of two lines, two character data start points and two data values are extracted.

When the process of analyzing and extracting histogram information (300) is completed, the server 100 performs masking on the set area using the secret key 400 to obtain a de-identified image 600 and a QR in which histogram information of the original image is stored. The code 500 is generated and provided together. The histogram information of the original image is not limited to the QR code form and can be provided in other forms of meta data, and FIG. 3 shows a method of using the QR code as an example.

Thereafter, the QR code 500 including the histogram of the original image and the de-identified image 600 are distributed online or offline and delivered to the user, and the user receives the original image collected from the QR code through the client device 800. Based on the histogram information 700 of the image, while unmasking is performed on the de-identified image 600, restoration performance is improved to restore text with improved visibility (900).

The secret key 400 used in the masking and unmasking algorithm is a security policy such as how the server 100 and the client device 800 set and use the same value in advance and how to safely receive and use it through the key management server It can be set in various ways depending on

4 is a block diagram showing the configuration of an image de-identification server according to an embodiment of the present invention.

Referring to FIG. 4 , the server 100 includes a de-identification area setting module 110 that sets a region requiring de-identification in an original image and a histogram information extraction module 120 that extracts and analyzes character area histogram information of the original image. ), a key management module 130 that manages users' secret keys, a masking module 140 that masks the selection area delivered by the non-identification area setting module 110 using a secret key, and stores the histogram of the original image To the QR code and non-identification image generation module 150 that generates the result of the masking module 140 and the QR code to generate the non-identified image form, and finally to the database 160 that safely stores and manages users' private keys System internal modules are configured.

5 is a block diagram showing the configuration of a de-identification image restoration apparatus according to an embodiment of the present invention.

Referring to FIG. 5, the client device 800 that restores the de-identified image by unmasking the camera module 810 that scans the QR code and the de-identified image, and obtains original image histogram information after scanning the QR code. QR code scan and original image histogram information acquisition module 820 that is stored by scanning QR codes, and non-identification area detection and correction module 830 that detects non-identification areas in images for unmasking and corrects items such as size and tilt ), the unmasking module 840 that unmasks the area finally extracted from the non-identification area detection and correction module 830, and the original image histogram information is used to restore the non-identification area in the unmasking module 400. A restored image performance improvement module 850 that provides a function to adjust the character accuracy performance of the restored image, a key management module 860 that manages the secret key, a secure storage module 870 that safely stores the secret key, and finally a client device 800 ) and an output module 880 displaying the restored image on the screen.

Considering the security aspect, the server 100 allocates the client device 800 ID and user ID when registering the client device 800 and the user, and based on the ID, the user secret key 400 and device-related unique information. It is safely managed on the database 160. In addition, the client device 800 can safely receive the user ID and user secret key 400 from the server upon user registration or login, store them in the device, and then use them.

Masking methods that make it impossible to identify personal information or important information contained in images and documents are largely divided into methods that provide only de-identification functions without considering restoration and methods that enable both de-identification and restoration based on encryption. The masking module 140 of the server 100 and the unmasking module 840 of the client device 800 mentioned in the present invention restore and verify the de-identified image only for authorized users. For this purpose, an encryption-based masking algorithm capable of both de-identification and restoration is used. Therefore, the image protected by masking in the masking module 140 of the server 100 is restored through the unmasking module 840 of the client device 800, and the masking module 140 and the unmasking module 840 are identical. It is assumed that the masking algorithm is used.

6 shows the overall structure of a de-identification image generation and restoration process using original image histogram information according to an embodiment of the present invention.

Referring to FIG. 6, in order to de-identify personal information or important information in an image, the server 100 responsible for the masking function inputs (S900) an image or document file containing text-type personal information and important information. A region requiring identification is set (S901), character histogram information of the set region is extracted (S902), and then masking (S903) is performed.

When the masking process is completed, a QR code including extracted character histogram information and a de-identified image file are generated (S904), and the QR code and de-identified image are delivered or distributed to users online or offline.

The client device 800 first scans the QR code, obtains character histogram information included in the set area in the original image (S905), detects and corrects the unidentified area (S906), and then unmasks the detection area (S907). ) is performed. It is determined whether the character histogram value of the restored image obtained as a result of unmasking is equal to or greater than the threshold value (S908), and if it is greater than or equal to the threshold value, the restored character is displayed on the screen (S909). Returning to step S906), the detection area unmasking (S907) and comparison with the threshold size (S908) are repeated again.

In this case, the threshold value may be set as a ratio of the original image character histogram value. In FIG. 6, as an example, a value obtained by multiplying the original image character histogram value by 0.7 is set as the threshold value.

At this time, as shown in the example of FIG. 6, the threshold value may be set to one, such as the ratio of the total value of the character histogram of the original image, but may be subdivided by character area of the original image and set to several threshold values.

7 is a diagram illustrating a process of analyzing a character histogram of an original image, extracting parameters, and storing a QR code.

Referring to FIG. 7, when a character area to be de-identified is set in the original image, an image inversion is performed on the set area, a horizontal histogram is generated, and the horizontal histogram graph is analyzed to begin with a value other than 0. The index and the length at which non-zero values are maintained are set as the first start index and first length parameters, respectively. Similarly, set the index at which the second non-zero value starts and the length at which the non-zero value is maintained as the second start index and second length parameters, respectively, and set the total value of non-zero values in the histogram in the horizontal direction as the total histogram value parameter. set as a variable When the histogram information analysis is completed, all extracted parameters are finally saved in the QR code.

8 is a diagram conceptually illustrating a process of analyzing and restoring a character histogram of a restored image.

9 is a diagram exemplarily illustrating a threshold value comparison process for increasing character restoration accuracy.

When restoring the non-identification area input through the camera of the client device 800, it is not possible to obtain clear character information like the original image due to various environmental variables, so the original image stored in the QR code as shown in FIGS. 8 and 9 After detecting the non-identification region based on the histogram information of , a stochastic threshold value based on the total histogram value of the original image is calculated for each of the true histogram value and false histogram value among the values displayed on the horizontal histogram graph of the unmasked reconstructed image. It is set, and the restoration text is displayed on the screen of the client device 800 only when all the set conditions are satisfied.

In this case, a histogram value that exists only in the restored image may exist in the histogram graph in the horizontal direction of the reconstructed image due to noise.

True Histogram Value and False Histogram Value are calculated using the first start index, first length, second start index, and second length values extracted from the character histogram graph of the original image.

Since the True Histogram Value of the restored image is generally smaller than the total histogram value of the original image, the True Histogram Value of the restored image is as close as possible to the total histogram value of the original image, and the False Histogram Value of the restored image is By displaying the reconstructed image on the screen at this small measured moment, it is possible to provide the reconstructed image with improved visibility to the user.

Th_p and Th_f values in FIG. 9 are threshold probability parameters used to set the threshold values to be used when determining the size of True Histogram Value and False Histogram Value. It is a configured value that can be used by adjusting it appropriately.

10 is a diagram showing the configuration of a computer system according to an embodiment.

An image compression device for machine vision according to an embodiment may be implemented in the computer system 1000 such as a computer-readable recording medium.

Computer system 1000 may include one or more processors 1010, memory 1030, user interface input devices 1040, user interface output devices 1050, and storage 1060 that communicate with each other over a bus 1020. can In addition, computer system 1000 may further include a network interface 1070 coupled to network 1080 . The processor 1010 may be a central processing unit or a semiconductor device that executes programs or processing instructions stored in the memory 1030 or the storage 1060 . The memory 1030 and the storage 1060 may be storage media including at least one of volatile media, nonvolatile media, removable media, non-removable media, communication media, and information delivery media. For example, memory 1030 may include ROM 1031 or RAM 1032 .

The specific implementations described herein are examples and do not limit the scope of the present invention in any way. For brevity of the specification, description of conventional electronic components, control systems, software, and other functional aspects of the systems may be omitted. In addition, the connection of lines or connecting members between the components shown in the drawings are examples of functional connections and / or physical or circuit connections, which can be replaced in actual devices or additional various functional connections, physical connection, or circuit connections. In addition, if there is no specific reference such as “essential” or “important”, it may not be a component necessarily required for the application of the present invention.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments and should not be determined, and all scopes equivalent to or equivalently changed from the claims as well as the claims to be described later are within the scope of the spirit of the present invention. will be said to belong to

100: image de-identification server
800: client terminal
1000: computer system 1010: processor
1020: bus 1030: memory
1031: Rom 1032: RAM
1040: user interface input device
1050: user interface output device
1060: storage 1070: network interface
1080: Network

Claims (1)

Receiving a QR code including an input image in which a non-identification area is masked and histogram information;
obtaining the histogram information based on the QR code;
detecting a masked region in the input image; and
performing unmasking on the masked area using the histogram information;
Histogram information-based de-identification image restoration method comprising a.

Images