KR102576826B1 - Electronic terminal apparatus which is able to identify personal information included in the image object, which is to be inserted into an electronic document, and the operating method thereof - Google Patents

Abstract

The present invention provides an electronic terminal device that can identify personal information contained in an image object inserted into an electronic document and an operating method thereof, thereby preventing personal information contained in an image object inserted in an electronic document from being leaked. there is.

Description

Electronic terminal device capable of identifying personal information included in an image object inserted into an electronic document and a method of operating the same , AND THE OPERATING METHOD THEREOF}

The present invention relates to an electronic terminal device capable of identifying personal information contained in an image object inserted into an electronic document and a method of operating the same.

Recently, as electronic terminal devices such as computers, smartphones, and tablet PCs have become widely available, various types of electronic document-related programs that allow users to view, create, and edit electronic documents using these electronic terminal devices have been released. .

These electronic document-related programs include word processors that support the creation and editing of basic documents, spreadsheets that assist in data input, arithmetic operations, and data management, and presentation programs that assist presenters in their presentations.

Users who create electronic documents using these electronic document-related programs may insert image objects into the electronic document using the image insertion function supported by the electronic document-related programs in order to make the contents of the electronic document more specific. many.

On the other hand, image objects inserted into electronic documents may contain personal information such as resident registration numbers or phone numbers. If such personal information is indiscriminately leaked to others, it may be misused for crimes, so make sure that personal information is not leaked. Be careful.

In this regard, previously, in order to prevent the leakage of personal information contained in image objects inserted into electronic documents, users had to directly identify personal information contained in image objects and individually retrieve personal information contained in image objects. After masking, the image object was often inserted into an electronic document.

However, this existing method not only takes a lot of time, but also has the problem that if the user fails to identify the personal information contained in the image object, the personal information contained in the image object can easily be leaked to the outside. .

If a technology capable of identifying personal information included in image objects inserted into electronic documents is introduced, it will be possible to more efficiently prevent personal information included in image objects inserted into electronic documents from being leaked.

Therefore, research is needed on technology that can identify personal information contained in image objects inserted into electronic documents.

The present invention seeks to prevent personal information contained in an image object inserted in an electronic document from being leaked by providing an electronic terminal device capable of identifying personal information contained in an image object inserted in an electronic document and a method of operating the same. .

An electronic terminal device capable of identifying personal information included in an image object inserted into an electronic document according to an embodiment of the present invention uses a plurality of different regular expressions preset to correspond to the notation format of the personal information. When an image insertion command instructing to insert a first image object at a first point in an electronic document is issued by a user to a regular expression storage unit in which they are stored, the first image is generated based on a preset text recognition model. A text extraction unit that performs text recognition on an object to extract the text included in the first image object, and when the text included in the first image object is extracted, the regular expression storage unit among the extracted texts. A string confirmation unit that checks whether a string having a notation format corresponding to the plurality of regular expressions stored in exists, a string having a notation format corresponding to the plurality of regular expressions among the extracted text, If it is confirmed that at least one first string exists, a query message asking the user whether to insert the first image object into the electronic document despite the fact that the first image object contains personal information. When a final image insertion command instructing to insert the first image object into the electronic document is issued by the user in response to the query message display unit and the query message that generates and displays on the screen, the electronic document and an image insertion unit that inserts the first image object into the first point.

In addition, a method of operating an electronic terminal device capable of identifying personal information included in an image object inserted into an electronic document according to an embodiment of the present invention includes a plurality of different regular rules preset to correspond to the notation format of the personal information. Maintaining a regular expression storage unit in which expressions are stored, when an image insertion command instructing to insert a first image object at a first point in the electronic document is issued by a user, based on a preset text recognition model, Extracting text included in the first image object by performing text recognition on the first image object; when the text included in the first image object is extracted, among the extracted text, the regular expression Checking whether a string having a notation format corresponding to the plurality of regular expressions stored in a storage unit exists, a string having a notation format corresponding to the plurality of regular expressions among the extracted text, If it is confirmed that at least one first string exists, a query message asking the user whether to insert the first image object into the electronic document despite the fact that the first image object contains personal information. generating and displaying on the screen and, in response to the query message, when a final image insertion command instructing to insert the first image object into the electronic document is issued by the user, and inserting the first image object at a first point.

The present invention provides an electronic terminal device that can identify personal information contained in an image object inserted into an electronic document and an operating method thereof, thereby preventing personal information contained in an image object inserted in an electronic document from being leaked. there is.

FIG. 1 is a diagram illustrating the structure of an electronic terminal device capable of identifying personal information included in an image object inserted into an electronic document according to an embodiment of the present invention.
2 to 7 are diagrams for explaining the operation of an electronic terminal device capable of identifying personal information included in an image object inserted into an electronic document according to an embodiment of the present invention.
FIG. 8 is a flow chart illustrating a method of operating an electronic terminal device capable of identifying personal information included in an image object inserted into an electronic document according to an embodiment of the present invention.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the attached drawings. This description is not intended to limit the present invention to specific embodiments, but should be understood to include all changes, equivalents, and substitutes included in the spirit and technical scope of the present invention. In describing each drawing, similar reference numerals are used for similar components, and unless otherwise defined, all terms used in this specification, including technical or scientific terms, are within the scope of common knowledge in the technical field to which the present invention pertains. It has the same meaning as generally understood by those who have it.

In this document, when a part "includes" a certain component, this means that it may further include other components rather than excluding other components, unless specifically stated to the contrary. Additionally, in various embodiments of the present invention, each component, functional block, or means may be composed of one or more subcomponents, and the electrical, electronic, and mechanical functions performed by each component may be electronic. It may be implemented with various known elements or mechanical elements such as circuits, integrated circuits, and ASICs (Application Specific Integrated Circuits), and may be implemented separately or by integrating two or more into one.

Meanwhile, the blocks in the attached block diagram or the steps in the flow chart are computer program instructions that are mounted on the processor or memory of equipment capable of data processing, such as general-purpose computers, special-purpose computers, portable laptop computers, and network computers, and perform designated functions. It can be interpreted to mean. Because these computer program instructions can be stored in a memory provided in a computer device or in a computer-readable memory, the functions described in the blocks of a block diagram or the steps of a flow diagram can be produced as a manufactured product containing instruction means to perform them. It could be. In addition, each block or each step may represent a module, segment, or portion of code that includes one or more executable instructions for executing specified logical function(s). Additionally, it should be noted that in some alternative embodiments, it is possible for functions mentioned in blocks or steps to be executed in a different order. For example, two blocks or steps shown in succession may be performed substantially simultaneously or in reverse order, and in some cases, some blocks or steps may be omitted.

FIG. 1 is a diagram illustrating the structure of an electronic terminal device capable of identifying personal information included in an image object inserted into an electronic document according to an embodiment of the present invention.

Referring to FIG. 1, the electronic terminal device 110 according to an embodiment of the present invention includes a regular expression storage unit 111, a text extraction unit 112, a string confirmation unit 113, a query message display unit 114, and Includes an image insertion unit 115.

The regular expression storage unit 111 stores a plurality of different regular expressions that are preset to correspond to the notation format of personal information.

Here, regular expression refers to a formal language used to express a set of strings with specific rules, such as '111-2222-3333' or '123456-4567890'.

For example, the regular expression storage unit 111 may store a plurality of different regular expressions that are preset to correspond to the notation format of personal information as shown in Table 1 below.

Format of personal information regular expression Mobile phone number notation format ^\d{3}-\d{3,4}-\d{4}$ Phone number notation format ^\d{2,3}-\d{3,4}-\d{4}$ Resident registration number notation format ^\d{6}-\d{7}$ ... ...

When an image insertion command instructing to insert a first image object at a first point of an electronic document is applied by the user 130, the text extraction unit 112 generates the first image object based on a preset text recognition model. Text recognition on the image object is performed to extract text included in the first image object. Here, the text recognition module refers to a text recognition module pre-built based on optical character recognition (OCR) technology or deep learning technology that can be obtained by recognizing text in an image or video.

When the text included in the first image object is extracted by the text extraction unit 112, the string confirmation unit 113 selects the plurality of regular expressions stored in the regular expression storage unit 111 from among the extracted text. Check whether a string with a notation format corresponding to the expressions exists.

When the query message display unit 114 confirms that at least one first string exists as a string having a notation format corresponding to the plurality of regular expressions among the extracted texts by the string confirmation unit 113, For the user 130, a query message asking whether to insert the first image object into the electronic document despite the fact that the first image object contains personal information is generated and displayed on the screen.

When a final image insertion command instructing to insert the first image object into the electronic document is applied by the user 130 in response to the query message, the image insertion unit 115 operates on the first image object in the electronic document. Insert the first image object at point 1.

Hereinafter, the operations of the text extraction unit 112, the string confirmation unit 113, the query message display unit 114, and the image insertion unit 115 will be described in detail using examples.

First, let us assume that an image insertion command instructing to insert 'Image Object 1' at the first point of the electronic document has been applied to the electronic terminal device 110 by the user 130.

At this time, let's assume that 'Image Object 1' is the same as the picture indicated by reference numeral 210 in FIG. 2.

Then, the text extraction unit 112 performs text recognition on 'Image Object 1 (210)' based on a preset text recognition model, and converts the text included in 'Image Object 1 (210)' into 'Hong Gil-dong'. You can extract text such as ', contact information: 010-1234-5678'.

Then, the string confirmation unit 113 selects the plurality of regular expressions stored in the regular expression storage unit 111 as shown in Table 1 among the extracted text such as 'Hong Gil-dong, contact information: 010-1234-5678'. You can check whether a string with the corresponding notation format exists.

In this regard, when comparing the extracted text with the plurality of regular expressions with reference to the regular expression storage unit 111 as shown in Table 1, the string indicated by reference numeral 211 among the extracted text is 'mobile phone'. It can be confirmed that it has a notation format corresponding to the ‘number notation format’.

Accordingly, the string confirmation unit 113 can confirm that the string 211 exists among the extracted texts as a string having a notation format corresponding to the plurality of regular expressions.

Then, the query message display unit 114 informs the user 130 whether to insert 'Image Object 1 (210)' into the electronic document even though 'Image Object 1 (210)' contains personal information. You can create a query message and display it on the screen.

At this time, let us assume that in response to the query message, a final image insertion command instructing to insert 'Image Object 1 (210)' into the electronic document is granted by the user 130 to the electronic terminal device 110. .

Then, the image insertion unit 115 can insert 'Image Object 1 210' into the first point of the electronic document.

According to one embodiment of the present invention, the electronic terminal device 110 may further include a dummy image generator 116 and a masking processor 117.

After the first image object is inserted into the first point of the electronic document by the image insertion unit 115, the dummy image generator 116 generates information about the first image object by the user 130. When a masking processing command is applied, a dummy image of the same size as the first image object is created.

At this time, according to an embodiment of the present invention, the dummy image generator 116 is a specific configuration for generating the dummy image, and includes an authentication matrix storage unit 117, a reference image storage unit 118, and a password setting request unit. (119), it may include a vector extraction unit 120, an operation matrix generation unit 121, and a dummy image generation processing unit 122.

The authentication matrix storage unit 117 stores an authentication matrix of size n (n is a natural number greater than or equal to 2) x n, which is preset and composed of components of '1' and '0'.

For example, if n is '3', the authentication matrix storage unit 117 contains ' An authentication matrix of size '3 x 3' such as 'may be stored.

The reference image storage unit 118 stores a first reference image that is pre-designated to correspond to a component of '1' and a second reference image that is pre-designated to correspond to a component of '0'.

For example, let's say that 'reference image 1', which is pre-designated to correspond to the component of '1', is the same as the picture indicated by reference numeral 310 in FIG. 3, and 'reference image 2', which is pre-designated to correspond to the component of '0', is shown in FIG. If it is the same as the picture indicated by reference numeral 320 in 3, 'reference image 1 (310)' and 'reference image 2 (320)' may be stored in the reference image storage unit 118.

After the first image object is inserted into the first point of the electronic document by the image insertion unit 115, the password setting request unit 119 provides information about the first image object by the user 130. When the masking processing command is approved, a password setting request message requesting the user 130 to set a password for demasking is displayed on the screen.

When the first password is set by the user 130 in response to the password setting request message, the vector extractor 120 performs a modulo operation with n as a divisor on the first password, After calculating the first operation value x, from the authentication matrix stored in the authentication matrix storage unit 117, a first column vector located in the x+1th column and a first row vector located in the x+1th row Extract .

Here, the modulo operation refers to an operation that performs division by dividing the dividend by the divisor and calculates the remainder.

When the first column vector and the first row vector are extracted by the vector extractor 120, the operation matrix generator 121 multiplies the first column vector and the first row vector to generate an operation matrix of size n x n. do.

When the operation matrix is generated by the operation matrix generator 121, the dummy image generation processing unit 122 divides an image area of the same size as the first image object into n × n sections to generate n × n divided areas, Among the n x n divided regions, the first reference image is inserted into each divided region located at a point corresponding to the point where the component '1' is located in the authentication matrix, and the component '0' in the authentication matrix is inserted. The dummy image is generated by inserting the second reference image into each divided area located at a point corresponding to this location.

Hereinafter, the operations of the password setting request unit 119, the vector extractor 120, the operation matrix generator 121, and the dummy image generation processor 122 will be described in detail using an example.

First, as in the above-described example, n is assumed to be '3' and 'Image Object 1 (210)' is inserted into the first point of the electronic document by the image insertion unit 115, and thereafter, Let us assume that a masking processing command for 'Image Object 1 (210)' is applied to the electronic terminal device 110 by the user 130.

Then, the password setting request unit 119 may display a password setting request message on the screen requesting the user 130 to set a password for unmasking.

At this time, in response to the password setting request message, when '1230' is set as the first password by the user 130, the vector extractor 120 uses '3' as a divisor for '1230'. By performing the modulo operation, the first operation value can be calculated as '0'.

Then, the vector extractor 120 extracts the authentication matrix '' stored in the authentication matrix storage unit 117. ', the first column vector located in the '1'th column is ' ' and ', the first row vector located in the '1'th row. ' can be extracted.

Then, the operation matrix generator 121 generates the first column vector ' ' and the first row vector ' ' is matrix multiplied to create an operation matrix of size '3 x 3'. It can be created like this:

Then, as shown in FIG. 4, the dummy image generation processing unit 122 divides the image area 410 of the same size as 'Image Object 1 210' into '3 x 3' pieces, so that '9' Two partition areas (411, 412, 413, 414, 415, 416, 417, 418, 419) can be created.

After that, the dummy image generation processing unit 122 selects the '1' component in the operation matrix among the '9' partition areas (411, 412, 413, 414, 415, 416, 417, 418, 419). 'Reference image 1 (310)' is inserted into each segment located at the point corresponding to the point, and 'reference image 1 (310)' is inserted into each segment area located at the point corresponding to the point where the component '0' is located in the authentication matrix. By inserting 'image 2 320', the dummy image can be created.

In relation to this, the operation matrix ' ', the point where the component of '1' is located is 'row 1, column 1, row 1, column 2, row 3, column 1, row 3, column 2', and '9' partition areas (411, 412, 413, 414, 415, 416, 417, 418, 419), the partition areas located at the points corresponding to 'row 1, column 1, row 1, column 2, row 3, column 1, and row 3, column 2' are 'partition areas 411'. , partition area 412, partition area 417, partition area 418', so the dummy image generation processing unit 122 generates partition area 411, partition area 412, partition area 417, partition area ‘Reference Image 1 (310)’ can be inserted into each ‘(418)’.

In addition, the operation matrix ' ', the point where the component of '0' is located is '1st row 3 columns, 2nd row 1 column, 2nd row 2 columns, 2nd row 3 columns, 3rd row 3 columns', and '9' partition areas (411 , 412, 413, 414, 415, 416, 417, 418, 419), located at the point corresponding to ‘row 1, column 3, row 2, column 1, row 2, column 2, row 2, column 3, and row 3, column 3’. Since the partition areas are 'partition area 413, partition area 414, partition area 415, partition area 416, and partition area 419', the dummy image generation processing unit 122 is 'partition area 413'. ), 'reference image 2 (320)' can be inserted into each of the divided areas 414, 415, 416, and 419'.

In this way, the dummy image generation processing unit 122 can generate a dummy image of the same size as 'Image Object 1 210', as shown by reference numeral 510 in FIG. 5.

In this way, when the dummy image is generated by the dummy image generator 116, the masking processor 117 is placed on the upper layer of the first image object inserted at the first point of the electronic document, By overlapping and displaying the dummy images, the first image object is masked.

For example, as in the above-described example, when the dummy image 510 is generated by the dummy image generation processing unit 122, the masking processing unit 117 generates 'image object 1' inserted at the first point of the electronic document. 'Image object 1 (210)' can be masked by displaying the dummy image 510 overlaid on the upper layer of '(210)'.

At this time, according to an embodiment of the present invention, the electronic terminal device 110 includes a password input request unit 123, a vector re-extraction unit 124, an operation matrix regeneration unit 125, a dummy matrix generation unit 126, and It may further include a masking release unit 127.

After the first image object has been masked by the masking processing unit 117, the password input request unit 123 receives a command to unmask the first image object from the user 130, and the user ( 130), a password input request message is displayed on the screen requesting that a password be entered to remove masking.

When the first password is input by the user 130 in response to the password input request message, the vector re-extraction unit 124 performs a modulo operation with n as a divisor on the first password, After calculating the first operation value x, from the authentication matrix stored in the authentication matrix storage unit 117, the first column vector located in the x+1th column and the first column vector located in the x+1th row Re-extract the first row vector.

When the first column vector and the first row vector are re-extracted by the vector re-extraction unit 124, the operation matrix regeneration unit 125 performs the operation of size n x n by multiplying the first column vector and the first row vector by a matrix. Regenerate the matrix.

When the operation matrix is regenerated by the operation matrix generation unit 125, the dummy matrix generator 126 divides the dummy image into n x n, generates n x n dummy division regions, and then creates n x n dummy division regions. Among them, a component of '1' is assigned to each of the dummy division areas in which the first reference image is inserted, and a component of '0' is assigned to each of the dummy division areas in which the second reference image is inserted, Create a dummy matrix of size n x n consisting of '1' and '0' components.

When the dummy matrix is generated by the dummy matrix generator 126, the unmasking unit 127 compares the dummy matrix with the operation matrix, and when it is confirmed that the dummy matrix matches the operation matrix, the first 1 By removing the dummy image displayed overlapping on the upper layer of the image object, the masking applied to the first image object is released.

In the following, the operations of the password input request unit 123, vector re-extraction unit 124, operation matrix generator 125, dummy matrix generator 126, and unmasking unit 127 will be described in detail using examples. I decided to do it.

First, as in the above-described example, n is assumed to be '3' and 'Image Object 1 (210)' is masked by the masking processor 117. After that, the user ( Let us assume that the unmasking command for 'Image Object 1 (210)' is approved by 130).

Then, the password input request unit 123 may display a password input request message on the screen requesting the user 130 to input a password for unmasking.

At this time, when the first password '1230' is input by the user 130 in response to the password input request message, the vector re-extraction unit 124 divides '1230' by '3'. After calculating the first operation value '0' by performing a modulo operation, the authentication matrix stored in the authentication matrix storage unit 117 is '0'. ', the first column vector located in the '1'th column is ' ' and ', the first row vector located in the '1'th row. ' can be re-extracted.

Then, the operation matrix regeneration unit 125 generates the first column vector ' ' and the first row vector ' ' is matrix multiplied to obtain the above operation matrix of size '3 x 3'. ' can be regenerated.

Then, as shown in FIG. 6, the dummy matrix generator 126 divides the dummy image 510 into '3 After creating '514, 515, 516, 517, 518, 519), 'reference image 1( By assigning a component of '1' to each dummy division region in which '310)' is inserted and a component of '0' to each dummy division region in which 'reference image 2 (320)' is inserted, ' A dummy matrix of size '3 x 3' consisting of components of '1' and '0' can be created.

In relation to this, among the '9' dummy segmentation areas (511, 512, 513, 514, 515, 516, 517, 518, 519), the dummy segmentation areas in which 'reference image 1 (310)' is inserted are 'dummy segmentation areas'. region 511, dummy partition region 512, dummy partition region 517, and dummy partition region 518', so the dummy matrix generator 126 has 'dummy partition region 511, dummy partition region 512'. , a component of '1' can be assigned to each of the dummy partition area 517 and the dummy partition area 518.

In addition, among the '9' dummy partition areas (511, 512, 513, 514, 515, 516, 517, 518, 519), the dummy partition areas in which 'reference image 2 (320)' is inserted are 'dummy partition areas'. (513), dummy partition area 514, dummy partition area 515, dummy partition area 516, dummy partition area 519', so the dummy matrix generator 126 has 'dummy partition area 513, A component of '0' may be assigned to each of the dummy partition area 514, dummy partition area 515, dummy partition area 516, and dummy partition area 519.

In this way, the dummy matrix generator 126 creates a dummy matrix of size '3 x 3' composed of components of '1' and '0'. It can be created like this:

Then, the unmasking unit 127 uses the dummy matrix ' ' is the operation matrix ' After confirming that the dummy matrix matches the operation matrix by comparing with ', the dummy image 510 displayed overlapping on the upper layer of 'Image Object 1 (210)' is removed, thereby creating 'Image Object 1 ( The masking applied to '210)' can be canceled.

Additionally, according to one embodiment of the present invention, the electronic terminal device 110 may further include a thumbnail display unit 128.

After the first image object is masked by the masking processing unit 117, the thumbnail display unit 128 displays the pointer of the input device connected to the electronic terminal device 110 by the user 130 by hovering over the dummy image. When hovering, a thumbnail image is generated by lowering the resolution of the first image object by a preset ratio, and the thumbnail image is displayed at a point adjacent to the point where the pointer hovers.

For example, as in the above-described example, it is assumed that 'Image Object 1 (210)' is masked by the masking processor 117, and thereafter, as shown in FIG. 7, the user 130 uses the electronic terminal Assume that the pointer 520 of the input device connected to the device 110 is hovered over the dummy image 510.

At this time, when the preset ratio is '20%', the thumbnail display unit 128 generates a thumbnail image 530 with the resolution of 'Image Object 1 210' reduced to '20%', and the pointer 520 A thumbnail image 530 may be displayed at a point adjacent to the hovered point.

In relation to this, when the resolution of 'Image Object 1 (210)' is '360dpi', the thumbnail display unit 128 sets the resolution of 'Image Object 1 (210)' to '72dpi', which is '20%' of '360dpi'. The image lowered to ' can be created as a thumbnail image (530).

FIG. 8 is a flow chart illustrating a method of operating an electronic terminal device capable of identifying personal information included in an image object inserted into an electronic document according to an embodiment of the present invention.

In step S810, a regular expression storage unit is maintained in which a plurality of different regular expressions preset to correspond to the notation format of personal information are stored.

In step S820, when an image insertion command instructing to insert a first image object at a first point of the electronic document is applied by the user, text for the first image object is generated based on a preset text recognition model. Recognition is performed to extract text included in the first image object.

In step S830, when the text included in the first image object is extracted, a string having a notation format corresponding to the plurality of regular expressions stored in the regular expression storage unit exists among the extracted text. Check whether it is working or not.

In step S840, if it is confirmed that at least one first string exists as a string having a notation format corresponding to the plurality of regular expressions among the extracted text, the first image object is sent to the user. A query message asking whether to insert the first image object into the electronic document despite the fact that it contains personal information is generated and displayed on the screen.

In step S850, when a final image insertion command instructing to insert the first image object into the electronic document is applied by the user in response to the query message, the first image object is placed at the first point of the electronic document. Insert the first image object.

At this time, according to an embodiment of the present invention, the method of operating the electronic terminal device includes inserting the first image object into the first point of the electronic document and then, by the user, the first image object. When a masking processing command is applied, generating a dummy image of the same size as the first image object; and when the dummy image is generated, an upper layer of the first image object inserted at the first point of the electronic document The method may further include masking the first image object by overlapping and displaying the dummy image.

At this time, according to one embodiment of the present invention, in the step of generating the dummy image, an authentication matrix of size n (n is a natural number of 2 or more) x n, which is preset and composed of components of '1' and '0', is stored. maintaining an authentication matrix storage unit, a reference image storage unit storing a first reference image pre-designated to correspond to a component of '1' and a second reference image pre-designated to correspond to a component of '0'; maintaining, after the first image object is inserted at the first point of the electronic document, if a masking processing command for the first image object is applied by the user, unmasking for the user; Displaying a password setting request message on the screen requesting to set a password for, when a first password is set by the user in response to the password setting request message, setting n as a divisor for the first password. After calculating the first operation value x by performing a modulo operation, from the authentication matrix stored in the authentication matrix storage unit, a first column vector located in the x+1th column and Extracting a located first row vector, once the first column vector and the first row vector are extracted, multiplying the first column vector and the first row vector by a matrix to generate an operation matrix of size n x n, and the operation matrix When created, an image area of the same size as the first image object is divided into n The first reference image is inserted into each divided region located at a point corresponding to the point, and the second reference image is inserted into each divided region located at a point corresponding to the point where the component '0' in the operation matrix is located. It may include generating the dummy image by inserting .

At this time, according to one embodiment of the present invention, the method of operating the electronic terminal device is when, after the first image object has been masked, a command to unmask the first image object is applied by the user, Displaying a password input request message on the screen requesting the user to enter a password for unmasking, when the first password is input by the user in response to the password input request message, After calculating the first operation value x by performing a modulo operation with n as a divisor on the first password, from the authentication matrix stored in the authentication matrix storage unit, the Re-extracting the first column vector and the first row vector located in the x+1th row. When the first column vector and the first row vector are re-extracted, the first column vector and the first row vector are Regenerating the operation matrix of n 1 By assigning a component of '1' to each of the dummy division areas in which the reference image is inserted, and assigning a component of '0' to each of the dummy division areas in which the second reference image is inserted, '1' and Generating a dummy matrix of n x n size composed of elements of '0'; and once the dummy matrix is generated, comparing the dummy matrix with the operation matrix, and if it is confirmed that the dummy matrix matches the operation matrix, The method may further include removing the masking applied to the first image object by removing the dummy image displayed overlapping on an upper layer of the first image object.

In addition, according to one embodiment of the present invention, the method of operating the electronic terminal device is such that after the first image object is masked, the pointer of the input device connected to the electronic terminal device is moved by the user to the dummy image. When hovered, the method may further include generating a thumbnail image in which the resolution of the first image object is reduced by a preset ratio and displaying the thumbnail image at a point adjacent to the point where the pointer is hovered.

Above, with reference to FIG. 8, a method of operating an electronic terminal device capable of identifying personal information included in an image object inserted into an electronic document according to an embodiment of the present invention has been described. Here, the operating method of the electronic terminal device capable of identifying personal information included in an image object inserted into an electronic document according to an embodiment of the present invention is the method of operating the electronic terminal device 110 described using FIGS. 1 to 7. Since it may correspond to the configuration of the operation, a more detailed description thereof will be omitted.

A method of operating an electronic terminal device capable of identifying personal information contained in an image object inserted into an electronic document according to an embodiment of the present invention may be implemented as a computer program stored in a storage medium for execution through combination with a computer. You can.

In addition, the method of operating an electronic terminal device capable of identifying personal information included in an image object inserted into an electronic document according to an embodiment of the present invention is implemented in the form of program instructions that can be executed through various computer means. Can be recorded on a readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and constructed for the present invention or may be known and usable by those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

As described above, the present invention has been described with specific details such as specific components and limited embodiments and drawings, but this is only provided to facilitate a more general understanding of the present invention, and the present invention is not limited to the above embodiments. , those skilled in the art can make various modifications and variations from this description.

Accordingly, the spirit of the present invention should not be limited to the described embodiments, and the scope of the patent claims described below as well as all modifications that are equivalent or equivalent to the scope of this patent claim shall fall within the scope of the spirit of the present invention. .

110: Electronic terminal device capable of identifying personal information contained within an image object inserted into an electronic document
111: regular expression storage unit 112: text extraction unit
113: String confirmation unit 114: Query message display unit
115: image insertion unit 116: dummy image creation unit
117: authentication matrix storage unit 118: reference image storage unit
119: Password setting request unit 120: Vector extraction unit
121: Operation matrix generation unit 122: Dummy image generation processing unit
123: Password input request unit 124: Vector re-extraction unit
125: Operation matrix regeneration unit 126: Dummy matrix generation unit
127: Masking release unit 128: Thumbnail display unit
130: user

Claims (12)

In the electronic terminal device capable of identifying personal information contained in an image object inserted into an electronic document,
a regular expression storage unit storing a plurality of different regular expressions preset to correspond to the notation format of personal information;
When an image insertion command instructing to insert a first image object at a first point in an electronic document is applied by a user, text recognition is performed on the first image object based on a preset text recognition model. a text extraction unit that extracts text included in the first image object;
When the text included in the first image object is extracted, it is checked whether a string having a notation format corresponding to the plurality of regular expressions stored in the regular expression storage unit exists among the extracted text. String confirmation unit;
If it is confirmed that at least one first string exists as a string having a notation format corresponding to the plurality of regular expressions among the extracted text, personal information is included in the first image object for the user. a query message display unit that generates a query message inquiring whether to insert the first image object into the electronic document despite the presence of the first image object and displays it on the screen; and
In response to the query message, when a final image insertion command instructing to insert the first image object into the electronic document is issued by the user, the first image object is inserted at the first point of the electronic document. Image insertion section
Electronic terminal device including. According to paragraph 1,
After the first image object is inserted into the first point of the electronic document, when a masking processing command for the first image object is applied by the user, an image object of the same size as the first image object is applied. A dummy image generator that generates a dummy image; and
When the dummy image is generated, masking the first image object by overlapping and displaying the dummy image on an upper layer of the first image object inserted at the first point of the electronic document. masking processor
An electronic terminal device further comprising: According to paragraph 2,
The dummy image generator
An authentication matrix storage unit that stores an authentication matrix of size n (where n is a natural number greater than or equal to 2) x n, which is preset and composed of components of '1' and '0';
a reference image storage unit storing a first reference image pre-designated to correspond to a component of '1' and a second reference image pre-designated to correspond to a component of '0';
After the first image object is inserted into the first point of the electronic document, if a masking processing command for the first image object is applied by the user, a password for unmasking may be set for the user. a password setting request unit that displays a password setting request message on the screen;
In response to the password setting request message, when the first password is set by the user, the first operation value x is calculated by performing a modulo operation with n as a divisor on the first password. , a vector extractor that extracts a first column vector located in the x+1th column and a first row vector located in the x+1th row from the authentication matrix stored in the authentication matrix storage unit;
When the first column vector and the first row vector are extracted, an operation matrix generator generates an operation matrix of nxn size by matrix multiplying the first column vector and the first row vector; and
When the operation matrix is created, an image area of the same size as the first image object is divided into nxn sections to create nxn division areas, and then, among the nxn division areas, the element '1' in the operation matrix The first reference image is inserted into each of the divided areas located at a point corresponding to this location, and the first reference image is inserted into each of the divided areas located at a point corresponding to the point where the component '0' in the operation matrix is located. 2 A dummy image generation processing unit that generates the dummy image by inserting a reference image
Electronic terminal device including. According to paragraph 3,
After the first image object is masked by the masking processing unit, when a command to unmask the first image object is applied by the user, the user is prompted to enter a password for unmasking. a password input request unit that displays a password input request message on the screen;
In response to the password input request message, when the first password is input by the user, the first operation value x is calculated by performing a modulo operation with n as a divisor on the first password. , a vector re-extraction unit that re-extracts the first column vector located in the x+1-th column and the first row vector located in the x+1-th row from the authentication matrix stored in the authentication matrix storage unit;
When the first column vector and the first row vector are re-extracted, an operation matrix regeneration unit that reproduces the operation matrix of nxn size by matrix multiplying the first column vector and the first row vector;
When the operation matrix is regenerated, nxn dummy division regions are generated by dividing the dummy image into nxn pieces, and then, among the nxn dummy division regions, each dummy division region in which the first reference image is inserted is By assigning a component of '1' and assigning a component of '0' to each dummy segment area in which the second reference image is inserted, a dummy matrix of nxn size composed of components of '1' and '0' A dummy matrix generator that generates; and
When the dummy matrix is generated, the dummy matrix is compared with the operation matrix, and if it is confirmed that the dummy matrix matches the operation matrix, the dummy image displayed overlaid on the upper layer of the first image object is displayed. A masking release unit that releases the masking applied to the first image object by removing it.
An electronic terminal device further comprising: According to paragraph 2,
After the first image object is masked by the masking processor, when the pointer of an input device connected to the electronic terminal device is hovered on the dummy image by the user, the first image object A thumbnail display unit that generates a thumbnail image with the resolution reduced to a preset ratio and displays the thumbnail image at a point adjacent to the point where the pointer is hovered.
An electronic terminal device further comprising: In a method of operating an electronic terminal device capable of identifying personal information contained in an image object inserted into an electronic document,
maintaining a regular expression storage unit in which a plurality of different regular expressions preset to correspond to the notation format of personal information are stored;
When an image insertion command instructing to insert a first image object at a first point in an electronic document is applied by a user, text recognition is performed on the first image object based on a preset text recognition model. extracting text included in the first image object;
When the text included in the first image object is extracted, it is checked whether a string having a notation format corresponding to the plurality of regular expressions stored in the regular expression storage unit exists among the extracted text. step;
If it is confirmed that at least one first string exists as a string having a notation format corresponding to the plurality of regular expressions among the extracted text, personal information is included in the first image object for the user. generating a query message inquiring whether to insert the first image object into the electronic document despite the presence of the first image object and displaying it on the screen; and
In response to the query message, when a final image insertion command instructing to insert the first image object into the electronic document is issued by the user, the first image object is inserted at the first point of the electronic document. steps to insert
A method of operating an electronic terminal device comprising: According to clause 6,
After the first image object is inserted into the first point of the electronic document, when a masking processing command for the first image object is applied by the user, an image object of the same size as the first image object is applied. generating a dummy image; and
When the dummy image is generated, masking the first image object by overlapping and displaying the dummy image on an upper layer of the first image object inserted at the first point of the electronic document. step
A method of operating an electronic terminal device further comprising: In clause 7,
The step of generating the dummy image is
maintaining an authentication matrix storage unit in which an authentication matrix of size n (where n is a natural number equal to or greater than 2) x n, which is preset and composed of components of '1' and '0', is stored;
maintaining a reference image storage unit in which a first reference image pre-designated to correspond to a component of '1' and a second reference image pre-designated to correspond to a component of '0' are stored;
After the first image object is inserted into the first point of the electronic document, if a masking processing command for the first image object is applied by the user, a password for unmasking may be set for the user. displaying a password setting request message on the screen;
In response to the password setting request message, when the first password is set by the user, the first operation value x is calculated by performing a modulo operation with n as a divisor on the first password. , extracting a first column vector located in the x+1th column and a first row vector located in the x+1th row from the authentication matrix stored in the authentication matrix storage unit;
When the first column vector and the first row vector are extracted, generating an operation matrix of nxn size by matrix multiplying the first column vector and the first row vector; and
When the operation matrix is created, an image area of the same size as the first image object is divided into nxn sections to create nxn division areas, and then, among the nxn division areas, the element '1' in the operation matrix The first reference image is inserted into each of the divided areas located at a point corresponding to this location, and the first reference image is inserted into each of the divided areas located at a point corresponding to the point where the component '0' in the operation matrix is located. 2. Creating the dummy image by inserting a reference image
A method of operating an electronic terminal device comprising: According to clause 8,
After the first image object is masked, if a command to unmask the first image object is applied by the user, a password input request message requesting the user to enter a password for unmasking displaying on the screen;
In response to the password input request message, when the first password is input by the user, the first operation value x is calculated by performing a modulo operation with n as a divisor on the first password. , re-extracting the first column vector located in the x+1-th column and the first row vector located in the x+1-th row from the authentication matrix stored in the authentication matrix storage unit;
When the first column vector and the first row vector are re-extracted, regenerating the operation matrix of nxn size by matrix multiplying the first column vector and the first row vector;
When the operation matrix is regenerated, nxn dummy division regions are generated by dividing the dummy image into nxn pieces, and then, among the nxn dummy division regions, each dummy division region in which the first reference image is inserted is By assigning a component of '1' and assigning a component of '0' to each dummy segment area in which the second reference image is inserted, a dummy matrix of nxn size composed of components of '1' and '0' generating a; and
When the dummy matrix is generated, the dummy matrix is compared with the operation matrix, and if it is confirmed that the dummy matrix matches the operation matrix, the dummy image displayed overlaid on the upper layer of the first image object is displayed. releasing the masking applied to the first image object by removing it;
A method of operating an electronic terminal device further comprising: According to clause 8,
After the first image object is masked, when the pointer of an input device connected to the electronic terminal device is hovered on the dummy image by the user, the resolution of the first image object is adjusted to a preset ratio. Creating a lowered thumbnail image and displaying the thumbnail image at a point adjacent to the point where the pointer is hovered.
A method of operating an electronic terminal device further comprising: A computer-readable recording medium recording a computer program for executing the method of any one of claims 6 to 10 through combination with a computer. A computer program stored in a storage medium for executing the method of any one of claims 6 to 10 through combination with a computer.

Images