JP6933356B2 - Image processing methods, devices, and systems - Google Patents

Description

The present invention relates to a technique for invalidating a biometric information image used for biometric authentication included in an image captured by an imaging device.

In recent years, many technologies that use biometric authentication have been developed when performing authentication processing in computers and network systems.
For example, it is known to utilize fingerprints, DNA, palm shapes, retinas, irises, faces, blood vessels, ear shapes, and the like as those that utilize the physical characteristics of the user.
In addition, as a behavioral feature, research is also being conducted on a method that utilizes habits such as changes in handwriting (trajectory, speed, and pressure during writing. An authentication method that estimates the rotation of the wrist and the length of the finger. ), Keystroke authentication (method using keyboard keystroke speed and timing habits), lip movement (method using lip movement habits during speech), blinking (method of measuring the amount of change in the black eye area due to blinking) , Walking (authentication method using human walking), etc. have been proposed.

Most of the information related to such biometric authentication (hereinafter, biometric information) can be extracted from still images and moving images (hereinafter, images), and the extraction technique thereof is well known. For example, Patent Document 1 discloses a method for extracting the characteristics of a person.
Biological information has a history of being used because it is difficult to steal, but in the past, many focused on handling after acquisition. Patent Documents 2 to 5 are techniques disclosed that image extraction and subsequent deletion are performed.

On the other hand, recently, it has been regarded as a problem that a malicious third party extracts biometric information from an image taken carelessly and performs an act such as spoofing.

JP 2016-51981 JP 2015-158921 JP 2013-105424 JP 2011-048523 JP-A-2010-140499

The present invention is intended to solve conventional problems, and an object of the present invention is to prevent unintentional leakage of biometric information and to improve the security of biometric authentication.

The following means are provided to solve the above problems.
According to the first embodiment of the present invention, it is an image processing method of a computer that invalidates biometric information from an image containing biometric information used for biometric authentication, and an image input unit inputs target image data. The image input step, the biometric information recognition processing unit recognizes whether or not the biometric information is included from the image, and the biometric information invalidation processing unit includes at least the biometric information in the image. The feature is that the biometric information invalidation step of performing predetermined image processing on the image area to change the biometric information into invalidated image data that cannot be recognized, and the image output unit include at least an image output step of outputting the invalidated image data. An image processing method to be performed is provided.

According to the second embodiment, in the image processing method, the biometric information is at least one of an image relating to a fingerprint, a palm print, a palm shape, a retina, an iris, a face, a blood vessel, and an ear shape. The image processing method described is provided.

According to the third embodiment, the image processing provides an image processing method in which at least one of pixelization processing and color reduction processing is performed.

According to the fourth embodiment, in the image processing method, when at least the biometric information is included after the biometric information recognition step, the recognition result output unit recognizes whether or not the biometric information is included. The recognition result output step for outputting the result, the invalidation instruction receiving unit has an invalidation instruction acceptance step for receiving an instruction as to whether or not to invalidate the biometric information from the user, and the biometric information invalidation step is performed according to the instruction result. An image processing method for execution processing is provided.

According to a fifth embodiment of the present invention, an image processing device of a computer that invalidates biometric information from an image containing biometric information used for biometric authentication, and an image input unit for inputting target image data. , The biometric information recognition processing unit that recognizes whether or not biometric information is included from the image, and the image area that contains at least biometric information in the image is subjected to predetermined image processing, and the biometric information cannot be recognized. Provided is an image processing apparatus including at least a biological information invalidation processing unit for changing into image data and an image output unit for outputting invalidated image data.

According to the sixth embodiment of the present invention, it is an image processing method in a network system that invalidates biometric information from an image containing biometric information used for biometric authentication, and the network system communicates with a plurality of terminal devices. It is composed of a server device connected by a network, the image receiving unit of the server device includes an image receiving step of receiving the target image data from the terminal device, and the biometric information recognition processing unit of the server device includes biometric information from the image. The biometric information recognition step that recognizes whether or not the image has been created, the biometric information invalidation processing unit of the server device performs predetermined image processing on the image area containing at least the biometric information in the image, and the biometric information cannot be recognized. The biometric information invalidation step of changing to invalidated image data, the image output unit of the server device includes at least an image output step of storing the invalidated image data in the server device or transmitting it to the terminal device. An image processing method is provided.

According to the seventh embodiment, in the image processing method, if at least the biometric information is included after the biometric information recognition step, whether or not the result transmission unit of the server device includes the biometric information. In the recognition result transmission step of transmitting the recognition result to the terminal device, the invalidation instruction receiving unit of the server device receives the user's instruction as to whether or not to invalidate the biometric information from the terminal device. Provided is an image processing method having a step and executing a biometric information invalidation step according to an instruction result.

According to the present invention, it is possible to prevent unintentional leakage of biometric information and improve the security of biometric authentication. In particular, it is difficult to recognize whether or not the biometric information is included in the image, but according to the present invention, the biometric information is automatically invalidated even if the user captures the image without paying special attention. Therefore, it is possible to improve the usability and the security of biometric authentication at the same time.

Furthermore, if the present invention is implemented in a server device that provides an SNS or the like, it is possible to prevent unintended exposure of biometric information to the network, improve the security of the network, and contribute to the improvement of the added value of the service. do.

It is a block diagram of the image processing apparatus which concerns on this invention. It is a flowchart of the image processing method which concerns on this invention. It is a block diagram of the image processing system which concerns on this invention. It is a flowchart of the image processing method which concerns on this invention.

Hereinafter, the image processing method, apparatus, and system according to the present invention will be described. The present invention is carried out using a computer, and the computer may be any computer such as a known personal computer, a network server, a terminal device, a smartphone, a mobile phone, a game machine, or a digital camera having an image processing function. Can be used.
Further, the present invention is not limited to the following embodiment, and can be arbitrarily changed within the scope of the claims.

(Example 1)
The present invention provides a computer image processing method for invalidating biometric information from an image containing biometric information used for biometric authentication. FIG. 1 shows an image processing device 1 according to the present invention, and FIG. 2 shows an image processing method according to the present invention.
The image processing device 1 includes a computer CPU 10, which includes an image input unit 101, a biometric information recognition processing unit 102, a biometric information invalidation processing unit 105, and an image output unit 106. In addition, a recognition result output unit 103 and an invalidation instruction receiving unit 104 can be selectively provided. Each of these processes is realized by executing a program stored in a storage medium such as a memory or an external storage device (not shown).

Along with the CPU 10, a network interface 11 for network communication such as the Internet, a monitor 12, a card slot 13 for reading and writing a memory card for storing image data, and an image acquisition unit 14 such as an image pickup means and a sensor means can be provided. All of these are selectively provided.

As shown in FIG. 2, the image input unit 101 first performs the image input step S1 for inputting the target image data. As a method of inputting image data, an image acquisition unit 14 such as an image sensor provided in the apparatus 1 itself, an external digital camera, a fingerprint sensor, a palm print sensor, a retinal sensor, an iris sensor, an arbitrary sensor such as a blood vessel sensor, etc. The image data is input from the image data, or the image data stored in the memory card is read out using the card slot 13.
Further, the image input unit 101 may input the image data by acquiring the image data through the network interface 11.

As the input image, fingerprint, palm print, palm shape, retina, iris, face, blood vessel, ear shape, handwriting, lip movement, blinking, walking state, etc. can be used. The image includes a still image and a moving image, and any one frame may be used in the moving image, or a plurality of transitioning images may be used.

When the image is input, the biological information recognition processing unit 102 performs the biological information recognition step S2 for recognizing whether or not the biological information is included from the image.
The method of recognizing the biometric information input in the image is known and detailed description thereof will be omitted, but a known method can be applied as a biometric authentication technique. It should be noted that the present invention does not perform authentication itself, but aims to invalidate the biometric information required for authentication, and therefore does not necessarily require the same high-precision processing as authentication.

For example, in the case of fingerprint / palm print authentication, in general fingerprint / palm print technology, feature points included in a pattern are extracted, or the direction and change of the pattern are focused on, but in the present invention, these are extracted. It suffices if it is possible to obtain whether the extractable information is included in the image. A technique for extracting whether or not biological information is contained in this way is also known, but for example, a sample image (fingerprint image, palm print image, etc.) of biological information to be invalidated is stored in advance and is constant with these. The biometric information recognition process can be performed by recognizing whether or not an image having similarities is included. Various methods are well known for measuring the similarity of images.

In addition, when palm shapes, retinas, irises, faces, blood vessels, ear shapes, etc. are used as biological information, it is extracted whether or not these shapes are included in the image. For example, in the case of the retina, iris, blood vessel, etc., image processing may be performed in advance at the time of biometric authentication, and then collation processing may be performed. It is also possible to recognize and process whether or not the information is included.

Also, when targeting handwriting, lip movement, blinking, walking, etc.
Using a plurality of images in a moving image, it is recognized whether or not these used information can be acquired.

When the input image data includes an image similar to a fingerprint, palm print, etc., this area in the image is called a biometric information image area, and the biometric information invalidation processing unit 105 sets the biometric information invalidation step. Execute S5. In the biometric information invalidation step S5, at least the biometric information image region in the image is subjected to predetermined image processing to change the biometric information into invalidated image data that cannot be recognized. The target of image processing may be not only the biometric information image region but also a region near the biometric information image region or the entire image data.

As image processing, pixelization processing generally called mosaic processing and color reduction processing for reducing the number of colors or reducing shades can be performed. According to the pixelation process, the resolution is reduced to reduce the distinctiveness, which makes biometric authentication impossible. Further, even in the color reduction process, biometric authentication becomes impossible because the color and darkness between the biometric information and other parts are reduced or eliminated.

The image data targeted by the present invention is not intended by the user to capture a fingerprint or the like in the first place, but is a detail that is unexpectedly reflected due to the high performance of the image sensor. Therefore, the pixelization process as described above is performed. Even if the biometric information image area is changed by color reduction processing, there is no concern, and there is an advantage that an image without a sense of discomfort can be provided.

The image processing method can be appropriately carried out based on the recognized biometric information, but it is also preferable to perform deformation in addition to pixelation and color reduction processing. That is, when fingerprints, palm prints, palm shapes, faces, blood vessels, ear shapes, handwritings, lip movements, blinks, walking states, etc. are used, it is difficult to perform biometric authentication by transforming the image into a predetermined shape. The deformation may be performed only by expanding and contracting in a predetermined direction, or may be a method of changing the position of each pixel according to a predetermined conversion formula. It is also possible to change only the biological information image region or its vicinity.

When targeting moving images such as handwriting, lip movement, blinking, and walking, changing the length of each frame changes the transition timing in the moving image, making it difficult to authenticate. You can also.

In the present invention, any other method can be appropriately adopted for the image processing. The method of image processing is not limited to one, and a plurality of methods can be combined.
The image processing is preferably irreversible, but it may be a reversible process that can be restored by a person having management authority.

Finally, the image output unit 106 performs the image output step S6 to output the invalidated image data. The mode of output is arbitrary, but it may be stored in a storage medium (not shown), the invalidated image may be transmitted to the outside through the network interface 11, or the invalidated image may be displayed from the monitor 12. Then, it may be saved in the memory card by the card slot 13.

As another embodiment, the method can also allow the user to intervene in whether or not to invalidate.
Therefore, after the biometric information recognition step S2, when at least the biometric information is included, the recognition result output unit 103 outputs the recognition result as to whether or not the biometric information is included in the recognition result output step S3. The invalidation instruction receiving unit 104 has an invalidation instruction acceptance step S4 for receiving an instruction from the user as to whether or not to invalidate the biometric information, and can execute and process the biometric information invalidation step according to the instruction result.

The recognition result output unit 103 outputs the recognition result when it is recognized that the biometric information is included as a result of the recognition by the biometric information recognition processing unit 102, or when it is determined that the biometric information is likely to be included. be able to. The latter possibility can be determined, for example, when the similarity of images is higher than a predetermined threshold value or when the accuracy is high by a known image recognition technique.
As a method of outputting the recognition result, it may be notified to the outside through the network interface 11 or may be displayed from the monitor 12.

The invalidation instruction receiving unit 104 receives an external instruction through the network interface 11 or an invalidation instruction from the user through an input means such as a keyboard, a mouse, or a touch panel (not shown), and receives, for example, an instruction requesting invalidation. If there is, it is possible to notify the biological information invalidation processing unit 105 to perform invalidation processing.

In the present invention, even if it is not recognized that the biometric information is included, it is possible to notify the user whether or not the biometric information is really included as well as determining that the biometric information is not included. In that case, even if the biometric information recognition processing unit 102 determines that the biometric information is not included, the biometric information invalidation processing unit 105 can perform the invalidation processing on the image data.

(Example 2)
According to the second embodiment of the present invention, it is possible to provide an image processing device 1 of a computer that invalidates biometric information from an image including biometric information used for biometric authentication.
The apparatus 1 includes an image input unit 101 for inputting target image data, a biometric information recognition processing unit 102 for recognizing whether or not biometric information is included from the image, and at least biometric information in the image. It is possible to include at least a biological information invalidation processing unit 105 that performs predetermined image processing on the image area and changes the biological information into invalidated image data that cannot be recognized, and an image output unit 106 that outputs invalidated image data. ..

The apparatus 1 can have the same configuration as the image processing apparatus 1 shown in the first embodiment, and may include, for example, a recognition result output unit 103 and an invalidation instruction receiving unit 104.
The same applies to the other examples.

(Example 3)
According to Example 3 of the present invention, it is possible to provide an image processing method in a network system that invalidates biometric information from an image containing biometric information used for biometric authentication. FIG. 3 shows a configuration diagram of a network system, and FIG. 4 shows a flowchart of processing.

As shown in the figure, the network system is composed of a plurality of terminal devices 30 and a server device 20 connected by a communication network 40, and an image receiving unit 201 of the server device receives target image data from the terminal device 30. Image receiving step S11, biometric information recognition processing unit 202 of the server device recognizes whether or not biometric information is included from the image, biometric information recognition step S12, biometric information invalidation processing unit 205 of the server device is an image. In the biometric information invalidation step S15, in which the image area including at least the biometric information is subjected to predetermined image processing to change the biometric information into invalidated image data that cannot be recognized, the image output unit 206 of the server device invalidates the image. Provided is an image processing method comprising at least an image output step S16 for storing image data in the server device 20 or transmitting the image data to the terminal device 30.

Further, in the above image processing method, whether or not the recognition result transmission unit 203 of the server device 20 includes the biometric information when at least the biometric information is included after the biometric information recognition step S12. In the recognition result transmission step S13 of transmitting the recognition result to the terminal device 30, the invalidation instruction receiving unit 204 of the server device 20 receives the user's instruction as to whether or not to invalidate the biometric information from the terminal device 30. It is also possible to have the conversion instruction receiving step S14 and execute the biometric information invalidation step according to the instruction result.

The third embodiment is the same as the respective processes in the first and second embodiments, and detailed description thereof will be omitted. However, particularly in the server device, the biological information recognition processing unit 102, the recognition result output unit 103, and the invalidity in the image processing device 1. The same processing as that of the conversion instruction receiving unit 104 and the biological information invalidation processing unit 105 can be performed.

By implementing the server device 20 according to this embodiment on the server of a social network system, for example, it is possible to invalidate the biometric information from the image uploaded by the registered user and improve the safety and convenience of the service at the same time. It also contributes to increasing the added value of services.

1 Image processing device 10 CPU
11 Network interface 12 Monitor 13 Card slot 14 Image acquisition unit 101 Image input unit 102 Biometric information recognition processing unit 103 Recognition result output unit 104 Invalidation instruction receiving unit 105 Biometric information invalidation processing unit 106 Image output unit

Claims (6)

It is an image processing method of a computer that invalidates biometric information from an image containing biometric information used for biometric authentication.
The biological information is an image relating to a fingerprint, a palm print, a palm shape, a retina, a face, a blood vessel, an ear shape, and a handwriting.
Image input step where the image input unit inputs the target image data,
A biometric information recognition step in which the biometric information recognition processing unit recognizes whether or not biometric information is included from the image.
A recognition result output step, which outputs a recognition result as to whether or not the recognition result output unit includes biometric information, at least when the recognition result output unit contains biometric information.
The invalidation instruction acceptance step in which the invalidation instruction receiving unit receives an instruction from the user as to whether or not to invalidate the biometric information.
Biometric information invalidation processing unit when receiving an instruction to invalidate is performed at least one of the image processing of at least biological information about the image area that contains the pixel processing or subtractive color processing in the image, The biometric information invalidation step of changing the biometric information into unrecognizable invalidated image data,
An image processing method, wherein the image output unit includes at least an image output step of outputting the invalidated image data. The biological information is a moving image of a lip movement, blinking, and walking instead of the image.
The image processing method according to claim 1. A computer image processing device that invalidates biometric information from images containing biometric information used for biometric authentication.
The biological information is an image relating to a fingerprint, a palm print, a palm shape, a retina, a face, a blood vessel, an ear shape, and a handwriting.
An image input unit for inputting target image data and
A biometric information recognition processing unit that recognizes whether or not biometric information is included from the image, and
When at least biometric information is included, the recognition result output unit outputs the recognition result of whether or not the biometric information is included, and the recognition result output unit.
The invalidation instruction receiving unit that receives instructions from the user as to whether or not to invalidate the biological information,
Perform at least one of the image processing of at least biological information about the image area that contains the pixel processing or subtractive color processing in the image when receiving an instruction to invalidate, invalidation biological information can not be recognized Biometric information invalidation processing unit that changes to image data,
An image processing apparatus including at least an image output unit that outputs the invalidated image data. The biological information is a moving image of a lip movement, blinking, and walking instead of the image.
The image processing apparatus according to claim 3. An image processing method in a network system that invalidates biometric information from an image containing biometric information used for biometric authentication.
The biological information is an image relating to a fingerprint, a palm print, a palm shape, a retina, a face, a blood vessel, an ear shape, and a handwriting.
The network system is composed of a plurality of terminal devices and a server device connected by a communication network.
An image receiving step in which the image receiving unit of the server device receives the target image data from the terminal device.
A biometric information recognition step in which the biometric information recognition processing unit of the server device recognizes whether or not biometric information is included from the image.
A recognition result transmission step of transmitting a recognition result of whether or not the result transmission unit of the server device includes biometric information to the terminal device when at least biometric information is included.
An invalidation instruction receiving step in which the invalidation instruction receiving unit of the server device receives a user's instruction as to whether or not to invalidate the biometric information from the terminal device.

Deactivate biological information invalidation processing unit of the server apparatus when receiving the instruction, at least one of an image of at least a biological information about the image area that contains the pixel processing or subtractive color processing in the image Biometric information invalidation step, which performs processing and changes the biometric information into unrecognizable invalidated image data.
An image processing method, wherein the image output unit of the server device includes at least an image output step of storing the invalidated image data in the server device or transmitting the invalidated image data to the terminal device. The biological information is a moving image of a lip movement, blinking, and walking instead of the image.
The image processing method according to claim 5.

Images