WO2015020709A2

WO2015020709A2 - Visual recognition system based on visually distorted image data

Info

Publication number: WO2015020709A2
Application number: PCT/US2014/037462
Authority: WO
Inventors: Nicholas P. De Luca
Original assignee: Cryovac, Inc.
Priority date: 2013-05-09
Filing date: 2014-05-09
Publication date: 2015-02-12
Also published as: WO2015020709A3

Abstract

Visual recognition system includes a video camera (e.g., CMOS or CCD array), a physical visual distortion system between the camera and incoming light, a mathematical characterization of the physical distortion device, a facial recognition or other biometric recognition algorithm, and a coupling of the analyzed images to an individual's identity. The distortion system provides a singular or paired effect of occluding, blurring, segmenting, multiplying, color filtering, lensing, mirroring, or other distortive effect that render the resulting distorted image unrecognizable to an observer. The identity of the individual from which the distorted image is generated is thereafter determined by electronic comparison of the distorted image against a catalog of stored distorted images obtained from identified individuals.

Description

Visual Recognition System Based on Visually Distorted Image Data

Field

[0001] The present invention relates to the field of visual recognition systems and processes, particularly as related to human identity.

Background

[0002] Video surveillance is a standard practice for the purpose of monitoring human actions and behaviors. Cameras are now ubiquitous within public as well as private areas and are monitored both automatically as well as by human beings. Cameras are manufactured by a large number of companies, including Sony, Toshiba, Trendnet, Cisco, Logitech, and Uniden.

[0003] Light entering these cameras first passes through a lens that focuses an image on a CMOS (complementary metal-oxide semiconductor) or CCD (charge- coupled device) sensor. Due to its architecture, a CCD sensor generally produces a higher quality image compared to a CMOS sensor, but both are commonly used. The image can be broken up into its respective color components and the intensity of the light at each pixel measured. A video signal is then created and transferred to a computer for processing.

[0004] In the case of using a 3-dimensional camera (3-D camera), depth

information is created using multiple image sensors. In some cases, infrared versus visible light is measured. For example, Microsoft Corporation sells a camera called Kinect that includes a depth sensor camera based on using an infrared sensor.

[0005] Once transferred to the computer, the image can be analyzed to detect a face.

[0006] Algorithms exist for the purpose of facial recognition which can be generally categorized into two classes: those that are image-based and those that are video- based. Image-based algorithms include Principal Component Analysis, Independent Component Analysis, Linear Discriminant Analysis, Evolutionary Pursuit Analysis, Elastic Bunch Graph Matching, Kernel Methods, Trace Transform, Radon Transform, Active Appearance Model, 3-D Morphable Model, 3-D Canonical Surface Data, Bayesian Framework, Support Vector Machine, Hidden Markova Models, Boosting and Ensemble Solutions. Video-based solutions may also combine one or more of the above-identified image-based algorithms to process a video sequence. The above-identified image-based algorithms require a focused image for processing. The image analyzed and processed can be easily discerned by a human being from the image captured by the camera. In some cases the image, once captured by the camera's sensor, may be encrypted prior to or after transmission to the computer.

[0007] Use of video cameras and video analytics within commercial and private settings is becoming more and more common. Images captured may be processed by humans or computers on site or at remote locations. The images transmitted may be encrypted prior to transmission but must then be de-encrypted prior to analysis. In many cases, the images are stored electronically (such as on a memory card or on magnetic tape) and accessed over time.

[0008] Sealed Air Corporation offers a system called VSS^® (Vision Safety

Solutions, powered by VTID™) that captures images of individuals in the process of washing their hands. The images of hand washing or sterilizing events are generally processed on a local computer. The identification of the individual performing the hand wash is determined using a radio frequency tag or antenna that is generally attached to a badge that is worn by the health care provider. However, it would be beneficial to avoid the use of radio frequency tags for identifying individuals, due to the inherent cost of the reader system, the complexity of mounting antennas, and the difficulty in obtaining a clear signal from a badge due to various badge positions near a body.

[0009] One alternative to using a radio frequency tag is the use of facial recognition to identify a person. In the case of the Vision Safety Solutions system described above, a camera could be placed at the monitored sink or hand sanitizing station and the image acquired compared to a database of facial images. The primary objection to using such a system relates to maintaining privacy laws and the risk that a breach of security would allow the link of biometric data to an individual. Summary of the Invention

[0010] A first aspect pertains to a facial recognition system comprising an optical image distorter, a camera, a processing algorithm, a data storage system, and a computer.

[0011] In an embodiment, the optical image distorter occludes, blurs, filters, multiplies, segments, mirrors, lenses, or otherwise distorts the image.

[0012] In an embodiment, the light entering the camera has passed through a lens just prior to entering the camera.

[0013] In an embodiment, the light entering the camera has bounced off a mirrored surface just prior to entering the camera. In an embodiment, the mirrored surface is not flat.

[0014] In an embodiment, only partial segments of the image enter the camera.

[0015] In an embodiment, one or more parts of the original image are sent to more than one camera.

[0016] in an embodiment, the light entering the camera has passed through one or more lenses to create a blurred image just prior to entering the camera.

[0017] In an embodiment, the light entering the camera passes through a diffraction grating just prior to entering the camera.

[0018] In an embodiment, the light entering the camera passes through a light diffuser just prior to entering the camera.

[0019] In an embodiment, the light entering the camera has passed through a multiple prism beam expander so as to distort the image just prior to entering the camera.

[0020] In an embodiment, the light entering the camera has passed through an achromatic lens so as to distort the image just prior to entering the camera.

[0021] in an embodiment, the light entering the camera has passed through color filters so as to distort the image just prior to entering the camera.

[0022] A second aspect of the invention is directed to a hand wash or hand sanitation compliance system wherein the association of an individual to the hand wash or hand sanitation compliance monitoring is done by using a facial recognition system that relies on a distorted image entering the camera. [0023] In an embodiment, the light entering the camera passes through one or more of the image distorters described herein.

[0024] A third aspect is directed to a facial recognition process, comprising: (A) acquiring an image of a face of an unidentified individual, the image being formed from light reflected from the face of the unidentified individual; (B) distorting the image of the face of the unidentified individual, by passing the reflected light through a distortion device which carries out an image distortion process to produce a distorted image corresponding with the face of the unidentified individual; (C) impinging the distorted image of the unidentified individual on a red green blue color filter array or a charge coupled device array, to form a digitized distorted image signal or an analog distorted image signal; (D) transmitting the digitized or analog distorted image signal to a computer for data storage and data processing, the computer having stored therein a plurality of digitized or analog distorted facial images of corresponding with the images of faces of a plurality of identified individuals; and (E) processing the digitized or analog distorted facial image of the unidentified individual in a manner so that the distorted facial image of the unidentified individual is compared with the stored plurality of digitized or analog distorted images of identified individuals, with the comparing being carried out in order to determine if the distorted image of the unidentified individual corresponds with a distorted image of an identified individual stored in the computer.

[0025] in an embodiment, the distorted image of the unidentified individual is encrypted, and the distorted images of the identified individuals are also encrypted.

[0026] In an embodiment, the distorted image of the unidentified individual is processed by at least one algorithm selected from the group consisting of Principal Component Analysis, Independent Component Analysis, Linear Discriminant Analysis, Evolutionary Pursuit Analysis, Elastic Bunch Graph Matching, Kernel Methods, Trace Transform, Radon Transform, Active Appearance Model, 3-D Morphable Model, 3-D Canonical Surface Data, Bayesian Framework, Support Vector Machine, Hidden Markova Models, and Boosting and Ensemble Solutions.

[0027] In an embodiment, the image distortion comprises at least one member selected from the group consisting of mirroring, occlusion, blurring, segmentation, color filtering, and shape distorting. [0028] In an embodiment, the distorted image is not recognizable when

unencrypted, the distorted image contains key attributes which are used to match the distorted image of the unidentified individual to a database of equally distorted images of identified individuals, so that a match can be made, resulting in the identification of the unidentified individual.

[0029] In an embodiment, the facial recognition process is used in a hand wash compliance system including a sink, a hand sanitizer and a camera acquiring a distorted facial image of an unidentified individual.

[0030] in an embodiment, the process is used in a hand wash compliance system including a sink, a hand sanitizer and a camera acquiring a visual identification of a tag or other transmission signal.

[0031] In an embodiment, the visual recognition system comprises a standard video camera (generally a CMOS or CCD array), a physical visual distortion system placed between the camera and incoming light, a mathematical characterization of the distortion system in the form of an algorithm, and a facial recognition or other biometric recognition system. The distortion system provides a singular or paired effect of occlusion, blurring, segmentation, multiplicity, color filtering, Sensing, mirroring, or other distortive effect that makes the camera output of human facial features or other biometric data impossible to recognize by a human being. The distortive effect can be further characterized mathematically in part or in whole, and this characterization can be used partially or in whole to allow for faciai recognition or other biometric recognition by a computer system. The system can be coupled to a database to correlate the analyzed images to the identification of people or physical objects.

[0032] In an embodiment, the system is coupled with a monitoring system. In an embodiment, the monitoring system includes a hand wash compliance system and/or a hand sanitizing compliance system.

[0033] In an embodiment, a recognition system is used to identify an individual within an environment that is sensitive to privacy, in an embodiment, the recognition system is used to identify the face of an individual within the privacy-sensitive environment. [0034] In an embodiment, light impinging the surface of the sensor is used to produce a distorted image of at least a portion of an individual, with the distorted image being unidentifiabie to a human observer thereof , but being identifiable to an automated identification system. In an embodiment, the automated identification system has the capacity to compare the distorted image to a catalog of distorted images of identified individuals. In a supplemental and/or alternative embodiment, the automated identification system has the capacity to identify the individual by reversing the distortion of the image to generate a reversed undistorted image, followed by comparing the reversed undistorted image to a catalog of undistorted images of identified individuals.

[0035] In an embodiment, the distorted image is, through means of a software system, capable of comparing the distorted image to other distorted images. The comparison can be made at high speed.

[0036] In an embodiment, light impinging on the surface of the sensor is used to produce a distorted image that does not resemble a human face. However, through electronic and/or mechanical means, the distorted image may be converted back into an undistorted image of the subject matter from which the distorted image was prepared.

[0037] In an embodiment, light impinging on the surface of the sensor is used to produce a distorted image that does not resemble the face of an individual to be identified, i.e., the distorted image cannot be used to create an identifiable biometric feature such as a human face.

[0038] In an embodiment, the images obtained by the camera may be correlated to an individual's name or create a unique data set associated with an individual.

[0039] In an embodiment, the processing of images obtained by the camera used in the following invention not require encryption prior to, during, or after processing by a local or remote computer.

[0040] In an embodiment, facial recognition algorithms are used to analyze the images obtained. Brief Description of the Drawings

[0041] Fig. 1 is a schematic diagram of a prior art process by which images are acquired and processed with a traditional facial recognition system.

[0042] Fig. 2 is a schematic diagram of a working embodiment in which distorted images are acquired and processed to form a facial recognition system.

[0043] Fig. 3 is a schematic of a camera using a distortion element as in the embodiment of Fig. 2.

[0044] Figs. 4A, 4B, 4C, 4D, 4E, 4F, and 4G are illustrative of images acquired using the camera of Fig. 3 with various distortive elements of Fig. 2 in a system according to Fig. 2.

[0045] Fig. 5 illustrates the process by which the distorted images acquired and processed in accordance with Fig. 2 are processed to obtain the identity of an individual

[0046] Fig. 6 illustrates a hand wash compliance system integrating the facia! recognition system of Fig. 2.

[0047] Fig. 7 is an illustration of the process by which the data of the hand wash compliance system in Fig. 6 is stored and the associated data is stored.

Detailed Description

[0048] Fig. 1 is a schematic diagram of a prior art process by which an image is acquired and processed with a traditional facial recognition system 1 . Light 2 entering the camera lens 3 impinges a RGB (i.e., red green blue) color filter array 4 or CCD array 4 and is generally digitized in step 5. This digitized or analog image signal is then transmitted to a computer 6 and further stored for processing on a storage medium such as a hard drive 7. The storage may be encrypted in step 14 and additional processing of the images may require an encryption key 15.

Algorithms such as Principal Component Analysis, Independent Component Analysis, Linear Discriminant Analysis, Evolutionary Pursuit Analysis, Elastic Bunch Graph Matching, Kernel Methods, Trace Transform, Radon Transform, Active Appearance Model, 3-D Morphabfe Model, 3-D Canonical Surface Data, Bayesian Framework, Support Vector Machine, Hidden Markova Models, Boosting and Ensemble Solutions are used in step 8 to process the image data with algorithm(s) immediateiy prior to storage on the computer or the images are retrieved and processed to detect a face. A database of images is accessed to compare the newly characterized image to the dataset and determine a relevant identity 9 or a non- matching or irrelevant identity 0. In step 1 1 , an encrypted image may be transmitted to a web site. This matched pair is then sent to a server 12 and may be accessed by a user for viewing through an encrypted process 15. In this traditional (prior art) process 1 a biasing factor 500 is not used.

[0049] Fig. 2 is a flow diagram of working process 300 wherein light 1 undergoes an image distortion process 301 prior to entering camera lens 3, with the resulting image data being received, digitized, transmitted to a computer, stored, encrypted (optional), retrieved, processed, and compared with images in a database, as per process 201 of Fig. 1. Distortions 305 may include mirroring effects 302, occlusion effects 303, blurring 304, segmentation 305, color filtering 306, distortive lens effects 307, and/or any alternative or further distortion effect. As an example, consider a blurred image 304 such as that created by using a pair of unfocused lenses. The image obtained may then be processed using an algorithm such as those used in step 8 of Fig. 1 with a biasing factor 500 included. In this way, although the image is not recognizable when unencrypted, the key attributes used to match the image to a database of equally unfocused images remain the same and allow for a match.

[0050] Fig. 3 shows the structure of a camera 800 wherein light 1 bounces off of individual 400 illustrated as face 500, with redirected light 402 (i.e., reflected light 402) passing to and through distorter 301. The segmentation 305 by distorter 301 produces distorted light 401 which passes through lens 3, thereafter impinging the CMOS sensor or CCD array 4. The array is read with processor 5 which also transmits the signal digitally or serially.

[0051] Figs. 4A, 4B, 4C, 4D, 4E, 4F, and 4G are illustrative of the image acquired using the camera of Fig. 3 with distortive elements 305 and of an image that would normally appear as shown in Fig. 4 using the process of Fig. 1 , Fig. 4A shows no distorter 301 in effect, while Fig. 4B shows mirroring effects 302, Fig. 4C shows occlusion effects 303, Fig. 4D shows blurring 304, Fig. 4E shows segmentation 305, Fig. 4F shows color filtering 306, and Fig. 4G shows distortive lens effects 307. When processing facial images that have been distorted, the biasing 500 will be turned on and used in conjunction with the algorithms 8 to create a characterized "neutral image". This "neutral image" although visually uncharacterizable by a human being and thus allowing for full privacy of any recorded image (whether decrypted or not) will have maintained the information required to form a

characterization that can be stored and then matched. Thus, if the computer storing the matched information is stolen, although an individual's name may be

decipherable, the facial characterization will not be.

[0052] Fig. 5 illustrates the process 300 by which the images obtained in the process illustrated in Fig. 2 are processed using a segmented distorted image 305 to obtain facial recognition characterization and identification 601.

[0053] Fig. 6 illustrates a hand wash compliance system integrating the facial recognition system of Fig. 2 consisting of a sink 802, a hand sanitizer 801 , a camera 800 that acquires one or more images of individual 400 with face 500 and hand/arm 804. RFID transmitter and/or receiver 803 detects an RFID badge that is used to identify the individual by name and further associate the individual with distorted image data set 12.

[0054] Fig. 7 is an illustration of the process 1001 by which the data of the hand wash compliance system in Fig. 6 is stored and processed and the associated data is stored. The light that enters the camera and process 800 has been distorted prior by distortion process 301 and in some cases segmented by 305. The data is processed as per the description of the various figures discussed above. The association of a distorted facial image to a hand wash event is then completed at step 13. The characterization of the actual hand wash event is generally done separately through various sensors including cameras, vibration sensors,

mechanical switches, or other electro-mechanical means. The association of a name to the characterized facial image can be done through visual identification of a tag, through passive or active RFID tag or other transmission signal.

Claims

WHAT IS CLAIMED IS:

1 . A facial recognition system comprising an optical image distorter, a camera, a processing algorithm, a data storage system, and a computer.

2. The facial recognition system of claim 1 , wherein the optical image distorter occludes, blurs, filters, multiplies, segments, mirrors, Ienses, or otherwise distorts the image.

3. The facial recognition system of claim 1 , wherein the light entering the camera has passed through a lens just prior to entering the camera.

4. The facial recognition system of claim 1 , wherein the light entering the camera has bounced off a mirrored surface just prior to entering the camera.

5. The facial recognition system of claim 4, wherein the mirrored surface is not flat.

6. The facial recognition system of claim 1 , wherein only partial segments of the image enter the camera.

7. The facial recognition system of claim 6, wherein one or more parts of the original image are sent to more than one camera.

8. The facial recognition system of claim 1 , wherein the light entering the camera has passed through one or more ienses to create a blurred image just prior to entering the camera.

9. The facial recognition system of claim 1 , wherein the fight entering the camera has passed through a diffraction grating just prior to entering the camera.

10. The facial recognition system of claim 1 wherein the light entering the camera has passed through a light diffuser just prior to entering the camera.

1. The facial recognition system of claim 1 wherein the light entering the camera has passed through a multiple prism beam expander so as to distort the image just prior to entering the camera.

12. The facial recognition system of claim 1 wherein the light entering the camera has passed through an achromatic lens so as to distort the image just prior to entering the camera.

13. The facial recognition system of claim 1 wherein the light entering the camera has passed through color filters so as to distort the image just prior to entering the camera.

14. The facial recognition system of claim 1 wherein the light entering the camera has passed through one or more of the distorters of claims 3-13.

15. A hand wash or hand sanitation compliance system wherein the association of an individual to the hand wash or hand sanitation compliance monitoring is done by using a facia! recognition system that relies on a distorted image entering the camera.

16. The hand wash or hand sanitation compliance system of claim 15 wherein the light entering the camera passes through one or more of the image distorters of claims 3-13.

17. A facial recognition process, comprising:

(A) acquiring an image of a face of an unidentified individual, the image being formed from fight reflected from the face of the unidentified individual;

(B) distorting the image of the face of the unidentified individual, by passing the reflected iight through a distortion device which carries out an image distortion process to produce a distorted image corresponding with the face of the unidentified individual;

(C) impinging the distorted image of the unidentified individual on a red green blue color filter array or a charge coupled device array, to form a digitized distorted image signal or an analog distorted image signal;

(D) transmitting the digitized or analog distorted image signal to a computer for data storage and data processing, the computer having stored therein a plurality of digitized or analog distorted facial images of corresponding with the images of faces of a plurality of identified individuals;

(E) processing the digitized or analog distorted facial image of the unidentified individual in a manner so that the distorted facial image of the unidentified individual is compared with the stored plurality of digitized or analog distorted images of identified individuals, with the comparing being carried out in order to determine if the distorted image of the unidentified individual corresponds with a distorted image of an identified individual stored in the computer.

18. The facial recognition process according to Claim 17, wherein the distorted image of the unidentified individual is encrypted, and the distorted images of the identified individuals are also encrypted.

19. The facial recognition process according to Ciaim 17, wherein the distorted image of the unidentified individual is processed by at least one algorithm selected from the group consisting of Principal Component Analysis, independent Component Analysis, Linear Discriminant Analysis, Evolutionary Pursuit Analysis, Elastic Bunch Graph Matching, Kernel Methods, Trace Transform, Radon Transform, Active Appearance Model, 3-D Morphable Model, 3-D Canonical Surface Data, Bayesian Framework, Support Vector Machine, Hidden Markova Models, and

Boosting and Ensemble Solutions.

20. The facial recognition process according to Claim 17, wherein the image distortion comprises at least one member selected from the group consisting of mirroring, occlusion, blurring, segmentation, color filtering, and shape distortion.

21. The facial recognition process according to Claim 17, wherein the distorted image is not recognizable when unencrypted, and the distorted image contains key attributes which are used to match the distorted image of the

unidentified individual to a database of equally distorted images of identified individuals, so that a match can be made, resulting in the identification of the unidentified individual.

22. The facial recognition process according to Claim 17, wherein the process is used in a hand wash compliance system including a sink, a hand sanitizer and a camera acquiring a distorted facial image of an unidentified individual.

23. The facial recognition process according to Claim 17, wherein the process is used in a hand wash compliance system including a sink, a hand sanitizer and a camera acquiring a visual identification of a tag or other transmission signal.