WO2023242875A1

WO2023242875A1 - Device, system and method for 3d face recognition and authentication door lock

Info

Publication number: WO2023242875A1
Application number: PCT/IN2023/050570
Authority: WO
Inventors: Suresh Kumar MATHA
Original assignee: Matdun Labs India Private Limited
Priority date: 2022-06-18
Filing date: 2023-06-15
Publication date: 2023-12-21

Abstract

The present invention relates generally to a door lock, and particularly to a 3D sensor module that comprises of a camera, a depth sensor among other sensors to recognize one or more faces in 2D and 3D space to authenticate Authorized users and provide access to locations and services. A smart door lock with authentication system includes one or more memories, sensors, or processor configured to identify an object in field of view and activate SoC (System on a chip)/Cloud to process one or more images of the object in 2D and 3D space. The smart lock recognizes face of the object and works with a custom sensor chip on authentication. The smart lock further calibrates match to the field of view and height of the device. A custom neural engine processes one or more authentication layer to identify the object images.

Description

DEVICE, SYSTEM AND METHOD FOR 3D FACE RECOGNITION AND AUTHENTICATION DOOR LOCK

FIELD OF INVENTION:

[0001] The present invention relates generally to a door lock, and particularly to a 3D sensor module configured with a camera with depth and other sensors to recognize one or more faces in 2D and 3D space for the purpose of authentication in door locks.

BACKGROUND OF THE INVENTION

[0002] With the development and widespread adoption of smart locks, the limitations of general fingerprint, password and radio frequency card smart locks have been increasing in market. Fingerprints of the old/aged with serious fingerprint abrasion, fingerprints of injured adults, fingerprints with water stains, etc. are difficult to solve by the existing fingerprint technology; the passwords are easy to peep by the hidden camera; the radio frequency cards are inconvenient to carry, and easy to copy, and all of the above systems cannot actively actively wake up the function of system identification, and are poor in convenience.

[0003] From the aforementioned standpoint there has been a need for 3D face recognition authentication and intelligent door lock to solve the prevailing issue. Therefore, there is a genuine need for improved 3D face recognition authentication and intelligent door lock to overcome all the above problems by developing an improved system and method, which is accurate, efficient, dynamic, and simple.

OBJECTS OF THE INVENTION

[0004] It is an objective of the present invention to recognize one or more face with a camera to authenticate for door locks. [0005] It is an objective of the present invention to identify an object in field of view and activate the SoC (System on a chip) to cloud to carry out the process of image capture.

[0006] It is an objective of the present invention to recognize the face and work with a custom sensor chip/board on authentication.

[0007] It is an objective of the present invention to processes different authentication layers to identify the object by use of custom neural engine.

[0008] It is an objective of the present invention to capture depth information by use of dot projectors.

[0009] It is an objective of the present invention to provide a calibrated match to the field of view and height of the device.

[00010] It is an objective of the present invention to provide a device, system and method to for an improved communication system.

[00011] It is an objective of the present invention to provide a device, system, and method to for an improved artificial intelligence system

[00012] It is an objective of the present invention to provide a device, system, and method for an improved information system.

[00013] It is an objective of the present invention to provide a device, system, and method for an improved survey system.

[00014] It is an objective of the present invention to provide a device, system, and method for an improved sensor device system.

[00015] It is an objective of the present invention to provide a device, system, and method for an improved verification system.

[00016] It is an objective of the present system to provide user profiles/information in a storage that can be on the Soc/cloud.

[00017] It is the objective of the system to store the usage permissions/policies set by the admin/super admin against each profile which acts as the database or active directory. [00018] It is the objective of the system to use the stored profiles and permissions for the purpose of authenticating/performing the operations of the door lock.

[00019] It is an objective of the present invention to provide a device, system and method for an improved tracking and monitoring system.

SUMM RY OF THE INVENTION

[00020] The present invention relates generally to a door lock, and particularly to a 3D sensor module configured with a camera with depth and other sensors to recognize one or more faces in 2D and 3D space for the purpose of authentication and to recognize one or more face and a camera to authenticate for door locks.

[00021] Although the present disclosure is explained with respect to a 3D door lock with authentication system and method, it would be appreciated that this application is used for illustration purposes, but it is not a limitation.

[00022] The present invention provides a 3D door lock configuration to identify an object in field of view and activate the SoC (System on a chip)/Cloud to process the 3D image. The door lock is capable to recognize the face and work with a custom sensor chip on authentication. The door lock is further configured to processes different authentication layers to identify the object by use of custom neural engine. In an embodiment the door lock is configured to capture depth and other information by use of dot projector, camera and other sensors.

[00023] The present invention provides easy repair mechanism. The mechanism provided is easy to work on thereby making the repair work very user friendly and easy. Since it is a modular type assembly whenever a part is damaged, it can be easily replaced and the whole damaged part can be replaced. This can be done in a short time and the can be done in a low cost. The present invention provides easy assembly.

[00024] Additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The aspects of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the appended claims. It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

[00025] The other objects, features and advantages will occur to those skilled in the art from the following description of the preferred embodiment and the accompanying drawings in which.

[00026] Fig. 1 illustrates a block diagram of entire door lock unit with the custom hardware.

[00027] Fig. 2 illustrates a flow diagram for operations performed in the door lock/unlock unit.

[00028] Fig. 3 illustrates a flow diagram for user’s authenticity and updates the lock state by examining the lock's status.

[00029] Fig. 4 illustrates a flow diagram for image capturing with projected pattern and sending it to processor for further processing.

[00030] Fig. 5 illustrates a block diagram of user’s face authentication functions of face recognition process.

[00031] Although specific features of the present invention are shown in some drawings and not in others, this is done for convenience only as each feature may be combined with any or all of the other features in accordance with the present invention.

DETAILED DESCRIPTION

[00032] Embodiments herein and the various features and advantageous details thereof are explained more fully with reference to the non-limiting embodiments that are illustrated in the accompanying figures and detailed in the following description. Descriptions of well-known components and processing techniques are omitted so as to not unnecessarily obscure the embodiments herein. The examples used herein are intended merely to facilitate an understanding of ways in which the embodiments herein may be practiced and to further enable those of skill in the art to practice the embodiments herein. Accordingly, the examples should not be construed as limiting the scope of the embodiments herein.

[00033] In the following description, numerous specific details are set forth in order to provide a thorough understanding of embodiments of the present invention. It will be apparent to one skilled in the art that embodiments of the present invention may be practiced without some of these specific details.

[00034] Embodiments of the present invention may include various steps, which will be described below. Steps may be performed by hardware components or may be embodied in machineexecutable instructions, which may be used to cause a general-purpose or special-purpose with the instructions to perform the steps. Alternatively, steps may be performed by a combination of hardware, software, and/or by human operators.

[00035] If the specification states a component or feature “may”, “can”, “could”, or “might” be included or have a characteristic, that component or feature is not required to be included or have the characteristic.

[00036] The foregoing illustrated and described embodiments of the invention are susceptible to various modifications and alternative forms, and it should be understood that the invention generally, as well as the specific embodiments described herein, are not limited to the particular forms or embodiments disclosed, but cover all modifications, equivalents and alternatives falling within the scope of the appended claims. By way of non- limiting example, it will be appreciated by those skilled in the art that particular features or characteristics in one embodiment may be combined as suitable with features or characteristics described in another embodiment. Moreover, the device and method for of the present invention can be a system used along with an automated or even with a manual system.

[00037] The present invention relates generally to a door lock, and particularly to a 3D sensor, a camera along with other sensors to recognize one or more faces to authenticate for door locks. [00038] Although the present disclosure is explained with respect to a camera along with a 3D and other sensors to recognize one or more faces to authenticate for door locks device or method, it would be appreciated that this application is used for illustration purposes, but it is not a limitation. This application may be also used in multiple gadgets, home appliances, OEM’s, etc. along with access to locations and services.

[00039] The various embodiments of the invention provide a 3D door lock configuration to identify an object in field of view and activate the SoC (System on a chip)/cloud to process the 3D image. The door lock is capable to recognize the face and work with a custom sensor chip on authentication. The door lock is further configured to processes different authentication layers to identify the object by use of custom neural engine. In an embodiment the door lock is configured to capture depth and other information by use of dot projector, camera and other sensors.

[00040] In an embodiment neural engine uses multiple different layers to process the depth information and accurately identify the user. In an embodiment the lock maintains a list of individual profiles that should and should not have access to the property. In an embodiment the lock can be configured to provide entry for house maintenance teams/individuals during specific time/scheduled hours. Yet in another embodiment the lock may provide guest access for a limited timeframe.

[00041] Fig. 1 illustrates a block diagram of entire door lock unit with the custom hardware. In an embodiment a 3d face recognition and authentication door lock (101) may be configured to capture the user’s face information to authenticate and automatically unlock the device in response to the outputs generated by custom deep learning models. A distance sensor (103) may be configured to measure and validate distance between object and door. Movement of the user approaching the door unit may be detected by motion sensor (102). In an embodiment IR sensor (106), RGB sensor(s), and pattern projector (107) may be configured to map the captured object face and obtain depth information. The one or more processors (104, 109, 110) and image capturing module may be triggered by the device when a user is in motion within a permissible range. In another embodiment, the authentication door lock (101) may be set up to capture sequence of facial images with a suitable pattern using camera sensor(s) and a projector and extract the depth information using novel method(s) in day and/or night weather conditions. In another embodiment, a custom processor may be employed to perform the user face authentication functions of face recognition, head pose estimation, and anti-spoofing while using depth data as an input.

[00042] Fig. 2 illustrates a flow diagram for operations performed in the door lock/unlock unit. The operation begins with step 200, where the door is open and not, or it is in idle state. In case the door is closed or open, process moves to next step 201, where one or more processors (104, 109, 110) may enable a motion detection based on an object movement, within a permissible range and the process moves to next step. In step 202 the distance sensor (103) measures and validates distance between object and door and process moves to next step. In step 203 the distance sensor (103) validates whether the distance is valid or not. In case the distance is valid the process moves to next step or else the process returns to the step 202.

[00043] In step 204 the process checks whether one or more sensors, such as the IR sensor 106, the PR (Pattern Projector) sensor 107 and the laser-based depth mapping system (108) are active or not. In case the one or more sensors are active in step 205 the process moves to step 206 or else the process returns back to the step 204. In step 206 the process may check whether face of the object is aligned or not. In case the face is aligned the process moves to next step or the process moves to 207 where the image sensor module (504) align face with optical axis and return to the step 206. In step 208 the one or more processors (104, 109, 110) capture the depth information and moves to the next step. In step 209 the one or more processors (104, 109, 110) analyzes the depth variation in facial region, where it checks face recognition in step 210, head post estimate in step 211, and anti-spoofing in step 212. In case it passes steps 210, 211, and 212 then the process moves to next step. In step 213, 214, and 215, the process checks for face verification, head pose verification and whether the object is real or not, and the process moves to last step. In step 216 the person may be authenticated, and the door may be unlocked. The process ends.

[00044] Fig. 3 illustrates a flow diagram for user’s authenticity and updates the lock state by examining the lock's status. The operation starts at 301, where it may be checked whether the object (person) is authenticated or not. In case the person is authenticated, process moves to step 302 and the one or more processors (104, 109, 110) unlocks the door and moves to next step. In step 304 the one or more processors (104, 109, 110) updates the lock status. In case the person is not authenticated in step 301, the process moves to step 303 where the one or more processors (104, 109, 110) locks the door and updates the lock status in the step 304. The process ends.

[00045] Fig. 4 illustrates a flow diagram for image capturing with projected pattern and sending it to processor for further processing. The operation begins at step 401, where an image sensor module (504) may check the connectivity with the one or more processors (104, 109, 110). In step 402, the process may sense ambient light level in day/night environment and moves to next step. In step 403 the process may check whether faces are aligned or not. In case the faces are aligned the process moves to next step or it again checks for face alignment. In step 404 the process may check whether the camera is active or not and moves to next step. In step 405 the one or more sensors captures the depth information and moves to next step. In step 406 the depth information may be shared with processors via connected hub and the process moves to next step. In step 407, once the data may be shared with the one or more processors, the communication may be disabled. The process ends.

[00046] Fig. 5 is a block diagram of user’s face authentication functions of face recognition process. In an embodiment, once the object is near by the door the motion sensor 501 detects the motion and the distance may be captured by a distance sensor 502. Measured distance the motion data may be passed to the illuminator 503 so that it may check the status of image sensor module 504. In an embodiment the image sensor module 504 may include an IR sensor and PR (Pattern Projector Sensor). The IR sensor may capture the image and the PR sensor may capture the DoT image. The image captured by IR sensor may be passed to face recognition module 505 and head pose module 506 for authentication. The DoT image captured by PR sensor may be passed to the anti-spoof module 507. The anti-spoof module 507 may communicate with the face recognition module 505 for image validation. In an embodiment, once the image may be validated by all the modules, the door unlock/lock operation may take place.

[00047] The present invention provides easy usage mechanism. The mechanism provided is easy to work on thereby making the device very user friendly and easy. [00048] In an embodiment, the access management can be used to control access of legitimate users comprising but not limited to commercial buildings; residential buildings; IT resources; cloud; home appliances;

[00049] In another embodiment, the access management can be used to allow one or more entity to authenticate, authorize and audit access to on-premises and cloud-based applications and IT systems

[00050] In another embodiment, the access management can be used to allow one or more entity to limit the information users can view or change to minimize the chances of improper activity.

[00051] In yet another embodiment, the access management can be used to allow one or more entity to grant authorized users the right to use a service or location, while preventing access to nonauthorized users.

[00052] In yet another embodiment, the access management can be used to allow one or more entity to control and monitor network/location/services access.

[00053] In yet another embodiment, the access management can be used to efficiently respond to requests for granting access to services and locations, changing access rights or restricting access, ensuring that the rights being provided or changed are properly granted.

[00054] In yet another embodiment, the access management can be used to oversee access to services/locations and ensure rights being provided are not improperly used, remove access when people leave/move out of the house or change jobs.

[00055] While embodiments of the present invention have been illustrated and described, it will be clear that the invention is not limited to these embodiments only. Numerous modifications, changes, variations, substitutions, and equivalents will be apparent to those skilled in the art, without departing from the spirit and scope of the invention, as described in the claim.

ADVANTAGES OF THE INVENTION:

[00056] The present invention provides an object identifier in field of view and activate the SoC (System on a chip)/cloud to process the 3D image. [00057] The present invention provides a face recognizer in 3D space and work with a custom sensor chip on authentication.

[00058] The present invention provides different authentication layers to identify the object by use of custom neural engine.

[00059] The present invention captures depth information by use of dot projectors.

[00060] The present invention calibrates match to the field of view and height of the device.

Claims

CLAIMS We Claim:

1. A device for 3d face recognition and authentication door lock (101) comprising: one or more processors (104, 109, 110) configured to: enable a motion detection based on an object movement, within a permissible range; measure and validate distance between object and door by a distance sensor (103) installed in the authentication door lock (101); capture sequence of facial images by an image sensor module (604) and extract the depth information in day and/or night weather conditions; and analyze one or more subsequent images to identify face of the object based on a facial profile match in one of the subsequent images that matches a predetermined facial profile of an authorized user of the device recorded by the image sensor module (604) and perform unlock operation.

2. The device for 3d face recognition and authentication door lock (101) as claimed in claim 1, wherein a laser-based depth mapping system (108) authenticates a user's face using depth input.

3. The device for 3d face recognition and authentication door lock (101) as claimed in claim 1, wherein movement of the user approaching the door unit is detected by motion sensor (102).

4. The device for 3d face recognition and authentication door lock (101) as claimed in claim 1, wherein the device includes IR sensor (106), RGB sensor(s), and pattern projector (107) to map the captured object face and obtain depth information.

5. The device for 3d face recognition and authentication door lock (101) as claimed in claim 1, wherein one or more processors (104, 109, 110) are further configured to operate the sensor and projector for capturing the depth information soon when user face is detected.

6. The device for 3d face recognition and authentication door lock (101) as claimed in claim 1, wherein one or more processors (104, 109, 110) are further configured to analyze the variation of depth information on the facial areas and identifying authorized and authentic faces by eliminating the imposters and unauthorized faces using deep learning approach.

7. The device for 3d face recognition and authentication door lock (101) as claimed in claim 1, wherein face authentication locking and unlocking the door is based on inputs provided by depthbased face recognition system custom hardware.

8. The device for 3d face recognition and authentication door lock (101) as claimed in claim 7, wherein closest user/face will be the one considered to authenticate the door lock among the users/faces in the permissible range to the device.

9. A method for 3d face recognition and authentication door lock (101) comprising: enabling, by one or more processors (104, 109, 110), a motion detection based on an object movement, within a permissible range; measuring and validating distance, one or more processors (104, 109, 110), between object and door by a distance sensor (103) installed in the authentication door lock (101); capturing sequence, by one or more processors (104, 109, 110), of facial images by image sensor module (604) and extract the depth information in day and/or night weather conditions; and analyzing, by one or more processors (104, 109, 110), one or more subsequent images to identify face of the object based on a facial profile match in one of the subsequent images that matches a predetermined facial profile of an authorized user of the device recorded by the image sensor module (604) and perform unlock operation.

10. The method for 3d face recognition and authentication door lock (101) as claimed in claim 1, wherein a laser-based depth mapping system (108) authenticates a user's face using depth input.

11. The method for 3d face recognition and authentication door lock (101) as claimed in claim 1, wherein movement of the user approaching the door unit is detected by motion sensor (102).

12. The method for 3d face recognition and authentication door lock (101) as claimed in claim 1, wherein the device includes IR sensor (106), RGB sensor(s), and pattern projector (107) to map the captured object face and obtain depth information.

13. The method for 3d face recognition and authentication door lock (101) as claimed in claim 1, wherein one or more processors (104, 109, 110) are further configured to operate the sensor and projector for capturing the depth information soon when user face is detected.

14. A system for 3d face recognition and authentication door lock (101) comprising: one or more processors (104, 109, 110) configured to: enable a motion detection based on an object movement, within a permissible range; measure and validate distance between object and door by a distance sensor (103) installed in the authentication door lock (101); capture sequence of facial images by an image sensor module (604) and extract the depth information in day and/or night weather conditions; and analyze one or more subsequent images to identify face of the object based on a facial profile match in one of the subsequent images that matches a predetermined facial profile of an authorized user of the device recorded by the image sensor module (604) and perform unlock operation.