US11205312B2 - Applying image analytics and machine learning to lock systems in hotels - Google Patents

Applying image analytics and machine learning to lock systems in hotels Download PDF

Info

Publication number
US11205312B2
US11205312B2 US17/057,129 US201917057129A US11205312B2 US 11205312 B2 US11205312 B2 US 11205312B2 US 201917057129 A US201917057129 A US 201917057129A US 11205312 B2 US11205312 B2 US 11205312B2
Authority
US
United States
Prior art keywords
access
credential
control device
access control
day
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
US17/057,129
Other languages
English (en)
Other versions
US20210201609A1 (en
Inventor
Santhosh Amuduri
Ramesh Lingala
Adam Kuenzi
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Carrier Corp
Original Assignee
Carrier Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Carrier Corp filed Critical Carrier Corp
Assigned to CARRIER CORPORATION reassignment CARRIER CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: UTC FIRE & SECURITY INDIA LTD.
Assigned to UTC FIRE & SECURITY INDIA LTD. reassignment UTC FIRE & SECURITY INDIA LTD. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: AMUDURI, SANTHOSH, LINGALA, RAMESH
Assigned to CARRIER CORPORATION reassignment CARRIER CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: KUENZI, ADAM
Publication of US20210201609A1 publication Critical patent/US20210201609A1/en
Application granted granted Critical
Publication of US11205312B2 publication Critical patent/US11205312B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G06K9/00268—
    • G06K9/00288—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
    • G07C9/00—Individual registration on entry or exit
    • G07C9/20—Individual registration on entry or exit involving the use of a pass
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
    • G07C9/00—Individual registration on entry or exit
    • G07C9/20—Individual registration on entry or exit involving the use of a pass
    • G07C9/22—Individual registration on entry or exit involving the use of a pass in combination with an identity check of the pass holder
    • G07C9/23—Individual registration on entry or exit involving the use of a pass in combination with an identity check of the pass holder by means of a password
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
    • G07C9/00—Individual registration on entry or exit
    • G07C9/20—Individual registration on entry or exit involving the use of a pass
    • G07C9/22—Individual registration on entry or exit involving the use of a pass in combination with an identity check of the pass holder
    • G07C9/25—Individual registration on entry or exit involving the use of a pass in combination with an identity check of the pass holder using biometric data, e.g. fingerprints, iris scans or voice recognition
    • G07C9/253—Individual registration on entry or exit involving the use of a pass in combination with an identity check of the pass holder using biometric data, e.g. fingerprints, iris scans or voice recognition visually
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
    • G07C9/00—Individual registration on entry or exit
    • G07C9/30—Individual registration on entry or exit not involving the use of a pass
    • G07C9/32—Individual registration on entry or exit not involving the use of a pass in combination with an identity check
    • G07C9/37—Individual registration on entry or exit not involving the use of a pass in combination with an identity check using biometric data, e.g. fingerprints, iris scans or voice recognition
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
    • G07C2209/00—Indexing scheme relating to groups G07C9/00 - G07C9/38
    • G07C2209/60—Indexing scheme relating to groups G07C9/00174 - G07C9/00944
    • G07C2209/63—Comprising locating means for detecting the position of the data carrier, i.e. within the vehicle or within a certain distance from the vehicle
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
    • G07C9/00—Individual registration on entry or exit
    • G07C9/00174—Electronically operated locks; Circuits therefor; Nonmechanical keys therefor, e.g. passive or active electrical keys or other data carriers without mechanical keys
    • G07C9/00563—Electronically operated locks; Circuits therefor; Nonmechanical keys therefor, e.g. passive or active electrical keys or other data carriers without mechanical keys using personal physical data of the operator, e.g. finger prints, retinal images, voicepatterns
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
    • G07C9/00—Individual registration on entry or exit
    • G07C9/00174—Electronically operated locks; Circuits therefor; Nonmechanical keys therefor, e.g. passive or active electrical keys or other data carriers without mechanical keys
    • G07C9/00817—Electronically operated locks; Circuits therefor; Nonmechanical keys therefor, e.g. passive or active electrical keys or other data carriers without mechanical keys where the code of the lock can be programmed

Definitions

  • the subject matter disclosed herein generally relates to the field of access control systems, and more particularly to an apparatus and method for applying image analytics and machine learning to lock systems in hotels.
  • Hotel customers may exit and enter their hotel rooms several times a day, and each time that they enter their room they are required to provide a credential to unlock a lock on the room door. Hotel customers may also be required to provide a credential when they enter other restricted areas of the hotel such as a health club or parking garage or entry doors or an elevator car.
  • the credential may be located on an access card provided by the hotel or on a mobile device of the user, both of which the hotel customer puts proximate to the lock (e.g., in a card reader or other short range reader) to unlock a lock on a restricted area. Both the access card and the mobile device require the hotel customer to take an action to unlock the lock, and they both require the hotel customer to remember to bring their access card or mobile device when they exit their hotel room.
  • a method of using image analytics and machine learning in a lock system includes receiving data describing access actions that are performed at an access control device based at least in part on a credential.
  • the data includes, for each access action, a description of the access action, a timestamp, and an image of a person presenting the credential.
  • the data is analysed to identify patterns of access. Facial characteristics of the person presenting the credential are identified.
  • the credential is associated with the facial characteristics. It is detected, based at least in part on facial recognition and the facial characteristics, that the person is proximate to the access control device.
  • An access action is performed at the access control device based on the detecting and the patterns of access, where the facial recognition is used in place of the credential to provide authorization to perform the access action.
  • further embodiments of the method may include that the patterns include a time of day, and the performing is further based on a current time of day being within a threshold of the time of day in the patterns.
  • further embodiments of the method may include that the patterns include a day of week, and the performing is further based on a current day of week being within a threshold of the day of week of the patterns.
  • further embodiments of the method may include that facial characteristics of a plurality of people are identified as presenting the credential and the credential is associated with the plurality of people.
  • further embodiments of the method may include the access action being an unlock or lock.
  • further embodiments of the method may include that the access action is enter security mode or enter office mode.
  • further embodiments of the method may include that additional access actions are performed subsequent to the performing as long as a confidence level of the associating is above a threshold.
  • further embodiments of the method may include that the facial characteristics are updated based on images of the person received after the associating.
  • a system is configured to use image analytics and machine learning in a lock system.
  • the system includes a processor and a memory having computer-executable instructions that, when executed by the processor, cause the processor to perform operations.
  • the operations include receiving data describing access actions, the access actions performed at an access control device based at least in part on a credential.
  • the data includes, for each access action, a description of the access action, a timestamp, and an image of a person presenting the credential.
  • the data is analysed to identify patterns of access. Facial characteristics of the person presenting the credential are identified.
  • the credential is associated with the facial characteristic. It is detected, based at least in part on facial recognition and the facial characteristics, that the person is proximate to the access control device.
  • the access action is performed at the access control device based on the detecting and the patterns of access, and facial recognition is used in place of the credential to provide authorization to perform the access action.
  • further embodiments of the system may include that the patterns include a time of day, and the performing is further based on a current time of day being within a threshold of the time of day in the patterns.
  • further embodiments of the system may include that the patterns include a day of week, and the performing is further based on a current day of week being within a threshold of the day of week of the patterns.
  • further embodiments of the system may include that facial characteristics of a plurality of people are identified as presenting the credential and the credential is associated with the plurality of people.
  • further embodiments of the system may include the access action being an unlock or lock.
  • further embodiments of the system may include that the access action is enter security mode or enter office mode.
  • further embodiments of the system may include that additional access actions are performed subsequent to the performing as long as a confidence level of the associating is above a threshold.
  • further embodiments of the system may include that the facial characteristics are updated based on images of the person received after the associating.
  • a method of using image analytics and machine learning in a two-step authentication system includes receiving data describing access actions that were performed at an access control device based at least in part on a credential,
  • the data includes, for each access action, a description of the access action, a timestamp, and an image of an identifier of a person presenting the credential.
  • the data is analyzed to identify patterns of access. Characteristics of the identifier of a person presenting the credential are identified, and the credential is associated with the characteristics of the identifier. It is detected, based at least in part on image recognition and the characteristics of the identifier, that the person is proximate to the access control device.
  • a personal identification number (PIN) entered at a keyboard of the access control device is validated with an expected PIN of the person.
  • An access action is performed at the access control device based on the expected PIN of the person being the same as the PIN entered at the keyboard of the access control device.
  • further embodiments of the method may include that the characteristics include facial characteristics.
  • further embodiments of the method may include that the characteristics include license plate characteristics.
  • further embodiments of the method may include the access action being an unlock.
  • Technical effects of embodiments of the present disclosure include the ability to use facial features of a user in place of a credential that must be swiped or presented by the user. Additional technical effects include the ability to eliminate two factor authentication.
  • FIG. 1 illustrates a general schematic system diagram of a system that applies image analytics and machine learning to lock systems, in accordance with an embodiment of the disclosure
  • FIG. 2 illustrates a block diagram of an access control device, mobile device and server of the access control system of FIG. 1 , in accordance with an embodiment of the disclosure
  • FIG. 3 is a flow diagram illustrating a method of applying image analytics and machine learning to lock systems, in accordance with an embodiment of the disclosure
  • FIG. 4 illustrates a general schematic system diagram of a system that applies image analytics and machine learning to two-factor authentication lock systems, in accordance with an embodiment of the disclosure.
  • FIG. 5 is a flow diagram illustrating a method of applying image analytics and machine learning to two-factor authentication lock systems, in accordance with an embodiment of the disclosure.
  • facial recognition can be used in place of a credential to unlock a lock of an access control device, such as a lock on a hotel room door (or any other door that a person returns to often such in an office door, a building entrance door, an apartment door, a home door, etc.).
  • Image analytics and machine learning are dynamically applied to accesses by a hotel customer when the hotel customer is using a credential located, for example, on an access card provided by the hotel or on a mobile device of the user. Once a pattern of access to the hotel room by the hotel customer is established, the lock on the hotel room door can be automatically unlocked when the hotel customer is detected by a camera located proximate to or in the access control device on the hotel room door of the hotel customer.
  • the hotel customer can continue to gain access to the hotel room based on facial recognition and without having to show a credential. In this manner, the picture, or image, of the hotel customer that is taken by the camera is used in place of the credential.
  • machine learning is used to determine when the lock can be unlocked based on a picture of the hotel customer captured by the camera.
  • the picture of the person is not prior associated with their credential in the access control system.
  • the system dynamically learns the access pattern and the person's face and once it is learned, facial recognition can be used. Accesses to the door can be tracked while an access pattern is being established.
  • the camera takes a picture of the hotel customer performing the access.
  • the access control device that contains the lock informs the camera to take the picture using a short range wireless communication method such as, but not limited to Bluetooth Low Energy (BTLE), ZigBee, and/or other wireless method.
  • BTLE Bluetooth Low Energy
  • ZigBee ZigBee
  • the camera can be integrated into the lock or separate from the lock (e.g., an already existing security camera).
  • a record of the access event e.g., timestamp and access event
  • the corresponding picture of the hotel customer is sent to a server for analysis/machine learning. Records can be captured continuously by the camera and machine learning applied to the record.
  • the image analytics software e.g., facial recognition software
  • the camera can be enabled to grant authorization automatically and to inform the lock to open without requiring presentation of a credential.
  • the lock has to be authenticated by the hotel customer manually using a credential in an access card or mobile device until a new pattern of accesses is established and identified.
  • an embodiment can be used in a home environment.
  • a pattern of children coming home from school every day around 3:00 PM can be established and the camera can analyze the background images to determine whether to unlock the door automatically.
  • Rules for unlocking the door based on image analytics can be applied.
  • the system may unlock the door only if one child, the one recognized by facial recognition as being expected in that time frame is in the picture.
  • the system may unlock the door as long as one of the children in the field of view of the camera is recognized by facial recognition.
  • the first example does not unlock the door if the child brings friends home
  • the second example does unlock the door if the child brings friends home.
  • a high confidence can be associated with a request from the child to unlock the lock at 3 PM on the next weekday
  • a lower confidence can be associated with a request from the child to unlock the lock at 6 PM on the next week day or on a weekend day.
  • One or more embodiments described herein can be utilized to eliminate two-factor authentication.
  • two-factor authentication a person typically has to present/swipe a credential using for example, a key card and then enter a personal identification number (PIN) into a keypad to cause the lock to unlock.
  • PIN personal identification number
  • by applying analytics using a camera to automate the first level authentication the swiping of the key card is eliminated.
  • details provided by the person such as car license plate or physical key card number (or name) can be identified by the camera.
  • the camera can be configured to look for the car license plate or the physical key card number (or name) to understand physical presence of the person, to fetch the corresponding expected PIN from the server.
  • the lock unlocks when a PIN entered by the person matches the expected PIN fetched by the camera.
  • FIG. 1 a general schematic system diagram of a system 100 that applies image analytics and machine learning to lock systems is generally shown in accordance with an embodiment of the disclosure.
  • the system 100 generally includes a server 14 , an access control device 16 , a user device 106 , an analytical service component 110 , and a camera 38 . Also shown in FIG. 1 are pictures 112 , a room door 116 , and hotel customers 118 . It should be appreciated that, although particular systems are separately defined in the schematic block diagrams, each or any of the systems may be otherwise combined or separated via hardware and/or software.
  • computer instructions to implement the analytical service component 110 can be located on and executed by the server 14 .
  • the computer instructions to implement the analytical service component 110 can also be located on mobile phone 112 , camera 38 , and/or access control device 16 .
  • the access control device 16 includes a lock and controls access to a hotel room through room door 116 . It should be appreciated that the access control system 100 may include any number of doors and rooms. Further, there may be multiple doors and access control devices 16 for each room. As shown in FIG. 1 , the access control device 16 communicates access events to the camera 38 and can receive requests from the camera 38 to perform an access event such as lock or unlock. In embodiment, the access events are sent to the hotel server 14 by the access control device 16 which sends them to the camera 38 . In an embodiment, an access event can also include entering a security mode and/or an office mode.
  • the camera 38 takes a picture 112 of a hotel customer requesting the access event when the access control device 16 communicates an access event to the camera 38 .
  • the camera is taking an image stream and a timestamp of the access event and the image stream are used to select a picture of the hotel customer making the access request.
  • the picture can be selected for example, based on a time stamp captured by the camera 38 or based on an access time stamp sent to the camera 38 by the access control device 16 .
  • the camera is streaming video to the hotel server 14 and the time stamp is sent from access control device 16 to the hotel server 14 ; and the hotel server 14 selects the image based on the timestamp from the recorded video.
  • the image stream is initiated as long as the camera 38 detects nearby motion.
  • the camera 38 shown in FIG. 1 can also request an access event (e.g., unlock a lock on the room door 116 ) for hotel customer 118 a and not for hotel customer 118 b , based for example on previously detected patterns and contents of the pictures 112 taken by the camera 38 .
  • the analytics are running on the hotel server 14 and are processing a continuous video stream looking for faces, the hotel server 14 can request an access event for particular customers.
  • the camera 38 is a camera sensor with low power that is very small in size such as, but not limited to an Ov7251 global shutter image sensor from OmniVision; and a visual sensor from Emza Visual Sense.
  • the camera sensor is located within the access control device 16 and communicates to the access control device 16 via a short-range wireless communication method and/or a near field communication method.
  • a field of view of the camera is focused on an expected location of user requesting an access action (e.g., swiping a key card to request the lock to unlock).
  • camera 38 communicates with the server 14 via any short or long-range wireless communication method known in the art over wired or wireless networks such as, but not limited to the Internet, a local area network (LAN), and a wide area network (WAN).
  • LAN local area network
  • WAN wide area network
  • the user device 106 associated with a hotel customer may be granted access to one or more access control devices 16 (e.g., the door lock on a hotel room assigned to the hotel customer).
  • access control devices 16 e.g., the door lock on a hotel room assigned to the hotel customer.
  • a hotel customer uses the user device 106 to unlock and/or lock the access control device 16 operably connected to their assigned room.
  • the state of the access control device 16 can be changed from locked to unlocked through an access request from the user device 106 being sent to the access control device 16 via a short-range wireless communication interface such as BTLE and/or via a near field communication interface (NFC).
  • a short-range wireless communication interface such as BTLE and/or via a near field communication interface (NFC).
  • the user device 106 may store credentials to unlock and/or lock the access control device 16 .
  • Some credentials may be used for multiple access control devices 16 if there are multiple access control devices 16 for a single assigned room or the hotel customer is assigned access to multiple rooms.
  • an access control device 16 operably connected to a hotel room and an access control device 16 operably connected to a hotel pool or other location within the hotel premises may respond to the same credential.
  • Other credentials may be specific to a single access control device 16 .
  • the user device 106 may be a physical key card and/or a mobile device.
  • the user device 106 may transmit an access request to the access control device 16 via a short-range radio transmission interface when the user device 106 is placed proximate to (i.e., within the range of the short-range radio transmission interface) the access control device 16 .
  • an access request can be transmitted to the access control device 16 by the user device 106 being inserted into or presented to the access control device 16 for the access control device 16 to read the key card (e.g., an encoded magnetic stripe, RFID, smart chip card, etc.).
  • the physical key card is capable of being encoded with card data, such as, for example, a magnetic stripe or RFID chip.
  • the data may include credentials to grant access to a specific access control device 16 .
  • the user device 106 may be granted access to a specific access control device 16 .
  • the user device 106 is a wireless capable handheld device such as a smartphone or tablet or smart watch.
  • the access control device 16 may be a wireless-capable, restricted-access, or restricted-use device such as a wireless lock, access control reader for building entry, and other restricted-use machine.
  • the user device 106 submits credentials to the access control device 16 , thereby selectively permitting a user to access or activate functions of the access control device 16 .
  • a user device 106 may, for example, submit a credential to an electromechanical lock to unlock it, and thereby gain access to a room at the hotel.
  • the access control device 16 is configured to generate records each time that a lock is locked (or closed) and each time that the lock is unlocked (or opened).
  • each record includes a picture 112 taken by the camera 38 at the time of the access event as well as access event information such as, but not limited to: a timestamp, an indicator of a state of locked or unlocked, and/or an identifier of a location (e.g., room number) of the access control device 16 .
  • the record can also include an identifier of the credential used when sending the access request.
  • the records are transmitted from the access control device 16 to the server 14 via the camera 38 .
  • the records are transmitted to the server 14 and/or camera 38 periodically at a selected interval.
  • a record is transmitted to the server 14 and/or camera after each access event.
  • the server 14 transmits the records to the analytical service component 110 via any short or long-range wireless communication method known in the art over wired or wireless networks such as, but not limited to the Internet, a local area network (LAN), and a wide area network (WAN).
  • the analytical service component 110 performs analysis on the records in an attempt to identify patterns of access events by the hotel customer 118 at the access control device 16 . Once a pattern of successful access events by the same person (as determined for example, by applying facial recognition to the pictures in the records) using the credential has been identified, then the hotel customer 118 can be provided access based on the camera 38 recognizing the hotel customer 118 proximate to the access control device 16 within an expected time frame.
  • the analytical server component 110 is an application that is executing on the server 14 . In another, embodiment the analytical server component 110 is executing on a mobile device. In another embodiment, the analytical server component 110 is executing on a processor remote from the server 14 .
  • a block diagram of an example electronic lock system 200 includes the access control device 16 , the mobile device 12 of the hotel customer (e.g., user device 106 of FIG. 1 ), and the server 14 .
  • the access control device 16 generally includes a lock actuator 22 , a lock controller 24 , a lock antenna 26 , a lock transceiver 28 , a lock processor 30 , a lock memory 32 , a lock power supply 34 , a camera 38 , a lock card reader 90 and a credential module 36 .
  • the access control device 16 may have essentially two readers, one reader 90 to read a physical key card 104 (e.g., and the credential module 36 to communicate with the mobile device 12 via the lock processor 30 and the transceiver 28 and antenna 26 .
  • the access control device 16 is responsive to credentials from the mobile device 12 , and may, for example, be the lock of a turnstile or a door lock.
  • credentials may be transmitted from a mobile device to an access control device so as to identify the user to an online system or validate user access rights or permissions in an offline system will benefit herefrom.
  • the lock controller 24 Upon receiving and authenticating an appropriate credential from the mobile device 12 using the credential module 36 , or after receiving card data from lock card reader 90 , the lock controller 24 commands the lock actuator 22 to lock or unlock a mechanical or electronic lock. In this manner, a lock state of the access control device 16 changes from one state to another state.
  • the lock controller 24 and the lock actuator 22 may be parts of a single electronic or electromechanical lock unit, or may be components sold or installed separately.
  • the lock transceiver 28 is capable of transmitting and receiving data to and from at least the mobile device 12 .
  • the lock transceiver 28 may, for instance, be a near field communication (NFC), Bluetooth, infrared, Zigbee, or Wi-Fi transceiver, or another appropriate wireless transceiver.
  • the lock antenna 26 is any antenna appropriate to the lock transceiver 28 .
  • the lock processor 30 and lock memory 32 are, respectively, data processing, and storage devices.
  • the lock processor 30 may, for instance, be a microprocessor that can process instructions to validate credentials and determine the access rights contained in the credentials or to pass messages from a transceiver to a credential module 36 and to receive a response indication back from the credential module 36 .
  • the lock memory 32 may be RAM, EEPROM, or other storage medium where the lock processor 30 can read and write data including but not limited to lock configuration options and the lock audit records.
  • the lock audit records described herein may be a unified audit trail that includes events initiated by accessing the lock via a mobile device 12 or key card 104 .
  • the lock power supply 34 is a power source such as line power connection, a power scavenging system, or a battery that powers the lock controller 24 . In other embodiments, the lock power supply 34 may only power the lock controller 24 , with the lock actuator 22 powered primarily or entirely by another source, such as user work (e.g. turning a bolt).
  • FIG. 2 shows the lock antenna 26 and the transceiver 28 connected to the processor 30 , this is not to limit other embodiments that may have additional antenna 26 and transceiver 28 connected to the credential module 36 directly.
  • the credential module 36 may contain a transceiver 28 and antenna 26 as part of the credential module. Or the credential module 36 may have a transceiver 28 and antenna 26 separately from the processor 30 which also has a separate transceiver 28 and antenna 26 of the same type or different.
  • the processor 30 may route communication received via transceiver 28 to the credential module 36 .
  • the credential module may communicate directly to the mobile device 12 through the transceiver 28 .
  • the mobile device 12 generally includes a key antenna 40 , a key transceiver 42 , a key processor 44 , a key memory 46 , a GPS receiver 48 , an input device 50 , an output device 52 , and a key power supply 54 .
  • the key transceiver 42 is a transceiver of a type corresponding to the lock transceiver 28
  • the key antenna 40 is a corresponding antenna 26 .
  • the key transceiver 42 and the key antenna 40 may also be used to communicate with the server 14 .
  • one or more separate transceivers and antennas may be included to communicate with server 14 .
  • the key memory 46 is of a type to store a plurality of credentials locally on the mobile device 12 .
  • the mobile device 12 may also include a mobile device application 80 . Embodiments disclosed herein, may operate through the mobile device application 80 installed on the mobile device 12 .
  • FIG. 3 a flow diagram 300 illustrating a method of applying image analytics and machine learning to a locking system is generally shown, in accordance with an embodiment of the disclosure.
  • the processing shown in FIG. 3 can be performed by all or a subset of system 100 shown in FIG. 1 .
  • a hotel customer accesses a lock at an access control device, such as access control device 16 of FIG. 1 , using a credential contained in a physical key card or a mobile application.
  • the credential is assigned to the hotel customer by the hotel and allows the hotel customer to perform selected access events at selected access control devices.
  • the access control device requests a camera, such as camera 38 of FIG.
  • the access control device associates a picture taken by the camera with the person based on a timestamp.
  • information about the access event is sent to the camera, and the camera creates a record that includes the information about the access event and the picture corresponding to the access event.
  • the camera sends the record to a server, such as hotel server 14 of FIG. 1 .
  • the access event includes a timestamp that indicates when the access event occurred as well as a description of the type of access event that occurred (e.g., unlock).
  • analytics uses machine learning to analyze the records corresponding to the access control device and the credential.
  • the analysis includes looking for patterns in the time frame, the type of access action, and the person performing the access action.
  • the type of access action can include, but is not limited to: lock, unlock, set security mode, and/or set office mode.
  • the time frame can include, but is not limited to: time of day, day of week, week of month, and/or day of month.
  • the pictures included in the records are analyzed using facial recognition software that includes algorithms to identify facial features by extracting landmarks, or features, from an image of the person's face.
  • an algorithm may analyze the relative position, size, and/or shape of the eyes, nose, cheekbones, and jaw. A probe image is then compared with the face data.
  • the facial recognition software can be used for later recognition by the camera of the hotel customer, or person(s) that they have authorized to use their credential.
  • processing continues by looping back to block 302 .
  • block 312 is performed and the analytics grants permission for the hotel customer to perform the access action based on the camera recognizing the hotel customer in its field of view.
  • recognition of the hotel customer's face in the field of view of the camera is used in place of the credential.
  • people other than the hotel customer who was assigned the credential are also granted permission to perform the access action based on the analysis.
  • a credential e.g., key card
  • a confidence level of the association between the credential and the customer's face has fallen below a specified threshold.
  • the threshold can be set based on factors such as, but not limited to the level of security required (e.g., hotel room may have a higher required security level than a hotel gym), and a tolerance level of the hotel customer for a security breach.
  • the confidence level can remain high if the hotel customer is observed by the camera requesting an access action that is the same as access actions performed in the past by the hotel customer and within the same time frame.
  • the confidence level can be lowered if a hotel customer is observed by the camera requesting an access action that is different than an access action performed in the past by the hotel customer and/or within a different time frame.
  • a higher level of confidence can be assigned when the camera has a high level of confidence in the facial recognition results (e.g., have a clear reading of the hotel customer's face) and a lower level of confidence can be applied if the facial recognition results are ambiguous.
  • the confidence level can be lowered when facial characteristics change rapidly (e.g., a hotel customer shaves off a beard or mustache), and the confidence level may remain the same or close to the same when facial characteristics change more slowly (e.g., the machine learning can adapt when a hotel customer grows a beard and is seen by the camera every few days).
  • the confidence level reflects how sure the system is that the face in the camera matches the credential.
  • block 316 is performed and the analytics revokes permission for the hotel customer to perform the access action using facial recognition in place of the credential.
  • the access action is denied and processing continues at block 302 with the hotel customer accessing the lock using a credential. If it is determined at block 314 that the confidence level has not fallen below a threshold, then processing continues at block 312 and facial recognition is used to allow the hotel customer to perform the access action.
  • block 304 may include a continuous video stream to the server, and at block 306 a timestamp and event code is sent from the access control device to the server and in addition a picture is retrieved from the video stream at the time indicated by the timestamp to use in block 308 .
  • FIG. 4 a general schematic system diagram of a system 400 that applies image analytics and machine learning to a two-factor authentication locking system, is generally shown in accordance with an embodiment of the disclosure.
  • the system 400 generally includes a server 14 , an access control device 416 , an analytical service component 110 , and a camera 38 . Also shown in FIG. 4 are a license plate 420 , a hotel customer 118 , and an employee badge 422 .
  • Access control device 416 has similar components as access control device 16 of FIGS. 1 and 2 , with the addition of a keypad and logic to support two-factor authorization.
  • the access control device 416 requires both a credential and a PIN in order to allow a hotel customer to perform an access action such as accessing a hotel room or other restricted area.
  • the interaction and communication connections between the server 14 , the analytical service component 110 , the camera 38 , and the access control device 416 are the same as those described above with reference to FIG. 1 with the addition of functionality to support two-factor authorization.
  • an additional function performed by the camera 38 to support two-factor authorization includes fetching an expected PIN from the server based on recognizing a hotel customer 118 in the field of view of the camera 38 .
  • the camera 38 (or the server 14 ) sends the expected PIN to the access control device 416 so that it can verify a PIN entered by the hotel customer 118 .
  • the analytic service component 110 can also perform machine learning that associates a license plate(s) 420 or an employee badge 422 to a credential instead of or in addition, to the face of the hotel customer 118 .
  • FIG. 5 a flow diagram 500 illustrating a method of applying image analytics and machine learning to two-factor authentication locking systems is generally shown, in accordance with an embodiment of the disclosure.
  • the processing shown in FIG. 5 can eliminate the first step of two-factor authentication by not requiring the hotel customer to present a credential via a mobile device or by swiping a key card.
  • the processing shown in FIG. 5 is performed as part of block 312 of FIG. 3 once the analytics has granted permission for the hotel guest 118 to use facial recognition in place of the credential.
  • a camera such as camera 38 of FIG.
  • the camera identifies the face of a hotel customer in its field of view, and at block 504 , the camera fetches an expected PIN for the hotel customer from a server, such as server 14 of FIG. 4 .
  • a server such as server 14 of FIG. 4 .
  • the camera takes the picture, and the analytic component, such as analytic service component 110 of FIG. 4 , identifies the face of the hotel customer and then signals the server to send expected PIN via the camera or straight to the lock.
  • the camera sends the expected PIN to an access control device, such as access control device 416 of FIG. 4 .
  • the hotel customer enters PIN at a keypad of the access control device.
  • the hotel customer is provided access to the lock if the expected pin is the same as the actual pin entered by the hotel customer.
  • the camera takes the picture, and the analytic component, such as analytic service component 110 of FIG. 4 , identifies the face of the hotel customer and then signals the server to send expected PIN via the camera or straight to the lock.
  • the PIN is entered and sent to the server, and the camera data is sent to the server.
  • the analytical component determines the face, and the server looks up the expected PIN and compares it with the entered PIN and then sends a signal back to the access control to ‘open’.
  • the camera 38 can be trained to recognize a car license plate 420 and/or an employee badge 422 of the hotel customer 118 along with or instead of the face of the hotel customer using machine learning techniques as described above.
  • the license plate 420 and/or the employee badge 422 can be used in place of the credential to identify the hotel customer 118 .
  • Embodiments can be used in commercial domains such as, but not limited to: offices, schools, and bank lockers. Embodiments can also be used in elevator systems. For example, when a hotel customer 118 registers at a hotel his/her picture is taken and a server is updated with picture. Once a pattern of access is detected as described herein, the hotel customer 118 the hotel customer 118 can be granted access to the elevator car based on facial recognition and without having to show a credential.
  • embodiments can be in the form of processor-implemented processes and devices for practicing those processes, such as a processor.
  • Embodiments can also be in the form of computer program code containing instructions embodied in tangible media, such as network cloud storage, SD cards, flash drives, floppy diskettes, CD ROMs, hard drives, or any other computer-readable storage medium, wherein, when the computer program code is loaded into and executed by a computer, the computer becomes a device for practicing the embodiments.
  • Embodiments can also be in the form of computer program code, for example, whether stored in a storage medium, loaded into and/or executed by a computer, or transmitted over some transmission medium, loaded into and/or executed by a computer, or transmitted over some transmission medium, such as over electrical wiring or cabling, through fiber optics, or via electromagnetic radiation, wherein, when the computer program code is loaded into an executed by a computer, the computer becomes an device for practicing the embodiments.
  • the computer program code segments configure the microprocessor to create specific logic circuits.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Lock And Its Accessories (AREA)
US17/057,129 2018-07-10 2019-07-10 Applying image analytics and machine learning to lock systems in hotels Active US11205312B2 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
IN201811025760 2018-07-10
IN201811025760 2018-07-10
PCT/US2019/041135 WO2020014311A1 (fr) 2018-07-10 2019-07-10 Application d'une analyse d'image et d'un apprentissage machine à des systèmes de verrouillage dans des hôtels

Publications (2)

Publication Number Publication Date
US20210201609A1 US20210201609A1 (en) 2021-07-01
US11205312B2 true US11205312B2 (en) 2021-12-21

Family

ID=67441759

Family Applications (1)

Application Number Title Priority Date Filing Date
US17/057,129 Active US11205312B2 (en) 2018-07-10 2019-07-10 Applying image analytics and machine learning to lock systems in hotels

Country Status (3)

Country Link
US (1) US11205312B2 (fr)
EP (1) EP3821404A1 (fr)
WO (1) WO2020014311A1 (fr)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102501245B1 (ko) * 2018-02-14 2023-02-17 삼성전자주식회사 키를 제공하기 위한 전자 장치 및 방법
US11900744B2 (en) * 2019-03-22 2024-02-13 Eingot Llc Virtual intercom system
US20200372743A1 (en) * 2019-05-20 2020-11-26 Popid, Inc. Face based door entry
AU2020280017B2 (en) * 2019-05-21 2023-05-25 Hid Global Corp. Physical access control systems and methods
WO2021028745A1 (fr) * 2019-08-14 2021-02-18 Carrier Corporation Système et procédé de fourniture d'accès à un utilisateur
US11709828B2 (en) * 2019-10-31 2023-07-25 Genetec Inc Method and system for associating a license plate number with a user
JP7211399B2 (ja) * 2020-06-25 2023-01-24 横河電機株式会社 装置、方法およびプログラム
SE2051379A1 (en) * 2020-11-26 2022-05-27 Assa Abloy Ab Configuring access rights for an electronic key
US11763613B2 (en) * 2021-03-08 2023-09-19 Johnson Controls Tyco IP Holdings LLP Automatic creation and management of digital identity profiles for access control
US20220407852A1 (en) * 2021-06-21 2022-12-22 Jio Platforms Limited Access control system and method thereof
CN118119939A (zh) * 2021-10-07 2024-05-31 视觉数据有限公司 电子访问控制系统

Citations (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020191817A1 (en) 2001-03-15 2002-12-19 Toshio Sato Entrance management apparatus and entrance management method
US20040046641A1 (en) 2002-09-09 2004-03-11 Junqua Jean-Claude Multimodal concierge for secure and convenient access to a home or building
US7110580B2 (en) 2000-05-19 2006-09-19 Nextgenid, Inc. Distributed biometric access control method and apparatus
US20140240504A1 (en) 2011-11-10 2014-08-28 Hee Moon Cho Door lock apparatus
US20140294255A1 (en) 2013-03-26 2014-10-02 Diehl Aerospace Gmbh Method for controlling the authorization of a person to access a secure area
CN104573457A (zh) 2014-12-30 2015-04-29 东莞市高明企业服务有限公司 一种基于云平台的智能门控制系统及其控制方法
US9058025B2 (en) 2011-03-24 2015-06-16 Recludo Ab Standalone biometric authorization control device and method
CN104821022A (zh) 2015-03-30 2015-08-05 无锡市崇安区科技创业服务中心 一种基于指纹及手势识别的门锁控制系统
US20150326570A1 (en) * 2014-05-09 2015-11-12 Eyefluence, Inc. Systems and methods for discerning eye signals and continuous biometric identification
CN204791242U (zh) 2015-07-21 2015-11-18 吴志华 一种智能门锁系统
US9235943B2 (en) 2013-07-26 2016-01-12 Joseph Frank Scalisi Remote identity verification of lodging guests
CN205450596U (zh) 2015-12-30 2016-08-10 苏州绿尚智能科技有限公司 家庭智能访客管理系统
CN106203647A (zh) 2016-06-30 2016-12-07 李玉婷 一种酒店网络化管理方法
CN106327627A (zh) 2016-08-10 2017-01-11 朱哲宇 一种锁具管理方法、门禁终端、服务器及系统
CN205910753U (zh) 2016-08-10 2017-01-25 朱哲宇 一种电子锁系统
US9652917B2 (en) 2014-03-12 2017-05-16 August Home, Inc. Intelligent door lock system with automatic unlock
US9822553B1 (en) 2016-11-23 2017-11-21 Gate Labs Inc. Door tracking system and method
CN107424085A (zh) 2017-08-01 2017-12-01 深圳市益廷科技有限公司 一种酒店服务系统
CN107480438A (zh) 2017-08-01 2017-12-15 深圳市益廷科技有限公司 一种酒店客户健康状况监控方法
US9888216B2 (en) 2015-09-22 2018-02-06 SkyBell Technologies, Inc. Doorbell communication systems and methods

Patent Citations (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7110580B2 (en) 2000-05-19 2006-09-19 Nextgenid, Inc. Distributed biometric access control method and apparatus
US20020191817A1 (en) 2001-03-15 2002-12-19 Toshio Sato Entrance management apparatus and entrance management method
US20040046641A1 (en) 2002-09-09 2004-03-11 Junqua Jean-Claude Multimodal concierge for secure and convenient access to a home or building
US9058025B2 (en) 2011-03-24 2015-06-16 Recludo Ab Standalone biometric authorization control device and method
US20140240504A1 (en) 2011-11-10 2014-08-28 Hee Moon Cho Door lock apparatus
US20140294255A1 (en) 2013-03-26 2014-10-02 Diehl Aerospace Gmbh Method for controlling the authorization of a person to access a secure area
US9235943B2 (en) 2013-07-26 2016-01-12 Joseph Frank Scalisi Remote identity verification of lodging guests
US9652917B2 (en) 2014-03-12 2017-05-16 August Home, Inc. Intelligent door lock system with automatic unlock
US20150326570A1 (en) * 2014-05-09 2015-11-12 Eyefluence, Inc. Systems and methods for discerning eye signals and continuous biometric identification
CN104573457A (zh) 2014-12-30 2015-04-29 东莞市高明企业服务有限公司 一种基于云平台的智能门控制系统及其控制方法
CN104821022A (zh) 2015-03-30 2015-08-05 无锡市崇安区科技创业服务中心 一种基于指纹及手势识别的门锁控制系统
CN204791242U (zh) 2015-07-21 2015-11-18 吴志华 一种智能门锁系统
US9888216B2 (en) 2015-09-22 2018-02-06 SkyBell Technologies, Inc. Doorbell communication systems and methods
CN205450596U (zh) 2015-12-30 2016-08-10 苏州绿尚智能科技有限公司 家庭智能访客管理系统
CN106203647A (zh) 2016-06-30 2016-12-07 李玉婷 一种酒店网络化管理方法
CN106327627A (zh) 2016-08-10 2017-01-11 朱哲宇 一种锁具管理方法、门禁终端、服务器及系统
CN205910753U (zh) 2016-08-10 2017-01-25 朱哲宇 一种电子锁系统
US9822553B1 (en) 2016-11-23 2017-11-21 Gate Labs Inc. Door tracking system and method
CN107424085A (zh) 2017-08-01 2017-12-01 深圳市益廷科技有限公司 一种酒店服务系统
CN107480438A (zh) 2017-08-01 2017-12-15 深圳市益廷科技有限公司 一种酒店客户健康状况监控方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
International Search Report and Written Opinion for International Application No. PCT/US2019/041135, International Filing Date: Jul. 10, 2019; dated Oct. 17, 2019; 20 pages.

Also Published As

Publication number Publication date
US20210201609A1 (en) 2021-07-01
EP3821404A1 (fr) 2021-05-19
WO2020014311A1 (fr) 2020-01-16

Similar Documents

Publication Publication Date Title
US11205312B2 (en) Applying image analytics and machine learning to lock systems in hotels
US10997809B2 (en) System and method for provisioning a facial recognition-based system for controlling access to a building
CN107018124B (zh) 用于控制访问的远程应用
US11232666B1 (en) Verified access to a monitored property
EP3704642B1 (fr) Procede et systeme de controle de l'acces aux ressources d'entreprise sur la base d'un suivi
US7907753B2 (en) Access control system with symbol recognition
US10492066B2 (en) Access and automation control systems with mobile computing device
KR101814719B1 (ko) 이동 단말을 이용한 디지털 도어락 원격제어 시스템 및 방법
CN111401159A (zh) 酒店鉴权管理方法、系统、酒店管理系统以及用户终端
WO2008066130A1 (fr) Dispositif de contrôle d'accès, système de contrôle d'accès et procédé de contrôle d'accès
US10673844B2 (en) Method for providing an access code on a portable device and portable device
US20220254212A1 (en) Systems and techniques to provide smart access capabilities in a smart system environment
CN111373453A (zh) 具有无线电和人脸识别机制的入口监控系统
KR102143716B1 (ko) Rf-card 기반의 출입통제시스템
CN112714928A (zh) 受理终端机
TWI610273B (zh) 自主式訪客系統與其主機
US11113374B2 (en) Managing seamless access to locks with person/head detection
US20200026829A1 (en) Biometric access control identification card
KR20180095404A (ko) 무인 숙박시설 시스템 제어 방법
KR102019093B1 (ko) 코드표시를 통한 차량 출입통제 시스템 및 방법
JP2017173890A (ja) 入退室管理装置及び入退室管理方法
US20220198861A1 (en) Access control system screen capture facial detection and recognition
CN112689856A (zh) 管理系统及数据中心
JP7316982B2 (ja) 顔認証サーバ及び情報処理方法
US20240046729A1 (en) Auto-programming door and camera relationships for a security system

Legal Events

Date Code Title Description
AS Assignment

Owner name: CARRIER CORPORATION, FLORIDA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:UTC FIRE & SECURITY INDIA LTD.;REEL/FRAME:054425/0922

Effective date: 20190416

Owner name: UTC FIRE & SECURITY INDIA LTD., INDIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:AMUDURI, SANTHOSH;LINGALA, RAMESH;REEL/FRAME:054425/0894

Effective date: 20180712

Owner name: CARRIER CORPORATION, FLORIDA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:KUENZI, ADAM;REEL/FRAME:054425/0850

Effective date: 20180710

FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE