KR102233265B1 - A method for controlling access based on the user's biometric information based on deep learning, and an electronic lock system access control system for the same - Google Patents

Abstract

In an embodiment, provided is a verification device for sensing biometric information of a user in order to determine whether the door is locked or unlocked by wirelessly interlocking with the door lock device installed on the door. The device comprises the steps of: detecting an object based on a photographed image; determining, by the verification device, whether the detected object is in proximity; identifying, by the verification device, the adjacent object as a user and detecting a face region of the object; measuring a body temperature of the user by detecting a heat sensing region within the face region displayed within a guide line displayed in a display image to detect a temperature within the heat sensing region; and detecting whether the mask is photographed by analyzing the face region of the user to output an alarm message according to the non-wearing of the mask of the user.

Description

A method for controlling access based on the user's biometric information based on deep learning, and an electronic lock system access control system for the same}

The present invention relates to a general electronic device, and more particularly, to a device for identifying a user based on deep learning and approving a user's access between barriers based on the user's biometric information.

In the modern society, as technology develops day by day, not only our lives are convenient and prosperous, but also various technologies and systems are being developed so that anyone can enjoy a clean and hygienic life.

In particular, through industrialization in the 20th century, Korea builds social overhead capital such as electricity, telecommunications, water and sewage, roads, ports, and airports, expands social systems such as national health insurance, national pension, and compulsory education, and provides various industrial technologies, Due to the development of life technology and medical technology, the average life span of the Korean people has increased significantly, enabling them to lead a healthy and comfortable life. However, due to the SARS in 2003, MERS in 2015, and Corona virus 19 in 2020, many people living in the world as well as in Korea are suffering from viruses and new diseases. I realized that I couldn't be free from the dangers. In particular, in the era of globalization in the 21st century, there is a limit to effectively coping with new infectious diseases that continue to emerge only with the efforts of a single country, so cooperation and information disclosure between countries are increasing in importance.

On the other hand, after a certain period of time after the infectious disease occurs, the pandemic period comes. Until the vaccine is developed, the suspected infection is first selected to control access to the building and the identification information of people is managed to quickly establish a database for self-isolation in the future. The best way to prevent the spread of infectious diseases is to select and quarantine people suspected of being infected as much as possible.

These infectious diseases are manifested in various ways depending on the type, but since it is the most common infection symptom and causes high fever, thermal imaging cameras are installed at the entrances of places such as airports, buildings, government offices, hospitals, etc. The introduction of a method of separately detecting a high fever among those entering through the thermal image obtained by the method is increasing.

In addition, even if a high fever person is detected, the conventional method has a disadvantage that not only the time required for the follow-up response is delayed but also the quarantine efficiency is significantly lowered because the follow-up measures are not performed collectively, but individually performed by the manager.

Korean Registered Patent Publication No. 10-2136837 Korean Registered Patent Publication No. 10-2168651 Korean Registered Patent Publication No. 10-2135633

In the present invention, the door lock device and the verification device are interlocked to control entry and exit in the building, collect data for management of disease spread according to input/output history management, and quickly and accurately detect user identification information based on deep learning for disease management. We would like to provide a method and system that can be done.

In an embodiment, a verification device for sensing biometric information of a user in order to determine whether the door is locked or unlocked by wirelessly interlocking with a door lock device installed on a door includes: detecting an object based on a photographed image; The verification device determining whether the detected object is in proximity; The verification apparatus includes the steps of identifying the adjacent object as a user and detecting a face area of the object; Measuring a body temperature of the user by detecting a temperature in the heat sensing region by detecting a heat sensing region in the face region displayed within a guide line displayed in a display image; A method of controlling access based on the user's biometric information, including the step of detecting whether or not the mask is photographed by analyzing the user's face area in order to output an alarm message according to the user's non-wearing of the mask may be provided.

In another aspect, the verification device further comprises the step of recognizing the user's fingerprint through a fingerprint sensor provided on the upper keypad whenever the user inputs a signal to the keypad of the door lock device. This can provide a way to control access.

In another aspect, the verification device further comprises: estimating the height of the user, and based on the estimation result of the height, the user's fingerprint and the keypad are adjusted by adjusting an angle formed by the surface of the keypad with the ground. It is possible to provide a method of controlling access based on the user's biometric information that increases the contact area of the surface of the user.

In another aspect, when the body temperature of the user exceeds a preset temperature, a method of controlling access based on biometric information of the user adjusting the angle of the keypad so that the surface of the keypad faces the ground can be provided. have.

In another aspect, it is possible to provide a method of controlling access based on the user's biometric information, further comprising: detecting the user's identification information based on the user's fingerprint information and managing the entry/exit personnel history in the building. have.

In another aspect, sensing the body temperature of the user exiting the building; And if it is determined that the body temperature of the user is abnormal, maintaining the lock of the door lock device and displaying a message informing the movement to a pre-prepared place; providing a method of controlling access based on the user's biometric information further comprising I can.

In another aspect, identifying a user by comparing characteristic information of a user, which is detected and stored when entering and exiting the building, with characteristic information of a user exiting the building; And Providing a method for controlling access based on the user's biometric information further comprising the step of detecting information of the verified people to enter the building from the time of entering the building to the present by the identified user and displaying a warning message. can do.

The embodiment detects biometric information of a user entering and exits a building using a verification device, and controls entry and exit of a building by locking or unlocking a door lock device wirelessly connected to the verification device based on the detected biometric information. I can.

The embodiment may provide an entry/exit control system capable of detecting whether a user entering a building is wearing a mask and outputting a result message accordingly.

In the embodiment, through the interlocking of the door lock device and the verification device, while easily controlling the entry and exit of people into the building, the spread of disease infection can be minimized, and the entry and exit history of people can be managed.

In the embodiment, even in the case of a user wearing a mask, body temperature may be measured through a heat sensing region near the tear gland in the face region.

According to the embodiment, by accurately identifying the user by detecting the user's fingerprint information, negativity when identifying a user wearing a mask by facial recognition may be improved.

According to the embodiment, by adjusting the angle of the keypad according to the estimated height of the user, the contact area between the user's fingerprint and the keypad may be increased, thereby accurately detecting the user's fingerprint information.

In the embodiment, a user entering a building and a user leaving the building are identified and specified, and when the body temperature of the user is abnormal, other people during the entering/exiting time of the user are identified, and the corresponding other people are warned to immediately move to a predetermined place. By doing so, the spread of disease infection can be minimized.

In the embodiment, in the case of the body temperature of people entering the building from the outside of the building, the accuracy of the measured body temperature information is poorly affected by the temperature of the external environment, so that the building is constructed to solve the problem of being detected as normal even though the body temperature is abnormal. Provides a method and system for measuring body temperature for a person going out.

1 shows the relationship between the overall configurations of the access control system according to an embodiment of the present invention.
2 and 3 are exemplary views installed in a building in an access control system having a verification component.
4A is an exemplary block diagram of an access control system according to an embodiment of the present invention.
4B and 4C are exemplary diagrams for explaining an interaction between a verification device and a user.
5 is an exemplary view of an access control system according to another embodiment of the present invention.
6 is a flowchart illustrating a method of controlling access based on biometric information of a user according to an embodiment of the present invention.
7 and 8 schematically show that the driving system adjusts the angle of the keypad based on the user's height estimation result, and the user's face region and body temperature information are displayed in the verification device.
9 to 12 are exemplary diagrams of screens displayed on the display of the verification device.
13 is a flowchart illustrating a method of controlling access based on biometric information of a user according to an embodiment of the present invention.
14 is an exemplary diagram for various types of information displayed through a display of a verification device.
15 is for explaining management information on the entry/exit history on the management server.
17 and 18 are exemplary views of a user interface of software for a verification device provided to set a face database through a management server.
19 depicts a user interface according to the addition of a person.
20 depicts a user interface showing log views and access records.

Since the present invention can apply various transformations and have various embodiments, specific embodiments are illustrated in the drawings and will be described in detail in the detailed description. Effects and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various forms. In the following embodiments, terms such as first and second are used for the purpose of distinguishing one constituent element from other constituent elements rather than a limiting meaning. In addition, expressions in the singular include plural expressions unless the context clearly indicates otherwise. In addition, terms such as include or have means that the features or components described in the specification are present, and do not preclude the possibility of adding one or more other features or components in advance. In addition, in the drawings, the size of components may be exaggerated or reduced for convenience of description. For example, the size and thickness of each component shown in the drawings are arbitrarily shown for convenience of description, and thus the present invention is not necessarily limited to what is shown.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, and when describing with reference to the drawings, the same or corresponding components are assigned the same reference numerals, and redundant descriptions thereof will be omitted. .

1 shows the relationship between the overall configurations of the access control system according to an embodiment of the present invention. 2 and 3 are exemplary views installed in a building in an access control system having a verification component, and FIG. 4A is an exemplary block diagram of an access control system according to an embodiment of the present invention. In addition, FIGS. 4B and 4C are exemplary diagrams for explaining an interaction between a verification device and a user. In addition, Figure 5 is an exemplary view of an access control system according to another embodiment of the present invention.

Referring to FIGS. 1 to 5, the access control system 100 according to an embodiment of the present invention may include a door lock device 110, a verification device 120, and a network 300.

In some embodiments, the access control system 100 may further include a management server 200.

The door lock device 110 and the verification device 120 may be integrally configured, but are not limited thereto, and the door lock device 110 and the verification device 120 may be configured separately from each other, and they communicate wirelessly with each other. can do.

The verification device 120 may display operation state information of the user p and operation state information of the door lock device 110. In addition, the verification device 120 is one for providing a user interface for displaying the state of the user p, such as body temperature information, and providing an option for interacting with the user p or a greeting to the user p. The above touch or non-touch interactive panel or display 199 may be provided. In addition, the verification device 120 verifies the user p and provides a user interface for communicating with other electronic devices or various sensors (built-in, implantable sensors, chemical or biosensors, and wearable sensors) in addition to the door lock device 110. In addition, the verification device 120 may include a touch screen, a mechanical switch, a touch detection switch, a display, a graphic and/or a touch interface, a virtual assistant (eg, an AI device that can be operated by voice), a sensor, a camera, It may include a receiver, a transmitter, and the like.

The sensor provided in the verification device 120 is a device for detecting and collecting the user's biometric information through touch and non-touch interactions such as facial recognition of the user (p), palm characteristics, voice, body size, color, and shape. It may include a device for detecting other personal identification information such as, posture, and the like.

In addition, the sensor provided in the verification device 120 may further include other sensors such as, for example, a motion sensor, a camera and a proximity sensor, and an ambient light sensor as an environmental sensor.

In some embodiments, at least some of the above-described sensors may be provided in the door lock device 110.

In addition, the verification device 120 may be a device to which a deep learning facial recognition AI algorithm is applied. In addition, the verification device 120 may have a built-in device for measuring the temperature of the hot screen.

In addition, the verification device 120 may support face recognition, mask detection, face recognition comparison, temperature measurement, and the like. In addition, the verification device 120 may accurately select individual information such as face information through a camera viewing angle, body surface temperature, and whether or not to wear a mask.

In addition, the verification device 120 may effectively help prevent infectious diseases such as access rights, body surface temperature status monitoring, and the like for users (p).

In some embodiments, the access control system 100 may include various light emitting systems for providing convenient night lighting functions at the entrance in order to provide better lighting conditions for collecting biometric information and video images.

The access control system 100 includes one or more disposable batteries or rechargeable batteries, such as nickel cadmium (NiCd), lithium (Li), AA, AAA, and/or rechargeable capacitors, such as super capacitors (SC) or ultra A capacitor may be included, and power required for driving the door lock device 110 and the verification device 120 may be provided through this. According to some embodiments, the access control system 100 may receive power by wire.

The access control system 100 may have any common interface such as a Universal Serial Bus (USB) interface that enables communication between different electronic devices and provides support for power.

The access control system 100 may include a sensor device 130 for collecting authentication information and an appropriate hardware and communication function for processing the collected user interaction, input, authentication information, battery charging, and communication with an external device. I can.

The access control system 100 may perform one or more messaging, text, and video display and unlocking or locking operations.

The access control system 100 uses various touch and non-touch input devices to authenticate the user (p) using the biometric information of the user (p) and perform an operation based on the interaction and authentication information of the user (p). May include any computing device having. As the authentication information, various entity interactions such as face and image recognition information, other biometric information, auditory information, body size information, color information, shape information, and posture information may be possible.

The access control system 100 may include interfaces for various types of computer-readable media, such as, for example, non-transitory computer-readable media, and various other types of computer-readable media.

The sensor device 130 of the access control system 100 may include a camera 190. The camera 190 may include a thermal imaging camera 191, a depth measuring camera 192, and an RGB image camera 193.

One or more of the remote sensor devices 180 may partially or wholly include components of the sensor device 130 and may facilitate collection. The access control system 100 may be composed of an electronic device having a processor, memory, network interface, GPS, power supply, etc. communicatively coupled to one or more sensors.

For example, the access control system 100 may include a main processor 142 and a memory 150. The main processor 142 may retrieve and execute the instruction 151 and/or data 152 from the memory 150 to perform the processor according to the embodiment. The main processor 142 may be a single processor, a multi-core processor, or multiple processors in different implementations.

Commands and data for operating the access control system 100 may be stored, transmitted, or received from any computer-readable storage medium.

The memory 150 may include dynamic random access memory (DRAM) and/or read-only memory (ROM). Memory 150 may provide a temporary location for storing data 152 and instructions 151 retrieved and processed by main processor 142.

The memory 150 may store various types of authentication information or environment information collected, and may be transmitted to other devices through a wireless connection.

The access control system 100 is connected to the network 300 through a network interface 161. In some embodiments, the network interface 161 allows the wireless control unit 162 to be integrated into the network device 160 and coupled to an antenna.

The network interface 161 may include a radio frequency (RF) circuit. The RF circuit may optionally include networks such as the Internet, intranet, and/or wireless network, also referred to as the World Wide Web (WWW), such as a cellular telephone network, a wireless local area network (LAN), and/or a metropolitan network ( metropolitan area network (MAN), and other devices by wireless communication. Wireless communication, optionally, GSM (Global System for Mobile Communications), EDGE (Enhanced Data GSM Environment), HSDPA (high-speed downlink packet access), HSUPA (high-speed uplink packet access), EV-DO (Evolution, Data-Only), HSPA, HSPA+, DC-HSPDA (Dual-Cell HSPA), LTE (long term evolution), NFC (near field communication), W-CDMA (wideband code division multiple access), CDMA (code division multiple access) ), time division multiple access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g and/or IEEE 802.11n), voice over Internet Protocol (VoiP) , Wi-MAX, protocol for email (e.g., Internet message access protocol (IMAP) and/or post office protocol (POP)), instant messaging (e.g., extensible messaging and presence protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions), Instant Messaging and Presence Service (IMPS)), and/or Short Message Service (SMS), or any other suitable communication protocols not yet developed as of the filing date of this document. Multiple communication standards, protocols and technologies, including but not limited to communication protocols Use any of these.

The sensor device 130 may be connected to the access control system 100, formed thereon, or remotely connected thereto.

The sensor device 130 may include and control various sensor configurations for sensing authentication information and environmental information such as temperature, humidity, air quality, smoke, and various gases.

These sensor configurations use one or more condition sensors 135, feature sensors 135, motion sensors 135a, cameras 190, and other sensors 135c to monitor various environmental information and adjust sensors appropriately for authentication. Information can be collected. In some embodiments, the user's body temperature information detected through the thermal imaging camera 191 may be corrected based on external temperature information. In addition, the motion sensor 135a may recognize motion by detecting, for example, a direction and a moving speed of an object in front of the electronic locking system 100.

The access control system 100 may collect additional authentication information from one or more remote sensor devices 180.

The biometric information and the characteristic information of the user p may be analyzed by the biometric recognition/AI/neural processor 141.

The sensor device 130 may include a speaker 131 for displaying status information according to the operation of the door lock device 110 and the verification device 120, and determination result information according to the biosignal recognition of the user p. have. In addition, for example, a microphone 132 for collecting user p or external sound information may be provided. In addition, the sensor device 130 may include an audio encoder/decoder 133 and a codec 134 for processing sound.

In some embodiments, the sensor device 130 may further include an NFC (135d). The user can lock or unlock the door lock device 110 by exchanging an electrical signal with the NFC 135d of the door lock device 110 by using the NFC function of a terminal such as a tag of an NFC card or a smartphone. have.

The sensor device 130 may be controlled by the controller 136, and results collected from various internal components may be transmitted to the controller 136.

In some embodiments, the access control system 100 may further include a GPS processor 143 for analyzing the detected GPS information.

The access control system 100 may include a drive system 112 in order to adjust the angle of the surface of the keypad 111 on the door lock device 110 with the ground. The keypad 111 may be formed of a touch panel equipped with a fingerprint sensor. In addition, the biometric recognition/AI/neural processor 141 may analyze detection information of a fingerprint sensor on the keypad 111 when a finger is in contact with the keypad 111.

6 is a flowchart illustrating a method of controlling access based on biometric information of a user according to an embodiment of the present invention. 7 and 8 schematically show that the driving system adjusts the angle of the keypad based on the user's height estimation result, and the user's face region and body temperature information are displayed in the verification device.

Referring to FIG. 6, a method (S100) of controlling access based on a user's biometric information according to an embodiment of the present invention includes an object detection step (S110), an object proximity determination step (S120), object identification and face area. Detecting step (S130), height estimating step (S140), heat sensing area detecting step (S150) according to the face area, detecting the user's body temperature within the heat sensing area (S160), adjusting the position of the keypad according to the estimated height (S170), collecting the user's identification information (S180), and maintaining or releasing the lock of the door lock device (S190).

Hereinafter, each step will be described in detail.

-Object detection step (S110)

The access control system 100 may detect an object in front of the camera 190.

The biometric recognition/AI/neural processor 141 may determine whether an object is detected through a comparison between an image in front of the access control system 100 that has been learned in advance and an image currently captured by the camera 190.

In some embodiments, the main processor 142 may detect an object in front by analyzing an image captured by an RGB camera. In another aspect, the main processor 142 may detect an object in front using a depth measuring camera. In detail, a pixel whose depth value is maintained may be determined as a background pixel, and a pixel whose depth value is changed may be determined as a depth value of an object.

-Determining whether the object is in proximity (S120)

When an object in front is detected, the access control system 100 may determine whether the object is in proximity.

In detail, the access control system 100 may analyze a photographed image in front to determine whether or not the detected object is close to a predetermined speed or higher.

When the user p proceeds toward the door 10, the access control system 100 may determine that the object approaches the object at a preset speed or higher. In contrast, when the user p passes the door 10, the access control system 100 does not determine that the object is close.

In some embodiments, the main processor 142 may determine whether or not the detected object approaches the depth measurement camera at a predetermined speed or higher by analyzing a change in the depth value in the captured image from the depth measurement camera. In detail, when an object is detected on the background area, the depth value gradually decreases around the plurality of pixels in the detected object, and when the number of surrounding pixels of the plurality of pixels increases, it may be determined that the object approaches.

-Object identification and face region detection step (S130)

The access control system 100 may determine whether the object is a person by analyzing the image of the photographed object when it is determined that the object approaches the forward speed more than a preset speed. In this case, the learned data of an image of a person may be used.

In some embodiments, the biometric recognition/AI/neural processor 141 stores learning information about the images of people on the background in front, and whether a person is detected in the captured image of the camera 190 based on the learning information. Can be judged.

In addition, the biometric recognition/AI/neural processor 141 may detect a face region of a human object through a CNN neural network learned in advance from an image captured by the RGB image camera 193. In this case, a Mask R-CNN deep learning neural network can be applied to detect an accurate face region.

-Height estimation step (S140)

When the object is identified as a person, the access control system 100 may estimate the height of the user according to the height estimation algorithm.

The height estimation algorithm will be described in detail.

1) body detection stage

The access control system 100 may detect a body by comparing a depth screen acquired by real-time photographing through the depth measurement camera 192 and a background depth screen. The access control system 100 may generate a binarized screen by setting a pixel value of a pixel having a depth value equal to or greater than a preset value (binarization threshold) to 1 and a pixel value of another depth value to 0.

2) Estimate height by 3D transformation of depth image

The height can be estimated by transforming the body into a three-dimensional structure through depth information indicating distance.

Height can be calculated as a distance in a three-dimensional space between the head and toe points in the extracted body area. The head and toe points are detected as the topmost pixel of the detected face area and the bottommost pixel of the body area, respectively. If a plurality of pixels exist at the top of the face area, the head point may be detected as an intermediate pixel of the corresponding pixel. In addition, when a plurality of pixels are present at the lowermost end of the toe region, the toe may be detected as an intermediate pixel of the pixel.

Height can be estimated by measuring the real-world length between the head and toe points. In order to measure the length in the real world between the two pixels of the crown of the head and the tip of the toe, the 2D image coordinates of the two pixels of the crown of the head and the tip of the toe may be converted into 3D coordinates. In addition, the height can be estimated by calculating the Euclidean distance between the two pixels of the head and the tip of the head converted into 3D coordinates.

-Detecting the heat detection area according to the face area (S150)

The access control system 100 may designate a heat detection area within the detected face area and track the corresponding heat detection area.

In addition, the access control system 100 may detect the tip of the nose within the detected face area. The tip of the nose may be the point with the smallest depth value. In addition, the nose region including the tip of the nose may have a shape in which the depth value of the pixels at the periphery thereof decreases sharply around the tip of the nose. Accordingly, the access control system 100 determines whether the detected pixel corresponds to the pixel at the tip of the nose by determining whether the depth value of the pixel at the periphery is sharply decreased around the pixel at the point with the smallest depth value. I can.

In addition, the access control system 100 designates a box area in which the end point of the head of the head in the detected face area is the uppermost middle pixel, and the pixel at the detected nose end point is the lowermost middle pixel. It can be designated as a detection area.

The heat sensing area may be a rectangular area centered on a pixel corresponding to a point of 2/3 vertically downward from the end point of the head and 1/3 of the vertical top from the tip of the nose as a center. In addition, the access control system 100 may adjust the focus of the thermal imaging camera 191 while continuously tracking the blind area, which is the heat detection area, and maintain the focus with the blind area.

-Detecting the user's body temperature within the heat detection area (S160)

The access control system 100 may detect the body temperature of the user p by sensing heat in the vicinity of the tear glands of the user p within the blind area, which is the heat detection area of the user p.

The access control system 100 may detect a region near the tear gland in the captured image through a CNN neural network of a rectangular region, which is a previously learned RGB image.

The access control system 100 may detect body temperature a plurality of times in an area near the lacrimal gland and determine a value of the highest body temperature among the detected body temperatures as the user's body temperature.

-Adjusting the position of the keypad according to the estimated height (S170)

The access control system 100 may proceed with a user identification information collection procedure when the body temperature of the user p is less than a preset body temperature value according to the body temperature detection of the user p.

For the user (p)'s identification information collection procedure, the access control system 100 adjusts the position of the keypad 111 of the door lock device 110 based on the estimated height value of the user p The contact area of the surface of (111) can be increased.

In detail, the angle formed by the keypad 111 with the ground based on the height of the user p may be changed. If the height of the user p is small, as shown in FIG. 8, the driving system 112 can increase the angle formed by the surface of the keypad 111 with the ground. In contrast, as shown in FIG. 7, as the height of the user increases, the angle formed by the keypad 111 from the ground may decrease.

In some embodiments, the access control system 100 determines that the body temperature of the user p is abnormal, so that the surface of the keypad 111 is out of the field of view of the user p. By changing the angle of the keypad 111 so that it faces, it is possible to prevent a user from touching the surface of the keypad 111.

-Collecting user identification information (S180) and locking or unlocking the door lock device (S190)

A user with normal body temperature (p) may touch a finger on the surface of the keypad 111 in order to input a password on the keypad 111 of the door lock device 101.

The access control system 100 not only determines whether the password input to the keypad 111 matches the previously stored password for unlocking, but also detects fingerprint information of the finger when the finger is touched on the surface.

In addition, since the password consists of a plurality of numbers, a fingerprint can be detected from the user's finger whenever the user touches the keypad 111 a plurality of times.

The access control system 100 may determine whether the user is a pre-registered user by identifying the user's identity through the detected fingerprint.

In some embodiments, the access control system 100 may manage the history of personnel entering and leaving the building by identifying the user's identity.

In some embodiments, the access control system 100 may transmit the detected fingerprint information to a terminal possessed by the user p. The terminal determines whether the fingerprint information received from the access control system 100 matches the previously stored fingerprint information of the user (p), and if the fingerprint information matches, transmits the identification information of the user (p) to the access control system 100. I can.

The access control system 100 may manage the history of personnel entering and exiting the building based on the received identification information of the user p.

In some embodiments, the door lock device 110 may be installed on a door of a building in which the public enters and exits. In this case, the verification device 120 may display a message to touch a specific area on the keypad 111. In addition, the user p may input a signal by touching specific buttons on the keypad 111 rather than inputting a password. In addition, the access control system 100 may detect the identification information of the user p by recognizing the user's fingerprint when the user p touches specific buttons. When the detection of the identification information of the user p is completed, the access control system 100 unlocks the door lock device 110 so that the user p can enter the building.

Meanwhile, in the embodiment, a user may be identified according to the user's facial recognition, and entry/exit may be controlled through a two-step verification process by recognizing a user's fingerprint and inquiring for additional user identification information.

In some embodiments, the terminal determines whether the fingerprint information received from the access control system 100 matches the previously stored fingerprint information of the user (p), and if the fingerprint information matches, whether the user (p) is vaccinated against a disease. Information about may be transmitted to the verification device 120.

The verification device 120 may allow the door lock device 110 to be unlocked when the vaccination of the user p is confirmed based on information on whether or not the user p is inoculated against a disease received from the terminal. have.

The user's medical information, such as whether or not to be vaccinated with a high level of security, can be stored only on the terminal of the user. And, after the user identification according to the user's facial recognition and the additional user identification according to the user's fingerprint recognition, that is, the user verification process through step 2, the verification device 120 transmits information on whether the user is vaccination from the terminal. Can be received from In addition, the verification device 120 may unlock the door lock device 110 when it is confirmed that the user has inoculated the vaccine. In addition, the verification device 120 may protect sensitive personal information of the user by permanently deleting information received from the terminal regarding whether or not the user is vaccinated.

9 to 12 are exemplary diagrams of screens displayed on the display of the verification device.

In some embodiments, a guide line gl may be displayed on the display 199 of the verification device 120.

The guide line (gl) is for guiding the user's proper position so that the verification device 120 becomes a suitable position for sensing the user's biometric information.

The user can set an appropriate position so that the photographed facial image cp of the user enters the area enclosed by the guide line gl.

The verification device 120 may measure the above-described body temperature of the user by analyzing the captured image on the guide line gl.

The measured user's body temperature may be displayed on the first area a1 of the display 199.

In some embodiments, the verification device 120 extracts the user's facial recognition information by analyzing the image within the guide line gl, and displays the matching and pre-stored user's facial image information on the second area a2. can do.

In some embodiments, the verification device 120 analyzes an image of a user's face wearing a mask to extract facial recognition information, and displays the matching and pre-stored user's facial image information in the second area (a2). can do. In addition, the verification device 120 may analyze the captured image to analyze whether the user photographs the mask m.

In some embodiments, if the mask m is not detected in the captured image cp of the user, the verification device 120 may output a message instructing to wear the mask.

In some embodiments, the access control system 100 may maintain the locked state of the door lock device 110 when the user's wearing of the mask is not detected through the verification device 120.

13 is a flowchart illustrating a method of controlling access based on biometric information of a user according to an embodiment of the present invention.

Referring to FIG. 13, a method of controlling access based on the user's biometric information in the process of a user entering the building going out of the building will be described.

A method (S200) of controlling access based on the user's biometric information according to an embodiment includes an object detection step (S210), an object proximity determination step (S220), an object identification and face area detection step (S230), and a height estimation step. (S240), detecting a heat detection area according to the face area (S250), detecting a user's body temperature within the heat detection area (S260), unlocking the door lock device when the body temperature is normal (S270), and the body temperature is normal If not, an emergency mode execution step (S280) may be included.

Object detection step (S210), object proximity determination step (S220), object identification and face area detection step (S230), height estimation step (S240), heat detection area detection step (S250) according to the face area, heat detection area In the step of detecting the user's body temperature (S260), an object detection step (S110) in FIG. 6, an object proximity determination step (S120), an object identification and face region detection step (S130), a height estimation step (S140) The access control system 100 may be driven in the same manner as the heat detection area detection step S150 according to the face area and the user's body temperature detection step S160 within the heat detection area.

If it is determined that the currently detected user's body temperature is normal, the access control system 100 may unlock the door lock device 110 and allow the user to exit the building (S270). In addition, the user's entry/exit history in the building may be managed using the identified user's information.

In addition, in the sensing of the user's body temperature (S260), the access control system 100 may execute the emergency mode when it is determined that the user's body temperature is abnormal (S280).

In detail, the emergency mode execution step (S280) may include a step S281 of maintaining the lock of the door lock device, a step of displaying a moving message to a previously prepared location (S282), and a step of identifying a user (S283).

The access control system 100 may maintain the lock of the door lock device 110 when the user's body temperature is abnormal. In addition, the access control system 100 may display a message prompting the user to immediately move to a pre-prepared place (S282). In addition, the access control system 100 may search for pre-stored feature information matching the user's feature information, such as identification information of the currently detected object, image information on the face region, and estimated height information. Here, the pre-stored feature information is feature information detected from people entering the building in the manner described with reference to FIG. 6. In addition, the access control system 100 externally transmits the information of all people identified by the verification device 120 to enter the building for a period of time from the time when the user's access into the building is permitted to the time when the current user is identified. Can be sent to the server. In this case, the server may display a warning message to promptly move to a pre-prepared place while guiding basic identification information of all related people through the display device inside the building.

14 is an exemplary diagram for various types of information displayed through a display of a verification device.

On the display 199 of the verification device 120, a captured image of the user, current body temperature information of the user, and information on the number of persons entering the building may be displayed. In addition, the verification device 120 may display information on a dense area of people in the building. The information on the crowded area of the people here is information generated based on the estimated number of people for each area determined based on information such as the number of terminals detected through various surveillance cameras and Wi-Fi communication networks installed in the building. In addition, information on a pre-designated place for quarantine of people who have entered the building with abnormal body temperature or suspected of being infected can be displayed.

In some embodiments, the verification device 120 may further display detailed information of the selected information in response to a touch input of any one of the displayed information.

15 is for explaining management information on the entry/exit history on the management server.

The management server 200 may manage the user's biometric information received from the verification device 120 and input/output history information according to the detection time point of the biometric information.

For example, the management server 200 may manage a visitor log, and may manage image information of a user who enters and exits, and information about an entry/exit time point.

For example, the visitor log information stored in the management server 200 may be used as information for managing attendance of employees.

16 depicts an example of a user interface for registering a verification device.

Referring to FIG. 16, in order to register the verification device 120 with the network 300, the verification device 120 is connected to a computing device, and the IP of the verification device 120 is searched by executing a search tool provided in the computing device. can do. In addition, the user may modify the IP address of the verification device 120 by using a tool. The registration process of the verification device 120 may be described by applying the same to the registration of the door lock device 110.

17 and 18 are exemplary views of a user interface of software for a verification device provided to set a face database through a management server. And, Figure 19 depicts a user interface according to the addition of a person. In addition, FIG. 20 depicts a user interface showing log view and access records.

Referring to FIG. 17, a face database may be set using software for a verification device, and a real-time video preview, a real-time comparison result, a device log read, and a device parameter may be set.

A description of each function on the user interface is as follows.

1) Video preview window: If you click the [Device Connection] button, you can show the real-time video input from the camera.

2) IP input box Allows you to enter the IP address of the device.

3) Connected/Disconnected Device Button: If you press the “Connect Device” button to connect to the verification device 120, you can preview the real-time video, capture a real-time snapshot, and display the recognition result on the screen. I can.

You can end the preview by clicking the "End Connection" button.

4) Comparison result: When the captured face matches the device's face database, it is displayed on the screen, and basic face information, matching rate, temperature, and other values can be displayed. If the face database does not match, it may not be displayed here.

5) Snap display bar: In the video preview mode, the face image and body temperature captured on the current screen are displayed.

6) Face management: By pressing the face management button, you can add, delete, delete, check, and change face data.

7) Log view: You can view or output the face recognition record information stored in the management server 200, and store employee access records.

8) Visitor log: Visitor information detected from the verification device 120 and stored in the management server 200 can be viewed or output, and employee access records can be stored.

9) Parameter setting: Main parameters of face recognition of the verification device 120 can be set or modified.

10) Spare button: A button that enables the designation of additional functions may be provided in advance.

Managing the face database on the user interface will be described with reference to FIG. 18. The user can add, delete, check, correct, import and export facial data in the setting window depicted in FIG. 18.

A description of each function on the user interface is as follows.

1) Add: You can add face information to the device as a snapshot.

2) Delete all: All face information on the verification device 120 and/or the management server 200 is deleted.

3) Person registration: Person information is registered on the verification device 120 and/or the management server 200.

4) Face registration: Multiple face information is registered in the verification device 120 and/or the management server 200 at once.

5) Face storage: Face information is backed up from the verification device 120 and/or the management server 200 to a PC program.

6) Synchronization of face information: Face information of the verification device 120 and/or the management server 200 is brought into a program and displayed on the screen.

7) Synchronization of face photos: The verification device 120 and/or the management server 200 uploads face photos to the PC software.

8) Database export: The face information is backed up from the verification device 120 and/or the management server 200 to a PC.

9) Database import: The face information is sent from the PC to the verification device 120 and/or the management server 200.

10) Correction: Correct the characteristics of the current individual face information.

11) Deletion: The current face information is deleted from the verification device 120 and/or the management server 200.

Referring to FIG. 19, it is possible to start adding a person by pressing the "Add"] button. Face photos are captured directly from the camera by pressing the “Snap” button, and personal information is entered. If you have successfully entered the face information, click the “Confirm Input” button.

In addition, an additional user interface with a “face information synchronization button” provided to synchronize basic information of the face database may be displayed, and statistics of the number of faces entered in response to the click of the button from the user may be displayed. have.

Referring to Fig. 20, click "Log View", enter the interface to select the start time and end time, and provide the "Search" button to search the log so that the log view and access records can be viewed. . In addition, a button can be provided to select a storage path in order to automatically save access records as EXCEL.

The embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, and the like alone or in combination. The program instructions recorded in the computer-readable recording medium may be specially designed and constructed for the present invention or may be known and usable to those skilled in the computer software field. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magnetic-optical media such as floptical disks. medium), and a hardware device specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device can be changed to one or more software modules to perform the processing according to the present invention, and vice versa.

Specific implementations described in the present invention are examples, and do not limit the scope of the present invention in any way. For brevity of the specification, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of the systems may be omitted. In addition, the connection or connection members of the lines between the components shown in the drawings exemplarily represent functional connections and/or physical or circuit connections. It may be referred to as a connection, or circuit connections. In addition, if there is no specific mention such as “essential” or “importantly”, it may not be an essential component for the application of the present invention.

In addition, although the detailed description of the present invention has been described with reference to a preferred embodiment of the present invention, the spirit of the present invention described in the claims to be described later if one of ordinary skill in the relevant technical field or those of ordinary skill in the relevant technical field And it will be understood that various modifications and changes can be made to the present invention within a range not departing from the technical field. Therefore, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be determined by the claims.

Claims (7)

A verification device for sensing biometric information of a user in order to determine whether the door is locked or unlocked by wirelessly interlocking with a door lock device installed on a door, the method comprising: detecting an object based on a captured image;
The verification device determining whether the detected object is in proximity;
The verification apparatus includes the steps of identifying the adjacent object as a user and detecting a face area of the object;
Measuring a body temperature of the user by detecting a temperature in the heat sensing region by detecting a heat sensing region in the face region displayed within a guide line displayed in a display image;
Analyzing the user's face area to output an alarm message according to the user's non-wearing of the mask, and detecting whether or not the mask is photographed;
The verification device includes the steps of recognizing the user's fingerprint through a fingerprint sensor provided on the upper keypad whenever the user inputs a signal to the keypad of the door lock device;
Detecting the user's identification information based on the user's fingerprint information, and managing a history of personnel entering and leaving the building;
Sensing the body temperature of the user exiting the building; And
If it is determined that the body temperature of the user is abnormal, maintaining the lock of the door lock device and displaying a warning message notifying the movement to a pre-prepared place; including;
A method of controlling access based on the user's biometric information. delete The method of claim 1,
The verification device further includes estimating the height of the user,
Increasing the contact area between the user's fingerprint and the surface of the keypad by adjusting the angle that the surface of the keypad makes with the ground based on the estimation result of the height
A method of controlling access based on the user's biometric information. The method of claim 3,
When the user's body temperature exceeds a preset temperature, adjusting the angle of the keypad so that the surface of the keypad faces the ground.
A method of controlling access based on the user's biometric information. delete delete The method of claim 1,
Identifying a user by comparing characteristic information of a user, which is detected and stored when entering or exiting the building, with characteristic information of a user exiting the building; And
Further comprising: detecting information of verified people to enter the building from the point of time of entering the building by the identified user to the present, and displaying a warning message;
A method of controlling access based on the user's biometric information.

Images