KR102351856B1 - Security device and controlling method thereof - Google Patents

Abstract

obtaining security release information including voice information;
obtaining at least one piece of analysis information by analyzing the at least one piece of security release information;
obtaining at least one piece of similarity information based on the at least one piece of analysis information; and
and determining whether to release security based on the at least one piece of similarity information.

Description

Security device and controlling method thereof

The present invention relates to a security device and a method for controlling the same. Specifically, the present invention relates to a security device for determining whether to release security based on security information including voice information and facial information, and a control method thereof.

In addition, the present invention relates to a security system including the security device, the server, a user terminal, an external organization, and the like.

In addition, the present disclosure relates to an artificial intelligence (AI) system for simulating functions such as cognition and judgment of a human brain by using a machine learning algorithm and an application thereof.

Recently, with the development of various technologies, research on security device technologies such as digital door locks is increasing. Security devices are being implemented in various bio-authentication methods ranging from basic type to input password, to fingerprint recognition and facial recognition.

On the other hand, the conventional basic type for inputting a password is inconvenient in a situation where both hands are not free and there is a risk of password leakage. In the case of fingerprint recognition, one hand must be free at least and it has a weak security, and in the case of facial recognition, there is a disadvantage that the recognition rate varies greatly depending on the state of wearing glasses or makeup.

The present invention is in accordance with the necessity to compensate for the above-mentioned disadvantages, and unlike the conventional security devices that were weak in recognition rate and security, security is enhanced and convenience is improved through interworking with external devices such as user terminals. An object of the present invention is to provide an increased security device and security system.

However, these problems are exemplary, and the scope of the present invention is not limited thereto.

A method for controlling a security device according to an embodiment of the present invention includes: acquiring security release information including voice information; obtaining at least one piece of analysis information by analyzing the at least one piece of security release information; obtaining at least one piece of similarity information based on the at least one piece of analysis information; and determining whether to release security based on the at least one piece of similarity information.

Also, the obtaining of the at least one piece of similarity information may include obtaining by comparing the at least one piece of analysis information with at least one piece of pre-registered security information.

In addition, the step of obtaining the analysis information is obtaining voice analysis information for the voice information, and the step of obtaining the at least one similarity information is comparing the speech analysis information with the first registered voice security information. The first voice similarity information may be acquired.

In this case, the security release information further includes facial information in addition to the voice information, and obtaining the analysis information includes: obtaining facial analysis information for the facial information; and acquiring lip motion analysis information based on the facial analysis information, wherein the acquiring at least one similarity information includes comparing the lip motion information with pre-registered facial security information to obtain lip motion similarity. The method may further include: obtaining information; and releasing the security may include determining whether to release the security based on the first voice similarity information and the lip motion similarity information.

On the other hand, the step of obtaining the analysis information is to obtain voice analysis information for the voice information, and the step of obtaining the at least one similarity information is comparing the speech analysis information with the previously registered second voice security information. The second voice similarity information is acquired, and the method of controlling the security device may further include determining whether there is a special situation based on the second voice similarity information.

The computer program of the present invention may be stored in a medium to execute the method of controlling the security device of the present invention using a computer.

The recording medium of the present invention may be a computer-readable recording medium in which a program for executing a control method of a security device is recorded.

The security device of the present invention includes a microphone for obtaining security release information including voice information; and whether to obtain at least one piece of analysis information by analyzing the at least one piece of security release information, obtain at least one piece of similarity information based on the at least one piece of analysis information, and release security based on the at least one piece of similarity information It may include; a processor to determine the.

Other aspects, features and advantages other than those described above will become apparent from the following detailed description, claims and drawings for carrying out the invention.

According to an embodiment of the present invention made as described above, the present invention can increase the user recognition rate by using various security release information including lip movement information.

In addition, the present invention can increase the degree of freedom of security by controlling the opening and closing of the door in a special situation through communication such as streaming between the user terminal and the camera of the security device.

In addition, the security device of the present invention can serve as a notification when a special emergency situation occurs by transmitting an emergency signal and image to the central control room (security room in the case of an apartment) through the recognition of a specific voice.

Of course, the scope of the present invention is not limited by these effects.

1 is a system diagram illustrating a security system including a security device according to an embodiment of the present invention.
2 is a block diagram illustrating components of a security device according to an embodiment of the present invention.
3 is a simple flowchart for explaining a method of releasing security of a security device according to an embodiment of the present invention.
4 is a view for explaining that the security device according to an embodiment of the present invention obtains voice analysis information.
5A to 5C are diagrams for explaining that a security device acquires facial analysis information from facial information according to an embodiment of the present invention.
6 illustrates that the security device according to an embodiment of the present invention obtains facial analysis information.
7 is a view for explaining a method for the security device to obtain lip motion analysis information according to an embodiment of the present invention.

이하, 본 개시의 다양한 실시예가 첨부된 도면과 연관되어 기재된다. 본 개시의 다양한 실시예는 다양한 변경을 가할 수 있고 여러 가지 실시예를 가질 수 있는바, 특정 실시예들이 도면에 예시되고 관련된 상세한 설명이 기재되어 있다. 그러나 이는 본 개시의 다양한 실시예를 특정한 실시 형태에 대해 한정하려는 것이 아니며, 본 개시의 다양한 실시예의 사상 및 기술 범위에 포함되는 모든 변경 및/또는 균등물 내지 대체물을 포함하는 것으로 이해되어야 한다. 도면의 설명과 관련하여, 유사한 구성요소에 대해서는 유사한 참조 부호가 사용되었다.

Hereinafter, various embodiments of the present disclosure are described in connection with the accompanying drawings. Various embodiments of the present disclosure may be subject to various modifications and may have various embodiments, and specific embodiments are illustrated in the drawings and the related detailed description is described. However, this is not intended to limit the various embodiments of the present disclosure to specific embodiments, and should be understood to include all modifications and/or equivalents or substitutes included in the spirit and scope of the various embodiments of the present disclosure. In connection with the description of the drawings, like reference numerals have been used for like elements.

In various embodiments of the present disclosure, "comprises." Or "have." The term such as is intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features or number, step, operation, component, part or It should be understood that it does not preclude the possibility of the existence or addition of combinations thereof.

In various embodiments of the present disclosure, expressions such as “or” include any and all combinations of words listed together. For example, "A or B" may include A, may include B, or may include both A and B.

Expressions such as “first”, “second”, “first”, or “second” used in various embodiments of the present disclosure may modify various components of various embodiments, but do not limit the components. does not For example, the above expressions do not limit the order and/or importance of corresponding components, and may be used to distinguish one component from another.

When a component is referred to as being “connected” or “connected” to another component, the component may be directly connected to or connected to the other component, but the component and It should be understood that other new components may exist between the other components.

In an embodiment of the present disclosure, terms such as “module”, “unit”, “part”, etc. are terms for designating a component that performs at least one function or operation, and such component is hardware or software. It may be implemented or implemented as a combination of hardware and software. In addition, a plurality of "modules", "units", "parts", etc. are integrated into at least one module or chip, except when each needs to be implemented in individual specific hardware, and thus at least one processor. can be implemented as

Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the context of the related art, and unless explicitly defined in various embodiments of the present disclosure, ideal or excessively formal terms not interpreted as meaning

In the present disclosure, artificial intelligence (AI) may refer to a field for researching artificial intelligence or a methodology capable of making it, and machine learning is a field of artificial intelligence technology, in which a computing device is It can be an algorithm that allows a computer to analyze data as a technical method to learn through learning to understand a specific object or condition, or to find and classify patterns in data. Machine learning disclosed in the present invention may be understood as meaning including an operation method for learning an artificial intelligence model.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a system diagram illustrating a security system including a security device according to an embodiment of the present invention.

The security system 1 according to an embodiment of the present invention may include a security device 100 , a server 200 , and an external device 300 .

The security device 100 of the present invention may be an electronic device attached to a door of a facility to control the opening/closing state of the door. In this case, the facilities include all types of facilities that the user enters, such as apartments, villas, officetels, and offices. For example, the security device 100 may be implemented by being coupled to the door in the form of a digital door lock for access control of the door, but is not limited thereto, and may be implemented in various forms to enhance the security of the door. have.

The security device 100 of the present invention may acquire at least one piece of security release information from a current user who is attempting to access. In this case, the security release information may include the user's voice information and the user's face information.

The security device 100 according to an embodiment of the present invention may determine whether to release the security of the door based on the acquired security release information. Specifically, the security device 100 may obtain at least one piece of analysis information by analyzing the obtained security release information, and may determine whether to release the security of the door based on the analysis information. In this case, the analysis information may include voice analysis information related to the frequency and frequency change of the voice, facial analysis information related to the structure of the face, lip motion analysis information related to the shape of the lips and the change of the lip shape, and the like.

Meanwhile, the security registered user may register his/her own information (security information) in the security device 100 to determine security maintenance and security release of the security device 100 . The security device 100 of the present invention may acquire similarity information based on the analysis information and pre-registered security information. In this case, the pre-registered security information may include voice information, voice analysis information, face information, lip movement information, lip movement analysis information, etc. registered by the security registered user. The security device 100 may obtain similarity information by comparing the previously registered security information with the analysis information.

The server 200 according to an embodiment of the present invention may store information received from the security device 100, determine a security state, etc. based on the received information, and obtain security state information. Also, the server 200 may transmit a first signal corresponding to the security state information to the external device 300 , and may receive a second signal associated with the security device 100 from the external device 300 .

According to an embodiment of the present invention, the security device 100 may transmit the analysis information and the similarity information to the server 200 . The server 200 of the present invention may acquire the security state information of the door based on the analysis information and the similarity information obtained by the security device 100 . The server 200 may transmit a first signal corresponding to the security state information to the external device 300 .

In this case, the external device 300 may include a user terminal 310 , a device installed in the security company 320 , a device installed in the police station 330 , and the like. The user terminal 310 may be a terminal used by a security registered user. The user terminal 310 may be implemented as a portable device such as a smartphone, but is not limited thereto, and may be implemented as any type of device having a display function, such as a PC, a tablet PC, a PMP, or a PDA. Hereinafter, for convenience of description, it is assumed that the user terminal 310 is implemented as a portable device.

According to another embodiment of the present invention, when the security device 100 obtains the security release information that is the basis of the analysis information, the security device 100 may immediately transmit the acquired security release information to the server 200 .

In this case, the server 200 stores the security release information received from the security device 100 , obtains at least one piece of analysis information based on the received security release information, and information on the security state based on the analysis information can be obtained. Thereafter, the server 200 may transmit a first signal corresponding to the security state information to the external device 300 .

Meanwhile, the first signal may include at least one piece of analysis information and motion analysis information obtained based on the time information at which the security release information was obtained, the type of the security release information, and the security release information. Also, the first signal may include security state information of the security device 100 .

For example, when receiving the first signal, the user terminal 310 may provide the user with information about the current security state (security release attempt, emergency situation) around the security device 100 . In addition, when the user terminal 310 receives the first signal, a notification including security release attempt time information may be displayed on a display (not shown), and the obtained security release information (voice, facial image, etc.) may be displayed to the user. can be provided to

In particular, the first signal may include a streaming image acquired through the camera of the security device 100 . Through this, there is an advantage that the user of the external device 300 can identify the user attempting to release the security through additional information other than the security cancellation information, such as the streaming video.

In addition, the first signal may include an emergency signal and an image when an emergency situation occurs. Accordingly, the present invention can serve as a notification when a special emergency situation occurs by transmitting an emergency signal and image to the security company 320 and the police station 330 through the server 200 of the emergency situation.

According to an embodiment of the present invention, when the external device 300 receives the first signal through the server 200 , it may approve whether to release the security of the security device 100 . For example, upon receiving the first signal, the user terminal 310 may provide the user with a UI requesting whether to release the security of the security device 100 according to a setting. When the user confirms that the provided security release information is reliable, the user may approve the security release of the security device 100 through the user terminal 310 .

The second signal of the present invention is a signal transmitted from the external device 300 including the user terminal 310 to the server 200 , and may include various information on security of the security device 100 . For example, the second signal may include information on whether to approve the release of security as described above, and may include a command signal for the security device 100 . In this case, the command signal may be a signal instructing to output a specific voice or an emergency alarm through the speaker 120 of the security device 100 , but is not limited thereto.

Meanwhile, the security system 1 of the present invention may further include an external information processing device (not shown). According to an embodiment of the present invention, the external information processing device may receive and store security information, security release information, analysis information, etc. received from the security device 100 and/or the server 200, and through this, the security device It can be determined whether the security of (100) is released.

2 is a block diagram illustrating components of a security device according to an embodiment of the present invention.

Referring to FIG. 2 , the security device 100 may include a microphone 110 , a speaker 120 , a camera 130 , a communication unit 140 , a security module 150 , a memory 160 , and a processor 170 . can

The microphone 110 of the present invention is configured to acquire sound and voice information around the security device 100 . In particular, the microphone 110 may acquire voice information of a user who attempts to release the security of the door.

The speaker 120 of the present invention is configured to output sound and voice corresponding to the control command received from the processor 170 . For example, the speaker 120 may output an emergency alarm, and may output a voice for guiding the user of the situation.

The camera 130 of the present invention may acquire information on the face of the user who attempts to unlock the door security. In addition, the camera 130 may acquire image data about the situation around the door.

The communication unit 140 of the present invention may be configured to communicate with various types of external devices 300 and servers 200 according to various types of communication methods. The communication unit 140 may include at least one of a Wi-Fi chip, a Bluetooth chip, a wireless communication chip, and an NFC chip. In particular, each of the Wi-Fi chip and the Bluetooth chip may perform communication using a WiFi method or a Bluetooth method. In the case of using a Wi-Fi chip or a Bluetooth chip, various types of connection information such as an SSID and a session key are first transmitted/received, and then various types of information may be transmitted/received after communication connection using this. The wireless communication chip refers to a chip that performs communication according to various communication standards such as IEEE, Zigbee, 3rd Generation (3G), 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), and Fifth Generation (5G). The wireless communication chip according to the 5G communication standard can use not only the frequency (Below 6GHz) band such as 3.5GHz, but also the millimeter wave (mmWave) such as 26, 28, 38, 39, 60GHz, that is, the ultra high frequency band (Above 6GHz). .

The NFC chip refers to a chip operating in an NFC (Near Field Communication) method using a 13.56 MHz band among various RF-ID frequency bands such as 135 kHz, 13.56 MHz, 433 MHz, 860 to 960 MHz, and 2.45 GHz.

The security module 150 of the present invention may be configured to control the opening and closing of an installed door to which the security device 100 is coupled. The security module 150 may turn on/off the lock of the door under the control of the processor 170 .

The memory 160 of the present invention may store the security release information obtained through the microphone 110 , the camera 130 , and the like. The memory 160 according to an embodiment of the present invention may store the analysis information obtained based on the security release information. Also, the memory 160 may store the second signal received from the server 200 and a control command corresponding thereto. In addition, the memory 160 of the present invention may store various data for the overall operation of the security device 100 , such as a program for processing or controlling the processor 170 .

The memory 160 may store a plurality of application programs (or applications) driven in the security device 100 , data for the operation of the security device 100 , and commands. At least some of these application programs may be downloaded from the server 200 through wireless communication. In addition, at least some of these application programs may exist on the security device 100 from the time of shipment for a basic function of the security device 100 . The application program is stored in the memory 160 , and may be driven by the processor 170 to perform an operation (or function) of the security device 100 .

The memory 160 may be implemented as a non-volatile memory, a volatile memory, a flash-memory, a hard disk drive (HDD), or a solid state drive (SSD). The memory 160 is accessed by the processor 170 , and data write/read/erase/update may be performed by the processor 170 .

The memory 160 may be implemented as, for example, an internal memory such as a ROM and a RAM included in the processor 170 , or may be implemented as a memory separate from the processor 170 . In this case, the memory 160 may be implemented in the form of a memory embedded in the security device 100 or may be implemented in the form of a memory detachable from the security device 100 according to the purpose of data storage.

The processor 170 of the present invention is a configuration for controlling the security device 100 as a whole. Specifically, the processor 170 controls the overall operation of the security device 100 by using various programs stored in the memory 160 of the security device 100 .

According to an embodiment of the present invention, the processor 170 may acquire at least one piece of analysis information based on the security release information acquired through the microphone 110 , the camera 130 , and the like. The processor 170 may acquire at least one piece of similarity information based on the analysis information. Also, the processor 170 may determine whether to release the security based on the at least one piece of similarity information. Specifically, the processor 170 may acquire voice information as security release information, and may acquire voice analysis information based on this. In this case, the voice analysis information may include information on a frequency change and a frequency change pattern of the voice information.

The processor 170 of the present invention may obtain voice similarity information by comparing the voice analysis information with voice security information previously registered for security release. In this case, the pre-registered voice security information is stored in the memory 160 after the user inputs it into the security device 100 in advance to release the security, and may be about a specific word (or voice password). That is, the processor 170 may obtain the voice similarity information including the similarity with respect to the frequency and frequency change between the above-described voice analysis information and the voice security information. According to an example, the voice similarity information may be expressed as a score of 0 to 100, but is not limited thereto, and may include values of various means indicating similarity.

The processor 170 of the present invention may determine whether to release the security based on the voice similarity information. Specifically, when the value of the voice similarity information is equal to or greater than a preset value, the processor 170 may determine to release the security of the security device 100 . For example, if the obtained voice similarity information is equal to or greater than a preset value (eg, 80 points), the processor 170 may release the security of the security device 100 .

According to another embodiment of the present invention, the processor 170 may obtain facial analysis information for the user's facial information acquired through the camera 130 . In this case, the face information may be a face image of the user included in the image acquired through the camera. In addition, the facial analysis information may include recognition of objects such as lips, pharynx, nose, eyes, and forehead included in the facial information. In this case, the above-described objects are merely examples, and may include various objects such as nasolabial folds, eyebrows, face lines, and teeth, and of course, specific objects such as upper lip and lower lip may be distinguished among lips.

The processor 170 may acquire object motion analysis information based on the facial analysis information. In this case, the object motion analysis information may include whether the shape of each object changes, the acceleration of the shape change of the object, the speed of the shape change of the object, the relative position change between the objects, and information about each object.

For example, the object motion analysis information may be lip motion analysis information. In this case, the first lip motion analysis information may be a shape change of the lips, the second lip motion analysis information may be a shape change speed/acceleration of the lips, and the third lip motion analysis information may be a change in the shape of the lips according to the movement of the lips. It may be a relative position change between different objects, the fourth lip motion analysis information may be a shape change of another object corresponding to the shape change of the lips, and the fifth lip motion analysis information may be another object corresponding to the shape change speed of the lips. may be the rate/acceleration of the shape change of However, this is only an example, and the lip movement analysis information may include all types of information related to the movement of the lips and a relationship with other objects according to the movement.

The processor 170 may obtain object motion similarity information by comparing the object motion information with the facial security information previously registered by the security registered user. The security registration user may register his/her own information in the security device 100 to determine security maintenance and security release of the security device 100 .

In this case, the pre-registered facial security information may include facial information including an image of the security registered user's face, facial analysis information acquired based on the user's facial information, and object movement information of the user. The facial analysis information may include information on the shape of objects, such as lips, pharynx, nose, eyes, and forehead, of the registered security user, and information on relative positions between objects. The object motion analysis information includes whether the shape of each object changes during the time period when the security registered user speaks a specific word (or voice code) for security release, the shape change acceleration of the object, the shape change speed of the object, and the relative relationship between each object. It may include a change in location and the like.

Specifically, the memory 160 may store object movement information when a security registered user utters a specific word (eg, a word for releasing security, a word for informing an emergency situation), and the processor ( 170) compares the object motion analysis information obtained from the user with the pre-stored object motion information of the security registered user to obtain similarity information about the similarity.

The processor 170 of the present invention may determine whether to release security based on object motion similarity information. Specifically, when the value of the similarity information is equal to or greater than a preset value, the processor 170 may determine to release the security of the security device 100 . For example, when the similarity information includes a score of 0 to 100 points and the obtained similarity information is equal to or greater than a preset value (eg, 80 points), the security of the security device 100 may be released.

The processor 170 according to an embodiment of the present invention may determine whether to release security by comprehensively considering similarity information for various security release information, such as voice similarity information and object motion similarity information. For example, the processor 170 is a security device only when the obtained voice similarity information is equal to or greater than a first preset value for voice security information, and the obtained object motion similarity information is equal to or greater than a second preset value for facial security information. The security of (100) can be released.

The processor 170 according to another embodiment of the present invention may release security when the average value of each similarity information is equal to or greater than a third preset value. According to an embodiment of the present invention, the processor 170 may set a weight for each security information, and determine whether to release security based on the weight. For example, when a double weight is set for the object motion similarity information, the processor 170 performs security when the (object motion similarity information*2/3) + (voice similarity information*1/3) value is equal to or greater than the third value. can be turned off.

Thereafter, the processor 170 may transmit a control command corresponding to whether to release the security to the speaker 120 , the security module 150 , and the communication unit 140 .

Meanwhile, the processor 170 according to an embodiment of the present invention may obtain voice similarity information by comparing voice analysis information with previously registered voice security information for a special situation. At this time, the pre-registered voice security information is input to the security device 100 in advance and stored in the memory 160 to notify a special situation, such as an emergency situation, an emergency situation, etc. by a security registered user. may be information. The processor 170 may obtain voice similarity information including a frequency and a degree of similarity between the voice analysis information of the current user and the voice security information of the security registered user. According to an example, the voice similarity information may be expressed as a score of 0 to 100, but is not limited thereto, and may include values of various means indicating similarity.

Thereafter, the processor 170 may determine whether there is a special situation based on the voice similarity information. Specifically, when the value of the voice similarity information is equal to or greater than a preset value, the processor 170 may determine that the vicinity of the security device 100 is a special situation. For example, if the obtained voice similarity information is equal to or greater than a preset value (eg, 80 points), the processor 170 may determine that the vicinity of the security device 100 is a special situation.

Similarly, the processor 170 may transmit a control command corresponding to whether or not there is a special situation to the speaker 120 , the security module 150 , and the communication unit 140 .

The processor 170 of the present invention may be implemented as a digital signal processor (DSP), a microprocessor, or a time controller (TCON) for processing a digital signal. However, the present invention is not limited thereto, and the central central processing unit (CPU), micro controller unit (MCU), micro processing unit (MPU), controller, application processor (AP), or communication processor (CP) , may include one or more of the ARM processor, or may be defined by the term.In addition, the processor 170 may be implemented as a SoC (System on Chip) or LSI (large scale integration) with a built-in processing algorithm. , it may be implemented in the form of a field programmable gate array (FPGA).

According to an embodiment of the present invention, functions performed by the memory 160 and/or the processor 170 are included in the server 200 and an external information processing device (not shown) in addition to the security device 100 to perform similar functions. can be done

3 is a simple flowchart for explaining a method of releasing security of a security device according to an embodiment of the present invention.

Referring to FIG. 3 , the security device 100 may obtain security cancellation information from a current user attempting to cancel security ( S310 ). In this case, the security release information may include, but is not limited to, the user's voice information and the user's face information. The security device 100 may obtain the user's voice information through the microphone 110 , and may obtain the user's face information through the camera 130 .

The security device 100 of the present invention may acquire at least one piece of analysis information based on the acquired security release information (S320). The at least one piece of analysis information may include voice analysis information when the acquired security release information is voice information, and may include facial analysis information and object motion analysis information when the acquired security release information is facial information.

The security device 100 of the present invention may acquire similarity information based on the at least one piece of analysis information (S330). The security device 100 of the present invention may obtain by comparing the at least one piece of analysis information with at least one piece of pre-registered security information registered by a security registered user (S330).

For example, the security device 100 may obtain the first voice similarity information by comparing the voice analysis information for the user with the first voice security information previously registered by the security registered user. In this case, the first voice security information may include voice information and voice analysis information about a specific word for security release of the security device 100 by the security registered user. Specifically, when the specific word for releasing security is 'turtle', the security device 100 compares the voice analysis information for the user with the first voice security information previously registered for 'turtle' uttered by the security registered user. The first voice similarity information may be acquired.

As another example, the security apparatus 100 may obtain the second voice similarity information by comparing the voice analysis information for the user with the second voice security information previously registered by the security registered user. In this case, the second voice security information may include voice information and voice analysis information for a specific word for notifying a special situation around the security device 100 by the security registered user. Specifically, when a specific word for notifying a special situation such as an emergency situation is 'fire', the security device 100 provides voice analysis information for the user and a pre-registered system for 'fire' uttered by the security registered user. The second voice similarity information may be obtained by comparing the two voice security information.

The security device 100 of the present invention may determine whether to release the security based on the obtained similarity information (S340). The security device 100 may determine to release security when the similarity information is equal to or greater than a preset value. According to an embodiment of the present invention, when there are a plurality of similarity information, the security device 100 provides security only when a value included in each similarity information is greater than or equal to a value set for security release information corresponding to each similarity information. can be turned off

According to an embodiment of the present invention, the security device 100 may determine whether there is a special situation based on the obtained similarity information. When the value included in the above-described second voice similarity information is equal to or greater than the value set for special situation recognition, the security apparatus 100 may determine the special situation.

In this case, the security device 100 includes at least one piece of analysis information acquired based on the time information at which the security release information was obtained, the type of security release information, and the security release information, motion analysis information, emergency signal and streaming image information. may transmit a signal to the server 200 . The server 200 may transmit the signal to the external device 300 , and then the external device 300 may transmit a command corresponding to the signal to the server 200 .

According to this embodiment, there is an effect that the user of the external device 300 can recognize whether a situation around the security device 100 is a special situation.

4 is a view for explaining that the security device according to an embodiment of the present invention obtains voice analysis information. 4 illustrates an example in which a specific word (or voice password) for releasing the security of the security device 100 is 'turtle'.

Referring to FIG. 4 , the security device 100 may acquire a voice frequency in the first section A in which the user utters 'go'. Similarly, the security device 100 may obtain a voice frequency in the second section (B) in which the user utters 'book', and obtains the voice frequency in the third section (C) in which the user utters 'this'. can be obtained

Although FIG. 4 illustrates that the voice analysis information includes only voice frequency information, this is only an example and it goes without saying that the voice analysis information may include various types of information.

Specifically, the security device 100 may acquire information on the frequency change pattern obtained in the first to third sections. For example, the voice analysis information may include frequency change speed information, frequency change aspect information, and the like.

In this case, the frequency change pattern information may be information on the pattern of sequentially changing from the frequency of the first section (A) to the frequency of the third section (C) through the frequency of the second section (B). According to an example of the present invention, the frequency change aspect information is from the frequency of the first section (A) to the frequency of the second section (B), and from the frequency of the second section (B) to the frequency of the third section (C). It may include information on the time it takes for each change and the time it takes for the frequency to change from the frequency of the first section (A) to the frequency of the third section (C).

5A to 7 illustrate that the security device 100 of the present invention obtains facial analysis information and object motion analysis information from facial information.

In particular, FIGS. 5A to 5C are diagrams for explaining that the security device acquires facial analysis information from facial information according to an embodiment of the present invention.

Referring to FIG. 5A , in the user's face included in the facial information 510 , the security device 100 includes a region 511 corresponding to the eyebrows and the forehead, a region 512 corresponding to the nose periphery, and the periphery of the eyes. An object may be recognized based on at least one of the corresponding regions 513 .

In this case, the security device 100 may use the learned artificial intelligence model to recognize each region of the user's face, but is not limited thereto. At this time, the learned artificial intelligence model may be, for example, CNN, RNN, etc., and such a learned artificial intelligence model may be stored in the server 200 .

According to an embodiment of the present invention, the security device 100 recognizes the region 511 corresponding to the eyebrows and the forehead, and the region 512 corresponding to the periphery of the nose, the T zone of the user's face. can be recognized as When the security apparatus 100 recognizes the T zone of a specific user, even if another facial information of the corresponding user is input, the security apparatus 100 may recognize the same area as the T zone. Accordingly, even if the facial information is input at different times, the security apparatus 100 may recognize the same area as the T zone, and may recognize the object as the reference area.

According to another embodiment of the present invention, the security apparatus 100 may recognize the face line 514 included in the face information 510 after setting the reference area of the face information. Even in this case, the security device 100 may use the learned artificial intelligence model to recognize the user's face region, but is not limited thereto. At this time, the learned artificial intelligence model may be, for example, CNN, RNN, etc., and such a learned artificial intelligence model may be stored in the external server 200 .

According to another embodiment of the present invention, the security device 100 may determine a triangle block included in the user's face as the reference area.

Specifically, referring to FIG. 5B , the security apparatus 100 may generate the triangle block R based on the region 513 corresponding to the eye and the region 516 corresponding to the chin included in the facial information. However, this is only an embodiment, and the security device 100 may generate a triangle block based on the area 515 corresponding to the mouth and the area 513 corresponding to the eyes, and in addition, the point where the eyebrows end (not shown). A triangle block may be generated based on various regions, such as 511 , a region corresponding to the forehead).

According to an embodiment of the present invention, the security device 100 may search for the position of the eyes from the input facial information and calculate the coordinates of the two eyes. For example, the security device 100 may use an active shape model (ASM) method in the fields of computer vision and pattern recognition to recognize a face image and a face expression. The core algorithm of the ASM method may include two sub-models, namely, a global shape model and a partial shape model. The ASM method can quickly locate feature points of objects at the same time. That is, the security device 100 may search for positions of both ends of the eyelids and eyes in the face region and determine the positions.

The security device 100 may calculate the slope deviation of the feature in the face region image through the search process, and find the positions of the eyelids and both ends of the eyes included in the facial information by searching for positions matching the slope deviation of the features in the ASM model. can decide

However, the algorithm through the ASM model is only an embodiment and is not limited thereto, and the security device 100 may use various methods other than this. According to an embodiment of the present invention, the security device 100 may use the learned artificial intelligence model to recognize the region 513 corresponding to the eye and the region 516 corresponding to the chin included in the user's facial information. However, the present invention is not limited thereto. At this time, the learned artificial intelligence model may be, for example, CNN, RNN, etc., and such a learned artificial intelligence model may be stored in the external server 200 .

On the other hand, when the angle and size of the first face information and the second face information are different and correction is required, the security device 100 matches the triangle block R that is the reference area of the first face information and the second face information can do it At this time, the security device 100 may first match the three axes (x, y, z) of the triangle when matching each triangle block (R).

That is, the security device 100 aligns the triangle block R determined through the position of the eye, etc. at a predetermined position, rotates the facial information and moves it in the horizontal direction, and furthermore, the eye position and the eye of the facial information Normalized facial information of a fixed size may be obtained by scaling the facial information according to a relationship between positions.

On the other hand, if the security device 100 matches the size and angle of the first face information and the second face information through a three-axis (x, y, z) algorithm, each through a correction solution algorithm to match the pixel level The difference between the reference values of the image can be calculated.

In this case, the reference value may be a pixel level of an area corresponding to the triangle block R of each face information. For example, due to external environmental factors (lighting, shooting angle, etc.), the triangle block (R) pixel level value of the first facial information may be different from the triangle block (R) pixel level value of the second facial information, Accordingly, a level deviation may occur. The security device 100 may match the level values of the entire first face information and the second face information by matching the reference value for each triangle block R.

Even if the security device 100 matches the pixel level values of the entire image, pixels at the same position in the first face information and the second face information may not always have the same value. However, even in this case, the slope deviation of all pixels included in the first face information and the second face information may be the same. Specifically, since the deviation of the inclination of the pixel from the triangle block R to the face line 514 corresponds to the user's facial characteristics, it does not change even if the facial information changes.

Accordingly, the security device 100 may determine the pixel inclination deviation from the triangle block R to the face line 514, and may perform pixel-level matching based on this.

The security device 100 of the present invention can precisely recognize the position of the object even in the facial information obtained at different times through the method according to FIGS. 5A and 5B, and can acquire facial analysis information based on this . Furthermore, the security apparatus 100 may acquire object motion analysis information by precisely tracking the relative position and degree of change of the user's object during the time period for uttering the voice encryption.

In particular, the security device 100 according to an embodiment of the present invention may acquire the position of a region corresponding to the mouth among the objects included in the facial information, and may perform an analysis on the movement of the lips. This will be described in detail with reference to FIGS. 6 and 7 .

6 illustrates that the security device according to an embodiment of the present invention obtains facial analysis information.

Referring to FIG. 6 , the facial analysis information obtained by the security device 100 according to an embodiment of the present invention may include information on the position and shape of the lip line.

That is, as described above, the security device 100 of the present invention may identify the region 515 corresponding to the mouth in the face information 510 and extract the lip line. Specifically, the security apparatus 100 may acquire the first lip line L1 and the second lip line L2 in the region 515 corresponding to the mouth.

In this case, the security device 100 may use the artificial intelligence model learned to recognize the lip line. At this time, the learned artificial intelligence model may be, for example, CNN, RNN, etc., and such a learned artificial intelligence model may be stored in the server 200 .

7 is a view for explaining a method for the security device to obtain lip motion analysis information according to an embodiment of the present invention.

7 exemplarily shows that the security device 100 analyzes the movement of the lip object while uttering the voice code 'turtle'. Specifically, the first state (A) shows the movement of the lip line for igniting 'book' after uttering 'geo', and the second state (B) is for uttering 'lee' after uttering 'book'. The movement of the lip line is shown for the lip line, and the third state (C) shows the lip line after uttering 'tooth'.

Referring to Figure 7, the security device 100 in the first state (A), the left portion of the first lip line (L1) moves in the X, -Y direction, the central portion moves in the -Y direction, the right portion -X, It can be analyzed by moving in the -Y direction. In addition, the security device 100 analyzes that in the first state (A), the left part of the second lip line L2 moves in the X and Y directions, the central part moves in the Y direction, and the right part moves in the -X and Y directions. can do.

Similarly, in the security device 100, in the second state (B), the left end of the first lip line L1 moves in the -X direction, the left portion moves in the X, -Y direction, and the right portion moves in the -X, -Y direction. It can be analyzed as moving in the X direction and the right end of the movement in the X direction. Also, the security device 100 may analyze that the left end of the second lip line L2 moves in the -X direction, the center moves in the Y direction, and the right end of the second lip line L2 moves in the X direction.

In FIG. 7 , the lip motion analysis information displays only information on the movement direction of the lips, but this is only an example and the lip motion analysis information may include various types of information. For example, the lip movement analysis information includes shape change speed information of the lip lines L1 and L2, shape change acceleration information of the lip lines L1 and L2, shape change pattern information of the lip lines L1 and L2, and the lip line (L1, L2) and the relative position change information of the other object (eg, Injung) may be included.

At this time, the shape change pattern information of the lip lines L1 and L2 sequentially changes the shape of the lip line from the shape of the first state (A) through the shape of the second state (B) to the shape of the third state (C). It may be information about the aspect of doing it. According to an example of the present invention, the shape change aspect information of the lip line is from the shape of the first state (A) to the shape of the second state (B), and from the shape of the second state (B) to the third state (C). may include information on the time it takes for the lip line to change in the shape of , and the time it takes for the lip line to change from the shape of the first state (A) to the shape of the third state (C).

The security device 100 according to an embodiment of the present invention may obtain the lip movement analysis security information from the facial security information in which the security registered user utters the voice password by the method of FIGS. 5A to 7 . Thereafter, the security device 100 may acquire facial information uttering a voice password from a user attempting to release security, and then acquire lip movement analysis information.

The security device 100 of the present invention may obtain lip motion similarity information by comparing the lip motion analysis security information with the lip motion analysis information. In this case, the lip motion similarity information may be expressed as a score of 0 to 100 points, but is not limited thereto, and may include values of various means indicating similarity.

The security device 100 of the present invention may determine whether to release the security based on the lip motion similarity information. Specifically, when the value of the lip motion similarity information is equal to or greater than a preset value, the security device 100 may determine to release the security of the security device 100 . For example, when the acquired lip motion similarity information is equal to or greater than a preset value (eg, 80 points), the security apparatus 100 may release security.

Meanwhile, the above-described methods according to various embodiments of the present invention may be implemented in the form of an application that can be installed in an existing security device.

In addition, the above-described methods according to various embodiments of the present invention may be implemented only by software upgrade or hardware upgrade of an existing security device.

In addition, various embodiments of the present invention described above may be performed through an embedded server provided in the security device or an external server of the security device.

On the other hand, according to an embodiment of the present invention, the various embodiments described above are a recording medium (readable by a computer or similar device) using software, hardware, or a combination thereof. It may be implemented as software including instructions stored in a computer readable recording medium). In some cases, the embodiments described herein may be implemented by the processor itself. According to the software implementation, embodiments such as the procedures and functions described in this specification may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein.

Meanwhile, a computer or a similar device is a device capable of calling a stored command from a storage medium and operating according to the called command, and may include the device according to the disclosed embodiments. When the instruction is executed by the processor, the processor may directly or use other components under the control of the processor to perform a function corresponding to the instruction. Instructions may include code generated or executed by a compiler or interpreter.

The device-readable recording medium may be provided in the form of a non-transitory computer readable recording medium. Here, 'non-transitory' means that the storage medium does not include a signal and is tangible, and does not distinguish that data is semi-permanently or temporarily stored in the storage medium. In this case, the non-transitory computer-readable medium refers to a medium that stores data semi-permanently, rather than a medium that stores data for a short moment, such as a register, cache, memory, etc., and can be read by a device. Specific examples of the non-transitory computer-readable medium may include a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

As such, the present invention has been described with reference to the embodiments shown in the drawings, which are merely exemplary, and those skilled in the art will understand that various modifications and equivalent other embodiments are possible therefrom. . Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the appended claims.

100: security device
200: server
300: external device
310: user terminal
320: security vendor
330: police station

Claims (8)

In the control method of the security device,
obtaining security release information including voice information and facial information;
obtaining voice analysis information for the voice information and facial analysis information for the face information;
a similarity information acquisition step of obtaining first voice similarity information by comparing the voice analysis information with previously registered first voice security information, and obtaining facial similarity information by comparing the facial analysis information with previously registered facial security information; and
determining whether to release security based on the first voice similarity information and the facial similarity information;
The step of obtaining the facial analysis information is,
The first lip line shape corresponding to the upper lip part included in the facial information is identified as the first left part, the first central part, and the first right part, and the second lip line shape corresponding to the lower lip part is identified as the second left part, the second part identifying the central portion and the second right portion;
The state of the face information is set to the first left part, the first central part, the first right part, the second left part, the second central part, and the second right part so that the plurality of syllables included in the voice information correspond to the utterance times, respectively. classifying into at least one state according to the arrangement state; and
The first left portion, the first central portion, the first right portion, the second left portion, the second central portion, and the second right portion, respectively, as the first of the at least one state changes to the second of the at least one state Acquiring the change rate information and the change acceleration information of the facial analysis information; Control method of the security device, characterized in that it further comprises. delete delete delete The method of claim 1,
The step of obtaining the similarity information is to obtain second voice similarity information by comparing the voice analysis information with the previously registered second voice security information;
The control method of the security device,
determining whether there is a special situation based on the second voice similarity information; and
When the determination result is a special situation, transmitting a signal corresponding to the special situation to the server; Control method of the security device further comprising a. A computer program stored in a medium for executing the method of controlling a security device according to any one of claims 1 to 5 by using a computer. A computer-readable recording medium in which a program for executing the control method of any one of claims 1 and 5 is recorded. A security device comprising:
a microphone for obtaining security release information including voice information;
a camera for obtaining security release information including facial information; and
Obtaining voice analysis information for the voice information and facial analysis information for the facial information,
Obtaining first voice similarity information by comparing the voice analysis information with previously registered first voice security information, comparing the facial analysis information with previously registered facial security information to obtain facial similarity information,
a processor that determines whether to release security based on the first voice similarity information and the facial similarity information;
Obtaining the facial analysis information is,
The first lip line shape corresponding to the upper lip part included in the facial information is identified as the first left part, the first central part, and the first right part, and the second lip line shape corresponding to the lower lip part is identified as the second left part, the second part Identified as a central part and a second right part,
The state of the face information is set to the first left part, the first central part, the first right part, the second left part, the second central part, and the second right part so that the plurality of syllables included in the voice information correspond to the utterance times, respectively. divided into at least one state according to the arrangement state,
The first left portion, the first central portion, the first right portion, the second left portion, the second central portion, and the second right portion, respectively, as the first of the at least one state changes to the second of the at least one state A security device that acquires change rate information and change acceleration information of the facial analysis information.

Images