KR20240003145A - Method for providing security and personalization service through personal facial recognition based on artificial intelligence and apparatus therefor - Google Patents

Abstract

The present invention relates to a method of providing security and personalization services through artificial intelligence-based personal facial recognition and a device therefor. The method of providing security and personalization services in a profile authentication device according to an aspect of the present disclosure includes registering profile setting information. a step of recognizing a face from a camera-captured image and performing a face registration procedure corresponding to the profile setting information; locking a profile that has successfully registered the face; and operating the camera according to a specific menu selection to provide the current It may include performing facial authentication on a user-selected profile and unlocking the user-selected profile based on successful facial authentication. Therefore, the present invention has the advantage of providing enhanced security and personalized services based on artificial intelligence.

Description

Method for providing security and personalization service through personal facial recognition based on artificial intelligence and apparatus therefor}

The present invention relates to technology for enhancing security and personalized services through personal facial recognition based on artificial intelligence.

Set-Top Box (STB) literally means a box placed on the floor and generally refers to a device for receiving digital broadcasts.

A set-top box is a device that receives terrestrial broadcasting, cable broadcasting, and satellite broadcasting at home and watches them on a connected TV. Recently, it is a device that connects a set-top box and a phone line (or dedicated line) to watch broadcast programs on TV as well as view/select/watch various contents such as YouTube, movies, music, shopping, education, and games through Internet access. It is being utilized.

The digital broadcasting server compresses video, voice, and data signals into multiplexes and then modulates them and transmits them. The set-top box receives signals transmitted by a digital broadcasting server through a digital tuner and then decompresses and separates each signal through a demodulation process.

Recently, set-top boxes equipped with smart speakers (i.e., artificial intelligence speakers) have been released mainly by telecommunication companies. The set-top boxes recognize the user's voice input through the speaker through linkage with a dedicated voice recognition server, and the recognition results Based on this, it was possible to provide broadcast channel search services, music playback services, various content search services, Internet information search services, and lighting and home appliance control services.

However, as shown in Figure 1, the security technology applied to currently commercialized set-top boxes sets a numeric profile lock password through the remote control when registering a profile after entering the home screen, and then the user enters the set lock password. It has been implemented so that the lock setting for the profile can be unlocked by directly entering it through the remote control key button. Therefore, the lock password can be easily exposed to other people, resulting in a security vulnerability.

The purpose of the present invention is to provide a method for providing security and personalized services through artificial intelligence-based personal facial recognition, and a device and system therefor.

Another object of the present invention is to provide an on-device (On-device) service that allows a set-top box equipped with a profile authentication device to directly recognize and analyze faces captured by a camera without linking with an external server for personal facial recognition, thereby providing security and personalized services. -Device) It provides a method of providing security and personalization services based on facial recognition.

Another object of the present invention is to identify actual users through a facial recognition engine in a set-top box equipped with a profile authentication device without linking with a separate server, and to provide accurate and fast profile switching and personalized services on-device ( On-Device) It provides a set-top box equipped with facial recognition function.

Another purpose of the present invention is to enhance the security of personal profiles compared to the conventional password input method by setting and unlocking profile locks through personal facial recognition, as well as to provide customized personalization to customers through automatic profile switching through personal facial recognition. The aim is to provide a method and device for providing security and personalized services based on on-device facial recognition that can provide services more quickly and conveniently.

Another purpose of the present invention is to identify the age/gender of the actual user through artificial intelligence-based personal facial recognition, so that not only can minors access adult content in advance, but also automatically provide personalized content appropriate for age and gender. The goal is to provide a profile authentication device that can be provided and a method of providing security and personalization services using the same.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

A method of providing security and personalization services in a profile authentication device according to an aspect of the present disclosure includes the steps of registering profile setting information, recognizing a face from a camera captured image, and performing a face registration procedure corresponding to the profile setting information. and locking the profile for which the face registration has been successful, driving the camera according to a specific menu selection to perform face authentication on the profile currently selected by the user, and locking the profile selected by the user based on successful face authentication. It may include the step of canceling the settings.

In an embodiment, the method may further include configuring and outputting a personalized user interface screen corresponding to the user-selected profile based on the lock setting being released.

In an embodiment, the profile setting information may include at least one of setting information for limiting viewing age and viewing time and setting information for restricting viewing of adult content.

In an embodiment, the step of registering the profile setting information includes outputting a user interface screen for entering a mobile phone number according to a profile registration menu selection and creating a profile setting information database corresponding to the mobile phone number entered by the user. It may include outputting a profile setting user interface screen including a restriction setting menu and registering user-entered restriction setting information in the profile setting information database.

In an embodiment, the profile setting user interface screen includes a face registration menu, and the face registration procedure may be performed by driving the camera according to selection of the face registration menu.

In an embodiment, the method further includes generating and outputting a face registration failure message based on the face registration failure, and the profile in which the face registration fails may be maintained in an unlocked state.

In an embodiment, the step of performing the face authentication is based on the current profile face authentication procedure of performing face authentication only for the user-selected profile based on the profile automatic switching mode not being activated and the profile automatic switching mode being activated. This may include a full profile face authentication procedure that performs face authentication on all pre-registered profiles.

In an embodiment, the full profile facial authentication procedure includes performing facial recognition on the entire profile to identify a profile to automatically switch to, and automatically switching the user-selected profile to the identified profile, wherein the automatic A personalized user interface screen may be configured and displayed based on profile setting information corresponding to the converted profile.

In an embodiment, the method further includes setting a common setting value commonly applied to all profiles based on failure of the face authentication through any one of the current profile face authentication procedure and the full profile face authentication procedure. However, after the common settings are set, the home screen can be entered.

In an embodiment, the facial authentication may be performed using a facial recognition engine within the set-top box equipped with the profile authentication device without linking with an external server.

storing at least one computer program including instructions that, when executed by at least one processor according to another aspect of the present disclosure, cause the at least one processor to perform operations for providing security and personalization services in the profile authentication device; In the non-volatile computer-readable storage medium, the operations include registering profile setting information, recognizing a face from a camera captured image, and performing a face registration procedure corresponding to the profile setting information, and registering the face. Locking a successful profile, driving the camera according to a specific menu selection to perform face authentication on the currently user-selected profile, and unlocking the user-selected profile based on successful face authentication. may include.

A profile authentication device according to another aspect of the present disclosure registers profile setting information, recognizes the user's face from a camera captured image, performs a face registration procedure corresponding to the profile setting information, and selects a profile that has successfully registered the face. The camera can be set to lock, operate the camera according to a specific menu selection, perform face authentication for the profile currently selected by the user, and unlock the user-selected profile based on successful face authentication.

In an embodiment, the profile authentication device may configure and output a personalized user interface screen corresponding to the user-selected profile based on the lock setting being released.

In an embodiment, the profile setting information may include at least one of setting information for limiting viewing age and viewing time and setting information for restricting viewing of adult content.

In an embodiment, the profile authentication device configures and outputs a user interface screen for entering a mobile phone number based on the profile registration menu being selected, creates a profile setting information database corresponding to the mobile phone number entered by the user, and limits the user to a predetermined limit. A profile setting user interface screen including a setting menu may be configured and output, and user-entered restriction setting information may be registered in the profile setting information database.

In an embodiment, the profile setting user interface screen includes a face registration menu, and the face registration procedure may be performed by driving the camera according to selection of the face registration menu.

In an embodiment, the profile authentication device may output a face registration failure message based on the face registration failure, and the profile in which the face registration failed may be maintained in an unlocked state.

In an embodiment, the face authentication is performed based on the current profile face authentication procedure that performs face authentication only for the user-selected profile based on the profile automatic switching mode not being activated and the pre-registered profile based on the profile automatic switching mode being activated. It may include a full profile facial authentication procedure that performs facial authentication for the entire profile.

In an embodiment, the full profile facial authentication procedure may include performing facial recognition on the entire profile to identify a profile to be automatically converted, and automatically converting the user-selected profile to the identified profile.

In an embodiment, based on failure of the face authentication through any one of the current profile face authentication procedure and the full profile face authentication procedure, the profile authentication device sets a common setting value that is commonly applied to all profiles and then returns to the home page. You can enter the screen.

In an embodiment, the facial authentication may be performed using a facial recognition engine within a set-top box equipped with the profile authentication device without linking with an external server.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below. You will be able to.

The present invention has the advantage of providing a method and device for providing security and personalized services through artificial intelligence-based personal facial recognition.

In addition, the present invention is an on-device device that allows a set-top box to directly recognize and analyze faces captured by a camera without separate linkage with an external server for personal facial recognition, thereby providing security and personalization services for the profile. It has the advantage of providing a method for providing facial recognition-based set-top box security and personalization services, as well as devices and systems for the same.

In addition, the present invention has the advantage of enhancing personal security and user convenience by automatically and accurately identifying actual users through a personal facial recognition function within a set-top box equipped with a profile authentication device without linking to a separate server.

In addition, the present invention not only enhances the security of the personal profile compared to the conventional password input method by setting and unlocking the profile lock through personal facial recognition, but also provides customized personalized services to customers through automatic profile switching through personal facial recognition. It has the advantage of being able to be provided more quickly and conveniently.

Another purpose of the present invention is to automatically identify the age and gender of actual users through artificial intelligence-based personal facial recognition, thereby not only preventing minors from accessing adult content, but also providing personalized content appropriate for age and gender. It has the advantage of being able to provide automatically.

In addition, various effects that can be directly or indirectly identified through this document may be provided.

The drawings attached to this specification are intended to provide an understanding of the present invention, show various embodiments of the present invention, and together with the description of the specification, explain the principles of the present invention.
1 is a diagram illustrating a conventional password-based profile lock setting and unlocking method.
Figure 2 is a system configuration diagram for providing security and personalized services through personal facial recognition according to an embodiment.
Figure 3 is a configuration diagram of an IPTV system for providing security and personalized services through personal facial recognition according to an embodiment.
Figure 4 is a diagram for explaining the operation of a facial recognition engine according to an embodiment of the present disclosure.
Figure 5 is a diagram for explaining a profile face registration procedure according to an embodiment.
Figure 6 is a diagram for explaining the configuration of a profile setting pop-up UI according to an embodiment.
Figure 7 is a flowchart illustrating a profile registration procedure according to an embodiment of the present disclosure.
FIG. 8 is a diagram illustrating a facial recognition-based automatic profile switching procedure according to an embodiment of the present disclosure.
Figure 9 shows a face registration information pop-up UI according to an embodiment of the present disclosure.
Figure 10 shows a profile registration information pop-up UI according to an embodiment of the present disclosure.
11 is a flowchart illustrating a facial recognition-based profile authentication procedure according to an embodiment of the present disclosure.

Hereinafter, some embodiments of the present invention will be described in detail through illustrative drawings. When adding reference numerals to components in each drawing, it should be noted that identical components are given the same reference numerals as much as possible even if they are shown in different drawings. Additionally, when describing embodiments of the present invention, if detailed descriptions of related known configurations or functions are judged to impede understanding of the embodiments of the present invention, the detailed descriptions will be omitted.

In describing the components of the embodiments of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are only used to distinguish the component from other components, and the nature, order, or order of the component is not limited by the term. Additionally, unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by a person of ordinary skill in the technical field to which the present invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless clearly defined in the present application, should not be interpreted in an idealized or excessively formal sense. No.

In various examples of this disclosure, “/” and “,” should be interpreted as indicating “and/or.” For example, “A/B” can mean “A and/or B.” Furthermore, “A, B” may mean “A and/or B.” Furthermore, “A/B/C” may mean “at least one of A, B and/or C.” Furthermore, “A, B, C” may mean “at least one of A, B and/or C.”

In various examples of this disclosure, “or” should be interpreted as indicating “and/or.” For example, “A or B” may include “only A,” “only B,” and/or “both A and B.” In other words, “or” should be interpreted as indicating “additionally or alternatively.”

Hereinafter, embodiments of the present invention will be described in detail with reference to FIGS. 1 to 11.

1 is a diagram illustrating a conventional password-based profile lock setting and unlocking method.

Referring to FIG. 1, the user can select his/her profile image among profile images displayed on the TV screen (S110).

The user can set the lock setting password on the profile lock password setting screen (S120). At this time, the user can enter the password using the number button provided on the remote control.

With multiple profiles registered, the user can select the profile image he or she wants to unlock among the locked profile images displayed on the home screen (S130).

The user can unlock the profile by entering the preset lock password corresponding to the profile image selected on the profile unlock screen (S140). At this time, the user can enter the password using the number button provided on the remote control.

If the user does not remember the password and wishes to re-register the password corresponding to his or her profile, the user can use the delete button 131 to delete the profile and then reset the profile lock password through the re-registration procedure.

As described above, the conventional set-top box profile unlocking procedure provides a two-depth unlocking procedure of entering the home screen and entering the lock password on the profile unlock screen, which not only reduces user convenience but also reduces profile lock settings. Due to the change, there was a problem that interfered with TV viewing.

In addition, the security technology applied to the conventional set-top box had a problem of being very vulnerable to security because a password consisting of only numbers could easily use someone else's profile rights through theft and/or theft.

In addition, the security technology applied to the conventional septop box had the inconvenience of having to change the profile by entering the home screen and then selecting the profile change menu to change the profile.

In addition, the security technology applied to conventional set-top boxes does not determine who the actual user currently watching broadcast content is, so it is not possible to provide personalized services tailored to current viewers, such as adult content being exposed even if the user is a minor. There was this.

Figure 2 is a system configuration diagram for providing security and personalized services through personal facial recognition according to an embodiment.

Referring to Figure 2, the system 1 includes a TV 10, a camera 11, a set-top box 20, a remote control 30, a digital broadcasting server 40, a facial recognition server 50, and a communication network 60. It may be configured to include.

The set-top box 20 can be connected to the TV 10 through a dedicated line - for example, an HDMI cable - and can separate and decode the signal received from the digital broadcasting server 40 and provide it to the TV 10. .

As an example, the remote control 30 can be simultaneously paired with the set-top box 20, TV 10, and camera 11 to wirelessly transmit control signals to the set-top box 20, TV 10, and camera 11. It can be equipped with a function as an integrated remote control.

As another example, the remote control 30 is paired with the set-top box 20 and the camera 11, and a remote control (not shown) for controlling the TV 10 may be separately provided.

As another example, the remote control 30 may be paired with the set-top box 20, and the set-top box 20 may be paired with the camera 11.

As another example, the camera 11 is connected to the TV 10 by wire or wirelessly, and the image captured by the camera 11 can be transmitted to the TV 10. At this time, the TV 10 can provide the image captured by the camera 11 to the set-top box 20.

The set-top box 20 according to another embodiment may receive the image captured by the camera 11 through wireless communication, but this is only one embodiment, and the set-top box 20 according to another embodiment includes A separate wired connection terminal is provided so that images captured by the camera 11 can be received directly by wire.

As an example, the set-top box 20 may receive images captured by the camera 11 through Bluetooth communication or Wi-Fi communication, but this is only one embodiment, and a mobile communication network - for example, LTE, LTE It is also possible to receive images captured by the camera 11 through -A, including 5G NR (New Radio Access Technology), etc.

The remote control 30 receives a TV power button for controlling the power ON/OFF of the set-top box 20 and the TV 10, a camera power button for controlling the camera 11 power ON/OFF, and a user voice command. At least one of a push-to-talk button for transmission to the set-top box 20, a channel selection button, a volume control button, a search button, a display adjustment button, an external input control button, a home menu selection button, and a settings button. It may be configured to include one, but is not limited thereto.

The user can use the settings button provided on the remote control 30 to enter the settings menu on the home screen, and the user can select a specific item in the settings menu to perform profile registration/release procedures, face registration procedures for each profile, and facial recognition-based You can perform the automatic profile switching setting procedure, etc.

The set-top box 20 may be equipped with a microphone and a speaker, and when the set-top box 20 is activated - that is, when the voice recognition function is activated - the user's voice signal input through the microphone is transmitted to the communication network 60. It can be transmitted to the voice recognition server 50 through.

The voice recognition server 50 may perform voice recognition on the received user voice signal to identify the user command, and transmit a result corresponding to the identified user command to the set-top box 20 through the communication network 60. The set-top box 20 may output the received voice recognition results to speakers and/or the screen of the TV 10.

A user according to an embodiment may control the operation of the camera 11 through voice commands.

When the user presses a specific button provided on the remote control 30 - for example, a Push-to-Talk button - while the microphone of the set-top box 20 is activated for voice recognition, the remote control 30 performs a predetermined control. By transmitting a signal to the set-top box 20, the microphone provided in the set-top box 20 may be deactivated. A user voice command input through a microphone provided in the remote control 30 may be transmitted to the set-top box 20.

In an embodiment, when the set-top box 20 is activated, the user may activate the personal facial recognition function of the set-top box 20 through a predetermined voice command.

In an embodiment, the set-top box 20 may automatically activate the personal facial recognition function according to the user's menu settings. For example, the set-top box 20 may automatically operate the camera 11 when entering the home screen and then activate the personal facial recognition function.

In an embodiment, the set-top box 20 can control operations such as profile lock setting, profile unlock, and automatic profile switching through personal facial recognition.

The set-top box 20 according to the embodiment is equipped with an artificial intelligence-based personal facial recognition engine and is equipped with a profile authentication device (not shown) that performs personal facial recognition based on the user's face image captured by the camera 11. It can be. In the following description according to the embodiment, the set-top box 20 is used to perform profile lock control and automatic profile switching through personal facial recognition based on artificial intelligence, and to provide personalized content corresponding to the identified or switched profile. Although described as an example, this is only one embodiment, and in other embodiments, the corresponding functions may be performed through a profile authentication device mounted or mounted on the set-top box 20 or a user terminal (not shown).

As an example, in the following embodiment, it is explained that the screen and/or message generated by the set-top box 20 is output through the screen of the TV 10, but this is only one embodiment, and the profile authentication device is When mounted on a user terminal, a screen and/or message configured by the profile authentication device may be output on the screen of the user terminal.

The set-top box 20 can identify the user through personal facial recognition and automatically unlock the profile lock setting corresponding to the identified user.

In addition, the set-top box 20 can identify the user's age and gender through personal facial recognition, and adaptively configure and provide personalized content corresponding to the identified age and gender.

For example, if the set-top box 20 determines that the user is a minor through personal facial recognition, the set-top box 20 may block adult content from being displayed on the screen of the TV 10.

For example, when gender is identified through personal facial recognition, the set-top box 20 may generate a recommended content screen corresponding to the identified gender and output it on the screen of the TV 10.

For example, when a user is identified through personal facial recognition, the set-top box 20 continues content, recommended channel information, and recommended VOD content based on the identified user's past TV channel selection history and content playback history such as VOD. Information can be generated, and the generated information can be output on the TV 10 screen.

For example, when user identification through personal facial recognition fails, the set-top box 20 may automatically perform user identification based on voice keywords. To this end, the set-top box 20 may be equipped with an artificial intelligence-based voice keyword recognition engine.

For example, if voice keyword recognition fails, the set-top box 20 may automatically switch to a password-based user (profile) unlock screen.

According to an embodiment, when personal face recognition, voice keyword recognition, and password verification all fail, the set-top box 20 may generate a predetermined warning alarm message and transmit it to the terminal of a pre-registered user.

In another embodiment, when at least one of personal facial recognition, voice keyword recognition, and password verification fails, the set-top box 20 may generate a warning alarm message corresponding to the failure and transmit it to the terminal of a pre-registered user.

As an example, the voice keyword used for voice keyword recognition may be implemented to be the same for all users, but this is only one embodiment, and different voice keywords may be registered and used for each user depending on user settings.

For example, the voice keyword for setting the profile lock can be predefined as “Set the profile lock,” the voice keyword for unlocking the profile can be predefined as “Unlock the profile,” and the voice keyword for switching profiles can be predefined as “Switch profile.” However, this is only one embodiment, and each user may pre-select or enter a voice keyword corresponding to each operation and register it.

In an embodiment, while the user's profile is locked, the user can unlock the profile by uttering a predefined lock keyword while pressing a specific button on the remote control 30. Here, when a specific button for entering a lock keyword - for example, a erase button - is pressed, the microphone of the remote control 30 is activated, and the microphone of the set-top box 20 can be deactivated while the specific button is pressed. . In an embodiment, the remote control 30 button for entering the lock keyword may be predefined as a different button according to the design of a person skilled in the art, or may be changed through a predetermined setting menu.

In another embodiment, while the user's profile is locked, the user selects the profile change menu on the home screen and then presses a specific button on the remote control 30 - for example, the erase button - and utters a predefined lock keyword to You can unlock the profile. Here, when a specific button for inputting a lock keyword is pressed, the microphone of the remote control 30 may be activated, and the microphone of the set-top box 20 may be deactivated while the specific button is pressed.

In another embodiment, while the user's profile is locked, the user may select the profile change menu on the home screen and then utter the lock keyword without pressing a specific button on the remote control 30 to unlock the profile. there is. In this case, the voice uttered by the user can be directly input to the microphone of the set-top box 20 and immediately detected, and if the input user voice is a preset or pre-registered lock keyword, the set-top box 20 You can unlock the user's profile.

In an embodiment, a voice recording file for each user for a lock keyword may be pre-registered and maintained in a predetermined recording area of the set-top box 20. In a state in which the lock setting for a plurality of profiles is released, the set-top box 20 can recognize the lock keyword uttered by the user by voice. The set-top box 20 can authenticate and identify the speaker who uttered the lock keyword by comparing/analyzing pre-registered voice recording files for each user corresponding to the recognized lock keyword. The set-top box 20 can configure a customer-customized screen corresponding to the profile of the authenticated and identified speaker.

In an embodiment, a customer-customized screen may be dynamically configured based on service usage data and statistics collected in advance for each user. As an example, the customer-customized screen may include content continuation information, recommended content curation images, information on recommended genres, etc., but is not limited to this.

Through the above-described system 1, the present invention allows the user to press a specific button on the remote control 30, or enter a lock keyword after pressing a specific button, without entering the home screen when a profile change is necessary even while watching TV. You have the advantage of being able to receive personalized services by uttering a message and immediately changing it to your profile.

In addition, the present invention allows for a 1-depth profile change through personal facial recognition and/or voice keyword recognition, rather than a 2-depth profile change by selecting the conventional home screen entry->profile change menu, thereby improving the user scenario. There is an advantage.

Figure 3 is a configuration diagram of an IPTV system for providing security and personalized services through personal facial recognition according to an embodiment.

Referring to FIG. 3, the IP TV system 300 includes a TV 10, a camera 11, a set-top box 20, a remote control 30, a server 40, a user terminal 50, and a communication network 60. It can be configured to include.

The set-top box 320 equipped with the profile authentication device (or equipped with a profile authentication means) of the present disclosure can be connected to the TV 310 through a dedicated cable - for example, an HDMI cable - and can be connected to a communication network (LAN) through a LAN cable. 360) and can exchange various information with the server 340. As an example, the server 340 is a digital broadcasting server that provides digital broadcasting information, a content server that provides various contents such as VOD, games, and sound sources, and a cloud that performs various cloud-based distributed computing processing such as a voice recognition engine and search engine. It may include, but is not limited to, at least one of the servers.

As an example, the remote control 330 can be used to control the operation of the set-top box 20, TV 10, and camera 11.

The camera 311 may be integrated with the TV 10, but this is only one embodiment, and the camera 311 according to another embodiment may be connected to the TV 310 or the set-top box 320 by wire or wirelessly. is connected, and the image captured by the camera 311 can be transmitted to the TV 320 or the set-top box 320. At this time, when the camera 311 is integrally mounted on the TV 310 or connected to the TV 310 by wire or wirelessly, the image captured by the camera 311 is transmitted from the TV 310 to the set-top box 320. can be provided.

If the camera 311 is connected to the set-top box 320 by wire or wirelessly, the image captured by the camera 311 may be displayed on one side of the screen of the TV 310 by the set-top box 320.

As an example, the set-top box 320 may be connected to the camera 311 through short-range wireless communication (Bluetooth communication or Wi-Fi communication) and receive images captured by the camera 311, but this is only one embodiment. In addition, the image captured by the camera 311 may be received through a mobile communication network - including, for example, LTE, LTE-A, 5G NR (New Radio Access Technology), etc.

The remote control 330 receives a TV power button for controlling the power ON/OFF of the set-top box 20 and the TV 10, a camera power button for controlling the camera 311 power ON/OFF, and a user voice command. At least one of a push-to-talk button for transmission to the set-top box 320, a channel selection button, a volume control button, a search button, a display adjustment button, an external input control button, a home menu selection button, and a settings button. It may be configured to include one, but is not limited thereto.

The user can use the settings button provided on the remote control 330 to enter the settings menu on the IPTV home screen, and the user can select a specific item in the settings menu to register the profile key value and register/release the profile lock password. , at least one of a basic profile information setting procedure, an initial profile face registration procedure, a profile face deregistration procedure, a profile face re-registration procedure, and a facial recognition-based automatic profile switching setting procedure may be performed.

The set-top box 320 may be equipped with a microphone and a speaker, and when the set-top box 320 is driven and the voice recognition function is activated, the set-top box 320 transmits the user voice signal input through the microphone to the communication network 360. It can be transmitted to the server 340 through . The server 340 may perform voice recognition on the received user voice signal to identify the user command, and transmit the processed result corresponding to the identified user command to the set-top box 320 through the communication network 360. The set-top box 320 may output the received voice recognition results through speakers and/or the screen of the TV 310.

A user according to an embodiment may control the operation of the camera 311 through a voice command, but this is only one embodiment, and the camera 311 according to another embodiment may be operated according to the user's menu selection after running the IPTV app. It can be automatically driven in the set-top box 320. As an example, the set-top box 320 selects the “My” menu, selects the “Live TV” menu, selects the “VOD Home” menu, selects the “Profile Settings” menu, selects the “User Authentication” menu, and selects “VOD Content”. When motion is detected, the camera 311 can be automatically operated to perform user facial recognition. The scope of the screen (or menu) to which the function that recognizes the user's face in real time to authenticate the user or automatically switches the profile by identifying the user may be changed according to the design of a person skilled in the art.

When the automatic profile switching mode is disabled, the set-top box 320 according to the embodiment extracts facial feature data in real time by analyzing the facial image file captured and input by the camera 311 through the installed facial recognition engine, User authentication for the current profile can be performed by comparing/analyzing pre-registered facial feature data and extracted facial feature data corresponding to the current profile. If user authentication fails, the set-top box 320 may output a predetermined warning notification message on one side of the TV 310 screen indicating that face recognition has failed. If user authentication is successful, the set-top box 320 can perform the operation according to the corresponding menu selection without separately outputting a UI notifying the success of facial recognition.

In addition, when the automatic profile switching mode is activated, the set-top box 320 according to the embodiment analyzes the facial image file captured and input by the camera 311 through the installed facial recognition engine to extract facial feature data in real time. And, by comparing/analyzing the pre-registered facial feature data and extracted facial feature data for all profiles, the profile corresponding to the current user can be identified and the identified profile can be activated. If user identification fails, the set-top box 320 may output a predetermined warning notification message on one side of the TV 310 screen indicating that face recognition has failed. If user identification is successful, the set-top box 320 may activate the identified profile as the current profile without separately outputting a UI indicating successful facial recognition.

The set-top box 320 according to the embodiment may register or update a profile using the user's mobile phone number as a key value when registering a new profile (or changing an existing profile). Of course, facial feature data corresponding to the profile can also be stored and maintained inside the set-top box 320 by databaseizing the corresponding mobile phone number as a key value.

In another embodiment, the set-top box 320 sets a communication channel with the user terminal 350 corresponding to the mobile phone number entered when registering a new profile, and uses a camera (Mido) provided in the user terminal 350 through the set communication channel. You can also receive the user's facial image file taken by the city).

The set-top box 320 analyzes the user's facial image file received from the user terminal 350 through a built-in facial recognition engine to extract facial feature data, and uses the extracted facial feature data as a key value. You can set it and save it in the internal database. As an example, a facial recognition engine may be implemented based on deep learning technology that pre-trains a deep neural network structure composed of a very large number of layers using a large amount of data, but is not limited to this.

Figure 4 is a diagram for explaining the operation of a facial recognition engine according to an embodiment of the present disclosure.

The facial recognition engine according to the embodiment may be installed in a set-top box, but this is only one embodiment, and may also be installed in a user terminal such as a smartphone or laptop.

In detail, FIG. 4 is a diagram for explaining a facial recognition pipeline.

Referring to FIG. 4, the facial recognition engine 400 may be largely composed of a learning pipeline 410 and a verification pipeline 420.

The verification pipeline 420 may perform facial authentication and identification operations based on the model learned by the learning pipeline 410.

As shown in FIG. 4, the face recognition engine 400 includes a face detection unit ( 412, 422) and a face preprocessor (413, 423) that performs preprocessing such as face alignment, luminance correction, and image normalization using landmarks information extracted from the detected face area, and features (413, 423) from the preprocessed face image. Classifies faces using feature extraction units 414 and 415 that extract features, a classification model learning unit 415 that learns a classification model in the learning stage using the extracted features, and a learning model 416 in the verification stage. and a classification and identification unit 425 that performs user authentication and user identification.

The feature extraction units 414 and 424 may be implemented as a locally connected convolutional neural network composed of multiple layers to effectively extract local features within the face area.

The face recognition engine 400 can be largely divided into a face verification engine and a face identification engine depending on the application.

The face verification engine performs a 1:1 verification operation to determine whether the two input face images are the same person by comparing/analyzing them, and the face identification engine compares one input face image with N pre-registered faces. By comparing/analyzing face images, a 1:N verification operation can be performed to determine which person the face image corresponds to.

The face identification engine uses the learned model to verify probe images - for example, user face images captured by TV cameras - and gallery images of N people registered in advance - In other words, an operation is performed to compare N face images corresponding to the pre-registered profiles of N people.

Deep learning-based face recognition technology is capable of learning very high-dimensional features on its own by performing feature extraction and classification model learning end-to-end using large amounts of facial data constructed in various environments. Accordingly, face recognition performance in a wild environment is very high, and has the advantage of providing performance that exceeds human recognition ability.

Figure 5 is a diagram for explaining a profile face registration procedure according to an embodiment.

Referring to Figure 5, the user can register a new profile using the profile registration (311) button on the My Menu screen (310) of the IPTV app. Here, information about the registered profile may be maintained within the set-top box.

The user can use the profile registration (311) button to register the profile key value corresponding to the new profile. For example, the profile key value may be the user's mobile phone number, but is not limited to this, and other information such as ID, email, and landline phone number may be used as the profile key value according to the design of a person skilled in the art.

When the user's mobile phone number is registered, a newly added profile image 312 and a settings 313 button corresponding to the mobile phone number may be displayed on the My Menu screen 310.

When the settings 313 button is selected, a profile settings pop-up UI 320 may be displayed.

The profile settings pop-up UI 320 may include a settings menu button display area and a detailed settings screen display area 324 for each menu.

In the settings menu button display area, a viewing age limit setting button 321, an adult content suggestion setting button 322, and a face registration button 323 may be displayed.

When the face registration button 323 is selected, a face registration pop-up UI 330 may be displayed on one side of the screen.

The face registration pop-up UI 330 may include a guidance message display area 331 notifying that face registration is in progress for profile lock setting and a camera image display area 332 where an image captured by a camera is displayed.

When the face registration button 323 is selected, the set-top box can automatically operate the camera. The set-top box can recognize faces in images captured by a camera and extract facial feature data by extracting features from the recognized face images.

If the facial feature data is extracted normally, the set-top box may output the facial registration confirmation UI 340.

The face registration confirmation UI 340 may include a confirmation 341 button for accepting the user's face registration and a cancel 342 button for canceling the user's face registration.

The operation according to selection of the viewing age limit setting button 321 and the adult content suggestion setting button 322 will be explained in detail with reference to FIG. 6, which will be described later.

Figure 6 is a diagram for explaining the configuration of a profile setting pop-up UI according to an embodiment.

Referring to FIG. 6, when the user selects the viewing age restriction setting button 321 in the profile setting pop-up UI 320, a selection menu for setting the content viewing restriction level (age) is displayed, as shown in drawing number 610. may be displayed. Here, the viewing restriction rating can be divided into no restrictions, 19+ rating, 15+ rating, 12+ rating, 7+ rating, and Kids rating.

The set-top box according to an embodiment of the present disclosure may determine that the profile is a parent profile when the viewing restriction level is selected as no restriction, and may determine that the profile is a child profile when the remaining levels are selected.

When the user selects the adult content restriction setting button 322 in the profile setting pop-up UI 320, as shown in drawing number 620, a selection menu for setting whether to hide adult content may be displayed.

The set-top box according to the embodiment may be implemented so that the setting authority for adult content restrictions is granted only to the parent profile, but this is only one embodiment, and the setting authority is only granted to the parent profile and a profile with a viewing age limit of 19 years or older. It can be implemented to be granted. At this time, the remaining profiles may be set to hide adult content by default.

Depending on the adult content restriction settings, when the set-top box switches to a profile with restricted exposure to adult content through facial recognition, it can automatically process all adult content for that profile as non-exposure.

For example, when the profile is switched through facial recognition while the automatic profile switching mode is activated, the set-top box displays the content previously being watched on the screen or continues viewing based on the previous viewing history of the profile to be switched. You can create the proposed UI and display it on the device screen.

As another example, when the profile is switched through facial recognition while the automatic profile switching mode is activated, the set-top box creates a UI that recommends content corresponding to the genre preferred by the profile to be switched and displays it on the device screen. You can do it.

Figure 7 is a flowchart illustrating a profile registration procedure according to an embodiment of the present disclosure.

In detail, FIG. 7 is a diagram illustrating a procedure for registering a user profile including face registration through the IPTV main APP mounted on a set-top box.

For convenience of explanation, the IPTV main APP will be briefly referred to as “APP.”

The set-top box according to the embodiment may include a profile registration module that performs a series of user profile registration procedures in conjunction with the APP.

The profile registration module may be configured to include a facial login management module and a profile setting module. The face login management module can perform face registration procedures and face authentication procedures for each profile. The profile setting module can perform profile registration/delete/update operations and manage profile setting information by forming it into a database. The profile setting module can manage profile setting information using the user's mobile phone number as the key value.

Referring to FIG. 7, when My Menu is selected by the user on the home screen of the IPTV main APP, the set-top box enters the My Menu screen, and the profile registration menu on the My Menu screen can be selected (S701 to S702).

When the profile registration menu is selected, the APP can configure and output the mobile phone number input UI as a pop-up screen (S703).

The APP can transmit the mobile phone number entered by the user to the profile setting module - in detail, the facial login management module - (S704).

The profile setting module may create a profile setting information DB corresponding to the received mobile phone number and then transmit a mobile phone number setting completion message to the APP (S705 to S706).

When the mobile phone number setting is completed, the APP can configure a profile setting UI pop-up screen and output it on one side of the My Menu screen (S707).

When a restriction setting menu is set by the user, the APP may output a selection menu corresponding to the restriction setting menu on one side of the profile setting UI pop-up screen (S708).

As an example, the restriction setting menu may include the viewing age restriction setting menu and the adult content restriction setting menu described above.

The APP may transmit user-entered restriction setting information to the profile setting module corresponding to the selected restriction setting menu (S709).

The profile setting module may store the received restriction setting information in a profile setting information DB created corresponding to a pre-registered mobile phone number (S710). At this time, the profile setting module may transmit a predetermined response message to the APP indicating that the restriction setting information has been properly registered.

The profile setting module according to the embodiment may determine whether the profile is a parent or a child based on the viewing age suggested rating and store the determination result in the corresponding profile setting information DB (S711).

As an example, the profile setting module may determine that the profile is a parent profile when the viewing restriction level is selected as no restriction, and may determine that the profile is a child profile when the remaining levels are selected.

When the face registration menu is selected on the profile setting UI pop-up screen, the APP may request face registration from the face login management unit (S712 to S713).

The facial login management module can recognize the user's face from a camera captured image, extract the recognized facial features, and generate facial feature data (S714).

Here, facial feature data can be extracted using a pre-trained facial recognition engine. The specific configuration and operation of the facial recognition engine are replaced with the description of FIG. 4 above.

The face login management module may transmit a predetermined response message to the APP indicating that face registration corresponding to the profile has been successfully completed (S716).

When face registration is successfully completed, the APP can configure and output a face registration completion toast pop-up screen (S717).

When face registration is completed, the APP may perform profile lock settings for the profile and output a profile registration completion toast pop-up screen indicating that profile registration has been completed (S718 to S720).

If face registration is not completed or face registration fails, the APP may output a face registration failure message indicating that face registration has been completed normally (S721) and return to step 707 described above.

FIG. 8 is a diagram illustrating a facial recognition-based automatic profile switching procedure according to an embodiment of the present disclosure.

Figure 9 shows a face registration information pop-up UI according to an embodiment of the present disclosure.

Figure 10 shows a profile registration information pop-up UI according to an embodiment of the present disclosure.

As shown in Figure 8, the APP according to the embodiment provides any one of the menus 'My Menu', 'Real-time TV', 'VOD Home', 'VOD Content Selection', 'Settings', and 'User Authentication' to the user. If selected, the user's face can be photographed upon entering the menu and the camera is activated to perform facial recognition for the user.

When the automatic profile switching mode is disabled and the current profile is locked, the set-top box can determine whether the user's face recognized from the real-time camera video matches the face already registered according to the current profile. .

As a result of the judgment, if there is a match, the set-top box configures and outputs a screen corresponding to the menu selected by the user without separate notification. If it does not match, a predetermined warning alarm message is sent indicating that face recognition corresponding to the current profile has failed. Can be printed.

In the set-top box according to the embodiment, if the face of the user currently captured by the camera does not match the face of the current profile, as shown in FIG. 9, which will be described later, a predetermined profile registration information pop-up UI 910 is provided requesting registration of a new profile. can be generated and output.

Referring to FIG. 9, the profile registration information pop-up UI 910 may be configured to include a profile registration shortcut button 911 for directly selecting the profile registration 311 menu described above.

When the automatic profile switching mode is activated, the set-top box can identify the profile to be switched by comparing the face of the user recognized in real time from the camera image captured in real time with the faces of all already registered users. When the set-top box identifies a profile that matches the face of the user currently captured by the camera among the faces of all registered profiles, it automatically switches profiles to the identified profile without separate notification and then configures and outputs the user-selected menu screen. can do.

If it fails to identify a profile that matches the face of the user currently captured by the camera among the faces of all registered profiles, the set-top box outputs a warning alarm indicating that facial recognition-based profile switching has failed, and the current screen is displayed as is. It can be maintained.

If the set-top box according to the embodiment fails to identify a profile matching the face of the user currently captured by the camera, as shown in FIG. 10, which will be described later, a predetermined face registration information pop-up UI 920 is provided requesting registration of the user's face. It can be created and displayed on the screen. Referring to FIG. 10, the face registration information pop-up UI 920 may be configured to include a face registration shortcut button 921 for directly selecting the face registration menu 323 described above.

In the set-top box according to another embodiment of the present disclosure, when the automatic profile switching mode is activated and the current profile is locked, first, the user's face recognized in real time from the real-time captured camera image and the current profile are selected corresponding to the current profile. It is possible to determine whether the registered face matches. If the user's face recognized in real time from the camera video and the current profile face do not match, the set-top box compares the user's face recognized in real time from the camera video with the faces of all previously registered profiles to identify the profile to be converted. You may. At this time, if profile switching fails, the set-top box may generate and output the profile registration information pop-up UI 910 described above.

The face registration information pop-up UI 920 and/or the profile registration information pop-up UI 910 according to the embodiment may further include an end button (not shown) for terminating the corresponding pop-up UI. If the end button is selected by the user, the set-top box can be set to common settings and then enter the home screen.

11 is a flowchart illustrating a facial recognition-based profile authentication procedure according to an embodiment of the present disclosure.

Referring to FIG. 11, the APP can determine whether the current profile is locked when a specific menu on the screen is selected after the profile is selected (S1101 to S1103).

If the current profile is locked, the APP can determine whether the profile automatic switching mode is activated (S1104).

If the profile automatic switching mode is not activated, the APP can perform a face authentication procedure for the current profile in conjunction with the profile registration module (S1105).

If the APP succeeds in facial recognition for the current profile, it can enter the user-selected menu screen (S1106 and S1112).

If facial recognition for the current profile fails, the APP can enter the home screen by performing a common setting value setting procedure (S1116 to S1117).

As a result of the determination in step 1104, if the automatic profile switching mode is activated, the APP can perform a face authentication procedure for the entire profile (S1107).

As a result of performing the full profile face authentication procedure, if the conversion target profile is identified, the APP automatically switches the current profile to the identified profile and then changes the setting values for the conversion target profile to the profile registration module - details, profile settings module - It can be obtained by requesting it (S1108 to S1111).

If the conversion target profile is not identified as a result of performing the entire profile face authentication process, the APP may output a profile registration information pop-up UI (S1108 and S1113).

If you do not enter the profile registration process through the profile registration information pop-up UI, the APP can enter the home screen by performing the common setting value setting process (S1114, S1116, and S1117).

If you enter the profile registration process through the profile registration information pop-up UI, the APP can output the profile registration screen (S1115).

If the current profile is unlocked in step 1103, the APP can directly enter the selection menu screen.

The server described in relation to the embodiments disclosed in this specification is connected to a display, a transceiver for transmitting and receiving signals with network equipment and user devices - for example, a set-top box - through an external wired / wireless communication network, and the display and transceiver are connected to the overall It may be configured to include at least one processor that controls the operation and a memory in which a program for the operation of the processor is recorded.

The set-top box described in relation to the embodiments disclosed herein can be connected to an external device - for example, a digital broadcasting server, a voice recognition server, It may be configured to include a transceiver for transmitting and receiving signals to and from a remote control, TV, etc., at least one processor (or controller or control unit) that is connected to the transceiver and controls the overall operation, and a memory in which a program for the operation of the processor is recorded. there is.

The user terminal described in relation to the embodiments disclosed herein transmits and receives signals with an external device (e.g., a digital broadcasting server, a voice recognition server, etc.) through an external wired/wireless communication network and/or short-range wireless communication. It may be configured to include a transceiver, at least one processor (or controller, or control unit) connected to the transceiver to control the overall operation, and a memory in which a program for the operation of the processor is recorded.

The steps of the method or algorithm described in connection with the embodiments disclosed herein may be implemented directly as hardware, software modules, or a combination of the two executed by a processor. Software modules may reside in a storage medium (i.e., memory and/or storage) such as RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM.

An exemplary storage medium is coupled to a processor, the processor capable of reading information from and writing information to the storage medium. Alternatively, the storage medium may be integral with the processor. The processor and storage medium may reside within an application specific integrated circuit (ASIC). The ASIC may reside within the user device. Alternatively, the processor and storage medium may reside as separate components within the user device.

The above description is merely an illustrative explanation of the technical idea of the present invention, and various modifications and variations will be possible to those skilled in the art without departing from the essential characteristics of the present invention.

Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but rather to explain it, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope should be construed as being included in the scope of rights of the present invention.

Claims (21)

In a method of providing security and personalization services in a profile authentication device,
registering profile setting information;
Recognizing the user's face from a camera-captured image and performing a face registration procedure corresponding to the profile setting information;
locking the profile that successfully registered the face;
performing face authentication for the profile currently selected by the user by operating the camera according to a specific menu selection; and
Unlocking the user-selected profile based on successful facial authentication.
Method, including. According to paragraph 1,
A method comprising configuring a personalized user interface screen corresponding to the user selected profile based on the lock setting being unlocked. According to paragraph 1,
The profile setting information includes at least one of setting information for limiting viewing age and viewing time, and setting information for restricting viewing of adult content. According to paragraph 1,
The step of registering the profile setting information is,
outputting a user interface screen for entering a mobile phone number according to a profile registration menu selection;
generating a profile setting information database corresponding to the mobile phone number entered by the user;
outputting a profile settings user interface screen including a restriction settings menu; and
Registering user-entered restriction setting information in the profile setting information database.
Method, including. According to paragraph 4,
The profile setting user interface screen includes a face registration menu,
The method is characterized in that the face registration procedure is performed by driving the camera according to the selection of the face registration menu. According to paragraph 1,
The method further includes generating and outputting a face registration failure message based on the face registration failure, wherein the profile in which the face registration fails is maintained in an unlocked state. According to paragraph 1,
The step of performing the face authentication is,
A current profile face authentication procedure that performs face authentication only for the user-selected profile based on the profile automatic switching mode not being activated; and
Full profile facial authentication procedure that performs facial authentication for all pre-registered profiles based on the profile automatic switching mode being activated.
Method, including. In clause 7,
The full profile face authentication process is,
Identifying a profile to be automatically converted by performing facial recognition on all profiles;
Automatically converting the user-selected profile to the identified profile
Method, including. According to clause 8,
Based on failure of the face authentication through any one of the current profile face authentication procedure and the full profile face authentication procedure, further comprising setting a common setting value commonly applied to all profiles, wherein the common setting value A method characterized by entering the home screen after setting. According to paragraph 1,
The method is characterized in that the facial authentication is performed using a facial recognition engine within a set-top box equipped with the profile authentication device without linking with an external server. Non-volatile computer-readable storage storing at least one computer program containing instructions that, when executed by at least one processor, cause the at least one processor to perform operations for providing security and personalization services in the profile authentication device. In the media,
The above operations are:
registering profile setting information;
Recognizing a face from a camera-captured image and performing a face registration procedure corresponding to the profile setting information;
locking the profile that successfully registered the face;
performing face authentication for the profile currently selected by the user by operating the camera according to a specific menu selection; and
Unlocking the user-selected profile based on successful facial authentication.
Storage media, including. Register profile setting information, recognize the user's face from images captured by the camera, perform a face registration procedure corresponding to the profile setting information, lock the profile that has successfully registered the face, and activate the camera according to a specific menu selection. A profile authentication device that operates to perform face authentication on the profile currently selected by the user, and unlocks the profile selected by the user based on successful face authentication. According to clause 12,
A profile authentication device that configures a personalized user interface screen corresponding to the user-selected profile based on the lock setting being released. According to clause 12,
The profile setting information includes at least one of setting information for limiting viewing age and viewing time and setting information for restricting viewing of adult content. According to clause 12,
Based on the selection of the profile registration menu on the My Menu screen of the TV app, a user interface screen for entering a mobile phone number is configured and output, a profile setting information database corresponding to the mobile phone number entered by the user is created, and a predetermined limit is set. A profile authentication device that configures and outputs a profile setting user interface screen including a setting menu, and registers user-entered restriction setting information in the profile setting information database. According to clause 15,
The profile setting user interface screen includes a face registration menu,
The profile authentication device is characterized in that the face registration procedure is performed by driving the camera according to the selection of the face registration menu. According to clause 12,
A profile authentication device, characterized in that a face registration failure message is generated and output based on the face registration failure, and the profile in which the face registration fails is maintained in an unlocked state. According to clause 12,
The facial authentication is,
A current profile face authentication procedure that performs face authentication only for the user-selected profile based on the profile automatic switching mode not being activated; and
Full profile facial authentication procedure that performs facial authentication for all pre-registered profiles based on the profile automatic switching mode being activated.
A profile authentication device comprising: According to clause 18,
The full profile face authentication process is,
Identifying a profile to be automatically converted by performing facial recognition on all profiles;
Automatically converting the user-selected profile to the identified profile
A profile authentication device comprising: According to clause 19,
Based on the failure of the face authentication through either the current profile face authentication procedure or the full profile face authentication procedure, setting a common setting value that is commonly applied to all profiles and then entering the home screen. , profile authentication device. According to clause 12,
A profile authentication device, characterized in that the facial authentication is performed using a facial recognition engine within a set-top box equipped with the profile authentication device without linking with an external server.