WO2021118048A1 - Dispositif électronique et procédé de commande associé - Google Patents

Dispositif électronique et procédé de commande associé Download PDF

Info

Publication number
WO2021118048A1
WO2021118048A1 PCT/KR2020/014563 KR2020014563W WO2021118048A1 WO 2021118048 A1 WO2021118048 A1 WO 2021118048A1 KR 2020014563 W KR2020014563 W KR 2020014563W WO 2021118048 A1 WO2021118048 A1 WO 2021118048A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
input image
processed
difference
information
Prior art date
Application number
PCT/KR2020/014563
Other languages
English (en)
Inventor
Vitaly Sergeevitch GNATYUK
Alena Dmitrievna MOSKALENKO
Alexey Mikhailovitch FARTUKOV
Vladimir Petrovitch PARAMONOV
Gleb Andreevitch ODINOKIKH
Vladimir Alekseevitch EREMEEV
Ivan Andreevitch SOLOMATIN
Yury Sergeevitch EFIMOV
Ivan Sergeevitch PECHENKO
Viktor Evseevitch MOROZOV
Ignaty Arkadyevitch DUBYSHKIN
Juwoan YOO
Kwanghyun Lee
Heejun Lee
Yangsoo Lee
Original Assignee
Samsung Electronics Co., Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from RU2019140786A external-priority patent/RU2735629C1/ru
Application filed by Samsung Electronics Co., Ltd. filed Critical Samsung Electronics Co., Ltd.
Publication of WO2021118048A1 publication Critical patent/WO2021118048A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/30—Authentication, i.e. establishing the identity or authorisation of security principals
    • G06F21/31—User authentication
    • G06F21/32—User authentication using biometric data, e.g. fingerprints, iris scans or voiceprints
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G06F18/24133—Distances to prototypes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/40—Analysis of texture
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/24—Aligning, centring, orientation detection or correction of the image
    • G06V10/245—Aligning, centring, orientation detection or correction of the image by locating a pattern; Special marks for positioning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Definitions

  • the disclosure relates to electronic devices for performing user authentication through face recognition and controlling methods of the electronic devices.
  • face recognition is widely used for security purposes, for example, for unlocking a mobile device.
  • Most face recognition methods work well to recognize different people.
  • the existing face recognition methods do not provide sufficient security because a mobile device may be unlocked by using an image of a user, etc.
  • a mobile device may be unlocked by a twin and sibling of a user through face recognition.
  • even the most advanced mobile devices comprising a time-of-flight (ToF) camera may not successfully recognize between twins despite the use of additional depth measurement and depth-based information with regard to the face shape.
  • ToF time-of-flight
  • Chinese patent application 105046219A entitled “Face Identification System” discloses a technical solution that generates three-dimensional (3D) images on the basis of 2D face images, compares the 3D images generated from 2D face images with 3D images generated on the basis of photos in a photo database and outputs results.
  • this face identification system has the following drawbacks: high computational complexity is not suitable for mobile devices, and face recognition may be used only for recognition under indoor light conditions and may fail under outdoor conditions.
  • US Patent 9639740B2 entitled “Face Detection and Recognition” discloses a face recognition technique based on principal component analysis (PCA).
  • PCA principal component analysis
  • this face detection and recognition has the following drawbacks: calculations are performed in a cloud server due to high computational load, which prevents using this method in mobile devices, and eye coordinates are derived based on the face coordinates, which makes the method unstable for different face shapes and may cause recognition errors.
  • global histogram equalization is used in the method as a preprocessing technique, which may cause recognition errors under different lighting conditions.
  • FIG. 1 is a diagram for describing a user authentication operation according to an embodiment of the disclosure
  • FIG. 2 is a diagram for describing a pre-processing operation according to an embodiment of the disclosure
  • FIG. 3 is a diagram for describing an operation of extracting a structural difference of a face according to an embodiment of the disclosure
  • FIG. 4 is a diagram for describing an operation related to asymmetric information according to an embodiment of the disclosure.
  • FIG. 5 is a diagram for describing an operation of extracting asymmetric information according to an embodiment of the disclosure
  • FIG. 6 is a diagram for describing an operation of extracting asymmetric information according to an embodiment of the disclosure.
  • FIG. 7 is a diagram for describing an operation of extracting asymmetric information according to an embodiment of the disclosure.
  • FIG. 8 is a diagram for describing asymmetric information according to an embodiment of the disclosure.
  • FIG. 9 is a diagram for describing an operation of extracting depth information according to an embodiment of the disclosure.
  • FIG. 10 is a diagram for describing an operation of extracting liveness information according to an embodiment of the disclosure.
  • FIG. 11 is a diagram illustrating an electronic device that performs user authentication according to an embodiment of the disclosure.
  • a controlling method of an electronic device includes pre-processing a stored user image and an input image; determining a difference between the pre-processed user image and the pre-processed input image; extracting face texture information of each of the pre-processed user image and the pre-processed input image; and performing user authentication based on the extracted face texture information and the determined difference.
  • the pre-processing may include identifying a facial landmark in each of the stored user image and the input image; aligning the stored user image and the input image based on the identified facial landmark; and equalizing illumination conditions of the aligned user image and the aligned input image.
  • the difference between the pre-processed user image and the pre-processed input image may be a difference between corresponding pixel values.
  • the controlling method may further include extracting a structural difference of a face from the determined difference, and the performing of user authentication may include performing user authentication based on the extracted structural difference of the face.
  • the controlling method may further include extracting asymmetric information from each of the pre-processed user image and the pre-processed input image; and determining a difference between the extracted asymmetric information, and the performing of user authentication may include performing user authentication based on the determined difference between the asymmetric information.
  • the extracted asymmetric information may be extracted by using at least one of a histogram distance method, a block distance method, or a kurtosis method.
  • the asymmetric information of each of the pre-processed user image and the pre-processed input image may be asymmetric information of a facial landmark.
  • the controlling method may further include extracting depth information from each of the stored user image and the input image, and the performing of user authentication may include performing user authentication based on the extracted depth information.
  • the controlling method may further include extracting liveness information from the input image, and the performing of user authentication may include performing user authentication based on the extracted liveness information.
  • an electronic device includes a memory storing a user image; an input/output interface; and at least one processor configured to pre-process the stored user image and an input image transmitted from the input/output interface, determine a difference between the pre-processed user image and the pre-processed input image, extract face texture information of each of the pre-processed user image and the pre-processed input image, and perform user authentication based on the extracted face texture information and the determined difference.
  • the at least one processor may be further configured to identify a facial landmark in each of the stored user image and the input image; align the stored user image and the input image based on the identified facial landmark; and equalize illumination conditions of the aligned user image and the aligned input image.
  • the difference between the pre-processed user image and the pre-processed input image may be a difference between corresponding pixel values.
  • the at least one processor may be further configured to extract a structural difference of a face from the determined difference, and perform user authentication based on the extracted structural difference of the face.
  • the at least one processor may be further configured to extract asymmetric information from each of the pre-processed user image and the pre-processed input image, determine a difference between the extracted asymmetric information, and perform user authentication based on the determined difference between the asymmetric information.
  • the extracted asymmetric information may be extracted by using at least one of a histogram distance method, a block distance method, or a kurtosis method.
  • the asymmetric information of each of the pre-processed user image and the pre-processed input image may be asymmetric information of a facial landmark.
  • the at least one processor may be further configured to extract depth information from each of the stored user image and the input image, and perform user authentication based on the extracted depth information.
  • the at least one processor may be further configured to extract liveness information from the input image, and perform user authentication based on the extracted liveness information.
  • a non-transitory computer-readable recording medium having recorded thereon a program for executing a controlling method of an electronic device.
  • the expression "at least one of a, b or c" indicates only a, only b, only c, both a and b, both a and c, both b and c, all of a, b, and c, or variations thereof.
  • FIG. 1 is a diagram for describing a user authentication operation according to an embodiment of the disclosure.
  • Operation 100 is an operation of pre-processing a stored user image and an input image.
  • the pre-processing operation is a process of processing data such that operations after pre-processing may be performed accurately and quickly. A specific pre-processing operation will be described in FIG. 2.
  • Operation 110 is an operation of determining a difference 111 between the pre-processed user image and the pre-processed input image.
  • the difference 111 between the pre-processed user image and the pre-processed input image may be a difference between corresponding pixel values.
  • Face texture information means information about a facial skin.
  • the face texture information may include skin texture information indicating skin roughness or softness, or skin tone information indicating a skin color.
  • Operation 120 is an operation of extracting face texture information 121 of each of the pre-processed user image and the pre-processed input image.
  • the face texture information 121 may be a vector value extracted through a convolution layer of a trained convolutional neural network (CNN).
  • CNN convolutional neural network
  • the face texture information 121 in the cheek may be extracted by inputting pixel values of a cheek part into the trained CNN.
  • Operations 110 and 120 are described in order for convenience of understanding, but may be performed in parallel, and the user authentication operation according to the disclosure may be performed even when the order of operations 110 and 120 changes.
  • Operation 130 is an operation of performing user authentication based on the extracted face texture information 121 and the determined difference 111.
  • User authentication may be performed by transmitting the extracted face texture information 121 and the determined difference 111 between the user image and the input image to a fully connected layer of the trained CNN.
  • the operation of performing user authentication based on the extracted face texture information 121 and the determined difference 111 may be performed by a Siamese neural network using the same CNN with respect to different inputs.
  • the operation of performing user authentication may be used when identity verification is required.
  • the user may perform various functions such as terminal unlocking 131, viewing personal information, or money transfer, etc. through user authentication.
  • FIG. 2 is a diagram for describing a pre-processing operation according to an embodiment of the disclosure.
  • Facial landmarks mean parts that constitute a person's face.
  • facial landmarks may include eyes, noses, mouths, chins or eyebrows.
  • Operation 200 is an operation of identifying a facial landmark in each of a stored user image and an input image.
  • Each region of the identified facial landmarks may be expressed in at least one dot.
  • a facial landmark 201 in the stored user image and a facial landmark 202 in the input image may be expressed in a plurality of points on eyes, nose, and mouth parts.
  • a specific operation of identifying the facial landmarks may be implemented using the technology disclosed in Korean Patent No. 10-1795264 related to a face landmark detection device and a verification method thereof.
  • Operation 210 is an operation of aligning the stored user image and the input image based on the identified facial landmarks.
  • the stored user image and the input image may be arranged such that the facial landmark 202 in the input image is located at a position corresponding to the facial landmark 201 in the user image.
  • the stored user image and the input image may be aligned based on an arbitrary line passing through the identified facial landmarks.
  • the stored user image and the input image may be aligned 211 and 212 with respect to a line parallel to the nose.
  • User authentication through image comparison is performed by comparing objects included in an image.
  • the pixel values of the image change according to the illumination conditions of an object, for example, an image capture angle, a shadow, a light intensity, indoor/outdoor, etc.
  • user authentication may not be performed when the illumination conditions change.
  • the user may not set the same illumination condition every time for authentication. Therefore, it is necessary to equalize the illumination conditions of images between the images to be compared.
  • Operation 220 is an operation of equalizing the illumination conditions of the aligned user image and the aligned input image.
  • the illumination conditions in the aligned user image and the aligned input image may be equalized through a histogram equalization method and a single/multi-scale retinex method.
  • the illumination conditions in the aligned user image and the aligned input image may be removed through a self-quotient image (SQI) method.
  • SQL self-quotient image
  • a user image 221 and an input image 222 from which the illumination conditions are removed may perform user authentication through object comparison of images regardless of the illumination conditions.
  • the pre-processing operation enables to accurately and quickly perform an operation after pre-processing by aligning images and equalizing the illumination conditions, for example, operation 110 of determining a difference between a user image and an input image. Accordingly, because the computing resources required in the operation after pre-processing are reduced, the computing load may be reduced.
  • FIG. 3 is a diagram for describing an operation of extracting a structural difference of a face according to an embodiment of the disclosure. Operations redundant with FIGS. 1 and 2 will be briefly described.
  • a stored user image and an input image are pre-processed, and a difference between the pre-processed user image and the pre-processed input image is determined.
  • face texture information of each of the pre-processed user image and the pre-processed input image is extracted.
  • a pre-processing operation includes operations of identifying facial landmarks in the stored user image and the input image, aligning the stored user image and the input image based on the identified facial landmarks, and equalizing illumination conditions of the aligned user image and the aligned input image.
  • Operation 310 is an operation of extracting the structural difference of the face from the determined difference.
  • the structural difference of the face means a facial landmark, for example, a component such as an eye, nose, mouth or chin and a positional difference.
  • the difference in the component of the facial landmark includes a difference 311 in length of the eyebrows and differences 314 and 315 whether or not a mole is present.
  • the difference in the position of the facial landmark includes a difference 312 in the position of the eye, a difference 313 in the position of the nose, or a difference 316 in the position of the mouth.
  • Operation 330 is an operation of performing user authentication based on the extracted face texture information and the extracted structural difference of the face.
  • user authentication may be performed by transmitting the extracted face texture information and the extracted structural difference of the face to a fully connected layer of a trained CNN.
  • user authentication may be performed by a Siamese neural network.
  • FIG. 4 is a diagram for describing an operation related to asymmetric information according to an embodiment of the disclosure. Operations redundant with FIGS. 1 and 2 will be briefly described.
  • a stored user image and an input image are pre-processed, and a difference between the pre-processed user image and the pre-processed input image is determined.
  • face texture information of each of the pre-processed user image and the pre-processed input image is extracted.
  • a pre-processing operation includes operations of identifying facial landmarks in the stored user image and the input image, aligning the stored user image and the input image based on the identified facial landmarks, and equalizing illumination conditions of the aligned user image and the aligned input image.
  • Operation 410 is an operation of extracting the asymmetric information from each of the pre-processed user image and the pre-processed input image.
  • the asymmetric information means a degree to which left and right images match or mismatch with respect to an arbitrary line.
  • the asymmetric information includes a degree 415 to which the left and the right match with respect to a line parallel to the nose.
  • the asymmetric information may be extracted by using various methods. A specific operation of extracting the asymmetric information in this regard will be described with reference to FIGS. 5 to 7.
  • Operation 420 is an operation of determining a difference between the asymmetric information extracted from the user image and the asymmetric information extracted from the input image.
  • the asymmetric information may be extracted by using various methods.
  • the difference between the asymmetric information extracted from the user image and the asymmetric information extracted from the input image may be a difference between asymmetric information extracted by using the corresponding method.
  • Operation 430 is an operation of performing user authentication based on the extracted face texture information, the determined difference between the user image and the input image, and the determined difference between the asymmetric information.
  • user authentication may be performed by transmitting the extracted face texture information, the determined difference between the user image and the input image, and the determined difference between the asymmetric information to a fully connected layer of a trained CNN.
  • user authentication may be performed by a Siamese neural network.
  • Face recognition is generally performed based on a structural difference in the face.
  • face recognition based on the structural difference is likely to cause errors for people with small structural differences, such as twins, or siblings. Therefore, face recognition based on the face texture information, the asymmetric information, and the structural difference of the face may perform user authentication without error even on persons who have a small structural difference.
  • FIGS. 5 to 7 are diagrams for describing an operation of extracting asymmetric information according to an embodiment of the disclosure.
  • a histogram distance method 500 separates an input image 505 with respect to a line perpendicular to a row for row-by-row comparison.
  • Row-by-row comparison means comparing image histograms (e.g., 530 and 540, 570 and 580) in the same row (e.g., 510 and 520, 550 and 560).
  • the asymmetric information extracted by using the histogram distance method 500 may be a vector having a difference value between histograms in the same row as a component.
  • the difference value between the image histogram 530 (or 570) and the image histogram 540 (or 580) is determined for each row as the average value of a pixel value difference
  • the asymmetric information is a vector having the average value of the pixel value difference as a component.
  • a block distance method 600 divides an input image 605 into set grids 610 and calculates distances 615, 620, 625, and 630 between the corresponding grids 610.
  • the asymmetric information extracted by using the block distance method 600 may be a vector having the distances 615, 620, 625, and 630 between the corresponding grids 610 as components.
  • the distances 615, 620, 625, and 630 between the corresponding grids 610 are calculated for all grids and only some of them are shown in the drawing.
  • a kurtosis method calculates skewness of histograms 715 and 720 for each window size 710 set in an input image 705.
  • the asymmetry information extracted by the kurtosis method may be a vector having the skewness of the histograms 715 and 720 as components.
  • the number of histograms is determined by the size of the input image 705 and the set window size 710, and only some of them are shown in the drawing.
  • a method of extracting the asymmetric information is described above but is not limited thereto, and the asymmetric information may be extracted using various methods such as the Negentropy based method, the Harris detector based method, the Harris-Laplace detector based method, the Fourier transform based method, the Bhattacharyya distance based method, the Intersection based method, etc.
  • FIG. 8 is a diagram for describing asymmetric information according to an embodiment of the disclosure. Operations redundant with FIGS. 5 to 7 will be briefly described.
  • the asymmetric information is extracted by using various methods such as a histogram distance method, a block distance method, or a Kurtosis method.
  • the histogram distance method separates an input image with respect to a line perpendicular to a row, and calculates a difference value between image histograms of the same row.
  • the block distance method divides the input image into set grids and calculates distances between the corresponding grids.
  • the Kurtosis method calculates skewness of a histogram for each window size set in the input image.
  • the asymmetric information may only be extracted with respect to facial landmarks.
  • the asymmetric information may only be extracted with respect to a right forehead 810 and a left forehead 815, a right eye 820 and a left eye 825, a right nose 840 and a left nose 845, a right cheek 830 and a left cheek 835, a right chin 850 and a left chin 855 or a mouth 860.
  • the asymmetric information may be extracted by using different methods for each facial landmark.
  • the asymmetric information of the right forehead 810 and the left forehead 815 may be extracted by using the histogram distance method
  • the asymmetric information of the right eye 820 and the left eye 825 may be extracted by using the block distance method
  • the asymmetric information of the right nose 840 and the left nose 845 may be extracted by using the Kurtosis method.
  • the asymmetric information may be extracted by using different methods for the same facial landmark.
  • the asymmetric information of the right forehead 810 and the left forehead 815 may be extracted by using the histogram distance method, the block distance method, and the Kurtosis method.
  • FIG. 9 is a diagram for describing an operation of extracting depth information according to an embodiment of the disclosure. Operations redundant with FIGS. 1 and 2 will be briefly described.
  • a stored user image and an input image are pre-processed, and a difference between the pre-processed user image and the pre-processed input image is determined.
  • face texture information of each of the pre-processed user image and the pre-processed input image is extracted.
  • a pre-processing operation includes operations of identifying facial landmarks in the stored user image and the input image, aligning the stored user image and the input image based on the identified facial landmarks, and equalizing illumination conditions of the aligned user image and the aligned input image.
  • Operation 900 is an operation of extracting the depth information from each of the stored user image and the input image.
  • the depth information means a distance 905 of a facial landmark, for example, an eye, a nose, a mouth, or a chin, from a camera that captures an image.
  • the depth information may be extracted from an image through a convolution layer of a trained CNN.
  • Operation 950 is an operation of performing user authentication based on the extracted face texture information, the extracted depth information, and the determined difference.
  • user authentication may be performed by transmitting the extracted face texture information, the extracted depth information, and the determined difference between the user image and the input image to a fully connected layer of a trained CNN.
  • user authentication may be performed by a Siamese neural network.
  • user authentication may be performed based on the extracted face texture information, the extracted depth information, the determined difference between the user image and the input image, and a difference between asymmetric information.
  • face recognition may be performed without error even on persons who have a small structural difference such as twins or siblings.
  • FIG. 10 is a diagram for describing an operation of extracting liveness information according to an embodiment of the disclosure. Operations redundant with FIGS. 1 and 2 will be briefly described.
  • a stored user image and an input image are pre-processed, and a difference between the pre-processed user image and the pre-processed input image is determined.
  • face texture information of each of the pre-processed user image and the pre-processed input image is extracted.
  • a pre-processing operation includes operations of identifying facial landmarks in the stored user image and the input image, aligning the stored user image and the input image based on the identified facial landmarks, and equalizing illumination conditions of the aligned user image and the aligned input image.
  • Face recognition needs to block spoofing attacks such as a 3D mask 1001, a face photo 1002 or a video 1003. Therefore, it is necessary to perform user authentication based on the liveness information which is information used to determine whether a user in the image is living being.
  • Operation 1000 is an operation of extracting the liveness information from the input image.
  • the liveness information may be a vector value extracted through a convolution layer of a trained CNN.
  • the convolution layer may extract the liveness information by comparing a user and a background of the face photo 1002.
  • Operation 1050 is an operation of performing user authentication based on the extracted face texture information, the extracted liveness information, and the determined difference.
  • user authentication may be performed by transmitting the extracted face texture information, the extracted liveness information, and the determined difference to a fully connected layer of a trained CNN.
  • user authentication may be performed by a Siamese neural network.
  • FIG. 11 is a diagram illustrating an electronic device 1100 that performs user authentication according to an embodiment of the disclosure.
  • the electronic device 1100 may include an input/output interface 1110, a memory 1120, and at least one processor 1130.
  • an element included in the electronic device 1100 may be implemented by one of software, hardware, and firmware, or a plurality of elements may be implemented by one component, or one element may include a plurality of components.
  • the electronic device 1100 may be implemented in various forms.
  • the electronic device 1100 described in the present specification may include a smartphone, a tablet personal computer (PC), a mobile phone, a video phone, an e-book reader, a desktop PC, a laptop PC, a netbook computer, a copyrighted workstation, a server, a personal digital assistant (PDA), a portable multimedia player (PMP), an MP3 player, a mobile medical apparatus, a camera, a wearable device, a lamp, a weighing scale, or a navigation device, but the disclosure is not limited thereto.
  • the electronic device 1100 may be home appliance.
  • the home appliance may include, for example, a TV, a digital video disk (DVD) player, an audio system, a refrigerator, an air conditioner, a vacuum cleaner, an oven, a microwave, a washing machine, an air cleaner, a set-top box, a home automation control panel, a security control panel, a game console, an electronic key, a camcorder, etc., but the disclosure is not limited thereto.
  • DVD digital video disk
  • the input/output interface 1110 may include an interface capable of at least one of input or output.
  • the input/output interface 1110 may include a display, a touch screen, a touch pad, an audio input/output portion, a camera, an HDMI, or a USB, or a combination thereof, but the disclosure is not limited thereto.
  • the memory 1120 may store programs for processing and controlling the processor 1130, and pieces of input/output information (e.g., an image of a user, an input image, etc.)
  • the memory 1120 may include at least one type of storage media such as a flash memory type, a hard disk type, a multimedia card micro type, a card type memory, for example, secure digital (SD) or extreme digital (XD) memory, random access memory (RAM) static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), a magnetic memory, a magnetic disc, or an optical disc.
  • SD secure digital
  • XD extreme digital
  • RAM random access memory
  • ROM read-only memory
  • EEPROM electrically erasable programmable read-only memory
  • PROM programmable read-only memory
  • the electronic device 1100 may use a web storage or a cloud server that provides a storage function of the memory 1120 on the Internet.
  • the electronic device 1100 communicating with a web storage or a cloud server, may store a signal or data in the web storage or the cloud server, or receive a signal or data from the web storage or the cloud server.
  • the processor 1130 may control the overall operation of the electronic device 1100.
  • the processor 1130 may generally control the input/output interface 110, etc. by executing programs stored in the memory 1120.
  • the processor 1130 may include one or more processors.
  • the processor 1130 pre-processes the image of the user stored in the memory 1120 and the input image transmitted from the input/output interface 1110. Pre-processing is a process in which the processor 1130 processes data such that operations after pre-processing may be performed accurately and quickly.
  • pre-processing includes identifying facial landmarks in the user image stored in the memory 1120 and the input image, aligning the stored user image and the input image based on the identified facial landmarks, and equalizing illumination conditions of the aligned user image and the aligned input image.
  • the processor 1130 determines a difference between the pre-processed user image and the pre-processed input image.
  • the difference between the pre-processed user image and the pre-processed input image may be a difference between corresponding pixel values.
  • the processor 1130 may extract a structural difference of the face from the determined difference.
  • the processor 1130 extracts face texture information of each of the pre-processed user image and the pre-processed input image.
  • the face texture information may be a vector value extracted through a convolution layer of a trained CNN.
  • the processor 1130 may extract depth information from each of the user image stored in the memory 1120 and the input image.
  • the processor 1130 may extract liveness information from the input image transmitted from the input/output interface 1110.
  • the processor 1130 may extract asymmetric information from each of the pre-processed user image and the pre-processed input image and determines a difference between the extracted asymmetric information.
  • the asymmetric information may be extracted by using various methods.
  • the difference between the asymmetric information extracted from the user image and the asymmetric information extracted from the input image may be a difference between asymmetric information extracted by using the corresponding method.
  • the processor 1330 performs user authentication based on the extracted face texture information and the difference between the pre-processed user image and the pre-processed input image.
  • User authentication may be performed by transmitting the extracted face texture information and the determined difference between the user image and the input image to a fully connected layer of the trained CNN.
  • the processor 1130 may perform user authentication by a Siamese neural network using the same CNN with respect to different inputs based on the extracted face texture information and the determined difference.
  • the processor 1130 may perform user authentication by at least one of the extracted structural difference of the face, the determined difference between asymmetric information, the extracted depth information, or the extracted liveness information. .
  • the embodiments of the disclosure may be implemented as an S/W program that includes instructions stored on computer-readable storage media. Furthermore, the embodiments of the disclosure may be implemented as a computer-readable storage medium storing a computer program.
  • a computer is a device capable of calling a stored instruction from a storage medium and operating according to the embodiment of the disclosure according to the called instruction, and may include an electronic device according to the embodiment of the disclosure.
  • a device-readable storage medium may be provided in the form of a non-transitory storage medium.
  • non-transitory storage medium means a tangible device and does not include a signal, e.g., an electromagnetic wave, and this term does not distinguish between the case where data is stored semi-permanently and temporarily.
  • the "non-transitory storage medium” may include a buffer where data is temporarily stored.
  • the electronic device and the method of operation according to the embodiments of the disclosure may be provided by being included in a computer program product.
  • the computer program products are products that may be traded between sellers and buyers.
  • the computer program products may be distributed in the form of a S/W program, a computer-readable storage medium where the SW program is stored, and a computer-readable storage medium, e.g., a compact disc read only memory (CD-ROM), through an application store, e.g., PlayStoreTM, or directly between two user devices, e.g., smartphones, or online, e.g., download or upload.
  • a computer program product e.g., a downloadable application
  • a computer program product may include a storage medium of a server or a storage medium of a terminal in a system including a server and a terminal, e.g., an electronic device, a portable electronic device, a wearable device, etc.
  • a third device e.g., a smartphone
  • the computer program product may include a storage medium of the third device.
  • the computer program product may be transmitted from the server to the terminal or the third device, or may include a S/W program that is transmitted from the third device to the terminal.
  • server, one of the terminal and the third device may perform a method according to the embodiments of the disclosure by executing the computer program product.
  • the method according to the embodiments of the disclosure may be performed in a distributed fashion.
  • a server for example, a cloud server or an artificial intelligence server, etc., may execute a computer program product stored in the server to control the terminal communicatively connected to the server to perform the method according to the embodiments of the disclosure.
  • the third device may execute a computer program product to control the terminal communicatively connected to the third device to perform the method according to the embodiment of the disclosure.
  • the third device may remotely control the electronic device to perform a controlling method of the electronic device.
  • the third device may download the computer program product from the server, and execute a downloaded computer program product.
  • the third device may execute a computer program product provided in a preloaded state to perform the method according to the embodiments of the disclosure.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Human Computer Interaction (AREA)
  • Computer Security & Cryptography (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Computer Hardware Design (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Collating Specific Patterns (AREA)

Abstract

Un procédé de commande d'un dispositif électronique consiste à prétraiter une image stockée d'utilisateur et une image d'entrée ; à déterminer une différence entre l'image prétraitée d'utilisateur et l'image prétraitée d'entrée ; à extraire des informations de texture de visage de chacune de l'image prétraitée d'utilisateur et de l'image prétraitée d'entrée ; et à effectuer une authentification d'utilisateur en fonction des informations extraites de texture de visage et de la différence déterminée.
PCT/KR2020/014563 2019-12-10 2020-10-23 Dispositif électronique et procédé de commande associé WO2021118048A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
RU2019140786A RU2735629C1 (ru) 2019-12-10 2019-12-10 Способ распознавания близнецов и ближайших родственников для мобильных устройств и мобильное устройство его реализующее
RU2019140786 2019-12-10
KR1020200069927A KR20210073434A (ko) 2019-12-10 2020-06-09 전자 장치 및 그 제어 방법
KR10-2020-0069927 2020-06-09

Publications (1)

Publication Number Publication Date
WO2021118048A1 true WO2021118048A1 (fr) 2021-06-17

Family

ID=76330506

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2020/014563 WO2021118048A1 (fr) 2019-12-10 2020-10-23 Dispositif électronique et procédé de commande associé

Country Status (1)

Country Link
WO (1) WO2021118048A1 (fr)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120320181A1 (en) * 2011-06-16 2012-12-20 Samsung Electronics Co., Ltd. Apparatus and method for security using authentication of face
US20140307929A1 (en) * 2012-06-26 2014-10-16 Google, Inc. Facial recognition
US20150067787A1 (en) * 2013-08-30 2015-03-05 David Stanasolovich Mechanism for facilitating dynamic adjustments to computing device characteristics in response to changes in user viewing patterns
US20180211096A1 (en) * 2015-06-30 2018-07-26 Beijing Kuangshi Technology Co., Ltd. Living-body detection method and device and computer program product
US20180373859A1 (en) * 2015-12-15 2018-12-27 Applied Recognition Inc. Systems and methods for authentication using digital signature with biometrics
US20190228556A1 (en) * 2016-09-21 2019-07-25 Intel Corporation Estimating accurate face shape and texture from an image

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120320181A1 (en) * 2011-06-16 2012-12-20 Samsung Electronics Co., Ltd. Apparatus and method for security using authentication of face
US20140307929A1 (en) * 2012-06-26 2014-10-16 Google, Inc. Facial recognition
US20150067787A1 (en) * 2013-08-30 2015-03-05 David Stanasolovich Mechanism for facilitating dynamic adjustments to computing device characteristics in response to changes in user viewing patterns
US20180211096A1 (en) * 2015-06-30 2018-07-26 Beijing Kuangshi Technology Co., Ltd. Living-body detection method and device and computer program product
US20180373859A1 (en) * 2015-12-15 2018-12-27 Applied Recognition Inc. Systems and methods for authentication using digital signature with biometrics
US20190228556A1 (en) * 2016-09-21 2019-07-25 Intel Corporation Estimating accurate face shape and texture from an image

Similar Documents

Publication Publication Date Title
TWI714225B (zh) 注視點判斷方法和裝置、電子設備和電腦儲存介質
CN110232369B (zh) 一种人脸识别方法和电子设备
CN108764071B (zh) 一种基于红外和可见光图像的真实人脸检测方法及装置
WO2014069822A1 (fr) Appareil et procédé de reconnaissance de visage
JP6629513B2 (ja) ライブネス検査方法と装置、及び映像処理方法と装置
WO2017204596A1 (fr) Procédé et dispositif de correction de contours faciaux
WO2017010695A1 (fr) Appareil de génération de contenu tridimensionnel et procédé de génération de contenu tridimensionnel associé
WO2019050360A1 (fr) Dispositif électronique et procédé de segmentation automatique d'être humain dans une image
WO2016112630A1 (fr) Système et procédé de reconnaissance d'image
WO2016163755A1 (fr) Procédé et appareil de reconnaissance faciale basée sur une mesure de la qualité
WO2013048160A1 (fr) Procédé de reconnaissance de visage, appareil et support d'enregistrement lisible par ordinateur pour exécuter le procédé
CN110956114A (zh) 人脸活体检测方法、装置、检测系统及存储介质
CN111144366A (zh) 一种基于联合人脸质量评估的陌生人脸聚类方法
CN108108711B (zh) 人脸布控方法、电子设备及存储介质
WO2018012928A1 (fr) Procédé d'authentification d'un utilisateur au moyen de la reconnaissance de son visage et dispositif associé
US11734954B2 (en) Face recognition method, device and electronic equipment, and computer non-volatile readable storage medium
WO2021020814A1 (fr) Dispositif électronique de mise en place d'avatar et son procédé d'exploitation
WO2020006964A1 (fr) Procédé et dispositif de détection d'image
KR20190136349A (ko) 3d 영상 기반의 얼굴 인증 방법 및 장치
WO2021177544A1 (fr) Système et procédé de reconnaissance faciale capable de mettre à jour un modèle facial enregistré
WO2019168264A1 (fr) Dispositif électronique et son procédé de commande
WO2015102126A1 (fr) Procédé et système pour gérer un album électronique à l'aide d'une technologie de reconnaissance de visage
EP3459009A2 (fr) Procédé de quantification adaptative pour codage d'image d'iris
WO2014088125A1 (fr) Dispositif de photographie d'images et procédé associé
TW202105329A (zh) 人臉校驗方法、裝置、伺服器及可讀儲存媒介

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20898350

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20898350

Country of ref document: EP

Kind code of ref document: A1