WO2015102126A1

WO2015102126A1 - Method and system for managing electronic album using face recognition technology

Info

Publication number: WO2015102126A1
Application number: PCT/KR2013/012414
Authority: WO
Inventors: 정진욱
Original assignee: 주식회사 시어스랩
Priority date: 2013-12-30
Filing date: 2013-12-31
Publication date: 2015-07-09
Also published as: KR101534808B1

Abstract

Disclosed are a method and a system for managing an electronic album by means of a face recognition technology. A method for managing an electronic album by means of a face recognition system according to the present invention may comprise the steps of: analyzing a plurality of images stored in a terminal to recognize the face of a particular subject contained in the images; assigning a tag to the recognized face; and grouping the images containing the particular subject by means of the tag. The method for managing an electronic album by means of the face recognition system additionally comprises the step for photographing, and the photographing step can comprise: displaying a face recognition indicator on the face of a subject being photographed when in photograph mode; if the face has already been assigned a tag, then displaying the registration information matching the face; and after photographing, if a tag has already been assigned to the face, then adding the image to the group of images containing the face.

Description

Electronic album management method and system using face recognition technology

The present invention relates to a method and apparatus for managing data in an electronic album and for sharing such data.

Recently, digital image generation and dissemination have been facilitated by activating the dissemination of a portable digital camera and a terminal equipped with a camera, and the accuracy of face recognition technology has been improved with the development of correction and analysis techniques for digital images.

Face recognition is a biometric technology that can be used safely without fear of theft, loss, or forgetting.It is not only easy to use by using a camera that is already installed, but also separately manufactured sensors such as iris and fingerprint. Most IDs have picture data, so a large amount of data is readily available.

In addition, the face is a means by which a person identifies a person, so that a manager can check the result later and is not a touch type, and thus a technology that can be easily applied as a monitoring means in a large public facility.

Face recognition first extracts facial regions using edges, colors, and geometric components of facial components (eyes, noses, mouths), and extracts feature vectors that can distinguish subtle differences between different people.

The construction of a large-scale face database considering various environmental changes in face recognition is essential for improving and evaluating the performance of the face recognition system, and considering the color and direction of lighting, various expressions, various angles, and accessories.

Face recognition technology can be applied to access control using face recognition technology without the need for access control of offices, houses, passwords that can cause forgetting, keys and IDs that can be stolen or lost, and facial information. In addition to increasing the level of security and smart card security, it can also provide personal authentication solutions for ATMs and Internet banking.

In addition, PC and server security solutions within the corporate network, as well as personal authentication and e-commerce through mobile communication devices such as mobile phones and PDAs, remote personal authentication for electronic banking and e-learning using mobile communication is possible.

Such a face recognition has a wide range of uses, such as the auto focus function that focuses on the face at the time of shooting, the ability to authenticate a person by comparing photos, and the automatic data generation of features of the person. In areas where security is required, it is used to control access and screen criminals.

In addition, recently, a digital camera or a terminal with a camera is provided with a function of adding a face recognition function. The electronic album of the existing terminal camera may perform image cleaning through location-based or time-based data. However, the face-based cleanup is not supported through face recognition, and there is no service that can be automatically shared based on face recognition.

An object of the present invention is to provide a method and system for performing group-by-group arrangement of images in an electronic album through face-based data through face recognition, and automatically sharing them based on face recognition.

In one aspect, the electronic album management method using a face recognition technology proposed in the present invention by analyzing a plurality of images stored in the terminal to recognize the face of a specific target included in the image, the recognized face The method may include assigning a tag to the group, and grouping images including the specific object by using the tag.

Recognizing a face of a specific object included in the images may automatically recognize a face of the specific object included in at least a predetermined number of images among the images.

The step of assigning a tag to the recognized face may ask a user whether to assign a tag to the face, and if the user allows a tag, match the registration information corresponding to the face.

The step of assigning a tag to the recognized face may include registering a face of an object to be automatically shared among the images photographed in the photographing mode.

Grouping the images including the specific target by using the tag may include managing the grouped images in synchronization with account information of various SNS platforms such as a user's contact, Facebook, and LinkedIn. .

The electronic album management method using face recognition technology further includes a step of shooting, wherein the step of shooting includes displaying a face recognition mark on a face of a subject to be photographed in a shooting mode, and when a tag is previously assigned to the face, The method may include displaying registration information matched to the face, and adding the tag to a group of images including the face when a tag is previously assigned to the face.

In the photographing mode, displaying a face recognition mark on a face of a subject to be photographed may be distinguished and displayed according to whether a tag is previously assigned to the face.

If the face of the subject photographed in the photographing mode is a face registered with the automatic sharing, the method may further include automatically sharing the photographed image with the subject simultaneously with the photographing.

In another aspect, the electronic album management system using the face recognition technology proposed in the present invention is to assign a tag to the face of a specific target included in a plurality of images stored in the terminal and to control to group the images A control unit, a storage unit storing the tag and the registration information by matching the registration information corresponding to the face to which the tag is assigned, and an image photographed at the same time as the shooting when the face is registered when the tag is assigned to the face. The automatic sharing may include a communication unit for automatically transmitting to the registered target.

When the tag is assigned to the face of the specific target, the controller may control to pre-register the face of the target to be shared automatically. When the tag is assigned to the face photographed in the shooting mode, the controller matches the face. The registered information may be controlled to be displayed on the display.

Currently, terminal applications through face recognition are mostly applications for fun, such as decorating, decorating, and applying face location, while according to the embodiments of the present invention, users recognize their family members and friends through face recognition. It can be easier to find by automatically organizing and tagging images and managing images. In addition, the application automatically recognizes the subject in which the picture was taken and automatically shares an image with the subject.

1 is a flowchart illustrating an electronic album management method using face recognition technology according to an embodiment of the present invention.

2 is a view for explaining a process of recognizing a face according to an embodiment of the present invention.

3 is a view for explaining a step of photographing according to an embodiment of the present invention.

4 is a diagram illustrating a process of displaying a face recognition display according to an embodiment of the present invention.

5 is a diagram illustrating a process of synchronizing a grouped image with a contact of a user according to an exemplary embodiment of the present invention.

6 is a diagram illustrating a configuration of an electronic album management system using face recognition technology according to an embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

An electronic album management method using a face recognition technique includes analyzing a plurality of images stored in a terminal to recognize a face of a specific target included in the images (110), and assigning a tag to the recognized face (120). The method may include a step 130 of grouping images including a specific object by using a tag and a step 140 of photographing.

In operation 110, a plurality of images stored in the terminal may be analyzed to recognize a face of a specific target included in the images. A plurality of images may be stored in the electronic album of the terminal. The plurality of images may be photographs or videos and the like. Faces included in these images can be recognized. For example, the face recognition method extracts a face region by using edges, colors, and geometric components of face components (eyes, noses, and mouths), and extracts feature vectors that can distinguish subtle differences of different people. Can be. These vectors can be used to distinguish different faces. In this case, a face included in at least a predetermined number of images among the plurality of images may be automatically recognized. For example, when there are three or more pictures including a specific object, the face of the specific object may be recognized.

In operation 120, a tag may be assigned to the recognized face. A tag may be assigned by automatically recognizing a face included in at least a predetermined number of images among the plurality of images. For example, when there are three or more pictures including a specific object, a tag may be assigned by recognizing the face of the specific object. In this case, the user may be asked whether to assign a tag to a face of a specific target. When the user allows a tag on a face of a specific object, the registration information of the specific object corresponding to the face may be matched. Here, the registration information may include the name, phone number, contact information such as email, SNS account information such as Facebook, LinkedIn, and the like, and the like. For example, when a specific object is 'Mom', it may be matched with related information such as 'Mom's phone number, email, Facebook profile', and the like. In addition, in the step of assigning a tag to a face, a face of a target to automatically share the images photographed in the photographing mode may be registered in advance. For example, if a tag is assigned to a face of 'mom', 'mom' can be registered as an auto-sharing target. Then, when photographing the mother, the camera may recognize the mother's face and transmit the captured image to the mother at the same time.

In operation 130, a tag may be used to group images including a specific object. A face included in at least a predetermined number of images among the plurality of images may be automatically recognized and a tag may be assigned, and the images including the face to which the tag is assigned may be grouped. For example, when a face of 'mom' is included in a predetermined number of images and a tag is assigned to the face of 'mom', the images including 'mom' may be collected and grouped. For example, the group name may be 'Mom', which is a collection of only mother's photos, or may be 'Family'. Then, the 'family' group may store the pictures including the 'mom's face. In addition, the grouped images can be managed in synchronization with the user's contacts. For example, you can synchronize and manage the contacts and images of 'Mom' included in the 'Family' group. A process of synchronizing and managing the grouped images with the user's contacts will be described in detail with reference to FIG. 5.

In operation 140, imaging may be performed. In operation 140, a face recognition display may be displayed on the face of the subject to be photographed. When a tag is previously assigned to the face of the photographing target, registration information matched to the face may be displayed. For example, the registration information may be a name. Therefore, when a tag is previously assigned to the face of the photographing subject, a name matched to the face may be displayed. After photographing such an object, when a tag is previously assigned to a face, the captured image may be added to a group of images including the face. This will be described in more detail with reference to FIG. 3.

2 is a view for explaining a process of recognizing a face according to an embodiment of the present invention. Referring to FIG. 2A, a plurality of images 210a, 220a ... stored in the terminal are illustrated. The first image 210a is an image including three

objects

211a, 212a, and 213a. It is assumed that a specific object 211a among the objects included in the first image 210a is included in the plurality of images in addition to the first image 210a. In this way, a tag may be automatically recognized by recognizing a face of a specific object 211a included in at least a predetermined number of images among the plurality of images 210a, 220a. For example, when the predetermined number is 3, when the face of the specific target 211a is included in three or more images, a tag may be assigned to the face of the specific target 211a.

Referring to FIG. 2B, the face recognition display 212b may be displayed when the tag is assigned to the face of the specific target 211a. The user may be asked whether to assign a tag to the face of the specific target 211b. When the user allows a tag on the face of a specific object, when the name of the specific object 211b is 'kim ○○', the user may tag 'kim ○○'. In addition, the registration information of the specific target 211b corresponding to the face may be matched. For example, when the specific object 211b is 'kim ○○', it may be matched with related information such as a phone number, an e-mail, and an SNS account of 'Kim ○○'. In addition, when a tag is assigned to a face, a face of an object to automatically share images photographed in the shooting mode may be registered in advance. For example, if the specific object 211b is registered as an automatic sharing object, when the specific object 211b is photographed, the photographed image may be transmitted to the specific object 211b simultaneously with the photographing.

In addition, after assigning a tag of '○○○' to a specific object 211b, images may be grouped using the tag. For example, a specific object 211b may be included in the first image of the specific object 211b, and a specific object 221b may also be included in the second image of the specific object 211b. At this time, the specific object 211b and the specific object 221b are the same person. When the number of photographs including the specific object 211b 'kim ○○' is more than a predetermined number, the plurality of

photographs

210b, 220b ... may be grouped. If 'Kim ○○' is a friend, this group name may be 'Friends'. Then, the

photos

210b, 220b ... including the face of Kim ○○ may be stored in the 'friends' group. In addition, the grouped images can be managed in synchronization with the user's contacts. Here, the contact may include a contact stored in the user terminal or an address book stored in the SNS account and the like.

The photographing step may include displaying a face recognition mark on a face of a subject to be photographed in the photographing mode (310); when a tag is assigned to a face in advance; When the face of the photographed subject is a face registered for auto sharing, the method may automatically share the photographed image with a specific subject at the same time as the photographing, and if the tag is assigned to the face after the photographing, the image includes the face. It may include the step 340 to add to the group of.

In operation 310, the face recognition display may be displayed on the face of the subject to be photographed in the photographing mode. The tag may be distinguished and displayed according to whether a tag is previously assigned to the face of the photographing target. For example, when the face of the photographing target is a face of a specific target to which a tag is previously assigned, the face recognition display may be indicated by a green line. On the other hand, when the face of the photographing target is a face of a specific target to which a tag is not previously assigned, the face recognition mark may be indicated by a red line.

In operation 320, when a tag is previously assigned to a face, registration information matched to the face may be displayed. Herein, the registration information may include a contact information such as a name, a phone number, an e-mail, or the like of a specific target. For example, when the photographing target is 'Mom' and a tag is previously assigned to the 'Mom' face, the face recognition mark may be displayed with a green line, and the tag 'Mom' may be displayed together.

In operation 330, when the face of the subject photographed in the photographing mode is a face registered for automatic sharing, the captured image may be automatically shared with the specific subject simultaneously with the photographing. For example, when the photographing target is 'Mom' and 'Mom' is registered as an auto-sharing target, when the 'Mom' is photographed, the captured image may be automatically transmitted to the 'Mom' at the same time.

In operation 340, after the photographing, if a tag is previously assigned to the face, the tag may be added to a group of images including the face. For example, when the photographing target is 'Mom' and a tag is previously assigned to the 'Mom''s face, the group name including 'Mom' may be 'Family'. Then, after capturing the 'mother', the photographed image may be added to the 'family' group.

Referring to FIG. 4, the image includes a specific object 410 and an unregistered object. In the photographing mode in which the specific object 410 and the unregistered object are photographed, the tag may be distinguished and displayed according to whether a tag is previously assigned to the face of the photographing object. For example, a tag may be previously assigned to a specific object 410, and a tag may not be assigned to an unregistered object. Then, the face recognition mark may be displayed on the face of the specific object 410 by the dotted line 420. On the other hand, the face recognition mark may be displayed on the face of the unregistered object by the solid line 430. Here, the face recognition display of the dotted line and the solid line is only an example and is not limited thereto.

In addition, in the case of the specific object 410 to which the tag is previously assigned, the registration information matched to the face may be displayed together with the face display line. Herein, the registration information may include a contact information such as a name, a phone number, an e-mail, or the like of a specific target. For example, when the tag 'Kim ○○' 421 is pre-assigned to the shooting target, the face recognition mark is displayed by a dotted line 420, and the tag 'Kim ○○' 421 is displayed together. Can be.

In addition, when the face of the subject photographed in the photographing mode is a face registered for automatic sharing, the captured image may be automatically shared with a specific subject at the same time as the photographing. For example, when 'Kim ○○' is previously registered as an object of automatic sharing, when photographing 'Kim ○○', an image photographed at the same time as the photographing may be automatically transmitted to 'Kim ○○'. When 'Kim ○○', to which the tag is assigned in advance, is a friend, the group name of the images including the face of 'Kim ○○' may be 'friend'. In addition, the image taken after the photographing of 'Kim ○○' may be added to the 'Friends' group.

The contact of the user may include a contact stored in the user terminal or an address book stored in the SNS account and the like. Referring to FIG. 5, a contact list of a user is illustrated as an example. It is assumed that the contacts of the user include contacts of A, B, and C, and their faces have been previously assigned a tag. In addition, the user's contact information includes a group containing images of names A, B, and C (510), phone number 520, account 530, e-mail 540, and images including A, B, and C faces. 550, the information of the photo 560 included in the group may be included. For example, A's name 510, phone number 520, account 530, and e-mail 540 are stored in the user's contacts, and the face of A is managed by the electronic album management method according to the embodiment of the present invention. You can assign tags in advance. Then, the user's contact information includes a group 550 that contains images containing the face of A, in addition to the information of name 510, phone number 520, account 530, email 540, Photo information 560 of A may be displayed together. Also, tags, groups, names, phone numbers, accounts, emails, etc. assigned to A's face can be edited by the user.

If the relationship with the user of A is a friend, and the names of the plurality of photographs taken are X, Y, Z ..., the group field of the user contact is displayed as' friend ', and the photo field is' X, Y, Z. .. ' In this way, grouped images can be synchronized with the user's contacts.

6 is a diagram illustrating a configuration of an electronic album management system using a face recognition technology according to an embodiment of the present invention.

The electronic album management system proposed by the present invention may include a processor 600 including a controller 610, a storage 620, a communication unit 630, a memory 601, and a network interface 602.

The memory 601 may store a program including instructions corresponding to an operating system (OS) and an electronic album management system routine. The electronic album management system can efficiently manage electronic albums by synchronizing images with contacts using face recognition technology for recognizing faces of subjects to be photographed. Steps performed in the electronic album management system described with reference to FIGS. 1 to 5 may be executed by a program stored in the memory 601. For example, the memory 601 may be a hard disk, SSD, SD card, cloud storage and other storage media.

The network interface 602 may couple the electronic album management system to the network for communication with user terminals using the electronic album management system.

The processor 600 is a device for processing according to an instruction of a program stored in the memory 601, and may include a microprocessor such as a CPU. The processor 600 may include a control unit 610, a storage unit 620, and a communication unit 630.

The controller 610 may analyze a plurality of images stored in the terminal to recognize a face of a specific object included in the images, and may control to allocate a tag to the recognized face of the specific object. At this time, a face included in at least a predetermined number of images among the images may be automatically recognized. When assigning a tag to a face, the user may be asked whether to assign a tag to the face, and if the user allows the tag, the registration information corresponding to the face may be matched. Also, a face of an object to be automatically shared among the images photographed in the photographing mode may be registered in advance. In addition, when a tag is previously assigned to a face photographed in the photographing mode, it may be controlled to display registration information matching the face on the display. After the photographing, when a tag is previously assigned to the face, the photographed image may be added to a group of images including the face.

The storage unit 620 may store the tag and the registration information by matching the registration information corresponding to the face to which the tag is assigned. In addition, by using a tag assigned to a face of a specific object, grouped information of images including the specific object may be stored. The grouped images may be stored in synchronization with the user's contacts.

When assigning a tag to a face, the communicator 630 may automatically transmit a photographed image to a target to which auto-sharing is registered when the face is registered for automatic sharing.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be, for example, processors, controllers, arithmetic logic units (ALUs), digital signal processors, microcomputers, field programmable arrays (FPAs), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of explanation, one processing device may be described as being used, but one of ordinary skill in the art will appreciate that the processing device includes a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as parallel processors.

The software may include a computer program, code, instructions, or a combination of one or more of the above, and configure the processing device to operate as desired, or process it independently or collectively. You can command the device. Software and / or data may be any type of machine, component, physical device, virtual equipment, computer storage medium or device in order to be interpreted by or to provide instructions or data to the processing device. Or may be permanently or temporarily embodied in a signal wave to be transmitted. The software may be distributed over networked computer systems so that they may be stored or executed in a distributed manner. Software and data may be stored on one or more computer readable recording media.

The method according to the embodiment may be embodied in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the media may be those specially designed and constructed for the purposes of the embodiments, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Although the embodiments have been described by the limited embodiments and the drawings as described above, various modifications and variations are possible to those skilled in the art from the above description. For example, the described techniques may be performed in a different order than the described method, and / or components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different form than the described method, or other components. Or even if replaced or substituted by equivalents, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are within the scope of the claims that follow.

Claims

Analyzing a plurality of images stored in the terminal and recognizing a face of a specific target included in the images;

Assigning a tag to the recognized face; And

Grouping images containing the specific object using the tag;

Electronic album management method comprising a.
The method of claim 1,

Recognizing a face of a specific object included in the images,

Automatically recognizing a face of a specific object included in at least a predetermined number of images among the images

How to manage your electronic albums.
The method of claim 1,

Assigning a tag to the recognized face,

Asking the user whether to assign a tag to the face, and if the user allows the tag, matching the registration information corresponding to the face

Electronic album management method comprising a.
The method of claim 1,

Assigning a tag to the recognized face,

Steps to register in advance the face of the subject to be shared among the images taken in shooting mode

Electronic album management method comprising a.
The method of claim 1,

Grouping the images containing the specific object by using the tag,

To manage the grouped images in synchronization with the user's contacts

How to manage your electronic albums.
The method of claim 1,

Steps to shoot

More,

The photographing step,

Displaying a face recognition indication on a face of a photographing target in a photographing mode;

Displaying a registration information matched to the face when a tag is previously assigned to the face; And

Adding the tag to a group of images including the face when a tag is previously assigned to the face after the capturing

Electronic album management method comprising a.
The method of claim 6,

In the photographing mode, displaying a face recognition mark on a face of a subject to be photographed

Differentiating and displaying the tag according to whether the tag has been previously assigned to the face

How to manage your electronic albums.
The method of claim 6,

If the face of the subject photographed in the photographing mode is a face registered with the automatic sharing, automatically sharing the photographed image with the subject at the same time as the photographing;

Electronic album management method comprising more.
A controller configured to assign a tag to a face of a specific object included in a plurality of images stored in a terminal and to group the images;

A storage unit which matches the registration information corresponding to the face to which the tag is assigned and stores the tag and the registration information; And

When the tag is assigned to the face, if the face is registered for automatic sharing, the communication unit for automatically transmitting the image taken at the same time as the shooting to the subject registered with the automatic sharing

Electronic album management system comprising a.
The method of claim 9,

The control unit,

When assigning a tag to the face of the specific target, it is possible to control to pre-register the face of the target to be shared automatically,

When a tag is assigned to a face photographed in the photographing mode in advance, a control for displaying registration information matching the face is displayed on the display.

Electronic album management system.