KR20080113843A - Apparatus and method for user information retrieval using face detection in portable communication system - Google Patents

Abstract

An apparatus and a method for searching user information by using face recognition in a portable terminal are provided to set phonebook information corresponding to a face area of image data. An apparatus for searching user information by using face recognition in a portable terminal comprises a face recognizing unit(104) and a control unit(100). The face recognizing unit recognizes a face area from an image data by using characteristic information necessary for extracting the face area. The control unit stores tag information corresponding to a face of the face area recognized in the face recognizing unit. The control unit processes a process of providing tag information corresponding to the face of the face area of the image data.

Description

Apparatus and method for retrieving user information through face recognition in a portable terminal {APPARATUS AND METHOD FOR USER INFORMATION RETRIEVAL USING FACE DETECTION IN PORTABLE COMMUNICATION SYSTEM}

1 is a flowchart illustrating a configuration of a portable terminal for providing detailed information of a person corresponding to a face region of image data according to an exemplary embodiment of the present invention;

2 is a flowchart illustrating a process for tagging detailed information of a person corresponding to a face area of image data in a portable terminal according to an exemplary embodiment of the present invention;

3 is a diagram illustrating a format of tag information corresponding to a face region of image data according to an exemplary embodiment of the present invention;

4 is a flowchart illustrating a process for outputting detailed information of a person corresponding to a face region of image data in a portable terminal according to an embodiment of the present invention;

5 is a flowchart illustrating a process for outputting detailed information of a person corresponding to a face region of image data in a portable terminal according to another exemplary embodiment of the present invention;

FIG. 6 is a diagram illustrating a screen outputting detailed information of a person corresponding to a face area of image data according to an exemplary embodiment of the present invention. FIG.

The present invention relates to an apparatus and method for face recognition of image data in a portable terminal, and more particularly, to an apparatus and method for recognizing a face region in image data and providing personal information corresponding to the recognized face region.

Recently, with the rapid development of portable terminals, it is indispensable for modern people, regardless of age or gender, and is used as a medium for wireless voice calls and information exchange.

In the early stages of the spread of portable terminals, it was recognized that they could simply be carried and wirelessly made available. However, as the technology has developed, service providers and terminal manufacturers have competitively developed products (or services) to differentiate themselves from others. have.

For example, the portable terminal may include a phone book, a game, a short message, an e-mail, a morning call, an MP 3 layer, and a schedule management function. It also develops into a multimedia device capable of digital cameras and wireless internet services to provide various services.

The digital camera function as described above provides an advantage of taking an image at any time without carrying a separate camera, so that users can easily take an image at any time and store it, and check the captured image at any time without restriction of time and place.

However, when confirming the photographed image as described above, the user may simply confirm the photographed image and thus may not link the stored image photographed by the digital camera function with other additional functions.

For example, when a user of the portable terminal checks an image photographed a long time ago and wants to make a call with an acquaintance existing in the photographed image which is checked in the past, the user of the portable terminal is stored in a phone book. You must retrieve the telephone number that is present to make the call connection.

Accordingly, there is a need for an apparatus and method for interworking a digital camera function and a phonebook function of a portable terminal as described above.

SUMMARY OF THE INVENTION An object of the present invention is to provide an apparatus and method for recognizing a face region of image data stored in a portable terminal.

Another object of the present invention is to provide an apparatus and method for setting phonebook information corresponding to a face region of image data.

Another object of the present invention is to provide an apparatus and method for providing phonebook information corresponding to a face area in image data stored in a portable terminal.

According to a first aspect of the present invention for achieving the above objects, the apparatus for identifying user information through face recognition is a face recognition unit for recognizing the face area from the image data by using the feature information required for extraction of the face area And a process of storing tag information for setting tag information which is information of a user corresponding to a person of a face area recognized by the face recognition unit, and providing tag information corresponding to a person of a face area of the image data. It characterized in that it comprises a control unit for processing the process.

According to a second aspect of the present invention for achieving the above objects, a method for confirming user information through face recognition in a portable terminal is to set tag information for setting tag information which is information of a person corresponding to a face area of image data. And checking tag information of a person corresponding to a face area of the image data when checking the image data.

Hereinafter, the operating principle of the present invention will be described in detail with reference to the accompanying drawings. In the following description of the present invention, when it is determined that detailed descriptions of related known functions or configurations may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. Terms to be described later are terms defined in consideration of functions in the present invention, and may be changed according to intentions or customs of users or operators. Therefore, the definition should be made based on the contents throughout the specification.

The following description relates to an apparatus and method for a portable terminal for extracting and recognizing a face region from image data and providing personal information corresponding to the recognized face region.

In addition, the image data is meant to include both an image file such as a photo file and a video file.

1 is a block diagram illustrating a configuration of a portable terminal capable of shortening a text data input time according to an exemplary embodiment of the present invention.

Referring to FIG. 1, the portable terminal includes a control unit 100, an image input unit 102, a face recognition unit 104, a memory unit 106, a tag information storage unit 107, an input unit 108, and a display unit ( 110 and the communication unit 112 may be configured.

The controller 100 controls the overall operation of the portable terminal. For example, processing and control for voice call and data communication are performed, and in addition to the usual functions, tag information of the person corresponding to the face region of the image data is generated and corresponding to the face region of the image data according to the present invention. Process to output detailed information of the person.

That is, when generating the tag information, the controller 100 receives the image data through the image input unit 102, and then causes the face recognition unit 104 to extract a face region of the image data. To recognize it.

Thereafter, the controller 100 receives detailed information of a person corresponding to the recognized face area from a user, generates the tag information, and stores the tag information in the generated tag information storage unit 107.

In addition, when outputting detailed information of a person corresponding to the face area, the controller 100 receives the image data through the image input unit 102 and then checks tag information corresponding to the image data. do.

Thereafter, the face recognizing unit 104 is instructed to recognize the face area existing in the image data, and then the detailed information present in the tag information corresponding to the face area selection by the user is checked to display the display unit 110. Process to output to

The image input unit 102 receives the image data in a meaning including a camera unit which is a device for inputting the image data.

The face recognizing unit 104 recognizes the face area using the feature information from the image data received through the image input unit 102 under the instruction of the control unit 100.

Here, the feature information is information used as a reference for extracting and recognizing a face region existing in the image data, and may include information such as eyes, nose, and mouth.

The memory unit 106 includes a read only memory (ROM), a random access memory (RAM), and a flash ROM. The ROM stores microcode and various reference data of a program for processing and controlling the controller 100 and the face recognition unit 104.

The RAM is a working memory of the controller 100 and stores temporary data generated during execution of various programs. In addition, the flash ROM stores various updatable storage data such as a phone book, an outgoing message and a received message, and may include a tag information storage unit 107 for storing tag information according to the present invention. have.

The input unit 108 includes 0 to 9 numeric key buttons, a menu button (menu), a cancel button (clear), an OK button, a call button (TALK), an end button (END), an internet access button, a navigation key ( Or a plurality of function keys, such as arrow keys) buttons and text input keys, and provides the controller 100 with key input data (eg, face region selection) corresponding to a key pressed by the user.

The display unit 110 displays status information generated during the operation of the portable terminal, a limited number of characters, a large amount of video and still images, and the like. The display unit 110 may use a color liquid crystal display (LCD), and outputs the image data and detailed information according to the present invention.

The communication unit 112 performs a function of transmitting and receiving a radio signal of data input and output through an antenna (not shown). For example, in the case of transmission, after performing channel coding and spreading on the data to be transmitted, RF processing is performed to transmit the data. In the case of reception, the received RF signal is converted into a baseband signal. The baseband signal is de-spreaded and channel decoded to restore data.

The role of the face recognition unit 104 may be performed by the controller 100, but the configuration of the face recognition unit 104 is illustrated to distinguish each function separately. Therefore, in the case of actually implementing a product, all of them may be configured to be processed by the controller 100.

The foregoing has described a device for recognizing a face area from image data of a portable terminal and providing personal information of a person (user) corresponding to the recognized face area, and in the following description, the device according to an embodiment of the present invention. A method for providing personal information of a person (user) corresponding to a face area recognized by the portable terminal will be described using.

2 is a flowchart illustrating a process for tagging detailed information on a face area of image data in a portable terminal according to an exemplary embodiment of the present invention.

Referring to FIG. 2, the controller 100 of the portable terminal first checks the image data input through the image input unit 102, and then proceeds to step 203 to cause the face recognition unit 104 to perform the feature on the image data. Instructs to obtain information.

Here, the feature information refers to reference information for determining a face of a person among background elements including a subject included in the image data.

In other words, when describing image data photographed using a person sitting on a tree as a subject, for example, the face recognizing unit 104 may identify characteristic information for determining eyes, nose, and mouth.

In step 205, the control unit 100 instructs the face recognizing unit 104 to recognize a face region existing in the image data by using the checked feature information, and then proceeds to step 207. The pointer is output on the recognized face area.

Here, the pointer is a pointer for selecting a person (user) corresponding to the face area existing in the image data, and may be displayed as a rectangle or a circle having a predetermined size that may include the face area. According to the user's selection of the portable terminal, movement to all face areas existing in the image data is possible.

In step 209, the controller 100 receives a face region from the user of the portable terminal. In step 211, the controller 100 generates tag information.

Here, the tag information is coordinates of the face area selected by the user of the portable terminal and detailed information of the person corresponding to the face area, and may include personal information, phone number information, birthday information, and the like. The process of generating the tag information may be performed as follows.

The generating of the tag information may be performed by using name information previously stored in the phone book of the portable terminal or directly receiving tag information from a user of the portable terminal.

In operation 213, the controller 100 stores the generated tag information. Here, the tag information may be stored in the tag information storage unit 107 of the memory unit 106.

Thereafter, the controller 100 ends the present algorithm.

3 is a diagram illustrating a format of tag information corresponding to a face region of image data according to an exemplary embodiment of the present invention.

FIG. 3A illustrates a format of the tag information and may include face region coordinate information 301 of the image data and detailed information 303 of a person (user) corresponding to the face region. Here, the detailed information 303 is phonebook index information, and may include personal name information, phone number information, birthday information, etc. of a user corresponding to the face area.

3B is a diagram illustrating face region coordinates of the image data.

Referring to FIG. 3B, the image data is assumed to be an image of two people.

When the face region of each person 310 or 320 is recognized by the portable terminal in the image data, the portable terminal converts the coordinates of the face region to the face region coordinate information 301 of the tag information. Record it.

In this case, the face area may be displayed as a shape such as a rectangle or a circle having a size that may include an area determined as a human face.

In addition, when the face region coordinates are identified by the rectangular figure, the portable terminal may record and display the coordinates of the upper left portion and the lower right portion of the quadrangle. If the face region is identified by the circular figure, the portable terminal may record using the coordinates and radius values of the center of the circle.

3C is a diagram illustrating a format of tag information in which coordinate information and detailed information of the face region of the image data are recorded.

Referring to FIG. 3 (c), when the portable terminal recognizes a face region such as 312 in the image data shown in FIG. 3 (b), the portable terminal determines a face region coordinate region 301 of the tag information. ), (A, b) (c, d) can be recorded, and phonebook index information called XXX can be recorded in the detailed information area 303.

In the same manner, when the portable terminal recognizes a face region such as 322, the portable terminal records (a ', b') (c ', d') in the face region coordinate region 301 of the tag information. In the detailed information area 303, phonebook index information called XYY may be recorded.

Here, the phonebook index information of the detailed information area indicates information indicating phonebook information corresponding to the face area among phonebook information previously stored in the portable terminal. You can check the number information and birthday information.

4 is a flowchart illustrating a process for outputting detailed information corresponding to a face region of image data in a portable terminal according to an exemplary embodiment of the present invention.

Referring to FIG. 4, the control unit 100 of the portable terminal first receives image data through the image input unit 102 in step 401, and then proceeds to step 403 to search for the memory unit 106 to receive the input data. Check the tag information corresponding to the video data.

After that, the controller 100 checks the face area included in the image data by using the face area coordinates included in the checked tag information, and then proceeds to step 407 to check detailed information among the checked face areas. The area is selected by the user of the portable terminal.

In operation 409, the controller 100 checks detailed information of a person (user) corresponding to the selected face area. Here, the detailed information is personal information including personal information, phone number information, birthday information, and the like, which can be confirmed through a phonebook index corresponding to the face region coordinates of the tag information.

In operation 411, the control unit 100 outputs the image input information and the checked detailed information previously input to the display unit 110.

Thereafter, the controller 100 ends the present algorithm.

5 is a flowchart illustrating a process of automatically recognizing a person corresponding to a face region of image data and outputting detailed information in a portable terminal according to another exemplary embodiment of the present invention.

5 is a process for automatically recognizing a face (user) of the recognized face region after recognizing the face region in the portable terminal repeatedly performing the process of FIG.

Referring to FIG. 5, the control unit 100 of the portable terminal first receives the image data through the image input unit 102 in step 501, and then proceeds to step 503 and causes the face recognition unit 104 to perform the image data. Process to check the face area present in the.

In step 505, the controller 100 automatically recognizes a person corresponding to the identified face area, and then proceeds to step 507 to check detailed information corresponding to the recognized user.

Here, in step 505, a face recognizer is trained on tag information by the repetitive operation of FIG. 2 to directly recognize a person corresponding to the face region.

In operation 509, the controller 100 processes the input image information and the checked detailed information to be output to the display 110.

Thereafter, the controller 100 ends the present algorithm.

6 is a diagram illustrating a screen on which detailed information corresponding to a face area of image data is output according to an exemplary embodiment of the present invention.

Referring to FIG. 6, it is assumed that tag information is tagged in a recognized face region by extracting a face region of the image data.

Thereafter, when the portable terminal receives the image data, the portable terminal may recognize a face area of the image data and output a pointer for selecting the recognized face area.

If it is determined that one face region is selected by the user of the portable terminal (610), the portable terminal checks tag information of a person corresponding to the face region where the selected pointer is located and outputs detailed information (612). )do.

In FIGS. 4 to 6, a method of outputting detailed information by selecting a face region of the image data has been described. However, the present invention is in response to a user's request in the state of selecting the face region and outputting the detailed information. A call connection with the other party and a message (eg, SMS, MMS, etc.) can be performed.

Meanwhile, in the detailed description of the present invention, specific embodiments have been described, but various modifications may be made without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the scope of the following claims, but also by the equivalents of the claims.

As described above, the face region of the image data is extracted and recognized using the portable terminal according to the present invention, and tag information of a person (user) corresponding to the recognized face region is generated and stored in the portable terminal. When checking the image data, detailed information of a person (user) corresponding to a face area may be checked, and automatic recognition may be enabled by using the learning data for automatic recognition of the person.

Claims (17)

In the device for checking the user information through the face recognition, A face recognition unit recognizing the face region from the image data by using feature information necessary for extracting the face region; A process of storing tag information for setting tag information corresponding to a person in a face area recognized by the face recognition unit, and a process of providing tag information corresponding to a person in a face area of the image data; Apparatus comprising a control unit. The method of claim 1, The tag information, And detailed information including face region coordinate information indicating a position of a face region and user information corresponding to a person in the face region. The method of claim 2, The detailed information, And at least one of human name information, telephone number information, and birthday information. The method of claim 3, wherein The detailed information, Device for storing the stored phonebook data of the portable terminal. The method of claim 2, Coordinate information of the face area is In the case of displaying in a rectangular shape of a predetermined size, the coordinates of the upper left portion and the lower right portion of the rectangle, when the display in a circular figure, characterized in that the device can be represented by the coordinates of the circular center. The method of claim 1, The control unit, And recognizing a face region from the image data, and receiving information of a person corresponding to the face region, thereby storing the tag information. The method of claim 6, The control unit, Providing the tag information by identifying the face region of the image data using coordinate information of the face region included in the tag information and providing detailed information of tag information corresponding to the selected face region among the identified face regions. Apparatus characterized in that the process can be performed. The method of claim 7, wherein The control unit, And after providing the tag information, process to perform a call connection. The method of claim 1, The face recognition unit, When the region is used for learning the face recognizer by performing a repetitive tagging process, the apparatus corresponding to the face region may be automatically recognized after omitting the checking of the tag information. In the method for confirming user information through face recognition in a portable terminal, A tag information setting process of setting tag information which is information of a person corresponding to a face area of image data; And checking tag information of a person corresponding to a face area of the image data when checking the image data. The method of claim 10, The tag information setting process, Recognizing a face region from the input image data; Generating the tag information by inputting information of a person corresponding to the recognized face area; And storing the generated tag information. The method of claim 10, The tag information providing process, Checking tag information corresponding to the image data when the image data is checked; Recognizing a face region of the image data by using face region coordinate information included in the checked tag information; And identifying and outputting detailed information of the person corresponding to the recognized face area. The method of claim 11, The tag information providing process, The method may further include automatically recognizing a person corresponding to the face region after omitting the checking of the tag information when checking the image data. The method of claim 13, The tag information, And detailed information including face region coordinate information indicating a position of a face region and information of a person corresponding to the face region. The method of claim 14, The detailed information, And at least one of human name information, telephone number information, and birthday information. The method of claim 15, The detailed information, And pre-stored phonebook data of the portable terminal. The method of claim 14, Coordinate information of the face area is The method of claim 1, wherein when displaying a rectangular figure having a predetermined size, the coordinates of the upper left portion and the lower right portion of the rectangle may be represented by the circular center coordinates.

Images