KR100705177B1 - Mobile communication terminal and method for classifying photograph using the same - Google Patents

Mobile communication terminal and method for classifying photograph using the same Download PDF

Info

Publication number
KR100705177B1
KR100705177B1 KR1020060002165A KR20060002165A KR100705177B1 KR 100705177 B1 KR100705177 B1 KR 100705177B1 KR 1020060002165 A KR1020060002165 A KR 1020060002165A KR 20060002165 A KR20060002165 A KR 20060002165A KR 100705177 B1 KR100705177 B1 KR 100705177B1
Authority
KR
South Korea
Prior art keywords
picture
method
mobile communication
communication terminal
image
Prior art date
Application number
KR1020060002165A
Other languages
Korean (ko)
Inventor
기현종
김성현
변성찬
이은실
이일용
Original Assignee
엘지전자 주식회사
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 엘지전자 주식회사 filed Critical 엘지전자 주식회사
Priority to KR1020060002165A priority Critical patent/KR100705177B1/en
Application granted granted Critical
Publication of KR100705177B1 publication Critical patent/KR100705177B1/en

Links

Images

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61HPHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
    • A61H3/00Appliances for aiding patients or disabled persons to walk about
    • A61H3/06Walking aids for blind persons
    • A61H3/068Sticks for blind persons
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61HPHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
    • A61H3/00Appliances for aiding patients or disabled persons to walk about
    • A61H3/06Walking aids for blind persons
    • A61H3/061Walking aids for blind persons with electronic detecting or guiding means
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61HPHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
    • A61H2201/00Characteristics of apparatus not provided for in the preceding codes
    • A61H2201/50Control means thereof
    • A61H2201/5058Sensors or detectors

Abstract

The present invention relates to a mobile communication terminal having a camera, and more particularly, to a mobile communication terminal for classifying photographic images photographed through a camera and a method of classifying photographic images using the same.
In accordance with another aspect of the present invention, there is provided a method of classifying photographic images, the method comprising: detecting a face region by performing a reference photo and performing a preprocessing process; Extracting a feature through the preprocessing and generating a learning DB for classifying a photo; Thereafter, when the picture is taken, detecting a face area of the taken picture and performing a preprocessing process; Extracting a feature through the preprocessing process and comparing the similarity with the learning DB to automatically classify the picture.
Photo Search, SVM

Description

Mobile communication terminal and method for classifying photographic images using same {mobile communication terminal and method for classifying photograph using the same}

1 is a view showing the configuration of a mobile communication terminal according to an embodiment of the present invention.

2 is a flowchart illustrating a learning method according to an exemplary embodiment of the present invention.

3 is a view showing a learning mode shooting according to an embodiment of the present invention.

4 is a view showing a picture taken for learning according to an embodiment of the present invention.

5 is a view showing a preprocessing process of a picture according to an embodiment of the present invention.

6 is a view showing a pre-processed picture for learning according to an embodiment of the present invention.

7 is a view showing a classification through the SVM in accordance with an embodiment of the present invention.

8 is a flowchart illustrating a recognition method according to an embodiment of the present invention.

9 is a flowchart illustrating a classification method according to an exemplary embodiment of the present invention.

<Explanation of symbols for the main parts of the drawings>

11 ... Wireless processing unit 12 ... Key input unit

13 Camera 14 Display

15 ... storage 16 ... image recognition

17.Control part

The present invention relates to a mobile communication terminal having a camera, and more particularly, to a mobile communication terminal for classifying photographic images photographed through a camera and a method of classifying photographic images using the same.

The camera phone is a mobile communication terminal that adds the function of a digital camera to a general mobile phone. The user can directly check the photographs taken with the digital camera through the LCD of the camera phone or upload it on a PC to edit and customize it according to his / her preferences. The prevalence is rapidly expanding in that it can be processed and provided to others by convenient means such as e-mail.

In addition, as the storage capacity for storing a photographic image in the camera phone increases, the number of photographs taken by the camera also increases innumerably.

Therefore, instead of searching for previously stored photo images one by one, the multi-view function, which shows a number of photos in the screen window, makes it easier to search photos.

However, as the storage capacity becomes larger and the number of pictures that can be stored increases, the above-described method for retrieving a picture does not have a great effect.

The present invention has been made to solve the above problems, to provide a mobile communication terminal and a method for classifying a photographic image using the same, which enables the user to easily search by learning the photographed photograph and automatically sort the photograph using the photograph.

Photo image classification method according to the present invention for achieving the above object,

Taking a reference picture to detect a face region and performing a preprocessing process;

Extracting a feature through the preprocessing and generating a learning DB for classifying a photo;

Thereafter, when the picture is taken, detecting a face area of the taken picture and performing a preprocessing process;

Extracting a feature through the preprocessing process and comparing the similarity with the learning DB to automatically classify the picture.

In the present invention, the reference picture for the learning is characterized in that is updated.

In the present invention, the reference picture is characterized in that the user-centered face picture.

In the present invention, the reference picture is characterized in that at least one or more are taken, including different expressions, lighting, poses.

In the present invention, the preprocessing step comprises the steps of: processing the detected face region as a gray scale image;

Processing the converted image into a light-corrected image using histogram smoothing;

The method may further include reducing the illumination corrected image to a predetermined size.

In the present invention, the feature extraction is characterized by using a direct linear discriminant analysis method (D-LDA).

In the present invention, the picture classification is characterized by using a support vector machine.

In the present invention, the picture classification is characterized in that it is automatically classified into a user-centered face picture and another person's face picture.

The method may further include checking and storing a reservoir bit of a JPEG when the photograph classification result is classified as a user face photograph.

In addition, the mobile communication terminal according to the present invention for achieving the above object,

A camera which takes a picture and takes a reference picture for comparing with the picture;

A storage unit for storing a learning DB obtained by analyzing the reference picture and performing a learning process;

And a video recognition unit for automatically classifying the photographs by comparing the photographed photographs with the learned reference photographs.

In the present invention, the image recognition unit is characterized in that for processing the reference picture and the photographed picture and extract the features of the face.

In the present invention, the image recognition unit is characterized in that to automatically classify the photograph using a support vector machine.

In the present invention, the storage unit is characterized in that for storing at least one or more comparison picture to extract the characteristics of the reference picture.

Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

1 is a view showing the configuration of a mobile communication terminal according to an embodiment of the present invention.

Referring to FIG. 1, a mobile communication terminal according to the present invention includes a wireless processing unit 11 for wireless communication with a base station, a key input unit including a keypad and various buttons to enable key manipulation for use of the communication terminal. 12, a camera 13 for photographing photographic images, a display unit 14 for displaying the photographic image, a storage unit 15 for storing the photographic image, and the photographed photographic image are recognized. And a control unit 17 for controlling the unit and the overall operation of the mobile communication terminal, and the image recognition unit 16 is integrally connected to the control unit 17. It can be configured to include.

The wireless processor 11 performs wireless communication with the base station by receiving a wireless signal (for example, voice data, video data, etc.) from the mobile communication base station and transmitting voice signals and text data for a call to the base station. do.

The key input unit 12 is a device that receives a user control command from a user. The key input unit 12 may be implemented as a device such as a keypad of a communication terminal. The key input unit 12 may be a mode such as taking a learning mode or taking a general picture mode for taking a picture to be used for learning. Enter a command to select.

The camera 13 is a device that receives and inputs a subject that the user wants to photograph, such as a face image, from the user, and may include a camera auxiliary device such as lighting.

The display unit 14 displays image data such as a photograph taken by the camera 13 or outputs image data that is already stored. In addition, it outputs so that the image data transmitted through the wireless processor 11 can be displayed.

The storage unit 15 stores a picture taken by the camera 13, a picture required for learning, and the like, and stores learning information of the picture.

When the image recognition unit 16 receives a photograph taken in the learning mode from the camera 13, after preprocessing it, the image recognition unit 16 normalizes between 0 and 1 together with the photo DBs stored in the storage unit for learning and extracts features. Do this.

Here, for the feature extraction, it is possible to apply various known techniques such as direct linear discriminant analysis (D-LDA).

Thereafter, the extracted feature vectors extracted by the D-LDA are generated through a support vector machine (hereinafter referred to as SVM), where a trained DB can be distinguished from a photograph taken in a learning mode and another photograph.

For example, if the picture taken in the learning mode is the user's own picture, a learning DB for distinguishing the user's picture from the picture of another person is generated.

Next, when the user's own picture is taken and input in the picture mode from the camera 13, the feature extraction is performed after preprocessing the taken picture as in the learning mode.

At this time, by comparing the similarity with the learned DB by using the SVM can be classified whether the photograph taken is a user's picture or another person's picture, and the tag is stored in the storage unit 14 with a tag on the classified user picture.

The tag is such that when a picture is stored as a JPEG, only the user picture can be easily retrieved by checking and storing a reserve bit of the JPEG.

The control unit 17 controls the processing of call connection, data communication, multimedia content, etc., as well as storing the captured picture in the storage unit 14, and an image recognition unit for processing the picture as a learning process and a recognition process ( 16) to control the overall operation of the mobile communication terminal.

2 is a flowchart illustrating a learning method according to an exemplary embodiment of the present invention.

The photograph taken to explain an embodiment of the present invention is defined as a portrait photograph.

Referring to FIG. 2, in order to automatically classify pictures, the present invention processes photographed pictures by dividing them into a learning process and a recognition process, extracts a feature of a picture to be classified through the learning process, and adjusts to converge the reference data. Create a learning DB.

First, the learning process will be described. As shown in FIG. 3, a learning mode is selected using a mobile communication terminal and a picture to be learned is taken (S11).

For example, if a user wants to learn and classify his or her own picture, the user may take a picture of a user's face to be learned at any time using the learning mode menu.

The reason is that the face of the person is changed slightly over time, the shape can be changed by the hair style, molding, etc., the present invention has the advantage of taking a picture that you want to learn at any time.

In addition, at least 10 pictures to be used in the learning should be taken and if the user's own pictures to learn, as shown in FIG. Will be.

Subsequently, a preprocessing process is performed to extract a feature of a face from the taken photo (S12).

In order to extract the face region from the photographed picture, the face region is extracted based on the screen window 22 indicated by the dotted line in the learning mode screen window 21 of FIG. 3, or the eye region or the center region of the pupil is referred to. Facial regions to be used for learning can be extracted.

In addition, an AdaBoost algorithm may be used to detect the face region.

5 is a view showing a pretreatment process according to an embodiment of the present invention.

Referring to FIG. 5, the face areas 32 and 32-1 detected in the images 31 and 31-1 photographed in the learning mode are converted into gray scale images 33 and 33-1.

In addition, histogram equalization is performed to convert the image to the light compensated image 34, 34-1, and to reduce the size to 1024 (32 * 32) dimension image 35, 35-1 for feature extraction. .

Thereafter, as illustrated in FIG. 6, the portrait pictures and the taken portrait pictures stored in the storage unit of the terminal are pre-processed for learning to be normalized from 0 to 1, and features of the portrait picture are extracted (S13).

In the normalization, the same pixels are normalized to store the maximum value and the minimum value for each pixel. For example, in a 1024-dimensional photograph, 1024 maximum and minimum values can be obtained.

In addition, although any facial feature extraction algorithm can be used as the feature extraction method, in this embodiment, a direct linear discriminant analysis (D-LDA) method is used and the D-LDA method is used. The extracted vector is limited to 50 dimensions to generate a D-LDA matrix.

The D-LDA method is suitable for use in mobile communication terminals and the like because it not only extracts features to distinguish between classes but also has a fast processing speed, and is well known and will not be described in detail anymore.

On the other hand, by using the feature vectors extracted by the D-LDA as an input, a learning is performed by a support vector machine (SVM), which is a binary pattern classifier.

That is, the user's photo and another person's photo can be distinguished by the SVM according to the difference of the input feature vectors.

The SVM analyzes the nonlinear high order of the input space by linearly projecting it in the feature space, thereby presenting an optimal boundary (that is, an optimal separation plane) 41 between the feature vectors as shown in FIG. 7. The method is already well known and will not be described in detail.

When the SVM learning is completed, a support vector (SV), a Lagrangian multiplier, and a bias value are obtained, and the values are stored in the mobile communication terminal. Through the above process, the user's face learning to be photographed for use in photo recognition is ended, and a learning DB is generated and stored (S14).

Next, when taking a picture using a mobile communication terminal, it is possible to automatically classify whether or not the user's own photo compared to the learning DB.

8 is a flowchart illustrating a recognition method according to an exemplary embodiment of the present invention.

Referring to FIG. 8, a picture is taken using a camera of a mobile communication terminal (S21), and a face region is detected from the taken picture as in the learning process.

The AdaBoost algorithm may be used to detect the face region.

In addition, the detected face region is converted to a gray scale image as in the learning process, and subjected to histogram equalization for light correction, and then pre-processed to reduce the image to 1024 (32 * 32) dimensions. After performing the extraction of the features of the picture (S22, S23).

At this time, the normalization is performed using the maximum and minimum values of the pixels obtained in the learning process, and the 50-dimensional feature vectors extracted using the D-LDA can be classified similarly by the SVM to automatically classify whether they are photographs of the user or not. It becomes (S24).

In addition, when the picture identified as the own picture is stored as a JPEG, the classified user picture is checked using a reserved bit that can be defined by the user as a tag (S25).

For example, the reserved bit becomes '1' when the captured picture is recognized as the user's own picture.

The reason for using the above method may be that the user may change the file name and store the file without using the file name designated by the terminal when storing the classified picture so as to easily search for the classified picture. For this case, check the reserve bit and make it easier to retrieve your own photos.

9 is a flowchart illustrating a classification method according to an exemplary embodiment of the present invention.

Referring to FIG. 9, a photograph to be learned is taken by using a camera module (S31), and a feature is extracted through a preprocessing process (S32). In the present invention, there is a feature that allows the user to take the picture to be learned and learn it.

The extracted feature generates a learning DB by storing parameters that enable the SVM to identify the person and others (S33).

The parameter includes a support vector (SV), a Lagrangian multiplier, a bias value, and the like, and is used as a feature vector.

On the other hand, if the user photographs not only himself or another person's photos using the camera module (S41), the feature is extracted from the photographed pictures through the preprocessing process as in the learning process (S42).

Then, the extracted feature is determined by the SVM for each individual in the face feature vector space consisting of feature vectors of face images stored in a learning DB, whether the feature vector belongs to its own region or another's region. His pictures are classified and stored (S43, 44).

As described above, the present invention has an effect of simply searching for a picture through a learning process and a recognition process to automatically classify the taken picture.

So far, the present invention has been described with reference to the embodiments, and those skilled in the art to which the present invention pertains may implement embodiments of the present invention in a different form from the detailed description of the present invention within the essential technical scope of the present invention. Could be. Here, the essential technical scope of the present invention is shown in the claims, and all differences within the equivalent range will be construed as being included in the present invention.

According to the mobile communication terminal according to the present invention and a method of classifying photographic images using the same, the user can conveniently search for a specific photograph by automatically classifying and storing the photographed photograph.

In addition, according to the present invention, there is an effect that the user can always take a picture that you want to be automatically sorted, stored and learned.

Claims (13)

  1. Taking a reference picture to detect a face region and performing a preprocessing process;
    Extracting a feature through the preprocessing and generating a learning DB for classifying a photo;
    Thereafter, when the picture is taken, detecting a face area of the taken picture and performing a preprocessing process;
    Extracting a feature through the preprocessing process, and comparing the similarity with a learning DB to automatically classify the picture.
  2. The method of claim 1,
    The reference picture for the learning is a picture image classification method using a mobile communication terminal, characterized in that for updating.
  3. The method of claim 1,
    The reference picture is a picture image classification method using a mobile communication terminal, characterized in that the user-centered face picture.
  4. The method of claim 1,
    The reference picture is a picture image classification method using a mobile communication terminal, characterized in that at least one or more including a different expression, lighting, poses.
  5. The method of claim 1,
    The preprocessing process may include processing the detected face region into a gray scale image;
    Processing the converted image into a light-corrected image using histogram smoothing;
    And reducing the illumination corrected image to a predetermined size.
  6. The method of claim 1,
    The feature extraction is a method for classifying photographic images using a mobile communication terminal (D-LDA).
  7. The method of claim 1,
    The picture classification is a picture image classification method using a mobile communication terminal, characterized in that using a support vector machine.
  8. The method of claim 1,
    The picture classification is classified into a picture image using a mobile communication terminal, characterized in that automatically classified into a user-centered face pictures and other people's face pictures.
  9. The method of claim 1,
    And classifying and storing a reservoir bit of JPEG when the photograph classification result is classified as a user face photograph.
  10. A camera which takes a picture and takes a reference picture for comparing with the picture;
    A storage unit for storing a learning DB obtained by analyzing the reference picture and performing a learning process;
    And a video recognition unit for automatically classifying pictures by comparing the taken pictures with the learned reference pictures.
  11. The method of claim 10,
    The image recognition unit is a mobile communication terminal, characterized in that for processing the reference picture and the photographed picture and extract the features of the face.
  12. The method of claim 10,
    The image recognition unit is a mobile communication terminal for retrieving a picture image, characterized in that to automatically classify the picture using a support vector machine.
  13. The method of claim 10,
    And the storage unit stores at least one photo to be compared to extract features of the reference photo.
KR1020060002165A 2006-01-09 2006-01-09 Mobile communication terminal and method for classifying photograph using the same KR100705177B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020060002165A KR100705177B1 (en) 2006-01-09 2006-01-09 Mobile communication terminal and method for classifying photograph using the same

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020060002165A KR100705177B1 (en) 2006-01-09 2006-01-09 Mobile communication terminal and method for classifying photograph using the same

Publications (1)

Publication Number Publication Date
KR100705177B1 true KR100705177B1 (en) 2007-04-06

Family

ID=38161243

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020060002165A KR100705177B1 (en) 2006-01-09 2006-01-09 Mobile communication terminal and method for classifying photograph using the same

Country Status (1)

Country Link
KR (1) KR100705177B1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101264797B1 (en) * 2006-10-16 2013-05-15 삼성전자주식회사 Method for searching photo by facial recognition in mobile terminal
KR101363017B1 (en) * 2007-08-23 2014-02-12 삼성전자주식회사 System and methed for taking pictures and classifying the pictures taken
US9542623B2 (en) 2014-06-11 2017-01-10 Samsung Electronics Co., Ltd. Image classification device, method for operating the same and electronic system comprising the image classification device
KR101858618B1 (en) * 2016-12-02 2018-05-17 (주)글로벌디지털콘텐츠중고왕 Identification and data management system for second hand goods

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005107885A (en) 2003-09-30 2005-04-21 Casio Comput Co Ltd Image classifying device and program
KR20050045773A (en) * 2003-11-12 2005-05-17 (주)버추얼미디어 Person verification and identification method and apparatus with 3d facial representation for mobile device

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005107885A (en) 2003-09-30 2005-04-21 Casio Comput Co Ltd Image classifying device and program
KR20050045773A (en) * 2003-11-12 2005-05-17 (주)버추얼미디어 Person verification and identification method and apparatus with 3d facial representation for mobile device

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101264797B1 (en) * 2006-10-16 2013-05-15 삼성전자주식회사 Method for searching photo by facial recognition in mobile terminal
KR101363017B1 (en) * 2007-08-23 2014-02-12 삼성전자주식회사 System and methed for taking pictures and classifying the pictures taken
US8866931B2 (en) 2007-08-23 2014-10-21 Samsung Electronics Co., Ltd. Apparatus and method for image recognition of facial areas in photographic images from a digital camera
US9542623B2 (en) 2014-06-11 2017-01-10 Samsung Electronics Co., Ltd. Image classification device, method for operating the same and electronic system comprising the image classification device
KR101858618B1 (en) * 2016-12-02 2018-05-17 (주)글로벌디지털콘텐츠중고왕 Identification and data management system for second hand goods

Similar Documents

Publication Publication Date Title
US20180025218A1 (en) Classification and organization of consumer digital images using workflow, and face detection and recognition
AU2017261537B2 (en) Automated selection of keeper images from a burst photo captured set
US8761446B1 (en) Object detection with false positive filtering
US8797448B2 (en) Rapid auto-focus using classifier chains, MEMS and multiple object focusing
US9652663B2 (en) Using facial data for device authentication or subject identification
US9104915B2 (en) Methods and systems for content processing
KR101424956B1 (en) Registration Determination Device, Control Method and Control Program Therefor, and Electronic Apparatus
US8199979B2 (en) Classification system for consumer digital images using automatic workflow and face detection and recognition
US8659697B2 (en) Rapid auto-focus using classifier chains, MEMS and/or multiple object focusing
US20160314344A1 (en) Image selecting device, image selecting method, image pickup apparatus, and computer-readable medium
JP4505362B2 (en) Red-eye detection apparatus and method, and program
US7558408B1 (en) Classification system for consumer digital images using workflow and user interface modules, and face detection and recognition
US7555148B1 (en) Classification system for consumer digital images using workflow, face detection, normalization, and face recognition
US7587068B1 (en) Classification database for consumer digital images
US7869658B2 (en) Representative image selection based on hierarchical clustering
JP5247480B2 (en) Object identification device and object identification method
JP4574249B2 (en) Image processing apparatus and method, program, and imaging apparatus
US7551755B1 (en) Classification and organization of consumer digital images using workflow, and face detection and recognition
JP5170961B2 (en) Image processing system, image processing apparatus and method, program, and recording medium
WO2016023336A1 (en) Method, apparatus and device for storing pictures in classified mode, and computer storage medium
JP4845755B2 (en) Image processing apparatus, image processing method, program, and storage medium
US8384792B2 (en) Imaging apparatus, method for controlling the same, and program
KR100947990B1 (en) Gaze Tracking Apparatus and Method using Difference Image Entropy
US8130281B2 (en) Information processing apparatus, eye open/closed degree determination method, computer-readable storage medium, and image sensing apparatus
KR101010081B1 (en) Media identification

Legal Events

Date Code Title Description
A201 Request for examination
E902 Notification of reason for refusal
E701 Decision to grant or registration of patent right
GRNT Written decision to grant
FPAY Annual fee payment

Payment date: 20120327

Year of fee payment: 6

FPAY Annual fee payment

Payment date: 20130326

Year of fee payment: 7

LAPS Lapse due to unpaid annual fee