TWI708189B - Electronic device, and method for analyzing face information in electronic device - Google Patents

Abstract

A method for analyzing face information in an electronic device is provided. The method includes detecting at least one face region from an image that is being captured by a camera module, zooming in the at least one detected face region, and analyzing the at least one detected and zoomed in face region according to at least one analysis item.

Description

Electronic device and method for analyzing facial information on the electronic device

The invention relates to an electronic device and a method for analyzing facial information on the electronic device. More specifically, the present invention relates to a method for analyzing facial information on an electronic device, wherein once facial information (for example, facial area) is recognized, the recognized area can be enlarged or can be displayed on the area during shooting The focus is automatically set without any user control action.

Various electronic devices have been developed to provide various functions. For example, electronic devices (such as smart phones) have been designed to provide not only the function of capturing images but also the function of identifying the captured objects.

The electronic device can be provided with a display so that the user can use or enjoy various functions more effectively. For example, the latest smart phones can be provided with a touch-sensitive display (such as a touch screen) at the front.

Various applications (or applications) can be installed and run in the electronic device. To run and control applications in the electronic device, various input units (for example, touch screen, buttons, mouse, keyboard, sensors, etc.) can be used.

In order to analyze the object being photographed (for example, the facial area), the electronic device can obtain an image in which the object is enlarged or finely photographed. On the other hand, in order to obtain magnified or finely shot images, the user can determine or identify the objects to be shot and determine whether to set the focus on the detailed area of each object, so as to improve the photography function (or shooting Function) empowerment.

The above information is presented as background information only to help understand the present invention. No judgment or assertion has been made as to whether any of the above can be used as the prior art related to the present invention.

The aspect of the present invention is to solve at least the above-mentioned problems and/or disadvantages and to provide at least the advantages described below. Therefore, the aspect of the present invention is to provide an electronic device and a method for analyzing facial information on the electronic device, wherein once facial information (for example, facial area) is recognized, the recognized area can be enlarged or can be located at all locations during shooting. The focus is automatically set on the area without any user control action.

According to an aspect of the present invention, a method for analyzing facial information on an electronic device is provided. The method includes: detecting at least one facial area from an image taken by a camera module; enlarging the at least one detected facial area; and analyzing the enlarged facial area according to at least one analysis item.

According to another aspect of the present invention, an electronic device is provided. The electronic device includes: a camera module; a display for displaying an image being shot by the camera module; and a controller for detecting at least one facial area from the image, Control the camera module to zoom in the at least one detected facial area, and analyze the at least one detected and zoomed facial area according to at least one analysis item.

According to another aspect of the present invention, a method for analyzing facial information on an electronic device is provided. The method includes: detecting at least one facial area from at least one stored image; and analyzing the at least one detected facial area according to at least one analysis item.

According to another aspect of the present invention, an electronic device is provided. The electronic device includes a camera module for shooting images; and a controller for detecting at least one facial region from at least one image captured and stored by the camera module; and analyzing according to at least one analysis item The at least one detected facial area.

Reading the following detailed description that discloses various embodiments of the present invention in conjunction with the accompanying drawings, other aspects, advantages and prominent features of the present invention will become apparent to those skilled in the art.

100: electronic device

110: Controller

111: Project Setting Unit

112: Photography Controller

113: Image Analyzer

114: District selector

115: Analysis result generator

120: camera module

130: input unit

140: display

150: storage

151: Skin Analysis Information

152: User identification information

210, 220, 230, 240: operation

310, 320, 330: Operation

410, 420, 430, 440, 450, 460, 470: Operation

500: image

501: eyebrows

501a: Wrinkles

502: right eye

502a: Wrinkles

503: nose

503a: Pores

504: The edge of the mouth

504a: Pigmentation

505: cheek

505a: Acne

506: left eye

506a: black circle

507: face

507a: Skin tone

600: Image

601: Acne

602: wrinkle

603: Pigmentation

710: captured image

711: Pigmentation

720: Video

721: Pigmentation part

810: captured image

811: Acne

820: Image

821: Acne Spot

910: captured image

911: Pores

920: Video

921: Color darkened area

1000: Image taken

1001: wrinkle line

1100: Image taken

1101: Skin color analysis area

1210: screen

1220: screen

1300: Analysis result screen

1310: eyes

1311: wrinkle

1320: nose

1321: Acne spots

1330: cheek

1331: Pigmentation part

1410: Analysis result screen

1411: View skin analysis history

1420: Skin analysis history screen

1421: skin tone measurement result

1422: acne measurement results

1423: Wrinkle measurement results

1424: Pigmentation measurement results

1425: Pore measurement results

1510: Image taken

1511: Wrinkle Mark

1520: Skin care recommendation screen

1521: Cosmetics that are good for wrinkles

1521a: **eye cream

1521b:##霜

1521c:oo eye cream

1522: Run button (execute button)

1523: Run button

1610: captured image

1611: Skin color analysis area

1620: Skin care recommendation screen

1621: Facial mask that is good for improving skin tone

1621a: **Mask

1621b: Mask

1622: Run button

1710: captured image

1711: Acne

1720: Advertising screen

1721: ``More'' run button

1810: Image taken

1811: Pore analysis area

1820: Skin care recommendation screen

1821: Information

1822: advertising

1900: network environment

1901: electronic device

1904: electronic device

1906: Server

1910: Bus

1920: processor

1930: memory

1931: core

1932: middle software

1933: Application Programming Interface

1934: Application

1940: Input/Output (I/O) interface

1950: display

1960: Communication interface

1962: Internet

1970: camera module

2001: front camera

2002: rear camera

2003: Color balance card

2110, 2120, 2130, 2140, 2150, 2160, 2170: operation

2210: image

2211: line

2212: part

2220: image

2230: image

2231: line

2240: Image

2301: line

2311: image

2312: Image

2321: Image

2322: Image

2410: Image

2420: Image

2501: electronic device

2510: application processor

2520: Communication module

2521: Honeycomb module

2523: Wi-Fi module

2524: Subscriber Identification Module (SIM) card

2525: Bluetooth module

2527: Global Positioning System (GPS) Module

2528: Near Field Communication (NFC) Module

2529: Radio Frequency (RF) Module

2530: memory

2532: internal memory

2534: External memory

2540: sensor module

2540A: Gesture sensor

2540B: Gyro sensor

2540C: Barometer

2540D: Magnetic sensor

2540E: accelerometer

2540F: Hold the sensor

2540G: proximity sensor

2540H: Color sensor

2540I: Biosensor

2540J: temperature/humidity sensor

2540K: Illumination sensor

2540M: Ultraviolet (UV) sensor

2550: input device

2552: Touch Panel

2554: (digital) pen sensor

2556: Button

2558: Ultrasonic Input Device

2560: display

2562: Panel

2564: Holographic device

2566: Projector

2570: Interface

2572: High-Definition Multimedia Interface (HDMI)

2574: Universal Serial Bus (USB)

2576: Optical interface

2578: D-subminiature (D-sub) interface

2580: Audio Module

2582: Speaker

2584: receiver

2586: headphones

2588: Microphone

2591: camera module

2595: Power Management Module

2596: battery

2597: indicator

2598: Motor

2610: Run screen

2611: Gallery button

2612: Camera button

2620: Image selection screen

2630: Analysis result screen

2631: District

2632: District

2633a: Summary button

2633b: Wrinkle button

2633c: Question button

2633d: Pigmentation button

2700: Analysis result screen

2710: Image area

2711: reference point

2712, 2713, 2714, 2715, 2716: facial area/area

2720: Analysis result area

2721: schema

2722: score

2730: Analysis project selection area

2800: Analysis result screen

2810: Image area

2811: Use input

2812: Enlarged screen

2820: Analysis result area

2821: schema

2822: score

2823: schema

2824: schema

2830: Analysis project selection area

2831: Wrinkle button

2900: Analysis result screen

2901: User input

2902: User input

2910: screen

2920: Analysis result area

2931: wrinkle button

3000: Analysis result screen

3010: Image area

3011: User input

3012: Enlarged screen

3020: Analysis result area

3021: schema

3022: score

3023: schema

3024: schema

3030: Analysis project selection area

3031: Question button

3100: Analysis result screen

3101: User input

3102: User input

3110: screen

3120: Analysis result area

3131: Question button

3200: Analysis result screen

3201: User input

3202: Enlarged screen

3210: Image area

3220: Analysis result area

3221: schema

3222: score

3223: schema

3224: schema

3230: Analysis project selection area

3231: Pigmentation button

3300: Analysis result screen

3301: User input

3302: User input

3311: screen

3312: screen

3320: Analysis result area

3331: Pigmentation button

3400: Analysis result screen

3401: User input

3402: Enlarged screen

3410: Image area

3420: Analysis result area

3421: Schema

3422: score

3423: schema

3424: schema

3431: Pore Button

3500: Analysis result screen

3510: Image area

3520: Analysis result area

3521: schema

3522: score

3523: schema

3524: schema

3530: Analysis project selection area

3531: Chroma button

3600: Analysis result screen

3601: Menu

3602: Management change function

3610: Image area

3620: Analysis result area

3621: schema

3630: analysis project

3631: summary

3632: wrinkle

3633: problem

3634: Pigmentation

3700: Analysis result screen

3701: User input

3731: summary

3800: Analysis result screen

3810: Image area

3820: Analysis result area

3831: wrinkle

3900: Analysis result screen

3910: image area

3920: Analysis result area

3931: problem

4000: Analysis result screen

4010: Image area

4020: Analysis result area

4031: Pigmentation

4100: Analysis result screen

4101: User input

4110: Image area

4111: Image

4112: Image

4113: Image

4120: Analysis result area

4121: first time

4122: second time

4123: third time

4131: summary

4200: Analysis result screen

4201: User input

4210: Image area

4211: image

4212: image

4213: Image

4220: Analysis result area

4221: first time

4222: second time

4223: specific time/third time

4231: wrinkles

4300: Analysis result screen

4301: User input

4310: Image area

4311: Image

4312: Image area

4320: Analysis result area

4321: schema

4322: schema

4331: problem

4400: Analysis result screen

4401: User input

4410: Image area

4411: the first image

4412: second image

4413: third image

4420: Analysis result area

4421: first time

4422: second time

4423: third time

4431: problem

4500: Analysis result screen

4501: User input

4510: Image area

4511: The first image

4512: second image

4513: Third Image

4520: Analysis result area

4521: first time

4522: second time

4523: third time

4531: Pigmentation

4600: facial image

4601: reference point

4700: Analysis result screen

4701: User input

4710: Image area

4712: separate window

4713: facial area

4714: Legend

4720: Analysis project area

4721: Total

4722: Pigmentation

4730: Analysis result area

4731: emoticon image

4732: improvement

4733: schema

A: cheek area

B: Bottom area/jaw area

C: area

CL: left area

CR: Right area

L: Left area / eyebrow area

M: area

ML: Left area

MR: Right area

R: Right area

T: top zone

Reading the following description in conjunction with the accompanying drawings, the above and other aspects, features, and advantages of certain embodiments of the present invention will become more obvious. In the accompanying drawings: FIG. 1 illustrates an electronic device according to various embodiments of the present invention. Block diagram of the configuration.

2, 3, and 4 are flowcharts illustrating the operation of analyzing skin information on the electronic device according to various embodiments of the present invention.

Figure 5A, Figure 5B, Figure 5C, Figure 5D, Figure 5E, Figure 5F, Figure 5G, and Figures 5H illustrates the situation where the facial area and the facial area in it are partially enlarged or finely photographed according to various embodiments of the present invention.

Figure 6 illustrates facial analysis items according to various embodiments of the present invention.

Figures 7, 8, 9, 10, and 11 illustrate operations of analyzing facial analysis items according to various embodiments of the present invention.

FIG. 12 illustrates a photographing operation for face analysis according to various embodiments of the present invention.

13A, 13B, and 13C illustrate screens showing the results of facial analysis according to various embodiments of the present invention.

FIG. 14, FIG. 15, FIG. 16, FIG. 17, and FIG. 18 illustrate screens showing the results of skin analysis according to various embodiments of the present invention.

Figure 19 illustrates a network environment according to various embodiments of the present invention.

20A, 20B, and 20C illustrate a situation in which an electronic device analyzes a face according to various embodiments of the present invention.

FIG. 21 is a flowchart illustrating the operation of analyzing skin information on an electronic device according to various embodiments of the present invention.

22A and 22B illustrate photographing operations for facial analysis according to various embodiments of the present invention.

23A and 23B illustrate photographing operations for facial analysis according to various embodiments of the present invention.

FIG. 24 illustrates a photographing operation for face analysis according to various embodiments of the present invention.

FIG. 25 is a block diagram illustrating the configuration of an electronic device according to various embodiments of the present invention.

FIG. 26 illustrates a screen of an application program for analyzing facial information based on stored images according to various embodiments of the present invention.

27, 28, 29, 30, 31, 32, 33, 34, and 35 illustrate screens showing facial analysis results according to various embodiments of the present invention.

FIG. 36 illustrates an application screen showing the history of facial information obtained by analyzing stored images according to various embodiments of the present invention.

37, 38, 39, 40, 41, 42, 43, 44, and 45 illustrate screens showing the history of analyzed facial information according to various embodiments of the present invention.

FIG. 46 illustrates a photographing operation for face analysis according to various embodiments of the present invention.

47A and 47B illustrate screens showing the results of skin analysis according to various embodiments of the present invention.

Throughout the drawings, the same reference numbers will be understood to refer to the same parts, components, and structures.

The following description is provided with reference to the accompanying drawings to help a broad understanding of various embodiments of the present invention defined by the scope of patent application and its equivalent scope. The following description package Various specific details are included to help understanding, but these specific details are only regarded as examples. Therefore, those with ordinary knowledge in the art will realize that various changes and modifications can be made to the various embodiments described herein without departing from the scope and spirit of the present invention. In addition, for the sake of clarity and conciseness, descriptions of well-known functions and structures may be omitted.

The terms and words used in the following description and the scope of the patent application are not limited to the bibliographic meaning, but are only used by the inventor to achieve a clear and consistent understanding of the present invention. Therefore, those skilled in the art should understand that the following descriptions of various embodiments of the present invention are provided for illustrative purposes only and not for limiting the present invention defined by the scope of the appended patent application and its equivalent scope.

It will be understood that unless the context clearly dictates otherwise, the singular forms "一 (a, an)" and "the (the)" also include plural indicators. Thus, for example, reference to "component surface" includes reference to one or more of such surfaces.

When the term "substantially" is used, it means that the characteristic, parameter, or value does not need to be accurately obtained, but a deviation or change in the amount that does not exclude the effect that the characteristic is intended to provide (including, for example, familiarity with the technology) Tolerance, measurement error, measurement accuracy limit, and other factors that are known to the reader).

The term "comprise and/or comprising" as used herein refers to the existence or addition of the disclosed function, operation, or component, but does not exclude the existence or addition of one or more other functions, operations, or components. It will be understood that when the terms "include" and/or "have" are used in this specification, it refers to the existence of the features, integers, operations, elements, and/or components, but does not exclude one or more other Characteristics, whole The existence or addition of numbers, operations, elements, components, and/or groups thereof.

The term "A or B" or "at least one of A and/or B" as used herein includes any and all combinations of one or more of the related listed items. For example, "A or B" or "at least one of A or/and B" may include A or B or both A and B, respectively.

The ordinal numbers used herein (for example, "first", "second", etc.) can modify various components of various embodiments of the present invention, but do not limit these components. For example, these terms do not limit the order and/or importance of the components. These terms are only used to distinguish each component. For example, the first user device and the second user device are different user devices from each other. For example, according to various embodiments of the present invention, without departing from the scope of the present invention, the first component may be represented by the second component, and vice versa.

When a component is “connected to” or “coupled to” another component, the component may be directly connected or coupled to the other component, or another (other) component may be interposed therebetween. In contrast, when a component is "directly connected to" or "directly coupled to" another component, other intervening components may not be intervened in between.

The terms used herein are provided only to illustrate certain embodiments of the present invention, but not to limit the present invention.

Unless otherwise defined, all terms used herein including technical and scientific terms have the same meanings as commonly understood by those with ordinary knowledge in the technology to which the various embodiments of the present invention belong. It will be better understood that terms (such as terms defined in commonly used dictionaries) should be interpreted as having the relevant technical context The meaning in is consistent with the meaning, and unless explicitly defined as such in this article, it should not be interpreted as having an idealized or overly formal meaning.

The electronic device according to various embodiments of the present invention may be a camera module. For example, the electronic device may include at least one of the following: smart phone, tablet personal computer (PC), mobile phone, video phone, e-book reader, desktop personal computer , Laptop personal computer, netbook computer, personal digital assistant (PDA), portable multimedia player (PMP), animation expert group stage 1 or stage 2 (MPEG-1 or MPEG-2) Audio Layer 3 (Moving Picture Experts Group phase 1 or phase 2 audio layer 3, MP3) players, mobile medical devices, cameras, or wearable devices (for example, head-mounted devices) -mounted-device, HMD), such as electronic glasses, electronic clothes, electronic bracelets, electronic necklaces, electronic application accessories (or application accessories), electronic tattoos, smart watches, etc.).

In various embodiments of the present invention, the electronic device may be a smart home appliance with a camera module. Smart home appliances may include at least one of the following: for example, television (TV), digital video disc (DVD) players, audio players, refrigerators, air conditioners, cleaners, ovens, microwave ovens , Washing machine, air purifier, set-top box, TV box (for example, Samsung HomeSync ^TM , Apple TV ^TM or Google TV ^TM ), game console, electronic dictionary, Electronic key, video camera, or electronic photo frame.

In various embodiments of the present invention, the electronic device may include a camera module, and may further include at least one of the following: various medical devices (for example, magnetic resonance angiography (MRA), magnetic resonance imaging ( magnetic resonance imaging (MRI), computed tomography (CT), camcorders, ultrasonic devices, etc.), navigation devices, global positioning system (GPS) receivers, event data recorders ( event data recorder (EDR), flight data recorder (flight data rocorder, FDR), in-vehicle entertainment information device, ship electronic device (for example, ship navigation device, turning compass, etc.), avionics, security device (security device) , Car head unit (car head unit), industrial or household robots, automatic teller's machine (ATM) used in banks, and point of sale (POS) used in stores, etc.

In various embodiments of the present invention, the electronic device may include at least one of the following: furniture or a part of a building/structure, an electronic board, an electronic signature receiving device, a projector, or various measuring instruments (for example, water meters, Electric meters, gas meters, radio wave meters, etc.), each of the devices includes a camera module. The electronic device according to various embodiments of the present invention may be one or a combination of the above-mentioned devices. In addition, the electronic device according to various embodiments of the present invention may be a flexible device. Those skilled in the art will understand that the electronic devices according to various embodiments of the present invention are not limited to the above-mentioned devices.

According to various embodiments of the present invention, the facial information that can be analyzed by the electronic device may include skin information of facial regions (for example, eyes, nose, eyebrows, cheeks, etc.).

According to various embodiments of the present invention, the electronic device can analyze skin information according to at least one analysis item. For example, items for analyzing facial information may include items for measuring various skin conditions, such as pores, acne, pigmentation, skin tone, dark circles, wrinkles, and so on.

According to various embodiments of the present invention, the operation in which the electronic device performs fine photography may include: in order to finely photograph any area in a specific image, an operation in which focus is set on the area during shooting and the area is enlarged .

Hereinafter, electronic devices according to various embodiments of the present invention will be explained with reference to the accompanying drawings. The term "user" as used herein can refer to a person who uses an electronic device or a device that uses an electronic device (for example, a smart electronic device).

FIG. 1 is a block diagram illustrating the configuration of an electronic device according to various embodiments of the present invention.

1, the electronic device 100 may include at least one of a controller 110, a camera module 120, an input unit 130, a display 140, or a storage 150.

The controller 110 may include at least one of a project setting unit 111, a photographing controller 112, an image analyzer 113, a region selector 114, or an analysis result generator 115, and may control the overall operation of the electronic device 100.

The item setting unit 111 can recognize a specific facial area in the captured image, and select at least one item for analyzing the skin condition of the recognized facial area. For example, if the area surrounding the eyes is recognized in the captured image, the item setting unit 111 may set wrinkles, black circles, etc. as items for analyzing the skin condition of the area surrounding the eyes. As another example, if the cheek area is recognized in the captured image, the item The setting unit 111 can set pigmentation, dark spots, freckles, skin color, etc. as items for analyzing the skin condition of the cheek area.

When a specific facial area is recognized in the photography mode, the photography controller 112 can control the camera module 120 so that the recognized facial area can be enlarged or finely photographed. For example, the camera controller 112 can control the camera module 120 to set the focus or zoom in on at least one area recognized as a facial area during the shooting of the captured image recognized by the camera module 120. At least one district.

The image analyzer 113 can select at least one item for analyzing a specific facial area in the image, and analyze the facial area according to the selected item. For example, the image analyzer 113 can analyze the area around the eyes identified in the captured image according to the selected item (for example, wrinkles or black circles). For example, the image analyzer 113 may analyze the depth or number of wrinkles or the width or color of the black circle in the recognized area around the eye.

The region selector 114 can determine whether a facial region is recognized in the specific image. According to various embodiments of the present invention, if a specific facial region is recognized, the region selector 114 can control the camera module 120 to set focus on the facial region or enlarge the facial region during shooting.

The analysis result generator 115 can determine the result of the facial information analyzed by the image analyzer 113, generate a screen including the determined result, and display the generated screen.

According to various embodiments of the present invention, the analysis result generator 115 can represent the analysis items or analysis results of each facial area in various ways to generate a screen of the analyzed facial information. For example, the analysis result generator 115 can generate facial information Screen, so that each analysis item can be displayed in a different color (for example, acne can be displayed in red, and pigmentation can be displayed in blue). In addition, by determining the previous analysis result, the analysis result generator 115 can display the improved facial area in green or blue and the deteriorated facial area in red to generate a screen of facial information.

According to various embodiments of the present invention, if an area corresponding to a specific facial area is selected from the captured image or an area distinguished by different colors according to analysis items or analysis results, the analysis result generator 115 can provide the result of magnification or analysis. The image obtained by selecting the face area. For example, the analysis result generator 115 can control the display 140 to display the generated image.

The camera module 120 can capture images. According to various embodiments of the present invention, the camera module 120 may be controlled by the controller 110 to set a focus on an area recognized as a specific face area during shooting and enlarge the area.

The display 140 can display an image being captured by the camera module 120.

According to various embodiments of the present invention, if a specific area is selected from the displayed image, the display 140 can display an image obtained by zooming in or finely shooting the area. For example, when the electronic device 100 selects a specific area, the enlarged image may be an image obtained by zooming in and shooting the selected area. When the electronic device 100 selects a specific area, the enlarged image may be an image in which the focus is set on the selected area and the selected area is enlarged during shooting.

The storage 150 may include skin analysis information 151 or user identification information 152. The skin analysis information 151 may include skin analysis information of a specific facial area based on the analysis item.

The skin analysis information 151 may include information obtained by analyzing the skin condition of a specific area of each item. For example, the skin analysis information 151 may include information obtained by determining the presence/absence of pigmentation, skin tone problems, dark spots, or acne in a specific facial area, previously analyzed information for each item, or in chronological order Shows the historical information of the skin analysis results for each facial area.

The user identification information 152 may include facial information of a specific user. For example, the image analyzer 113 can use the user identification information 152 to compare the “eye” area identified in the captured image with the previously stored “eye” image of another user. For the user identification information 152, based on the comparison result in the image analyzer 113, the analysis result of the facial area recognized in the captured image can be stored in the facial information about the user including the same facial area.

For example, the electronic device according to various embodiments of the present invention may include: a camera module; a display for displaying an image being shot by the camera module; and a controller for detecting at least from the image A face area, controlling the camera module to enlarge the at least one detected face area, and analyzing the enlarged face area according to at least one analysis item.

The at least one facial region according to various embodiments of the present invention may include a region surrounding at least one of eyes, nose, cheeks, mouth, or eyebrows.

The at least one analysis item according to various embodiments of the present invention may include at least one of wrinkles, acne, pigmentation, skin tone, dark circles, or pores.

The controller according to various embodiments of the present invention can analyze the thickness, size, number, width, or color of each analysis item in the enlarged face area. One less.

When an input gesture to a specific area is detected in the image, the controller according to various embodiments of the present invention can control the display to display an image obtained by enlarging the area where the gesture is detected.

The controller according to various embodiments of the present invention can detect the facial area of at least one predetermined analysis item from an image, and analyze the detected facial area according to the predetermined analysis item.

When the at least one facial region is detected, the controller according to various embodiments of the present invention may control the display to display a screen of the enlarged facial region.

The controller according to various embodiments of the present invention can control the camera module to finely photograph the at least one detected facial area when a predetermined motion is detected.

The predetermined action according to various embodiments of the present invention may include any one of the following: an action of running an application in which the camera module is enabled, an action of energizing an electronic device in a locked state, or an electronic device The device 100 is positioned at a specific angle or a specific direction of action.

The controller according to various embodiments of the present invention may determine a pre-stored analysis result of a specific face area or analysis item, compare the determined pre-stored analysis result with the result of the analysis, and control the display to display the result including the comparison Screen.

2 is a flowchart illustrating the operation of analyzing skin information on an electronic device according to various embodiments of the present invention.

2, in operation 210, the electronic device can detect from the captured image Facial area.

In operation 220, the electronic device may select an analysis item from the photographed facial area.

In operation 230, the electronic device may finely photograph the selected item. For example, the electronic device can set the focus on the selected item and zoom in on the selected item during shooting.

In operation 240, the electronic device may analyze the enlarged image. According to various embodiments of the present invention, the electronic device can compare the enlarged image with the previously stored result value. For example, the electronic device can compare the enlarged image with the previously stored result value to analyze the facial area corresponding to the enlarged image.

According to various embodiments of the present invention, if a user captures an image including facial regions, the captured output image may include an image obtained by zooming in or finely capturing each facial region included in the captured image, or include An image that is the result of analyzing an enlarged or finely shot image. For example, by photographing the facial area, the user can obtain the image obtained by zooming in or finely photographing each facial area and the result of analyzing the enlarged or finely photographed image without having to do anything for the face Additional actions for analysis.

At least one of operations 210 to 240 shown in FIG. 2 may be omitted, or at least one other operation may be added between operations 210 and 240. In addition, operations 210 to 240 shown in FIG. 2 may be performed in the order shown, or at least one of operations 210 to 240 may be changed in the execution order.

For example, an electronic device according to various embodiments of the present invention The method for analyzing facial information may include detecting at least one facial region from an image captured by a camera module, enlarging the at least one detected facial region, and analyzing the enlarged facial region according to at least one analysis item. The at least one facial region may include a region surrounding at least one of eyes, nose, cheeks, mouth, or eyebrows. The at least one analysis item may include at least one of wrinkles, acne, pigmentation, skin tone, dark circles, or pores.

The analysis according to various embodiments of the present invention may include analyzing at least one of thickness, size, number, width, or color for each analysis item in the enlarged face region.

The method according to various embodiments of the present invention may further include: when an input gesture to a specific area is detected in the image, displaying an image obtained by enlarging the area where the gesture is detected.

The method according to various embodiments of the present invention may further include: detecting a facial area of at least one predetermined analysis item from an image, and analyzing the detected facial area according to the predetermined analysis item.

The method according to various embodiments of the present invention may further include: when the at least one facial region is detected, displaying a screen obtained by enlarging the detected facial region.

The method according to various embodiments of the present invention may further include: enlarging the at least one detected facial area when the predetermined motion is detected. The predetermined action may include any of the following: the action of running the application in which the camera module is enabled, the action of energizing the electronic device in the locked state, and the positioning of the electronic device in a specific angle or direction Actions.

The method according to various embodiments of the present invention may further include: determining a pre-stored analysis result of a specific facial area or analysis item, and comparing the determined pre-stored analysis result with the result of the analysis, and displaying the result of the comparison .

3 is a flowchart illustrating the operation of analyzing skin information on an electronic device according to various embodiments of the present invention.

3, in operation 310, the electronic device may detect the facial area from the captured image.

In operation 320, the electronic device may recognize at least one focused area in the photographed facial area.

In operation 330, the electronic device may zoom in or finely photograph at least one focused area. For example, when zooming in or finely shooting the focused area, the output image may be an image obtained by setting the focus on a specific area during shooting and/or zooming in on the specific area.

Although according to various embodiments of the present invention, the electronic device has performed the operation of recognizing at least one focused area in operation 320, the electronic device may zoom in or fine-tune the face area in the self-photographed image in operation 310. Shoots the detected facial area carefully. In this case, operation 320 may be omitted.

At least one of operations 310 to 330 shown in FIG. 3 may be omitted, or at least one other operation may be added between operations 310 and 330. In addition, operations 310 to 330 shown in FIG. 3 may be performed in the order shown, or at least one of operations 310 to 330 may be changed in the execution order.

FIG. 4 is a diagram illustrating an analysis of an electronic device according to various embodiments of the present invention The flow chart of the operation of skin information.

Referring to FIG. 4, in operation 410, the electronic device may determine whether to run any application in the non-photographic mode.

In operation 420, the electronic device may determine whether a predetermined user input is detected.

If it is determined that the predetermined user input is not detected, the electronic device may perform operation 410 again.

If it is determined that a predetermined user input is detected, in operation 430, the electronic device may control the camera module to operate in the background.

In operation 440, the electronic device may determine whether a facial area is recognized in the captured image.

If it is determined that the facial area is not recognized, the electronic device may perform operation 410 again.

If it is determined that the facial area is recognized, in operation 450, the electronic device can select at least one analysis item of the recognized image. For example, the at least one analysis item in the image may include at least one item for measuring various skin conditions, such as pores, acne, pigmentation, skin tone, dark circles, wrinkles, and the like.

In operation 460, the electronic device may control the camera module to zoom in on the selected item or set the focus on the selected item.

In operation 470, the electronic device may analyze the enlarged image.

At least one of operations 410 to 470 shown in FIG. 4 may be omitted, or at least one other operation may be added between operations 410 to 470. In addition, Operations 410 to 470 shown in FIG. 4 may be performed in the order shown, or at least one of operations 410 to 470 may be changed in the execution order.

5A, 5B, 5C, 5D, 5E, 5F, 5G, and 5H illustrate a situation where the facial area and its middle facial area are partially enlarged or photographed in detail according to various embodiments of the present invention.

5A, 5B, 5C, 5D, 5E, 5F, 5G, and 5H, according to various embodiments of the present invention, it is assumed that the electronic device is photographing or photographing the entire face. The electronic device can recognize at least one facial area in the image 500 being shot.

5A, the electronic device can recognize at least one of the eyebrows 501, the right eye 502, the left eye 506, the nose 503, the edge of the mouth 504, the cheek 505, or the face 507 in the image 500 being shot.

According to various embodiments of the present invention, when at least one facial area is detected in the image 500 being shot, the electronic device may enlarge the area to capture an image obtained by partially zooming in or finely photographing the area.

5B, when the eyebrows 501 are recognized in the image 500 being shot, the electronic device may enlarge the area of the eyebrows 501 to partially enlarge or finely photograph the area. The electronic device can obtain skin analysis information based on at least one analysis item from the captured image obtained by partially magnifying the area of the eyebrow 501. For example, the skin analysis information may include information about wrinkles 501a in the region of the eyebrows 501.

5C, when the right eye 502 is recognized in the image 500 being shot, the electronic device can enlarge the area of the right eye 502 to partially enlarge or finely photograph the Area. The electronic device can obtain skin analysis information based on at least one analysis item from the captured image obtained by partially magnifying the area of the right eye 502. For example, the skin analysis information may include information about wrinkles 502a in the area of the right eye 502.

Referring to FIG. 5D, when the nose 503 is recognized in the image 500 being shot, the electronic device may enlarge the area of the nose 503 to partially enlarge or finely photograph the area. The electronic device can obtain skin analysis information based on at least one analysis item from the captured image obtained by partially magnifying the area of the nose 503. For example, the skin analysis information may include information about the pores 503a in the area of the nose 503.

Referring to FIG. 5E, when the edge 504 of the outlet is recognized in the image 500 being shot, the electronic device can enlarge the area of the edge 504 of the mouth to partially enlarge or finely photograph the area. The electronic device can obtain skin analysis information based on at least one analysis item from the captured image obtained by partially magnifying the area of the edge 504 of the mouth. For example, the skin analysis information may include information about the pigmentation 504a in the region of the edge 504 of the mouth.

5F, when the cheek 505 is recognized in the image 500 being shot, the electronic device may enlarge the area of the cheek 505 to partially enlarge or finely photograph the area. The electronic device can obtain skin analysis information based on at least one analysis item from the captured image obtained by partially magnifying the area of the cheek 505. For example, the skin analysis information may include information about acne 505a in the area of the cheek 505.

Referring to FIG. 5G, when the left eye 506 is recognized in the image 500 being shot, the electronic device may enlarge the area of the left eye 506 to partially enlarge or finely photograph the area. The electronic device can be taken by partially magnifying the area of the left eye 506 It is like obtaining skin analysis information based on at least one analysis item. For example, the skin analysis information may include information about the black circle 506a in the area of the left eye 506.

Referring to FIG. 5H, when the face 507 is recognized in the image 500 being shot, the electronic device may enlarge the area of the face 507 to partially enlarge or finely photograph the area. The electronic device can obtain skin analysis information based on at least one analysis item from the captured image obtained by partially magnifying the area of the face 507. For example, the skin analysis information may include information about the skin color 507a in the region of the face 507.

Figure 6 illustrates facial analysis items according to various embodiments of the present invention.

Referring to FIG. 6, the electronic device has captured an image 600 including at least one facial area, and can identify at least one analysis item in the captured image 600. For example, the electronic device can recognize acne 601, wrinkles 602, or pigmentation 603 as analysis items.

FIG. 7 illustrates the operation of analyzing facial analysis items according to various embodiments of the present invention.

Referring to FIG. 7, the electronic device can analyze the pigmentation 711 in each analysis item in the captured image 710.

According to various embodiments of the present invention, the electronic device can generate an image 720 obtained by inverting the color of the pigmented part in the captured image. The electronic device can determine a color-inversed region in the image 720, thereby identifying at least one pigmented portion 721. For example, the electronic device can determine whether the pigmentation has been improved or worsened based on the number of pigmented parts.

FIG. 8 illustrates the operation of analyzing a face analysis item according to various embodiments of the present invention.

Referring to FIG. 8, the electronic device can analyze the acne 811 in each analysis item in the captured image 810.

According to various embodiments of the present invention, the electronic device can generate an image 820 obtained by inverting the color of the portion where acne is recognized in the captured image. For example, the electronic device can determine the color inversion area in the image 820, thereby identifying at least one acne point 821. For example, the electronic device can determine whether the acne has increased or decreased based on the number of acne spots.

FIG. 9 illustrates an operation of analyzing a face analysis item according to various embodiments of the present invention.

Referring to FIG. 9, the electronic device can analyze the pores 911 in each analysis item in the captured image 910.

According to various embodiments of the present invention, the electronic device can generate an image 920 obtained by darkening the color of the portion where the pores are recognized in the captured image. For example, the electronic device can determine the darkened area 921 in the image 920 to identify at least one pore. For example, the electronic device can determine whether the pores have increased or decreased based on the number of pores.

FIG. 10 illustrates the operation of analyzing a face analysis item according to various embodiments of the present invention.

10, the electronic device can analyze the wrinkles in each analysis item in the captured image 1000.

According to various embodiments of the present invention, the electronic device may indicate the wrinkle line 1001 in the area where the wrinkle is recognized in the captured image 1000. For example, the electronic device can determine the presence/absence of wrinkles by determining at least one wrinkle line 1001 in the captured image 1000.

According to various embodiments of the present invention, the electronic device can determine the thickness or number of at least one wrinkle line represented in the captured image. For example, the electronic device can determine whether the wrinkle has become deeper than before based on the thickness of the at least one wrinkle line, and can determine whether the wrinkle has increased or decreased based on the number of wrinkle lines.

FIG. 11 illustrates the operation of analyzing facial analysis items according to various embodiments of the present invention.

Referring to FIG. 11, the electronic device can analyze the skin color in each analysis item in the captured image 1100.

According to various embodiments of the present invention, the electronic device may set a part of the skin in the captured image 1100 as the skin color analysis area 1101. For example, by determining at least one skin color analysis area 1101 in the captured image 1100, the electronic device can compare the value obtained by measuring the at least one skin color analysis area 1101 with the previously measured skin color value , Or obtain the measured value of the skin color analysis area.

According to various embodiments of the present invention, the electronic device can analyze the skin color analysis area 1101 in the captured image 1100. For example, the electronic device can measure the color, contrast, brightness, color temperature, etc. of the skin color analysis area, thereby determining whether the measured skin color looks darker or brighter than the previously measured skin color.

FIG. 12 illustrates a photographing operation for face analysis according to various embodiments of the present invention.

Referring to FIG. 12, it is assumed that the electronic device is photographing or photographing a facial region for facial analysis.

According to various embodiments of the present invention. The electronic device can determine the characteristics of the image currently being shot (for example, resolution, image size, white balance, etc.). The electronic device can compare the characteristics of the image currently being shot with the characteristics of the previously shot image. For example, if the characteristics of the currently shot image (for example, white balance) are not similar to those of the previously shot image, the electronic device can synchronize the characteristics of the currently shot image with the characteristics of the previously shot image ( synchronize).

12, the electronic device may display a screen 1210 indicating that the electronic device is matching the white balance of the image currently being shot with the white balance of the previously shot image. After synchronizing the characteristics of the currently photographed image with the characteristics of the previously photographed image, the electronic device may display a screen 1220 indicating that the synchronization of the white balance has been completed and photography is possible.

According to various embodiments of the present invention, the user can confirm on the screen 1220 that the current photographing condition of the electronic device is set to be similar to the previous photographing condition of the image for facial analysis. Therefore, it is possible to prevent a situation in which the analysis result of face information may be different due to factors such as lighting, indoor/outdoor environment, weather, etc. despite the same skin condition.

13A, 13B, and 13C illustrate screens showing the results of facial analysis according to various embodiments of the present invention.

13A, 13B, and 13C, according to various embodiments of the present invention, when at least one facial region is recognized in the captured image, the electronic device can zoom in or finely capture the at least one facial region. For example, the electronic device may generate the analysis result screen 1300 including the enlarged or finely photographed area in the facial area corresponding to the captured image. For example, the analysis result screen 1300 may include an enlarged or finely photographed area of each of the eyes 1310, the nose 1320, and the cheeks 1330 that are recognized as at least one facial area.

Referring to FIG. 13A, the user can select an eye 1310 on the analysis result screen 1300. According to various embodiments of the present invention, the electronic device can output an image obtained by zooming in or shooting the eye 1310 in detail. For example, in an image obtained by zooming in or finely shooting the eye 1310, a line representing at least one wrinkle 1311 around the eye 1310 may be displayed.

Referring to FIG. 13B, the user can select the nose 1320 on the analysis result screen 1300. According to various embodiments of the present invention, the electronic device can output an image obtained by zooming in or finely photographing the nose 1320. For example, in an image obtained by magnifying or finely photographing the nose 1320, at least one acne point 1321 around the nose 1320 can be displayed.

Referring to FIG. 13C, the user can select the cheek 1330 on the analysis result screen 1300. According to various embodiments of the present invention, the electronic device can output an image obtained by zooming in or finely photographing the cheek 1330. For example, in an image obtained by zooming in or finely photographing the cheek 1330, the color of at least one pigmented portion 1331 around the cheek 1330 may be displayed in a bright manner.

FIG. 14 illustrates a screen showing the result of skin analysis according to various embodiments of the present invention.

14, the analysis result screen 1410 may include a selection item of View Skin Analysis History 1411 to show the history of the skin analysis result of each analysis item.

According to various embodiments of the present invention, if the skin analysis history 1411 is selected to be viewed, the electronic device may switch to the skin analysis history screen 1420.

The skin analysis history screen 1420 according to various embodiments of the present invention may show skin analysis items (for example, skin tone, acne, wrinkles, pigmentation, pores, etc.) measured in a specific period (for example, from June to August) The history of each skin analysis result.

For example, the skin color measurement result 1421 may show that the measured skin color is brighter than before. The acne measurement result 1422 may show that the measured acne has increased compared to before. The wrinkle measurement result 1423 may show that the number of wrinkles measured is similar to the number of previous wrinkles. The pigmentation measurement result 1424 can show that the color of the pigmented part is lighter than before. The pore measurement result 1425 can show that the number of pores has increased compared to before.

FIG. 15 illustrates a screen showing the result of skin analysis according to various embodiments of the present invention.

15, according to various embodiments of the present invention, the electronic device may analyze a specific facial area, and recommend ways to improve the facial area according to the analysis result.

When the area surrounding the eye is identified in the captured image 1510, the electronic device can analyze the skin condition of the area surrounding the eye and display the analysis result. For example, As a result of the analysis, the electronic device can determine that new wrinkles are recognized or found around the eyes of a specific user, or the number of wrinkles has increased.

According to various embodiments of the present invention, the electronic device may provide a skin care recommendation screen 1520 including ways to improve wrinkles. For example, if the user selects a wrinkle mark 1511 in the captured image 1510, the electronic device can switch from the screen showing the captured image 1510 to the skin care recommendation screen 1520.

The skin care recommendation screen 1520 according to various embodiments of the present invention may include cosmetics 1521 that are good for wrinkles, a run button (or execution button) 1522 for searching for more eye creams, or for recommending massages that are good for wrinkles around the eyes The run button 1523 of the massage method. For example, cosmetics 1521 beneficial to wrinkles may include **eye cream 1521a, ##cream 1521b, or oo eye cream 1521c, and may further include an operation button that the user can use to evaluate each cosmetic or order a specific cosmetic.

FIG. 16 illustrates a screen showing the result of skin analysis according to various embodiments of the present invention.

Referring to FIG. 16, according to various embodiments of the present invention, the electronic device may analyze a specific facial area and recommend ways to improve the facial area based on the analysis result. For example, as a result of the analysis, the electronic device can recognize the part where the skin color around the cheek of the specific user changes.

When the area surrounding the cheek is recognized in the captured image 1610, the electronic device may display the skin color analysis area 1611 in which the skin color of the specific area of the cheek is analyzed.

According to various embodiments of the present invention, the electronic device may provide a skin care recommendation screen 1620 including ways to improve skin tone. For example, if the user selects the skin color analysis area 1611 in the captured image 1610, the electronic device can switch from the screen showing the captured image 1610 to the skin care recommendation screen 1620.

The skin care recommendation screen 1620 according to various embodiments of the present invention may include a mask pack 1621 for improving skin tone or a run button 1622 for searching for more mask packs. For example, a facial mask 1621 that is good for improving skin tone may include a **mask 1621a or a □□mask 1621b, and may further include an operation button that the user can use to jump to a screen showing the characteristics of each mask or order a specific mask.

FIG. 17 illustrates a screen showing the result of skin analysis according to various embodiments of the present invention.

Referring to FIG. 17, according to various embodiments of the present invention, the electronic device can analyze a specific facial area, and provide an advertisement screen of a product capable of improving the facial area according to the analysis result.

When at least one acne spot in the area surrounding the cheek or the area surrounding the cheek is identified in the captured image 1710, the electronic device may highlight the identified acne 1711.

According to various embodiments of the present invention, the electronic device may provide an advertisement screen 1720 that advertises products capable of improving acne. For example, if the user selects acne 1711 shown in the captured image 1710, the electronic device can switch from the screen showing the captured image 1710 to the advertisement screen 1720.

The advertising screen 1720 according to various embodiments of the present invention may include acne Treatment advertisement or "More" run button 1721 used to switch to the screen explaining acne treatment.

FIG. 18 illustrates a screen showing the results of skin analysis according to various embodiments of the present invention.

Referring to FIG. 18, according to various embodiments of the present invention, the electronic device may analyze a specific facial area, and recommend ways to improve the facial area according to the analysis result.

When the nose is recognized in the captured image 1810, the electronic device can display the pore analysis area 1811 in which the number of pores in the nose is analyzed.

According to various embodiments of the present invention, the electronic device may provide a skin care recommendation screen 1820 that includes a way to reduce the number of pores. For example, if the user selects the pore analysis area 1811 in the captured image 1810, the electronic device can switch from the screen showing the captured image 1810 to the skin care recommendation screen 1820.

The skin care recommendation screen 1820 according to various embodiments of the present invention may include information 1821 about dermatology clinics famous for pore treatment or advertisement 1822 of a specific dermatology clinic. For example, the advertisement 1822 of a specific dermatology clinic may include a screen showing the medical department of the dermatology clinic or a shortcut running button for switching to the information screen of the dermatology clinic.

Figure 19 illustrates a network environment according to various embodiments of the present invention.

19, the network environment 1900 may include at least one electronic device 1901 or 1904 or a server 1906. The at least one electronic device 1901 or 1904 can be connected to the server 1906 via the network 1962.

The electronic device 1901 may include a bus 1910, a processor 1920, a memory Body 1930, input/output (I/O) interface 1940, display 1950, communication interface 1960, and camera module 1970.

The bus 1910 may be a circuit that connects the above-mentioned components to each other, and transmits communication information (for example, control messages) between the above-mentioned components.

The processor 1920 can receive commands from the above-mentioned other components (for example, the memory 1930, the input/output interface 1940, the display 1950, the communication interface 1960, the camera module 1970, etc.) via the bus 1910, and decrypt the received commands. And based on the decryption command to perform operations or data processing.

The processor 1920 can control the overall operation of the electronic device 1901. According to various embodiments of the present invention, the processor 1920 may include at least one controller (for example, the controller 110).

The memory 1930 can store commands or commands received from the processor 1920 or other components (for example, the input/output interface 1940, the display 1950, the communication interface 1960, the camera module 1970, etc.) or generated by the processor 1920 or other components. data. The memory 1930 may include programming modules, for example, a core 1931, middleware 1932, application programming interface (API) 1933, or one or more application programs 1934. Each of the above-mentioned programming modules can be configured by software, firmware, hardware, or a combination thereof.

The core 1931 can control or manage the system resources (for example, bus 1910, processing system) used to execute operations or functions implemented in other programming modules (for example, middleware 1932, application programming interface 1933, or application 1934). 1920, memory 1930, etc.). In addition, the core 1931 can provide middleware 1932 The application programming interface 1933 or the application 1934 can be used to access the interface of individual components of the electronic device 1901 and control or manage the individual components.

The middleware 1932 can play an intermediary role, so that the application programming interface 1933 or the application 1934 can communicate with the core 1931 to exchange data with each other. In addition, in response to the work request received from the application 1934, the middleware 1932 can, for example, use the following method to perform control of the work request (for example, scheduling or load balancing): assign at least one of the application 1934 to Utilize the priority of the system resources of the electronic device 1901 (for example, the bus 1910, the processor 1920, the memory 1930, etc.).

The application programming interface 1933 may include at least one interface or function (for example, command) for file control, window control, image processing, character control, etc., as an application program 1934 for controlling the core 1931 or middleware 1932 provided Functional interface.

According to various embodiments of the present invention, the application 1934 may include a short message service (SMS)/multimedia messaging service (multimedia messaging service, MMS) application, email application, calendar application, alarm clock application, medical Health care applications (for example, applications for measuring exercise or blood sugar), or environmental information applications (for example, applications for providing information about pressure, humidity, temperature, etc.). Additionally or alternatively, the application program 1934 may be an application program related to information exchange between the electronic device 1901 and an external electronic device (for example, the electronic device 1904). Applications related to information exchange may include, for example, those used to relay specific information to external electronic devices. Notification relay application or device management application used to manage external electronic devices.

For example, notification relay applications may include other applications used to connect the electronic device 1901 (for example, SMS/multimedia messaging service applications, email applications, healthcare applications, environmental information applications, etc.) The function of transmitting the generated notification information to an external electronic device (for example, the electronic device 1904). Additionally or alternatively, the notification relay application may, for example, receive notification information from an external electronic device (eg, electronic device 1904) and provide the received notification information to the user. The device management application can, for example, manage (e.g., install, delete, or update) the function of at least one component of the external electronic device (e.g., electronic device 1904) that communicates with the electronic device 1901 (e.g., control to turn on/off the external electronic device itself (Or some of its components) or the brightness (or resolution) of the display), manage applications running on external electronic devices, or manage services provided on external electronic devices (for example, call service or messaging service) .

According to various embodiments of the present invention, the application 1934 may include an application specified according to the nature (for example, the type of electronic device) of the external electronic device (for example, the electronic device 1904). For example, when the external electronic device is an MP3 player, the application program 1934 may include an application program related to music playback. Similarly, when the external electronic device is a mobile medical device, the application program 1934 may include an application program related to medical care. In one embodiment of the present invention, the application 1934 may include an application specified in the electronic device 1901 or an application received from an external electronic device (for example, the server 1906 or the electronic device 1904) At least one.

The input/output interface 1940 can transmit commands or data received from the user via an input/output device (for example, a sensor, a keyboard or a touch screen) to the processor 1920, the memory 1930, and the communication via the bus 1910, for example. Interface 1960 or camera module 1970. For example, the input/output interface 1940 can provide the processor 1920 with data of the user's touch made on the touch screen. In addition, the input/output interface 1940 can output commands or data received from the processor 1920, the memory 1930, the communication interface 1960, or the camera module 1970 via the bus 1910, for example, via an input/output device (for example, a speaker or a display). For example, the input/output interface 1940 can output the voice data processed by the processor 1920 to the user through a speaker.

The display 1950 can display various information (for example, multimedia data, text data, etc.) to the user.

The communication interface 1960 can connect the communication with the electronic device 1901 to an external device (for example, the electronic device 1904 or the server 1906). For example, the communication interface 1960 can be connected to the network 1962 through wireless communication or wired communication to communicate with external devices. Wireless communication may include at least one of the following: for example, WiFi, Bluetooth (BT), near field communication (NFC), global positioning system (GPS) or cellular communication (for example, long-term evolution (long-term evolution) term evolution, LTE, LTE Advanced (LTE-A), code division multiple access (CDMA), wideband CDMA, WCDMA, general mobile telecommunications system (universal mobile telecommunications system, UMTS), wireless broadband (WiBro), global system for mobile communications (GSM), etc.). Wired communication may include at least one of the following: for example, universal serial bus (USB), high definition multimedia interface (HDMI), recommended standard-232 (RS) -232) or plain old telephone service (POTS).

The camera module 1970 can be controlled based on the information obtained from other components (for example, the processor 1920, the memory 1930, the input/output interface 1940, the communication interface 1960, etc.) to enlarge or refine the image being shot. To shoot a specific area.

In the embodiment of the present invention, the network 1962 may be a telecommunication network. The telecommunication network may include at least one of a computer network, the Internet, the Internet of things (IoT), or a telephone network. In one embodiment of the present invention, the communication protocol (for example, transport layer protocol, data link layer protocol, or physical layer protocol) used for the communication between the electronic device 1901 and the external device can be used by the application 1934 and the application programming interface 1933. Supported by at least one of, middleware 1932, core 1931, or communication interface 1960.

According to various embodiments of the present invention, the electronic device can use the automatic white balance function (or the white balance automatic setting function) to calculate the lighting environment of the image being shot. For example, using the calculated lighting environment, the electronic device can shoot images, so that batch white balance can be set. Besides, even if it doesn’t exist in it In a photography environment with an external light source, the electronic device can still shoot images by adjusting the light source (for example, a light emitting diode (LED) light source) included in the electronic device, so that the batch white balance can be set.

If the white balance is set according to the automatic white balance function during shooting, the face area in the shot image can be set to have a different color according to the white balance set to correspond to each shot image.

To set the color of the facial area of multiple images in batch, only the light source included in the electronic device can be used after blocking the external light source. However, the electronic device including the light source used to implement this method can be large and heavy for the user to carry.

According to various embodiments of the present invention, the electronic device may include a front camera or a rear camera. For example, if the front camera is used to shoot an image, the rear camera can calculate the shooting environment to set the white balance of the image to be shot, so that the image can be shot in a constant shooting environment. This will be described in detail below with reference to FIGS. 20A, 20B, 20C, and 21.

20A, 20B, and 20C illustrate a situation in which an electronic device analyzes a face according to various embodiments of the present invention.

20A, 20B, and 20C, the electronic device may include a front camera 2001 or a rear camera 2002, and the rear camera 2002 may include a color balance card 2003.

According to various embodiments of the present invention, the electronic device can use the color balance card 2003 to detect the lighting elements (or lighting components) of the surrounding environment in which the electronic device is shooting images. For example, the color balance card 2003 can be used for light scattering It is a device that photographs the corrected color of light, or it can be a device for color correction. Although the color balance card 2003 is installed in, for example, the rear camera 2002, the color balance card 2003 can be installed in various other modules that interact with electronic devices.

According to various embodiments of the present invention, the electronic device may determine that the brightness of an image obtained through at least one camera module (for example, the front camera 2001 or the rear camera 2002) is abnormal. For example, the situation in which the brightness of the image is determined to be abnormal may include: a situation in which the light is brighter or darker than a set value, or a situation in which a facial area cannot be obtained.

According to various embodiments of the present invention, if it is determined that the brightness of the image is abnormal, the electronic device may not store the image being captured by the camera module (for example, the front camera 2001 or the rear camera 2002), thereby determining that it is difficult to obtain the image.

In addition, according to various embodiments of the present invention, if it is determined that the brightness of the image is abnormal, the electronic device may display a screen indicating the abnormality of the photographing condition or output an audio alarm indicating the abnormality of the photographing condition. Therefore, the user can move the electronic device to a position where normal photography can be performed. If it is determined that normal photography can be performed, the electronic device can again output a screen or sound indicating that photography can be performed.

Referring to FIG. 20A, the electronic device may use the front camera 2001 to obtain an image including the face area, and use the color balance card 2003 of the rear camera 2002 to calculate the lighting elements.

According to various embodiments of the present invention, the electronic device may apply the calculated lighting element to the obtained image including the facial area. For example, for an image to which the calculated lighting element is applied, the setting is independent of the lighting composition of the photographic environment White balance, so the image can be shot to correspond to the white balance of the face area of the previously stored image.

Referring to FIG. 20B, it is assumed that the electronic device is using the front camera 2001 to capture or photograph an image. For example, the electronic device can analyze the image being captured by the front camera 2001.

According to various embodiments of the present invention, the electronic device can determine whether the lighting environment in which the electronic device is capturing the image is an extreme lighting environment (for example, backlighting, low illumination, over-bright lighting, etc.) based on the result of analyzing the image. For example, if the lighting environment where the electronic device is using the front camera 2001 to capture images is determined to be an extreme lighting environment, the electronic device may not store the images being captured by the front camera 2001, or control the front camera 2001 so that the photography function may not be performed Empowerment.

According to various embodiments of the present invention, if the lighting environment in which the electronic device is using the front camera 2001 to capture images is determined to be an abnormal lighting environment, the electronic device may use the color balance card 2003 to calculate the white balance, and the calculated white balance value Applied to the image being taken by the front camera 2001.

Referring to FIG. 20C, it is assumed that the camera 2002 captures or photographs an image after the electronic device is being used. For example, the electronic device can analyze the image being captured by the rear camera 2002.

According to various embodiments of the present invention, the electronic device can determine whether the lighting environment in which the electronic device is capturing the image is an extreme lighting environment (for example, backlighting, low illumination, over-bright lighting, etc.) based on the result of analyzing the image. For example, if the lighting environment in which the camera 2002 shoots an image after the electronic device is in use is determined to be In an extreme lighting environment, the electronic device may not store images being taken by the rear camera 2002, or control the rear camera 2002 so that the photography function may not be enabled.

According to various embodiments of the present invention, if the lighting environment in which the camera 2002 takes an image after the electronic device is being used is determined to be an abnormal lighting environment, the electronic device may use the color balance card 2003 to calculate the white balance, and then the calculated white balance value Applied to the image being shot by the rear camera 2002.

FIG. 21 is a flowchart illustrating the operation of analyzing skin information on an electronic device according to various embodiments of the present invention.

Referring to FIG. 21, in operation 2110, the electronic device may obtain images from at least one camera.

In operation 2120, the electronic device may analyze the obtained image.

In operation 2130, the electronic device may determine whether the brightness of the obtained image is abnormal.

If it is determined in operation 2130 that the brightness of the obtained image is not abnormal, the electronic device may perform operation 2110 again.

If it is determined in operation 2130 that the brightness of the obtained image is abnormal, the electronic device may determine in operation 2140 whether the camera that has taken the obtained image is a camera mounted on the front of the electronic device.

If it is determined in operation 2140 that the camera that has taken the obtained image is not a camera mounted on the front of the electronic device, the electronic device may calculate the white balance in operation 2150. For example, a module capable of calculating white balance (for example, a color balance card) can be used to calculate white balance, and the module is installed on the front of the electronic device. On the camera. In operation 2160, the electronic device may use the white balance calculated in operation 2150 to apply the white balance of the captured image as the calculated white balance.

If it is determined in operation 2140 that the camera that has captured the obtained image is a camera mounted on the front of the electronic device, the electronic device may apply white balance to the captured image in operation 2160. For example, the image to which the white balance is applied in operation 2160 may be a pre-stored image to which the white balance value is applied, because the color balance card is not set in the camera installed on the front of the electronic device.

In operation 2170, the electronic device may store the image. For example, the stored image may be an image with white balance applied.

At least one of operations 2110 to 2170 shown in FIG. 21 may be omitted, or at least one other operation may be added between operations 2110 and 2170. In addition, operations 2110 to 2170 shown in FIG. 21 may be performed in the order shown, or at least one of operations 2110 to 2170 may be changed in the execution order.

22A and 22B illustrate photographing operations for facial analysis according to various embodiments of the present invention.

22A and 22B, according to various embodiments of the present invention, the electronic device can determine whether the accessory is worn in the facial area being photographed.

Referring to FIG. 22A, it is assumed that a high school girl whose facial area is a photographic target is wearing glasses.

According to various embodiments of the present invention, the electronic device can determine whether the glasses are worn on the facial region according to whether the facial region being photographed includes a line 2211 corresponding to the edge of the glasses or a portion 2212 representing light reflected on the glasses.

According to various embodiments of the present invention, if it is determined that the glasses are worn on the facial area being photographed, the electronic device may instruct the user or high school girl to take off the glasses. For example, the electronic device may display words such as "Please take off your glasses before shooting" in the image 2210 being shot.

When seeing the wording, the user of the electronic device according to various embodiments of the present invention can take off the glasses. If the electronic device determines that the user has taken off the glasses, the electronic device may display words such as "You can shoot" in the image 2220 being shot.

Referring to FIG. 22B, it is assumed that a high school girl whose facial area is a photographic target is wearing a hat.

According to various embodiments of the present invention, the electronic device can determine whether the hat is worn on the face region according to whether the face region being photographed includes a line 2231 corresponding to the shape of the hat.

According to various embodiments of the present invention, if it is determined that the hat is worn on the face area being photographed, the electronic device may instruct the user or high school girl to take off the hat. For example, the electronic device may display words such as "Please take off your hat before shooting" in the image 2230 being shot.

When seeing the wording, the user of the electronic device according to various embodiments of the present invention can take off the hat. If the electronic device determines that the user has taken off the hat, the electronic device may display words such as "you can take a picture" in the image 2240 being shot.

Figures 23A and 23B illustrate various embodiments of the present invention Departmental analysis of photography operations.

Referring to FIGS. 23A and 23B, according to various embodiments of the present invention, a line 2301 corresponding to a facial region may be displayed on an electronic device, so that the user can recognize the direction or size of the facial region being photographed.

Referring to FIG. 23A, it is assumed that a high school girl whose facial area is a photographing target is photographing herself and her face is inclined.

According to various embodiments of the present invention, the electronic device can determine whether the facial region is being photographed at the correct position according to whether the facial region being photographed is outside the line 2301.

According to various embodiments of the present invention, if it is determined that the face area being photographed is outside the line 2301, the electronic device may instruct the user or high school girl to place his face in the correct position. For example, the electronic device may display words such as "Please put your face at the correct position" in the image 2311 being shot.

When seeing the wording, the user of the electronic device according to various embodiments of the present invention can position his face to be consistent with the line 2301. If the electronic device determines that the user has placed his face in the correct position, the electronic device may display words such as "you can take a picture" in the image 2312 being shot.

Referring to FIG. 23B, it is assumed that a high school girl whose face area is a photographic target photographs herself at a certain distance.

According to various embodiments of the present invention, the electronic device can determine whether the face area is at a certain distance according to whether the size of the face area being photographed matches the size of the line 2301 The shooting is still at close range.

According to various embodiments of the present invention, if it is determined that the face area being photographed is shot at a certain distance because the size of the face area is smaller than the size of the line 2301, the electronic device may instruct the user or high school girl to place his face closer . For example, the electronic device may display words such as "please place your face closer" in the image 2321 being shot.

When seeing the wording, the user of the electronic device according to various embodiments of the present invention may position his face to be consistent with the line 2301. If the electronic device determines that the user has placed the face closer, the electronic device may display words such as "you can take a picture" in the image 2322 being shot.

FIG. 24 illustrates a photographing operation for face analysis according to various embodiments of the present invention.

Referring to FIG. 24, it is assumed that an image 2410 is shakenly shot in an electronic device.

According to various embodiments of the present invention, when the electronic device cannot recognize the shape of the facial area in the image being shot or when the electronic device detects the shaking of the electronic device through the sensor, the electronic device can enable the image stabilization function Operation. For example, the image stabilization function may include a function of shooting an image without shaking by adjusting the shutter speed or aperture during the shooting of the image.

According to various embodiments of the present invention, after the image stabilization function is set, the electronic device may display words such as "you can take a picture" in the image 2420 being shot. For example, when seeing the wording, the user can take pictures of his face.

25 is a diagram illustrating the configuration of electronic devices according to various embodiments of the present invention Block diagram.

Referring to FIG. 25, the electronic device 2501 may constitute, for example, all or a part of the electronic device 100 or 1901 shown in FIG. 1 or FIG.

The electronic device 2501 may include at least one processor (for example, an application processor (AP) 2510), a communication module 2520, a subscriber identification module (SIM) card 2524, a memory 2530, and a sensor The device module 2540, the input device 2550, the display 2560, the interface 2570, the audio module 2580, the camera module 2591, the power management module 2595, the battery 2596, the indicator 2597, and the motor 2598.

The application processor 2510 can control multiple hardware or software components connected to the application processor 2510 by running an operating system or an application, and can process and calculate various data including multimedia data. The application processor 2510 may be implemented as, for example, a system on chip (SoC). In an embodiment of the present invention, the application processor 2510 may further include a graphics processing unit (GPU) (not shown).

The communication module 2520 (for example, the communication interface 1960) can be connected to the electronic device 2501 (for example, the electronic device 100 or the electronic device 1901) and other electronic devices (for example, the electronic device 1904 or the server 1906) connected to the electronic device 2501 via the network. ) Perform data transmission/reception in the communication between. In an embodiment of the present invention, the communication module 2520 may include a cellular module 2521, a Wi-Fi module 2523, a Bluetooth module 2525, a global positioning system (GPS) module 2527, and a near field communication (NFC) module. Group 2528, and radio frequency (RF) module 2529.

The cellular module 2521 can provide voice call services via communication networks (for example, Long Term Evolution, Advanced Long Term Evolution, Code Division Multiple Access, Broadband Code Division Multiple Access, Universal Mobile Telecommunications System, Wireless Broadband, Global Mobile Communication System, etc.) , Video call service, messaging service, or Internet service. In addition, the cellular module 2521 can use, for example, a user identification module (for example, a user identification module card 2524) to perform identification and authentication of electronic devices in a communication network. In an embodiment of the present invention, the cellular module 2521 can perform at least some functions that can be provided by the application processor 2510. For example, the cellular module 2521 can perform at least a part of the multimedia control function.

In the embodiment of the present invention, the cellular module 2521 may include a communication processor (CP). In addition, the honeycomb module 2521 can be implemented as a system chip, for example. Although components such as a cellular module 2521 (for example, a communication processor), a memory 2530, or a power management module 2595 are shown as separate components from the application processor 2510 in FIG. 25, according to the present invention In the embodiment, the application processor 2510 can be implemented to include at least some of the above-mentioned components (for example, the honeycomb module 2521).

In the embodiment of the present invention, the application processor 2510 or the cellular module 2521 (for example, a communication processor) can load the volatile memory received from non-volatile memory or another component connected to Order or data, and process the loaded order or data. In addition, the application processor 2510 or the cellular module 2521 can store data received from or generated by at least one of the other components in the non-volatile memory.

Each of the Wi-Fi module 2523, the Bluetooth module 2525, the global positioning system module 2527, or the near field communication module 2528 may include, for example, a processor for processing data transmitted or received via the corresponding module. Although the cellular module 2521, Wi-Fi module 2523, Bluetooth module 2525, GPS module 2527, or near field communication module 2528 are shown as separate blocks in FIG. 25, according to the present invention In an embodiment, at least some (for example, two or more) of the cellular module 2521, the Wi-Fi module 2523, the Bluetooth module 2525, the GPS module 2527, or the near field communication module 2528 may Contained in an integrated chip (IC) or integrated chip package. For example, at least some of the processors corresponding to the cellular module 2521, Wi-Fi module 2523, Bluetooth module 2525, GPS module 2527, or near field communication module 2528 (for example, with The communication processor corresponding to the wireless module 2521 or the Wi-Fi processor corresponding to the Wi-Fi module 2523 can be implemented as a system chip.

The radio frequency module 2529 can perform data transmission/reception, such as radio frequency signal transmission/reception. Although not shown, the radio frequency module 2529 may include, for example, a transceiver, a power amplifier module (PAM), a frequency filter, a low noise amplifier (LNA), etc. The radio frequency module 2529 may further include components (for example, conductors or wires) for transmitting and receiving electromagnetic waves in the free space of wireless communication. Although the cellular module 2521, Wi-Fi module 2523, Bluetooth module 2525, GPS module 2527, and near field communication module 2528 are shown as sharing one radio frequency module 2529 in FIG. 25, According to the embodiment of the present invention, the cellular module 2521, the Wi-Fi module 2523, the Bluetooth module 2525, the global fixed At least one of the bit system module 2527 or the near field communication module 2528 can perform radio frequency signal transmission/reception via a separate radio frequency module.

The user identification module card 2524 can be a card with a user identification module, and can be inserted into a slot formed in a specific position of the electronic device. The user identification module card 2524 may include unique identification information (for example, an integrated circuit card identifier (ICCID)) or user information (for example, an international mobile subscriber identity (IMSI)).

The memory 2530 (for example, the storage 150 or the memory 1930) may include an internal memory 2532 or an external memory 2534. The internal memory 2532 includes at least one of the following: for example, volatile memory (for example, dynamic random access memory (DRAM), static random access memory (static RAM, SRAM), synchronous Dynamic random access memory (synchronous dynamic RAM (SDRAM), etc.) or non-volatile memory (for example, one time programmable read only memory (OTPROM), programmable read only memory) (programmable ROM, PROM), erasable and programmable ROM (erasable and programmable ROM, EPROM), electrically erasable and programmable ROM (electrically erasable and programmable ROM, EEPROM), mask only Read memory, flash read-only memory, NAND flash memory, NOR flash memory, etc.).

In the embodiment of the present invention, the internal memory 2532 may be a solid state drive (SSD). The external memory 2534 may further include a flash drive, such as compact flash (CF), secure digital (secure digital, SD), micro security digital, mini security digital, extreme digital (xD), memory stick, etc. The external memory 2534 can be functionally connected to the electronic device 2501 via various interfaces. In an embodiment of the present invention, the electronic device 2501 may further include a storage device (or storage medium), such as a hard drive.

The sensor module 2540 can measure physical quantities or detect the operating status of the electronic device 2501, and convert the measured or detected information into electrical signals. The sensor module 2540 may include at least one of the following: for example, a gesture sensor 2540A, a gyroscope sensor 2540B, a barometer 2540C, a magnetic sensor 2540D, an accelerometer 2540E, and a grip sensor 2540F , Proximity sensor 2540G, color sensor (for example, red, green, blue (red, green, blue, RGB) sensor) 2540H, biological sensor 2540I, temperature/humidity sensor 2540J, illuminance Sensor 2540K or ultraviolet (ultra violet, UV) sensor 2540M. Additionally or alternatively, the sensor module 2540 may include, for example, an electronic nose sensor (not shown), an electromyography (EMG) sensor (not shown), an electroencephalogram (not shown), EEG) sensor (not shown), electrocardiogram (ECG) sensor (not shown), infrared (IR) sensor (not shown), iris sensor (not shown) , Fingerprint sensor (not shown), etc. The sensor module 2540 may further include a control circuit for controlling at least one or more sensors belonging to it.

The input device 2550 may include a touch panel 2552, a (digital) pen sensor 2554, buttons 2556, or an ultrasonic input device 2558. The touch panel 2252 may, for example, recognize touch in at least one of capacitive, resistive, infrared or ultrasonic methods. enter. The touch panel 2552 may further include a control circuit. In the case of the capacitive solution, the touch panel 2552 can recognize physical contact or proximity. The touch panel 2552 may further include a tactile layer. In this case, the touch panel 2552 can provide tactile or tactile feedback to the user.

The (digital) pen sensor 2554 can be implemented using, for example, the same or similar method to receiving the user's touch input or through a separate identification chip. The keys 2556 may include, for example, physical buttons, optical keys, or keypads. The ultrasonic input device 2558 is a device that the electronic device 2501 can use a microphone (for example, the microphone 2588) to detect sound waves through an input tool for generating ultrasonic signals to determine data. The ultrasonic input device 2558 empowers wireless recognition. In one embodiment of the present invention, the electronic device 2501 can use the communication module 2520 to receive user input from an external device (for example, a computer or a server) connected to it.

The display 2560 (for example, the display 140 or the display 1950) may include a panel 2562, a holographic device 2564, or a projector 2566. The panel 2562 may be, for example, a liquid-crystal display (LCD) panel or an active-matrix organic LED (AM-OLED) panel. The panel 2562 can be implemented to be flexible, transparent, or wearable, for example. The panel 2562 and the touch panel 2552 can be implemented as a module together. The holographic device 2564 can use the interference of light to show a stereoscopic image in the air. The projector 2566 can display images by projecting light on the screen. A screen may be arranged on the inside or outside of the electronic device 2501, for example. In an embodiment of the present invention, the display 2560 may further include a control circuit for the control panel 2562, the holographic device 2564, or the projector 2566.

The interface 2570 may include, for example, a high-definition multimedia interface (HDMI) 2572, a universal serial bus (USB) 2574, an optical interface 2576, or a D-subminiature (D-sub) interface 2578. The interface 2570 may be included in the communication interface 1960 shown in FIG. 19, for example. In addition or as another option, the interface 2570 may include, for example, a mobile high-definition link (MHL) interface, a secure digital card/multi-media card (MMC) interface, or an infrared data association (infrared data association) interface. data association, IrDA) interface.

The audio module 2580 can perform two-way conversion of audio signals and electrical signals. At least some components of the audio module 2580 may be included in the input/output interface 1940 shown in FIG. 19, for example. The audio module 2580 can process audio information received or output via, for example, the speaker 2582, the receiver 2584, the earphone 2586, or the microphone 2588.

According to an embodiment of the present invention, the camera module 2591 as a device capable of shooting still images and video may include one or more image sensors (for example, a front image sensor or a rear image sensor), a lens (not shown) Shown), an image signal processor (ISP) (not shown), or a flash lamp (not shown) (for example, a light emitting diode or a xenon lamp).

The power management module 2595 can manage the power of the electronic device 2501. Although not shown, the power management module 2595 may include, for example, a power management integrated circuit (PMIC), a charger integrated circuit, or a battery meter or a fuel meter.

The power management integrated circuit can be installed in, for example, an integrated circuit or a system chip semiconductor chip. The charging scheme can be divided into wired charging scheme and wireless charging scheme case. The charger integrated circuit can charge the battery and prevent overvoltage or overcurrent from flowing in from the charger. In one embodiment of the present invention, the charger integrated circuit may include a charger integrated circuit for at least one of a wired charging scheme or a wireless charging scheme. The wireless charging scheme may include, for example, a magnetic resonance scheme, a magnetic induction scheme, or an electromagnetic scheme. Additional circuits for wireless charging can be added (for example, coil loops, resonance circuits, rectifiers, etc.).

The battery meter or fuel meter can measure, for example, the remaining capacitance, charging voltage, charging current, or temperature of the battery 2596. The battery 2596 can store or generate electricity, and use the stored or generated electricity to supply power to the electronic device 2501. The battery 2596 may include, for example, a rechargeable battery or a solar battery.

The indicator 2597 can indicate a specific state (for example, activation state, message state, charging state, etc.) of the electronic device 2501 or its components (for example, the application processor 2510). The motor 2598 can convert electrical signals into mechanical vibrations. Although not shown, the electronic device 2501 may include a processing device (for example, a graphics processing unit) for mobile TV support. The processing device used for mobile TV support can process media data based on standards such as digital multimedia broadcasting (DMB), digital video broadcasting (DVB), or media FLO.

Each of the above-mentioned components of the electronic device according to various embodiments of the present invention may be configured with one or more components, and the name of the component may be changed according to the type of the electronic device. The electronic device according to various embodiments of the present invention may include at least one of the above-mentioned components, some of which may be omitted, or may further include additional other components. Components. In addition, some of the components of the electronic device according to various embodiments of the present invention can be configured as one entity by being combined, thereby performing the previous functions of the components in the same manner.

The term “module” used in various embodiments of the present invention may refer to a unit including, for example, one or a combination of hardware, software, or firmware. The term "module" can be used interchangeably with terms such as unit, logic, logic block, component, or circuit. The module can be an integrated structural component or the smallest unit of its components. The module may be the smallest unit or its component used to perform one or more functions. Modules can be implemented mechanically or electrically. For example, modules according to various embodiments of the present invention may include application-specific integrated circuit (ASIC) chips, field-programmable gate arrays (FPGA), or programmable gate arrays. For at least one of the logic devices, the above-mentioned module is known or will be developed in the future and the above-mentioned module performs certain operations.

At least one component (for example, a module or its function) or method (for example, an operation) of the device according to various embodiments of the present invention may be stored in a computer-readable storage medium in the form of, for example, a programmed module Command to implement. If the command is executed by one or more processors (for example, the controller 110 or the processor 1920), the one or more processors may perform a function corresponding to the command. The computer-readable storage medium may be, for example, the storage 150 or the memory 1930. At least a part of the programming module can be implemented (eg, executed) by, for example, the controller 110 or the processor 1920. At least a part of the programming module may include, for example, a module, a program, a routine, a set of instructions, or a processor for performing one or more functions.

Certain aspects of the present invention can also be implemented as a computer-readable code on a non-transitory computer-readable recording medium. A non-transitory computer-readable recording medium is any data storage device that can store data that can be read by a computer system later. Examples of non-transitory computer-readable recording media include Read-Only Memory (ROM), Random-Access Memory (RAM), and Compact Disc-ROM (Compact Disc-ROM). CD-ROM), magnetic tapes, floppy disks, and optical data storage devices. The non-transitory computer-readable recording medium can also be distributed on a network-coupled computer system to store and execute computer-readable codes in a distributed manner. In addition, the functional programs, codes, and code segments used to implement the present invention can be easily explained by programmers familiar with the technology of the present invention.

In this regard, it should be noted that the various embodiments of the present invention as described above generally involve the processing of input data and the generation of output data to some extent. This input data processing and output data generation can be implemented in hardware or in software combined with hardware. For example, specific electronic components may be employed in mobile devices or similar or related circuit systems for implementing functions associated with various embodiments of the present invention as described above. Alternatively, one or more processors operating in accordance with the stored instructions may implement the functions associated with the various embodiments of the invention as described above. If this is the case, it is within the scope of the present invention to store such instructions on one or more non-transitory processor-readable media. Examples of processor-readable media include read-only memory, random access memory, optical disc read-only memory, magnetic tape, floppy disk, and optical data storage device. The processor-readable medium can also be distributed on a network-coupled computer system to store and execute the instructions in a distributed manner. In addition, for implementing the present invention The functional computer programs, instructions, and instruction sections of the can be easily explained by programmers familiar with the technology of the present invention.

The modules or stylized modules according to various embodiments of the present invention may include at least one of the above-mentioned components, some of which may be omitted, or may further include additional other components. The operations performed by the modules, stylized modules, or other components according to various embodiments of the present invention may be performed in a sequential, parallel, iterative or heuristic manner. Certain operations can be performed in a different order or omitted, or other operations can be added.

According to various embodiments of the present invention, in a storage medium storing instructions, when the instructions are executed by at least one processor, the instructions are set to allow the at least one processor to perform at least one operation. The at least one operation may include: an operation of detecting at least one face region from an image captured by the camera module, an operation of enlarging the at least one detected face region, and analyzing the enlarged face according to at least one analysis item Area operation.

According to various embodiments of the present invention, the at least one operation may include: an operation of detecting a facial area from at least one stored image, and an operation of analyzing the at least one detected facial area according to at least one analysis item.

According to various embodiments of the present invention, the operation of analyzing facial information can be performed based on previously stored images. For example, the stored image may be an image shot at a preset distance or at a preset angle according to a preset fixed focus value. If the user's face is at a position corresponding to the set distance or angle, the electronic device can take a facial image or display a screen for requesting the user to take a facial area corresponding to the distance or angle.

According to various embodiments of the present invention, the stored images can be photographed and stored under predetermined conditions, so that the user may not perform photographing operations. For example, in a situation where the user observes the electronic device for a predetermined time or longer (for example, the user confirms a web browser or views a media file (for example, video)), or where the electronic device is located on the electronic device In the situation where the angle of the user’s face image is captured (for example, in the situation where the electronic device can shoot the user’s face in the forward direction according to the angle of the front camera), the electronic device can use the camera module in the background Operation and shooting the user's facial image.

According to various embodiments of the present invention, the electronic device can determine the stored image, remove the parts that do not need to be analyzed (for example, shadows or hair) from the image, and set in the area where the unnecessary parts are removed At least one reference point to determine the facial area. For example, the reference point can be set on the entire facial area in the image, and can be set based on the pre-analysis database or the previously analyzed facial image of a specific user, so that each facial area (e.g., C area, M area, U area, eyes, nose, cheeks, mouth, etc.).

According to various embodiments of the present invention, when facial regions are recognized in previously stored images, the electronic device can output the results of analyzing the skin conditions of each region through the analysis result screen.

According to various embodiments of the present invention, the operation of analyzing facial information can be performed based on previously stored images. For example, the electronic device may set at least one reference point on the image, and determine the facial area based on the set reference point.

According to various embodiments of the present invention, it is determined from the previously stored images In the face area, the electronic device can output the results of the analysis of the skin condition in each area through the analysis result screen.

FIG. 26 illustrates a screen of an application program for analyzing facial information based on stored images according to various embodiments of the present invention.

26, if an application for analyzing facial information is running, the execution screen 2610 may include a gallery button 2611 for running a gallery application to display images stored in an electronic device, or include a camera The camera button 2612 enabled by the module is switched to the screen for shooting images.

According to various embodiments of the present invention, the image displayed when the gallery button 2611 is selected may include selection items that the user can use to select the image, or may include filtered images stored in the electronic device that can analyze facial information. image. The image that can be analyzed for facial information may be an image in which the facial area is shot to correspond to a predetermined angle or size.

According to various embodiments of the present invention, if the gallery button 2611 is selected, an image selection screen 2620 can be displayed, and the user can select at least one image for which facial information is to be analyzed on the image selection screen 2620. For example, the images displayed on the image selection screen 2620 can be arranged in the order of final shooting or storage.

According to various embodiments of the present invention, if the user selects at least one of the images included in the image selection screen 2620 as the image to be analyzed for facial information, a display for displaying the analysis result of the selected image may be displayed The analysis result screen 2630.

According to various embodiments of the present invention, the analysis result screen 2630 may include an area 2631 in which the result of the analysis of facial information is displayed in the image, and an area 2632 in which the result of the analysis of the facial information is displayed in numbers or in a diagram. The user can use a button to select the analysis result of each analysis item, and can also include various other items related to face analysis. For example, the buttons that the user can use to select the analysis results of each analysis item may include a Summary button 2633a for displaying the analysis results of the entire facial area, and a Wrinkle button for displaying the analysis results of wrinkles. 2633b, Trouble button 2633c for displaying skin problems, or Pigmentation button 2633d for displaying analysis results of pigmentation, and other options for displaying analysis results of various other analysis items project.

FIG. 27 illustrates a screen showing facial analysis results according to various embodiments of the present invention.

27, the analysis result screen 2700 may include an image area 2710 in which the analyzed face area is displayed, an analysis result area 2720 in which an analysis result is displayed, or an analysis item selection area 2730 for selecting an analysis result of each analysis item.

In the image area 2710, the facial area can be detected from the image stored in the electronic device, and at least one reference point 2711 can be set in the detected facial area. Based on the at least one reference point 2711, the facial area to be analyzed (for example, 2712, 2713, 2714, 2715, or 2716) can be set according to at least one analysis item. The area 2712 may include the outer area of both eyes as the C area. The region 2713 may include the nose region, the region 2714 may include the cheek region, the region 2715 may include the region surrounding the nose, and the region 2716 Can include a region surrounding the mouth.

According to various embodiments of the present invention, when a user input (for example, a touch gesture) is performed, each of the facial regions can be selected, and for the selected facial region, the screen obtained by analyzing the facial region can be Enlarged and displayed.

In the analysis result area 2720, the result of analyzing the skin condition can be displayed digitally (for example, it can be displayed as being in a state of a score of 2722 or a diagram of 2721). For example, the graph 2721 may represent the score of the skin condition of each analysis item obtained by analyzing the specific facial area of each analysis item. The score 2722 may be a value obtained by averaging the scores of each analysis item for a specific facial area, and when the score 2722 is higher, the skin condition may be better.

The analysis item selection area 2730 may include at least one button for displaying the analysis result of each analysis item, and according to the selected button, the analysis result of a specific face area may be displayed in the image area 2710 or the analysis result area 2720. For example, if the summary button is selected, the entire face area can be displayed in the image area 2710, and the analysis result of the entire face area can be displayed in the analysis result area 2720.

FIG. 28 illustrates a screen showing facial analysis results according to various embodiments of the present invention.

28, the analysis result screen 2800 may include an image area 2810, an analysis result area 2820, or an analysis item selection area 2830 in which at least one of the analyzed facial areas can be selected.

According to various embodiments of the present invention, if the wrinkle button 2831 is selected in the analysis item selection area 2830, the wrinkle button 2831 can be selected in the image area 2810 or the analysis result area 2820 Shows the results of analyzing wrinkles in various analysis items.

In the image area 2810, based on the image stored in the electronic device, the result of analyzing wrinkles in at least one facial area (for example, the area of the C area or the U area) can be displayed in the image. For example, since each facial area is displayed in a different color according to the skin condition, the user can determine the skin condition based on the displayed skin color.

In the analysis result area 2820, the graph 2821 obtained by comparing the result of analyzing the wrinkles with the average score for each of the C area or the M area or the score 2822 of the wrinkles in the entire facial area can be displayed. .

According to various embodiments of the present invention, if the user uses input 2811 in a specific area (for example, the area in the left C area) to determine the analysis result, an enlarged screen 2812 can be displayed, and the analysis is performed on the enlarged screen 2812. User input area (for example, the area of the left C area).

According to various embodiments of the present invention, the analysis result area 2820 may provide a history of wrinkle changes in the entire facial area. For example, the analysis result area 2820 may include a graph 2823 obtained by comparing the wrinkle state measured one month ago with the current wrinkle state measured or that the wrinkle state of the C area or the M area has been improved compared to before How many schemas 2824.

FIG. 29 illustrates a screen showing facial analysis results according to various embodiments of the present invention.

Referring to FIG. 29, the user has selected the wrinkle button 2931 in the analysis item selection area. Therefore, the analysis result screen 2900 can include those obtained by analyzing wrinkles. Obtained image or analysis result area 2920.

In the analysis result area 2920, the wrinkle status of each of the specific areas (for example, the left area CL or the right area CR of the C area or the left area ML or the right area MR of the M area) can be scored and displayed or the wrinkle status in the entire facial area The state of wrinkles.

According to various embodiments of the present invention, the user can select an area of the analyzed facial area in which the user desires to determine the enlarged or detailed screen. For example, if the user input 2901 selects the area of the C area on the left, a screen 2910 can be displayed, and the left area of the C area on the upper face of the screen 2910 is zoomed. If the left area of the nose is selected by the user input 2902, a screen 2912 can be displayed on which the left area of the nose in the face is zoomed.

FIG. 30 illustrates a screen showing facial analysis results according to various embodiments of the present invention.

Referring to FIG. 30, if the user selects the question button 3031 in the analysis item selection area 3030 of the analysis result screen 3000, the image area 3010 or the analysis result area 3020 of the analysis result screen 3000 can be displayed in the analysis result screen 3000 for the problems in various analysis items. Results of the analysis.

In the image area 3010, based on the image stored in the electronic device, at least one facial area (for example, the right area R, the left area L, the top area T, or the bottom area B) analyzed for the problem can be displayed in the image. Results of the analysis. For example, since each facial area is displayed in a different color according to the skin condition, the user can determine the skin condition of the facial area based on the displayed skin color.

In the analysis result area 3020, it can be displayed by focusing on the right area R, left Each of the side area L, the top area T, or the bottom area B compares the result of analyzing the question with the average score to obtain a pattern 3021 or a score 3022 of the question in the entire face area.

According to various embodiments of the present invention, if the bottom of the face is selected by the user input 3011, an enlarged screen 3012 can be displayed, and the bottom area can be analyzed on the enlarged screen 3012. For example, based on the enlarged screen 3012, the user can determine the location and size of the problem in the bottom area.

According to various embodiments of the present invention, the analysis result area 3020 may provide a history of problem changes in the entire facial area. For example, the analysis result area 3020 may include a graph 3023 obtained by comparing the problem state measured one month ago with the current problem state measured or representing each area (for example, the right area R, the left area L. The problem state of the top area T or the bottom area B) has been improved to a much better degree than the previous diagram 3024.

FIG. 31 illustrates a screen showing facial analysis results according to various embodiments of the present invention.

Referring to FIG. 31, the user has selected the question button 3131 in the analysis item selection area. Therefore, the analysis result screen 3100 may include an image obtained by analyzing a problem or an analysis result area 3120.

According to various embodiments of the present invention, the user can select an area of the analyzed facial area in which the user desires to determine the enlarged or detailed screen. For example, if the left cheek area is selected by the user input 3101, a screen 3110 may be displayed, on which the left cheek of the face is enlarged. If the bottom is selected by the user input 3102, a screen 3112 can be displayed, and the bottom of the face on the screen 3112 is enlarged.

Figure 32 illustrates a screen showing facial analysis results according to various embodiments of the present invention.

32, if the user selects the pigmentation button 3231 in the analysis item selection area 3230 of the analysis result screen 3200, the image area 3210 or the analysis result area 3220 of the analysis result screen 3200 can display the pigments in various analysis items Calmly analyze the results.

In the image area 3210, based on the image stored in the electronic device, at least one facial area (for example, the right area R, the left area L, the top area T, or the bottom area B) analyzed for pigmentation can be displayed in the image. The results of the analysis. For example, since each facial area is displayed in a different color according to the skin condition, the user can determine the skin condition of the facial area based on the displayed skin color.

In the analysis result area 3220, a graph 3221 obtained by comparing the result of the analysis of the pigmentation with the average score for each of the right area R, the left area L, the top area T, or the bottom area B can be displayed Or a score of 3222 for pigmentation in the entire facial area.

According to various embodiments of the present invention, if the left cheek is selected by the user input 3201, an enlarged screen 3202 can be displayed, and the left cheek area can be analyzed on the enlarged screen 3202. For example, based on the enlarged screen 3202, the user can determine the position and size of the pigmentation point in the left cheek area.

According to various embodiments of the present invention, the analysis result area 3220 may provide a history of pigmentation changes in the entire facial area. For example, the analysis result area 3220 may include a method of comparing the pigmentation state measured one month ago with the current measured pigmentation state. The graph 3223 obtained by comparing the states or a graph 3224 representing how much the pigmentation state of each area (for example, the right area R, the left area L, the top area T, or the bottom area B) has been improved compared with before.

FIG. 33 illustrates a screen showing facial analysis results according to various embodiments of the present invention.

Referring to FIG. 33, the user has selected the pigmentation button 3331 in the analysis item selection area. Therefore, the analysis result screen 3300 may include an image or analysis result area 3320 obtained by analyzing pigmentation.

According to various embodiments of the present invention, the user can select an area of the analyzed facial area in which the user desires to determine the enlarged or detailed screen. For example, if the left cheek area is selected by the user input 3301, a screen 3311 may be displayed, on which the left cheek of the face is enlarged. If the nose area is selected by the user input 3302, the screen 3312 can be displayed, and the nose area is enlarged on the screen 3312.

FIG. 34 illustrates a screen showing facial analysis results according to various embodiments of the present invention.

Referring to Figure 34, if the user selects the Pore button 3431 in the analysis item selection area 3430 of the analysis result screen 3400, the analysis results screen 3400 can be displayed in the image area 3410 or analysis result area 3420 of the various analysis items. The results of the analysis of the pores.

In the image area 3410, based on the image stored in the electronic device, at least one facial area (for example, right area R, left area L, top area T, or bottom area B) analyzed for pores can be displayed in the image. Results of the analysis. For example In other words, since each facial area is displayed in a different color according to the skin condition, the user can determine the skin condition of the facial area based on the displayed skin color.

In the analysis result area 3420, a graph 3421 or a diagram obtained by comparing the results of the analysis of the pores with the average score for each of the right area R, the left area L, the top area T, or the bottom area B can be displayed The score of the pores in the entire facial area is 3422.

According to various embodiments of the present invention, if the right cheek is selected by the user input 3401, an enlarged screen 3402 can be displayed, and the right cheek area of the face can be analyzed on the enlarged screen 3402. For example, based on the enlarged screen 3402, the user can determine the position and size of the pores in the right cheek area.

According to various embodiments of the present invention, the analysis result area 3420 may provide a history of changes in pores of the entire facial area. For example, the analysis result area 3420 can include a graph 3423 obtained by comparing the pore condition measured one month ago with the current pore condition measured or a graph showing how much the pore condition of the entire facial area has improved compared to before. Scheme 3424.

FIG. 35 illustrates a screen showing facial analysis results according to various embodiments of the present invention.

35, if the user selects the Tone button 3531 in the analysis item selection area 3530 of the analysis result screen 3500, various analysis items can be displayed in the image area 3510 or the analysis result area 3520 of the analysis result screen 3500 The result of the analysis of the skin tone.

In the image area 3510, based on the image stored in the electronic device, The image shows the result of analyzing at least one facial area (for example, eyebrow area L, cheek area A, or jaw area B) analyzed for skin color. For example, since each facial area is displayed in a different color according to the skin condition, the user can determine the skin condition of the facial area based on the displayed skin color.

In the analysis result area 3520, it can be shown that the result obtained by comparing the uniformity of the skin color with the average score for each of the eyebrow area L, cheek area A, or jaw area B Schema 3521 or a score of 3522 for the uniformity of skin color in the entire facial area.

According to various embodiments of the present invention, if the user selects the score 3522, the analysis result area 3520 can provide the analysis result of the skin color change of the entire facial area. For example, the analysis result area 3520 may include the chromaticity state (or chromaticity uniformity state) measured one month ago for each facial area (for example, eyebrow area L, cheek area A, or jaw area B) ) The graph 3523 obtained by comparing with the currently measured chromaticity state or the graph 3524 indicating how much the chromaticity state of each facial area has been improved compared with before.

36 illustrates an application screen showing the history of facial information analyzed based on stored images according to various embodiments of the present invention.

Referring to FIG. 36, if an analysis result screen 3600 including a result of analyzing facial information using an application for analyzing facial information is displayed, a menu 3601 including functions related to the analyzed facial information may be displayed. For example, if the menu 3601 is selected, selection items can be displayed, through which the user can choose to analyze at least one selected image based on experiments (Analyze the whole image). Whole Image function, Manage Changes function 3602 for displaying the history of analyzed facial information, or Initialize DB function for initializing data in previously analyzed facial information. Various other functions related to the analysis of facial information can be included in the menu 3601.

According to various embodiments of the present invention, if the user selects the management change function 3602, the image area 3610 or the analysis result area 3620 of the analysis result screen 3600 can display the changes of the previously analyzed facial information. For example, the analysis result area 3620 may use the graph 3621 in the time period in which the facial information is analyzed to display the result of the facial analysis of each analysis item. In the image area 3610, an image obtained by analyzing the facial information at a specific time input by the user in the diagram 3621 can be displayed. The display part of the analysis item 3630 can display analysis items such as summary 3631, wrinkles 3632, problem 3633, and pigmentation 3634.

FIG. 37 illustrates a screen showing the history of analyzed facial information according to various embodiments of the present invention.

Referring to FIG. 37, the user can select the analysis item or the image area in which the user desires to determine the change of the analysis result on the analysis result screen 3700. For example, in order to determine the change of the analysis result of pigmentation, the user can select a summary button in each analysis item (for example, summary, wrinkle, problem, or pigmentation).

According to various embodiments of the present invention, the user can perform user input (for example, pinch in/out, or tap-and-drag) 3701 to zoom in/out displayed on the image The image in the area, or the image can be moved to set the image area where the user desires to determine the change of the analysis result.

According to various embodiments of the present invention, the analysis result screen 3700 can display the set image area, and use a graph to display the result of analyzing the displayed image area according to at least one analysis item (for example, pigmentation, problem, or wrinkle) Variety. For example, the diagram shows the time when the facial information is analyzed, and the analysis result corresponding to the time. If user input is performed at a specific time, an image analyzed for facial information at the time can be displayed.

FIG. 38 illustrates a screen showing the history of analyzed facial information according to various embodiments of the present invention.

Referring to FIG. 38, the analysis result screen 3800 may include an image area 3810 or an analysis result area 3820.

According to various embodiments of the present invention, in order to determine the change in the analysis result of wrinkles, the user may select wrinkle 3831 in each analysis item (for example, summary, wrinkle, problem, or pigmentation).

In the image area 3810, the result of analyzing wrinkles in the facial area of the image can be displayed. For example, the results of analyzing the wrinkles can be displayed in the form of lines, and the depth or length of the wrinkles can be expressed in terms of the thickness or length of the lines.

The analysis result area 3820 may use a graph to represent the previously analyzed time-based wrinkle state of each facial area (for example, the right area CR or the left area CL of the C area, or the right area MR or the left area ML of the M area) As a result of analyzing wrinkles.

According to various embodiments of the present invention, if the user performs user input (for example, tapping or swiping) at a specific time on the diagram, the image area 3810 An image for analyzing wrinkles at the time is displayed, and the result of analyzing wrinkles at the time may be displayed on the displayed image.

FIG. 39 illustrates a screen showing the history of analyzed facial information according to various embodiments of the present invention.

Referring to FIG. 39, the analysis result screen 3900 may include an image area 3910 or an analysis result area 3920.

According to various embodiments of the present invention, in order to determine the change of the analysis result of the problem, the user can select the problem 3931 among various analysis items (for example, summary, wrinkle, problem, or pigmentation).

In the image area 3910, the result of analyzing the problem in the face area of the image can be displayed. For example, the result of analyzing the problem can be displayed in a form corresponding to the problem, and the size or status of the problem can be expressed according to the size or color of the problem.

The analysis result area 3920 may use a diagram to represent the previously analyzed time-based problem status of each face area (for example, the right area R, the left area L, the top area T, or the bottom area B) as the result of the analysis of the problem .

According to various embodiments of the present invention, if the user performs user input (for example, tapping or sweeping) at a specific time on the diagram, an image for analyzing the problem at that time can be displayed in the image area 3910 , And the result of analyzing the problem at the time can be displayed on the displayed image.

FIG. 40 illustrates a screen showing the history of analyzed facial information according to various embodiments of the present invention.

Referring to FIG. 40, the analysis result screen 4000 may include an image area 4010 or an analysis result area 4020.

According to various embodiments of the present invention, in order to determine the change of the analysis result of the pigmentation, the user can select the pigmentation 4031 in each analysis item (for example, summary, wrinkle, problem, or pigmentation).

In the image area 4010, the result of analyzing the pigmentation in the facial area of the image can be displayed. For example, the result of the analysis of the pigmentation can be displayed in a form corresponding to the pigmentation, and the size or state of the pigmentation can be expressed according to the size or color of the pigmentation.

The analysis result area 4020 can use a graph to represent the previously analyzed time-based pigmentation state of each facial area (for example, the right area R, the left area L, the top area T, or the bottom area B) as the analysis of the pigmentation the result of.

According to various embodiments of the present invention, if the user performs user input (for example, tapping or swiping) at a specific time on the diagram, an image for analyzing pigmentation at the time may be displayed in the image area 4010 , And the results of the analysis of pigmentation at the time can be displayed on the displayed image.

FIG. 41 illustrates a screen showing the history of analyzed facial information according to various embodiments of the present invention.

Referring to FIG. 41, if the management change function (for example, 3602) is selected in the menu (for example, 3601) of the application for analyzing facial information, the analysis result screen 4100 can display the history of analysis results of various analysis items.

According to various embodiments of the present invention, the analysis result screen 4100 may include The image area 4110 or the analysis result area 4120. The image area 4110 includes images obtained by analyzing the facial information at a specific time for the selected analysis item. The analysis result area 4120 includes information indicating that each analysis item (for example, pigmentation , Problems, or wrinkles) as the result of the previous analysis of the skin condition score during analysis.

According to various embodiments of the present invention, if the user selects the summary 4131 in each analysis item (for example, summary, wrinkles, problems, or pigmentation), the image area 4110 can be displayed in the image area 4110 for analyzing the facial information. The image 4111 analyzed at the first time 4121. The result of analyzing the facial information of the image 4111 can be displayed in the analysis result area 4120 by using the schema.

According to various embodiments of the present invention, if the user input 4101 is performed at a position corresponding to the specific analysis time of the diagram, the image analyzed at the time when the user input 4101 is performed can be displayed in the image area 4110. For example, in the image area 4110, if the user input 4101 is performed at the position corresponding to the second time 4122, the image 4112 that analyzes the facial information can be displayed at the second time 4122, and if it corresponds to the third time The user input 4101 at the position of time 4123, the image 4113 for analyzing the facial information can be displayed at the third time 4123.

According to various embodiments of the present invention, the displayed image 4111, 4112, or 4113 is used to analyze facial information at the time of the user input 4101 (for example, the first time 4121, the second time 4122, or the third time 4123). Since the result of analyzing each analysis item at the time is displayed to overlap the displayed image, the user can determine the skin condition of the analysis item.

Figure 42 illustrates showing analyzed faces according to various embodiments of the present invention Information history screen.

Referring to FIG. 42, if the management change function (for example, 3602) is selected in the menu (for example, 3601) of the application for analyzing facial information, the analysis item (for example, wrinkle 4231) can be displayed on the analysis result screen 4200 Analyze the history of the results. For example, the analysis result screen 4200 may include an image area 4210 or an analysis result area 4220.

According to various embodiments of the present invention, the image displayed in the image area 4210 is the image 4211 whose facial information is analyzed at the first time 4221, and the image 4211 can be resized in response to user input (eg, pinch/open) (For example, zoom in). For example, based on the resized image, the user can determine the skin condition in the image analyzed at the specific time 4223.

According to various embodiments of the present invention, in the image area 4210, the settings of the image whose size or position changes in response to user input can be maintained at other times. For example, if the image is enlarged when the user input is performed on the image 4211, the zoom setting can also be applied to the image 4212 whose facial information is analyzed at the second time 4222. The image 4212 analyzed on the facial information at the second time 4222 may be displayed to correspond to the area in which the image was previously enlarged. According to various embodiments of the present invention, if the user input 4201 is performed at the position corresponding to the specific analysis time of the schema, the image analyzed at the time when the user input 4201 is performed can be displayed in the image area 4110. For example, the image 4213 analyzed on the facial information at the third time 4223 may be displayed to correspond to the area where the image 4212 was previously enlarged.

FIG. 43 illustrates showing analyzed faces according to various embodiments of the present invention Information history screen.

Referring to FIG. 43, if one of various analysis items is selected, the analysis result screen 4300 can display the analysis result of the selected analysis item in the image area 4310 or the analysis result area 4320.

According to various embodiments of the present invention, if the user selects the question 4331 in each analysis item (for example, summary, wrinkle, problem, or pigmentation), the image area 4310 can display the analysis of facial information for the problem Image 4311. The diagram 4321 can be used to display the results of the analysis of the problem of the image 4311 in the analysis result area 4320.

According to various embodiments of the present invention, the displayed image 4311 is an image used to analyze facial information at the time of user input. Since the shape corresponding to the analyzed problem is displayed to overlap the displayed image, the user can determine the location or situation of the problem.

According to various embodiments of the present invention, by performing user input (for example, tapping and dragging) 4301, the user can move the desired area in the image to the image area 4310. For example, the diagram 4322 can be used to display the analysis result of the problem in the image area 4312 in the analysis result area 4320.

According to various embodiments of the present invention, for the image displayed in the image area 4310, the size or position setting displayed in response to the user input can be maintained, and even if a specific time is selected, the analysis at that time The image can still be displayed according to the settings whose size or position is changed.

Figure 44 illustrates showing analyzed faces according to various embodiments of the present invention Information history screen.

Referring to FIG. 44, if one of various analysis items is selected (for example, question 4431), the analysis result screen 4400 may display the history of facial information of the selected analysis item in the image area 4410 or the analysis result area 4420.

According to various embodiments of the present invention, the image area 4410 can be moved or zoomed in a specific image, and even when an image analyzed at another time is displayed, the position in which the image area 4410 is moved or enlarged can be maintained. For example, the user may perform a user input (for example, a touch gesture) in the image area 4410 to move the image area 4410 to an area desired by the user or to enlarge the area where the image area 4410 is moved to.

According to various embodiments of the present invention, the user can select the question 4431 in each analysis item, and locate the position of the image area 4410 or the enlarged image area 4410 in the left cheek area. In the image area 4410, the image analyzed at the earliest time among the images analyzed for the problem 4431 can be displayed.

According to various embodiments of the present invention, when the user performs the user input 4401 at the first time 4421, the first image 4411 analyzed at the first time 4421 may be displayed in the image area 4410. For example, in response to the action of the user having moved or enlarged the image area 4410, the corresponding area in the position where the image area 4410 was moved or enlarged may be displayed in the first image 4411.

According to various embodiments of the present invention, if the user input 4401 is performed at the second time 4422, the second image 4412 analyzed at the second time 4422 can be displayed in the image area 4410. For example, when the user has moved or enlarged the image The action of the area 4410 can display the corresponding area in the position where the image area 4410 is moved or enlarged in the second image 4412.

According to various embodiments of the present invention, if the user input 4401 is performed at the third time 4423, the third image 4413 analyzed at the third time 4423 can be displayed in the image area 4410. For example, in response to the action of the user having moved or enlarged the image area 4410, the corresponding area in the position where the image area 4410 was moved or enlarged may be displayed in the third image 4413.

FIG. 45 illustrates a screen showing the history of analyzed facial information according to various embodiments of the present invention.

Referring to FIG. 45, if one of various analysis items is selected (for example, pigmentation 4531), the analysis result screen 4500 can display the history of facial information of the selected analysis item in the image area 4510 or the analysis result area 4520.

According to various embodiments of the present invention, the image area 4510 can be moved or zoomed in a specific image, and even when the image analyzed at another time is displayed, the position in which the image area 4510 is moved or enlarged can be maintained. For example, the user can perform user input (for example, touch gestures) in the image area 4510 to move the image area 4510 to the area the user desires to determine or to enlarge the area where the image area 4510 is moved to.

According to various embodiments of the present invention, the user can select the pigmentation 4531 in each analysis item, and locate the position of the image area 4510 or the position of the enlarged image area 4510 in the left cheek area. In the image area 4510, the image analyzed at the earliest time among the images analyzed for the pigmentation 4531 can be displayed.

According to various embodiments of the present invention, when the user performs a user input 4501 at the first time 4521, the first image 4511 analyzed in the first image 4521 may be displayed in the image area 4510. For example, in response to the user's movement or enlargement of the image area 4510, the corresponding area in the position where the image area 4510 is moved or enlarged may be displayed in the first image 4511.

According to various embodiments of the present invention, if the user input 4501 is performed at the second time 4522, the second image 4512 analyzed at the second time 4522 can be displayed in the image area 4510. For example, in response to the user's movement or enlargement of the image area 4510, the corresponding area in the position where the image area 4510 is moved or enlarged may be displayed in the second image 4512.

According to various embodiments of the present invention, if the user input 4501 is performed at the third time 4523, the third image 4513 analyzed at the third time 4523 can be displayed in the image area 4510. For example, in response to an action that the user has moved or enlarged the image area 4510, the corresponding area in the position where the image area 4510 is moved or enlarged may be displayed in the third image 4513.

FIG. 46 illustrates a photographing operation for face analysis according to various embodiments of the present invention.

Referring to FIG. 46, the electronic device can obtain the user's facial image by energizing the camera module installed on the front of the electronic device. For example, the electronic device can periodically obtain images in various situations such as a situation in which the user's face is photographed at a specific angle, or a situation in which the electronic device is tilted at a specific angle.

According to various embodiments of the present invention, when a user uses an electronic device to watch On the web browser screen, the electronic device can make the camera module run in the background, so that the user's facial image 4600 can be captured independently of the web browser screen.

According to various embodiments of the present invention, the electronic device can determine at least one reference point 4601 in the captured facial image 4600, and use the reference point to identify the facial area to analyze facial information.

47A and 47B illustrate screens showing the results of skin analysis according to various embodiments of the present invention.

According to various embodiments of the present invention, the analysis result screen 4700 can be configured in various layouts according to the size or rotation direction of the display.

Referring to FIGS. 47A and 47B, if the display is positioned horizontally or the electronic device is a tablet PC, the image area 4710 and the analysis item area 4720 can be arranged horizontally on the analysis result screen 4700.

According to various embodiments of the present invention, if the user selects the pigmentation 4722 in the analysis item area 4720, the result of analyzing the pigmentation in at least one facial area may be displayed in the image area 4710 or the analysis result area 4730.

In the image area 4710, when the user input 4701 is performed in a specific area (for example, the cheek area), the analysis result of the facial area in which the user input 4701 is performed can be enlarged and displayed in a separate window 4712.

According to various embodiments of the present invention, when a specific analysis item is selected in the analysis item area 4720, the analysis result of the facial information may be displayed in the analysis result area 4730. For example, the analysis result area 4730 may include an emoticon 4731 representing the analysis result of the selected pigmentation, a graph showing the score of each item. Formula, or pigmentation, is 4732 better than before.

According to various embodiments of the present invention, if the user selects Total 4721 in the analysis item area 4720, the analysis of various analysis items in at least one facial area can be displayed in the image area 4710 or the analysis result area 4730 result.

In the image area 4710, at least one face area 4713 analyzed according to various analysis items may be displayed, and may include a legend 4714 representing the skin condition corresponding to the color of each face area. For example, the user can determine the skin condition of a specific part by determining the color of each facial area.

According to various embodiments of the present invention, the analysis result area 4730 may include a graph 4733 representing a score obtained by comparing analysis results of various analysis items (for example, pores, wrinkles, pigmentation, or problems) with previous analysis results. .

It is obvious from the foregoing description that various embodiments of the present invention can provide an electronic device and a method for analyzing facial information on the electronic device, in which a part of the facial information that a user can obtain is enlarged or finely photographed by the electronic device. The image does not require any user action, and the analysis result of the obtained image can be determined.

Although the present invention has been illustrated and described with reference to various embodiments of the present invention, those skilled in the art will understand that without departing from the spirit and scope of the present invention as defined by the scope of the appended application and its equivalent scope. Next, various forms and details can be changed.

100: electronic device

110: Controller

111: Project Setting Unit

112: Photography Controller

113: Image Analyzer

114: District selector

115: Analysis result generator

120: camera module

130: input unit

140: display

150: storage

151: Skin Analysis Information

152: User identification information

Claims (10)

A method for analyzing facial information in an electronic device, the method comprising: detecting at least one facial region from an image containing a face by setting at least one reference point, wherein the image is an image taken by a camera module or an image stored Set at least one analysis item according to the detected at least one face area; analyze the at least one face area detected according to the at least one analysis item; and obtain the detected at a specific time Historical information of the skin analysis result of at least one facial area, wherein the at least one facial area includes at least one of an area surrounding the eyes, a nose area, a cheek area, an area surrounding the mouth, and an eyebrow area, wherein the The at least one analysis item includes at least one of wrinkles, acne, pigmentation, skin tone, dark circles, and pores, wherein the analysis includes for each of the at least one analysis item in the at least one enlarged facial region One is to analyze at least one of thickness, size, number, width, and color. The method for analyzing facial information on an electronic device as described in item 1 of the scope of the patent application further includes: arranging the previously analyzed images according to the analysis time of the previously analyzed images, and displaying them on the basis of user input. The image analyzed at the selected time; detecting the user input used to zoom in, zoom out, or move the displayed image And based on the at least one analysis item, outputting an analysis result of an image that changes in response to the detected user input. The method for analyzing facial information on an electronic device as described in claim 1 of the patent application further includes: selecting at least one of the detected at least one face area, wherein the at least one detected face is selected At least one of the areas includes at least one of: zooming in the at least one face area detected, setting the focus of the camera module on the at least one face area detected, and detecting Enlarge the detected at least one facial area when the action is reached, and wherein the action includes any one of the following: an action of running an application in which the camera module is enabled is in a locked state The electronic device performs an energizing action and an action of positioning the electronic device at a specific angle or a specific direction. As described in item 1 of the scope of patent application, the method for analyzing facial information on an electronic device further includes: determining a pre-stored analysis result of a specific facial area or analysis item; combining the determined pre-stored analysis result with the analysis result The results are compared; and the result of the comparison is displayed. The method of analyzing facial information on electronic devices as described in item 1 of the scope of patent application further includes: Determine whether the electronic device is in a situation for shooting an image; and shoot an image according to a result of the determination, wherein the situation includes a situation where the electronic device is located at a certain angle or a situation where a user input is performed. An electronic device comprising: a camera module; a display for displaying an image being shot by the camera module; and a controller for self-containing the image detection of a face by setting at least one reference point At least one face area, wherein the image is an image taken by the camera module or a stored image; at least one analysis item is set according to the detected at least one face area; and analysis is performed according to the at least one analysis item The at least one facial area; and obtaining historical information of the skin analysis result of the at least one facial area detected in a specific period, wherein the at least one facial area includes an area surrounding the eyes, a nose area, a cheek area, At least one of the area surrounding the mouth and the eyebrow area, wherein the at least one analysis item includes at least one of wrinkles, acne, pigmentation, skin tone, dark circles, and pores, wherein the controller is more For the at least one face zoomed in Each of the at least one analysis item in the area analyzes at least one of thickness, size, number, width, and color. The electronic device described in the scope of patent application, wherein the controller is further used to: arrange the previously analyzed images according to the analysis time of the previously analyzed images, and display the previously analyzed images according to the user input The image analyzed at the selected time; detecting user input for zooming in, reducing, or moving the displayed image; and based on the at least one analysis item, outputting a pair corresponding to the detected user input The result of the analysis of the changed image. The electronic device according to claim 6, wherein the controller is further used to select at least one of the detected at least one facial region by executing at least one of the following: controlling the The camera module enlarges the detected at least one face area, sets the focus of the camera module on the detected at least one face area, and controls the camera module to enlarge when motion is detected The detected at least one facial area, and wherein, the action includes any one of the following: an action of running an application program in which the camera module is enabled, to the electronic device in a locked state The device performs an energizing action and an action of positioning the electronic device at a specific angle or a specific direction. The electronic device described in item 6 of the scope of patent application, wherein the controller is further used for: Determine the pre-stored analysis result of a specific facial area or analysis item, compare the determined pre-stored analysis result with the result of the analysis, and control the display to display a screen including the result of the comparison. According to the electronic device described in item 6 of the scope of patent application, the controller is further used to determine whether the electronic device is in a condition for shooting images, and to control the camera module to perform the operation according to the result of the determination. An image is taken, and the situation includes: the electronic device is located at a certain angle, or a user input is performed.

Images