KR20170014919A - Electronic apparatus and method for detecting skin condition in electronic apparatus - Google Patents

Abstract

According to various embodiments of the present invention, an electronic apparatus comprises: a camera; and a processor for detecting at least one of a pigmentation area and a pore area by using a skin image photographed by the camera, and determining a skin condition corresponding to the skin image by using the detection result. Besides, various other embodiments are possible.

Description

TECHNICAL FIELD [0001] The present invention relates to a skin condition detection method for an electronic device and an electronic device,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an electronic device, and more particularly, to a method of detecting a skin condition in an electronic device.

Recently, as the interest in the skin condition has increased, more and more users are desiring to be diagnosed with the skin condition. Generally, the device for diagnosing the skin condition is designed as a complicated device and has a large volume, so that it is often installed in a professional dermatologist rather than a general home, so users have to visit a specialized dermatologist to diagnose the skin condition.

In recent years, a small-sized skin condition diagnostic device having a simple structure has been developed and widely available to the general public. However, it is inconvenient for ordinary people to carry it as a portable device. The diagnostic device for skin condition for dissemination is simple There is a problem that it is impossible to detect various skin conditions since it is possible to measure only the skin condition.

Therefore, it is an object of the present invention to provide a skin condition detection method in an electronic device and an electronic device which are convenient to carry and can detect various skin conditions.

Thus, according to various embodiments of the present invention, it is possible to provide a method of detecting a skin condition in an electronic device and an electronic device capable of detecting a skin condition through a portable electronic device such as a smart phone.

According to various embodiments of the present invention, an electronic device capable of detecting various skin conditions such as skin pigmentation, pores, roughness, and the like using a skin photographing image using a camera of a portable electronic device, Can be provided.

At least one of the above-mentioned objects of the present invention can be achieved by the following constituent elements.

According to various embodiments of the present invention, in an electronic device, at least one of a camera, a pigment deposition area and a pore area is detected using a skin image taken by the camera, and the detection result is used to correspond to the skin image And a processor for determining a skin condition to be performed.

According to various embodiments of the present invention, there is provided a method of detecting a skin condition in an electronic device, the method comprising: obtaining a skin image; detecting at least one of a pigmentation region and a pore region using the skin image; And determining the skin condition corresponding to the skin image using the skin image.

According to various embodiments of the present invention, there is provided a storage medium storing a program, the program causing the computer to perform the steps of: obtaining a skin image; detecting at least one of a pigmentation area and a pore area using the skin image; And an operation of determining the skin condition corresponding to the skin image using the detection result.

According to various embodiments of the present invention, various skin conditions can be detected through a portable electronic device, so that the user can diagnose the skin condition anytime and anywhere.

According to various embodiments of the present invention, when a user takes a skin photograph through a camera of a portable electronic device such as a smart phone, various skin condition information such as skin pigmentation, pore, roughness, It is convenient because it can receive.

1 illustrates a network environment including an electronic device according to various embodiments of the present invention.
2 is a block diagram of an electronic device according to various embodiments.
3 is a block diagram of a program module according to various embodiments.
4 is a view showing an example of skin imaging in an electronic device according to various embodiments.
5 is a block diagram of an electronic device for skin condition detection according to various embodiments.
6 is a view for explaining the concept of skin condition detection in an electronic device according to various embodiments.
7 is a flowchart illustrating an operation of providing skin condition information in an electronic device according to various embodiments.
8 is a flow diagram illustrating a pigmentation detection operation in accordance with various embodiments.
9 is a view showing an example of detection of pigment deposition by image domain according to various embodiments.
FIG. 10 is a diagram illustrating a histogram for detecting a boundary between a pigmented area and a normal skin area according to various embodiments.
11 is a view for explaining degree of pigmentation according to various embodiments.
12 is a view for explaining weights according to degree of pigment deposition according to various embodiments.
13 is a flowchart illustrating pore detection operations according to various embodiments.
14 and 15 are views for explaining the concept of pore detection according to various embodiments.
16 is a diagram illustrating a histogram for detecting a boundary between an initial pore region and a normal skin region according to various embodiments.
17 is a view for explaining the depth of a pore according to various embodiments.
18 is a view for explaining weights according to the degree of pore depth according to various embodiments.
19 is a flowchart illustrating skin roughness estimation operations according to various embodiments.
20 is a view for explaining the concept of skin roughness estimation according to various embodiments.
Figure 21 is a histogram for estimating skin roughness according to various embodiments.
FIGS. 22 to 24 are views showing a skin condition detection screen according to various embodiments.

Hereinafter, various embodiments of the present document will be described with reference to the accompanying drawings. It should be understood, however, that this invention is not intended to be limited to the particular embodiments described herein but includes various modifications, equivalents, and / or alternatives of the embodiments of this document . In connection with the description of the drawings, like reference numerals may be used for similar components.

In this document, the expressions "having," " having, "" comprising," or &Quot;, and does not exclude the presence of additional features.

In this document, the expressions "A or B," "at least one of A or / and B," or "one or more of A and / or B," etc. may include all possible combinations of the listed items . For example, "A or B," "at least one of A and B," or "at least one of A or B" includes (1) at least one A, (2) Or (3) at least one A and at least one B all together.

As used herein, the terms "first," "second," "first," or "second," and the like may denote various components, regardless of their order and / or importance, But is used to distinguish it from other components and does not limit the components. For example, the first user equipment and the second user equipment may represent different user equipment, regardless of order or importance. For example, without departing from the scope of the rights described in this document, the first component can be named as the second component, and similarly the second component can also be named as the first component.

(Or functionally or communicatively) coupled with / to "another component (eg, a second component), or a component (eg, a second component) Quot; connected to ", it is to be understood that any such element may be directly connected to the other element or may be connected through another element (e.g., a third element). On the other hand, when it is mentioned that a component (e.g., a first component) is "directly connected" or "directly connected" to another component (e.g., a second component) It can be understood that there is no other component (e.g., a third component) between other components.

As used herein, the phrase " configured to " (or set) to be "configured according to circumstances may include, for example, having the capacity to, To be designed to, "" adapted to, "" made to, "or" capable of ". The term " configured to (or set up) "may not necessarily mean" specifically designed to "in hardware. Instead, in some situations, the expression "configured to" may mean that the device can "do " with other devices or components. For example, a processor configured (or configured) to perform the phrases "A, B, and C" may be implemented by executing one or more software programs stored in a memory device or a dedicated processor (e.g., an embedded processor) , And a generic-purpose processor (e.g., a CPU or an application processor) capable of performing the corresponding operations.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of the other embodiments. The singular expressions may include plural expressions unless the context clearly dictates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by one of ordinary skill in the art. The general predefined terms used in this document may be interpreted in the same or similar sense as the contextual meanings of the related art and, unless expressly defined in this document, include ideally or excessively formal meanings . In some cases, even the terms defined in this document can not be construed as excluding the embodiments of this document.

An electronic device according to various embodiments of the present document may be, for example, a smartphone, a tablet personal computer, a mobile phone, a video phone, an e-book reader, A desktop personal computer, a laptop personal computer, a netbook computer, a workstation, a server, a personal digital assistant (PDA), a portable multimedia player (PMP) A medical device, a camera, or a wearable device. According to various embodiments, the wearable device may be of the accessory type (e.g., a watch, a ring, a bracelet, a bracelet, a necklace, a pair of glasses, a contact lens or a head-mounted-device (HMD) (E. G., Electronic apparel), a body attachment type (e. G., A skin pad or tattoo), or a bioimplantable type (e.g., implantable circuit).

In some embodiments, the electronic device may be a home appliance. Home appliances include, for example, televisions, digital video disc (DVD) players, audio, refrigerators, air conditioners, vacuum cleaners, ovens, microwaves, washing machines, air cleaners, set- Such as a home automation control panel, a security control panel, a TV box such as Samsung HomeSync ^TM , Apple TV ^TM or Google TV ^TM , a game console such as Xbox ^TM and PlayStation ^TM , , An electronic key, a camcorder, or an electronic frame.

In an alternative embodiment, the electronic device may be any of a variety of medical devices (e.g., various portable medical measurement devices such as a blood glucose meter, a heart rate meter, a blood pressure meter, or a body temperature meter), magnetic resonance angiography (MRA) Navigation systems, global navigation satellite systems (GNSS), event data recorders (EDRs), flight data recorders (FDRs), infotainment (infotainment) systems, ) Automotive electronic equipment (eg marine navigation systems, gyro compass, etc.), avionics, security devices, head units for vehicles, industrial or home robots, automatic teller's machines (ATMs) Point of sale, or internet of things (eg, light bulbs, various sensors, electrical or gas meters, sprinkler devices, fire alarms, thermostats, street lights, Of the emitter (toaster), exercise equipment, hot water tank, a heater, boiler, etc.) may include at least one.

According to some embodiments, the electronic device is a piece of furniture or a part of a building / structure, an electronic board, an electronic signature receiving device, a projector, Water, electricity, gas, or radio wave measuring instruments, etc.). In various embodiments, the electronic device may be a combination of one or more of the various devices described above. An electronic device according to some embodiments may be a flexible electronic device. Further, the electronic device according to the embodiment of the present document is not limited to the above-described devices, and may include a new electronic device according to technological advancement.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An electronic apparatus according to various embodiments will now be described with reference to the accompanying drawings. In this document, the term user may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

Referring to Figure 1, in various embodiments, an electronic device 101 in a network environment 100 is described. The electronic device 101 may include a bus 110, a processor 120, a memory 130, an input / output interface 150, a display 160, and a communication interface 170. In some embodiments, the electronic device 101 may omit at least one of the components or additionally include other components.

The bus 110 may include circuitry, for example, to connect the components 110-170 to one another and to communicate communications (e.g., control messages and / or data) between the components.

The processor 120 may include one or more of a central processing unit (CPU), an application processor (AP), or a communication processor (CP). The processor 120 may perform computations or data processing related to, for example, control and / or communication of at least one other component of the electronic device 101.

Memory 130 may include volatile and / or non-volatile memory. Memory 130 may store instructions or data related to at least one other component of electronic device 101, for example. According to one embodiment, the memory 130 may store software and / or programs 140. The program 140 may include one or more of the following: a kernel 141, a middleware 143, an application programming interface (API) 145, and / or an application program . ≪ / RTI > At least a portion of the kernel 141, middleware 143, or API 145 may be referred to as an operating system (OS).

The kernel 141 may include system resources used to execute an operation or function implemented in other programs (e.g., middleware 143, API 145, or application program 147) (E.g., bus 110, processor 120, or memory 130). The kernel 141 also provides an interface to control or manage system resources by accessing individual components of the electronic device 101 in the middleware 143, API 145, or application program 147 .

The middleware 143 can perform an intermediary role such that the API 145 or the application program 147 can communicate with the kernel 141 to exchange data.

In addition, the middleware 143 may process one or more task requests received from the application program 147 according to the priority order. For example, middleware 143 may use system resources (e.g., bus 110, processor 120, or memory 130, etc.) of electronic device 101 in at least one of application programs 147 Priority can be given. For example, the middleware 143 may perform the scheduling or load balancing of the one or more task requests by processing the one or more task requests according to the priority assigned to the at least one task.

The API 145 is an interface for the application 147 to control the functions provided by the kernel 141 or the middleware 143, Control or the like, for example, instructions.

The input / output interface 150 may serve as an interface by which commands or data input from, for example, a user or other external device can be transferred to another component (s) of the electronic device 101. Output interface 150 may output commands or data received from other component (s) of the electronic device 101 to a user or other external device.

Display 160 may include, for example, a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting diode (OLED) A microelectromechanical systems (MEMS) display, or an electronic paper display. Display 160 may display various content (e.g., text, image, video, icon, or symbol, etc.) to a user, for example. Display 160 may include a touch screen and may receive a touch, gesture, proximity, or hovering input using, for example, an electronic pen or a portion of the user's body.

The communication interface 170 establishes communication between the electronic device 101 and an external device (e.g., the first external electronic device 102, the second external electronic device 104, or the server 106) . For example, communication interface 170 may be connected to network 162 via wireless or wired communication to communicate with an external device (e.g., second external electronic device 104 or server 106).

Wireless communications may include, for example, cellular communication protocols such as long-term evolution (LTE), LTE Advance (LTE), code division multiple access (CDMA), wideband CDMA (WCDMA) mobile telecommunications system, WiBro (Wireless Broadband), or Global System for Mobile Communications (GSM). The wireless communication may also include, for example, local communication 164. The local area communication 164 may include at least one of, for example, wireless fidelity (WiFi), Bluetooth, near field communication (NFC), or global navigation satellite system (GNSS). GNSS can be classified into two types according to the use area or bandwidth, for example, Global Positioning System (GPS), Global Navigation Satellite System (Glonass), Beidou Navigation Satellite System (Beidou) And may include at least one. Hereinafter, in this document, "GPS" can be interchangeably used with "GNSS ". The wired communication may include at least one of, for example, a universal serial bus (USB), a high definition multimedia interface (HDMI), a recommended standard 232 (RS-232), or plain old telephone service (POTS). The network 162 may include at least one of a telecommunications network, e.g., a computer network (e.g., a LAN or WAN), the Internet, or a telephone network.

Each of the first and second external electronic devices 102, 104 may be the same or a different kind of device as the electronic device 101. According to one embodiment, the server 106 may comprise a group of one or more servers. According to various embodiments, all or a portion of the operations performed in the electronic device 101 may be performed in one or more other electronic devices (e.g., electronic devices 102, 104, or server 106). According to the present invention, when electronic device 101 is to perform a function or service automatically or on demand, electronic device 101 may perform at least some functions associated therewith instead of, or in addition to, (E.g., electronic device 102, 104, or server 106) may request the other device (e.g., electronic device 102, 104, or server 106) Perform additional functions, and forward the results to the electronic device 101. The electronic device 101 may process the received results as is or additionally to provide the requested functionality or services. For example, Cloud computing, distributed computing, or client-server computing techniques can be used.

2 is a block diagram of an electronic device 201 according to various embodiments. The electronic device 201 may include all or part of the electronic device 101 shown in Fig. 1, for example. The electronic device 201 may include one or more processors (e.g., an application processor (AP)) 210, a communication module 220, a subscriber identification module 224, a memory 230, a sensor module 240, an input device 250 A display 260, an interface 270, an audio module 280, a camera module 291, a power management module 295, a battery 296, an indicator 297, and a motor 298 have.

The processor 210 may control a plurality of hardware or software components connected to the processor 210, for example, by driving an operating system or an application program, and may perform various data processing and calculations. The processor 210 may be implemented with, for example, a system on chip (SoC). According to one embodiment, the processor 210 may further include a graphics processing unit (GPU) and / or an image signal processor. Processor 210 may include at least some of the components shown in FIG. 2 (e.g., cellular module 221). Processor 210 may load or process instructions or data received from at least one of the other components (e.g., non-volatile memory) into volatile memory and store the various data in non-volatile memory have.

The communication module 220 may have the same or similar configuration as the communication interface 170 of FIG. The communication module 220 includes a cellular module 221, a WiFi module 223, a Bluetooth module 225, a GNSS module 227 (e.g., a GPS module, a Glonass module, a Beidou module, or a Galileo module) An NFC module 228, and a radio frequency (RF) module 229.

The cellular module 221 can provide voice calls, video calls, text services, or Internet services, for example, over a communication network. According to one embodiment, the cellular module 221 may utilize a subscriber identity module (e.g., a SIM card) 224 to perform the identification and authentication of the electronic device 201 within the communication network. According to one embodiment, the cellular module 221 may perform at least some of the functions that the processor 210 may provide. According to one embodiment, the cellular module 221 may include a communication processor (CP).

Each of the WiFi module 223, the Bluetooth module 225, the GNSS module 227, or the NFC module 228 may include a processor for processing data transmitted and received through the corresponding module, for example. At least some (e.g., two or more) of the cellular module 221, the WiFi module 223, the Bluetooth module 225, the GNSS module 227, or the NFC module 228, according to some embodiments, (IC) or an IC package. According to various embodiments, each of the WiFi module 223, the Bluetooth module 225, the GNSS module 227, or the NFC module 228 may communicate with the electronic device 101 and other devices (e.g., electronic devices 102, 104, or the server 106).

The RF module 229 can, for example, send and receive communication signals (e.g., RF signals). The RF module 229 may include, for example, a transceiver, a power amplifier module (PAM), a frequency filter, a low noise amplifier (LNA), or an antenna. According to another embodiment, at least one of the cellular module 221, the WiFi module 223, the Bluetooth module 225, the GNSS module 227, or the NFC module 228 transmits / receives an RF signal through a separate RF module .

The subscriber identity module 224 may include, for example, a card containing a subscriber identity module and / or an embedded SIM and may include unique identification information (e.g., an integrated circuit card identifier (ICCID) Subscriber information (e.g., international mobile subscriber identity (IMSI)).

Memory 230 (e.g., memory 130) may include, for example, internal memory 232 or external memory 234. The built-in memory 232 may be implemented as, for example, a volatile memory (e.g., dynamic RAM, SRAM, or synchronous dynamic RAM), a non-volatile memory Programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash) A hard drive, or a solid state drive (SSD).

The external memory 234 may be a flash drive such as a compact flash (CF), a secure digital (SD), a micro secure digital (SD), a mini secure digital (SD) digital, a multi-media card (MMC), a memory stick, and the like. The external memory 234 may be functionally and / or physically connected to the electronic device 201 via various interfaces.

The sensor module 240 may, for example, measure a physical quantity or sense the operating state of the electronic device 201 to convert the measured or sensed information into an electrical signal. The sensor module 240 includes a gesture sensor 240A, a gyro sensor 240B, an air pressure sensor 240C, a magnetic sensor 240D, an acceleration sensor 240E, a grip sensor 240F, 240G, a color sensor 240H (e.g., an RGB (Red, Green, Blue) sensor), a living body sensor 240I, a temperature / humidity sensor 240J, And a sensor 240M. Additionally or alternatively, the sensor module 240 may include, for example, an E-nose sensor, an electromyography sensor, an electroencephalogram sensor, an electrocardiogram sensor, , An infrared (IR) sensor, an iris sensor, and / or a fingerprint sensor. The sensor module 240 may further include a control circuit for controlling at least one or more sensors belonging to the sensor module 240. In some embodiments, the electronic device 201 further includes a processor configured to control the sensor module 240, either as part of the processor 210 or separately, so that while the processor 210 is in a sleep state, The sensor module 240 can be controlled.

The input device 250 may include a touch panel 252, a (digital) pen sensor 254, a key 256, or an ultrasonic input device 258). As the touch panel 252, for example, at least one of an electrostatic type, a pressure sensitive type, an infrared type, and an ultrasonic type can be used. Further, the touch panel 252 may further include a control circuit. The touch panel 252 may further include a tactile layer to provide a tactile response to the user.

(Digital) pen sensor 254 may be part of, for example, a touch panel or may include a separate recognition sheet. Key 256 may include, for example, a physical button, an optical key, or a keypad. The ultrasonic input device 258 can sense the ultrasonic wave generated by the input tool through the microphone (e.g., the microphone 288) and confirm the data corresponding to the ultrasonic wave detected.

Display 260 (e.g., display 160) may include a panel 262, a hologram device 264, or a projector 266. Panel 262 may include the same or similar configuration as display 160 of FIG. The panel 262 may be embodied, for example, flexible, transparent, or wearable. The panel 262 may be composed of one module with the touch panel 252. [ The hologram device 264 can display a stereoscopic image in the air using interference of light. The projector 266 can display an image by projecting light onto a screen. The screen may be located, for example, inside or outside the electronic device 201. According to one embodiment, the display 260 may further comprise control circuitry for controlling the panel 262, the hologram device 264, or the projector 266.

The interface 270 may be implemented using a variety of interfaces including, for example, a high-definition multimedia interface (HDMI) 272, a universal serial bus (USB) 274, an optical interface 276, or a D- ) ≪ / RTI > The interface 270 may, for example, be included in the communication interface 170 shown in FIG. Additionally or alternatively, the interface 270 may be, for example, a mobile high-definition link (MHL) interface, a secure digital (SD) card / multi-media card (MMC) data association standard interface.

The audio module 280 can, for example, convert sound and electrical signals in both directions. At least some of the components of the audio module 280 may be included, for example, in the input / output interface 145 shown in FIG. The audio module 280 may process sound information input or output through, for example, a speaker 282, a receiver 284, an earphone 286, a microphone 288, or the like.

The camera module 291 may be, for example, a device capable of capturing still images and moving images, and may include one or more image sensors (e.g., a front sensor or a rear sensor), a lens, an image signal processor (ISP) , Or a flash (e.g., an LED or xenon lamp, etc.).

The power management module 295 can, for example, manage the power of the electronic device 201. [ According to one embodiment, the power management module 295 may include a circuit for charging the battery 296. According to one embodiment, the power management module 295 may include a power management integrated circuit (PMIC), a charger integrated circuit, or a battery or fuel gauge. The PMIC may have a wired and / or wireless charging scheme. The wireless charging scheme may include, for example, a magnetic resonance scheme, a magnetic induction scheme, or an electromagnetic wave scheme, and may further include an additional circuit for wireless charging, for example, a coil loop, a resonant circuit, have. The battery gauge can measure, for example, the remaining amount of the battery 296, the voltage during charging, the current, or the temperature. The battery 296 may include, for example, a rechargeable battery and / or a solar battery.

The indicator 297 may indicate a particular state of the electronic device 201 or a portion thereof (e.g., processor 210), e.g., a boot state, a message state, or a state of charge. The motor 298 can convert electrical signals to mechanical vibration and can generate vibration, haptic effects, and the like. Although not shown, the electronic device 201 may include a processing unit (e.g., a GPU) for mobile TV support. The processing unit for supporting the mobile TV can process media data conforming to standards such as digital multimedia broadcasting (DMB), digital video broadcasting (DVB), or media flow ( ^TM ).

Each of the components described in this document may be composed of one or more components, and the name of the component may be changed according to the type of the electronic device. In various embodiments, the electronic device may comprise at least one of the components described herein, some components may be omitted, or may further include additional other components. In addition, some of the components of the electronic device according to various embodiments may be combined into one entity, so that the functions of the components before being combined can be performed in the same manner.

3 is a block diagram of a program module according to various embodiments. According to one embodiment, program module 310 (e.g., program 140) includes an operating system (OS) that controls resources associated with an electronic device (e.g., electronic device 101) (E.g., application programs 147) running on the system. The operating system may be, for example, android, iOS, windows, symbian, tizen, or bada.

The program module 310 may include a kernel 320, a middleware 330, an application programming interface (API) 360, and / or an application 370. At least a portion of the program module 310 may be preloaded on the electronic device or may be downloaded from an external electronic device such as the electronic device 102 104 or the server 106,

The kernel 320 (e.g., the kernel 141) may include, for example, a system resource manager 321 and / or a device driver 323. The system resource manager 321 can perform control, allocation, or recovery of system resources. According to one embodiment, the system resource manager 321 may include a process manager, a memory manager, or a file system manager. The device driver 323 may include, for example, a display driver, a camera driver, a Bluetooth driver, a shared memory driver, a USB driver, a keypad driver, a WiFi driver, an audio driver, or an inter-process communication .

The middleware 330 may provide various functions commonly required by the application 370 or may be provided through the API 360 in various ways to enable the application 370 to efficiently use limited system resources within the electronic device. Functions can be provided to the application 370. According to one embodiment, middleware 330 (e.g., middleware 143) includes a runtime library 335, an application manager 341, a window manager 342, a multimedia manager 343, a resource manager 344, a power manager 345, a database manager 346, a package manager 347, a connectivity manager 346, (Not shown) 348, a notification manager 349, a location manager 350, a graphic manager 351, or a security manager 352 can do.

The runtime library 335 may include, for example, a library module that the compiler uses to add new functionality via a programming language while the application 370 is executing. The runtime library 335 may perform input / output management, memory management, or functions for arithmetic functions.

The application manager 341 can manage the life cycle of at least one of the applications 370, for example. The window manager 342 can manage GUI resources used in the screen. The multimedia manager 343 can recognize the format required for reproducing various media files and can encode or decode the media file using a codec suitable for the format. The resource manager 344 can manage resources such as source code, memory or storage space of at least one of the applications 370.

The power manager 345 operates together with a basic input / output system (BIOS), for example, to manage a battery or a power source, and can provide power information and the like necessary for the operation of the electronic device. The database manager 346 may create, retrieve, or modify a database for use in at least one of the applications 370. The package manager 347 can manage installation or update of an application distributed in the form of a package file.

The connection manager 348 may manage wireless connections, such as, for example, WiFi or Bluetooth. The notification manager 349 may display or notify events such as arrival messages, appointments, proximity notifications, etc. in a manner that is unobtrusive to the user. The location manager 350 may manage the location information of the electronic device. The graphic manager 351 may manage the graphic effect to be provided to the user or a user interface related thereto. The security manager 352 can provide all security functions necessary for system security or user authentication. According to one embodiment, when an electronic device (e.g., electronic device 101) includes a telephone function, middleware 330 further includes a telephony manager for managing the voice or video call capabilities of the electronic device can do.

Middleware 330 may include a middleware module that forms a combination of various functions of the above-described components. The middleware 330 may provide a module specialized for each type of operating system in order to provide differentiated functions. In addition, the middleware 330 may dynamically delete some existing components or add new ones.

The API 360 (e.g., API 145) may be provided in a different configuration depending on the operating system, for example, as a set of API programming functions. For example, for Android or iOS, you can provide one API set per platform, and for tizen, you can provide more than two API sets per platform.

An application 370 (e.g., an application program 147) may include, for example, a home 371, a dialer 372, an SMS / MMS 373, an instant message 374, a browser 375, The camera 376, the alarm 377, the contact 378, the voice dial 379, the email 380, the calendar 381, the media player 382, the album 383 or the clock 384, or one or more applications capable of performing functions such as health care (e.g., measuring exercise or blood glucose), or providing environmental information (e.g., providing atmospheric pressure, humidity, or temperature information, etc.).

According to one embodiment, an application 370 is an application that supports the exchange of information between an electronic device (e.g., electronic device 101) and an external electronic device (e.g., electronic devices 102 and 104) For convenience, an "information exchange application"). The information exchange application may include, for example, a notification relay application for communicating specific information to an external electronic device, or a device management application for managing an external electronic device.

For example, the notification delivery application may send notification information generated by other applications (e.g., SMS / MMS applications, email applications, health care applications, or environmental information applications) of the electronic device to external electronic devices , 104), respectively. Further, the notification delivery application can receive notification information from, for example, an external electronic device and provide it to the user.

The device management application may be configured to perform at least one function (e.g., turn-on or turn-off) of an external electronic device (e.g., an electronic device 102 or 104) (E.g., on / off-off, or adjusting the brightness (or resolution) of the display), managing applications on external electronic devices or functions provided on external electronic devices , Or updated).

According to one embodiment, the application 370 may include an application (e.g., a healthcare application of a mobile medical device, etc.) designated according to an attribute of an external electronic device (e.g., electronic device 102, 104). According to one embodiment, application 370 may include an application received from an external electronic device (e.g., server 106 or electronic device 102, 104) May include a preloaded application or a third party application downloadable from a server. The names of the components of the program module 310 according to the illustrated embodiment may include the type of operating system Therefore, it can be changed.

According to various embodiments, at least some of the program modules 310 may be implemented in software, firmware, hardware, or a combination of at least two of them. At least some of the program modules 310 may be implemented (e.g., executed) by, for example, a processor (e.g., processor 210). At least some of the program modules 310 may include, for example, modules, programs, routines, sets of instructions or processes, etc. to perform one or more functions.

4 is a view showing an example of skin imaging in an electronic device according to various embodiments. Referring to FIG. 4, the electronic device 401 may be portable by a user, and may photograph a user's skin using a camera in response to a photographing request input by a user. For example, the electronic device may include a first camera 410 on the front surface of the housing and a second camera 420 on the rear surface of the housing, and may include at least one camera of the first camera 410 or the second camera 420 The skin can be photographed. The photographed image can be displayed through the display 430 of the electronic device.

According to various embodiments, an electronic device includes at least one of a camera, at least one of a pigmentation area and a pore area using a skin image captured by the camera, and detects a skin condition corresponding to the skin image using the detection result And a processor that determines whether or not the received signal is a signal.

According to one embodiment, the electronic device further comprises a display, and the processor can control to superimpose the detection result on the skin image to be displayed on the display.

According to one embodiment, the processor may be configured to further determine skin roughness using the skin image and the detected pigmentation area and pore area.

According to one embodiment, the electronic device may further comprise an image processing module for correcting the brightness of the photographed skin image.

According to one embodiment, the display may include at least one of skin goodness and badness information determined by using degree of pigmentation, degree of pore, degree of skin roughness, extent of pigmentation, degree of pore, degree of skin roughness, As shown in FIG.

According to one embodiment, the processor can detect a chroma component by converting a domain of a skin image into a domain capable of chroma detection, and detect a pigment region using the detected chroma component distribution.

According to one embodiment, the processor can determine the pigmentation score using the area ratio and weight of the pigment deposit area.

According to one embodiment, the processor may detect a threshold value of the pigmented deposition area and the normal skinned area in the saturation component distribution using a bordering technique and detect a region having a saturation component larger than the threshold value as the pigmented deposition area have.

According to one embodiment, the saturation-detectable domain may be an HSV domain.

According to one embodiment, the processor converts the domain of the skin image into a domain capable of brightness detection, detects a brightness component in the converted domain, detects an initial pore region using the detected brightness component, The final pore region can be detected by excluding the pore-outside region such as a point in the region.

According to one embodiment, the processor may determine a pore score according to the pore depth and the number of pores in the final pore region.

According to one embodiment, the processor can detect a threshold value of the pore region and the normal skin region in the brightness component distribution using a bordering technique, and detect a region having a brightness component lower than the threshold value as a pore region.

According to one embodiment, the brightness-detectable domain may be an RGB domain.

According to one embodiment, the processor can detect a background skin area excluding the pigmentation area and the pore area in the skin image, and detect the skin roughness according to the brightness distribution of the background skin area.

5 is a block diagram of an electronic device for skin condition detection according to various embodiments. 5, an electronic device 500 may include a camera 510, an image processing module 520, a processor 530, and a display 540.

The camera 510 may, for example, capture still images and motion pictures, and may, according to one embodiment, include one or more image sensors (e.g., front or rear), a lens, an image signal processor flash (e.g., LED or xenon lamp, etc.). The camera 510 may include one or more cameras. The camera 510 may include more than 16 million pixels. However, the pixels of the camera 510 are not limited to 16 million pixels, and pixels of different numerical values can be used as long as the pixels can analyze the skin. For example, a first camera and a second camera when the camera 510 includes a plurality of cameras. The camera 510 may photograph the skin using at least one camera of the first camera or the second camera according to the user's selection.

The image processing module 520 can process the skin image photographed by the camera 510 using various image processing methods. According to one embodiment, the image processing module 520 may correct the brightness of the skin image taken using an illumination adjustment method. According to one embodiment, the image processing module 520 may be included in the processor 530, the processing operation of the image processing module 520 may be performed in an external server, and the processing result may be received through the communication unit.

The processor 530 may include one or more of a central processing unit (CPU), an application processor (AP), or a communication processor (CP). The processor 120 may perform computations or data processing related to, for example, control and / or communication of at least one other component of the electronic device 101. The processor 530 detects the pigmentation area using the corrected image, detects the pore area using the corrected image, and estimates the skin roughness using the area excluding the pigmentation area and the pore area in the corrected image can do. The processor 530 may output skin condition information using the pigmentation area detection result, the pore area detection result, and the skin roughness estimation result. According to one embodiment, the processor 530 may convert the domain of the corrected image to a hue saturation value (HSV) domain, detect the S (saturation) component in the HSV domain, and then detect the saturation component of the detected saturation component It is possible to detect a region having a saturation component larger than the threshold value as a pigmentation region and to calculate a pigmentation score according to the detected region of the pigmentation region. According to one embodiment, the processor 530 may convert the domain of the corrected image into RGB (Red Green Blue) domain, detect the B (Blue) component in the RGB domain, and then use the detected B component The initial pore region can be detected and the final pore region can be detected by excluding the pore outside region such as a point in the initial pore region and the pore point according to the detected final pore region can be calculated. According to one embodiment, the processor 530 can detect a background skin area excluding a pigmentation area and a pore area in a corrected image, estimate a skin roughness distribution, and calculate a skin roughness score have.

The display 540 may include, for example, a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting diode (OLED) A microelectromechanical systems (MEMS) display, or an electronic paper display. Display 160 may display various content (e.g., text, image, video, icon, or symbol, etc.) to a user, for example. Display 160 may include a touch screen and may receive a touch, gesture, proximity, or hovering input using, for example, an electronic pen or a portion of the user's body. Such display 540 may display skin condition information. According to one embodiment, the skin condition information may include a pigmentation score, a pore score, a skin roughness score, and may include a good or bad skin condition depending on a pigmentation score, a pore score, and a skin roughness score.

6 is a view for explaining the concept of skin condition detection in an electronic device according to various embodiments. Referring to FIG. 6, when the skin image a captured by the camera 510 is input, the brightness of the skin image a is corrected by the image processing module 520, . The corrected skin image (b) may be processed by the processor (530). The corrected skin image b may be processed by the processor 530 to produce a pigmented deposition area extracted image c for pigmented deposition area extraction and the corrected skin image b may be generated by processor 530 And a pore region extracted image (d) for extracting the pore region can be generated. A background skin area excluding the pigmentation area and the pore area in the skin image b corrected by the processor 530 can be detected and the skin roughness estimated image e can be generated using the background skin area. The processor 530 can detect the pigment deposition detection result from the pigment accumulation area extraction image c and can detect the pore accumulation result from the pore area extraction image d and calculate the skin roughness estimation image e The skin roughness estimation result can be detected.

According to various embodiments, a method for detecting a skin condition in an electronic device includes: acquiring a skin image; detecting at least one of a pigmentation area and a pore area using the skin image; And determining the skin condition corresponding to the skin image.

According to one embodiment, the method may further comprise superimposing the detection result on the skin image and displaying the result on the display.

According to one embodiment, the method may further include determining skin roughness using the skin image, the detected pigment deposition area, and the pore area.

According to one embodiment, the method may further include correcting brightness of the photographed skin image.

According to one embodiment, the method may further comprise at least one of skin goodness and badness information determined using the degree of pigmentation, the degree of pore, the degree of skin roughness, the extent of pigmentation, the degree of pore, And may further include an operation of displaying information.

According to one embodiment of the present invention, the operation of detecting the pigment deposition includes an operation of converting a domain of the skin image into a domain capable of detecting saturation to detect a saturation component, and a step of detecting a coloring region using the detected saturation component distribution And the like.

According to one embodiment, the method may further comprise determining a pigmentation score using an area ratio and a weight of the pigment deposit area.

According to one embodiment, the act of detecting the pigment deposit region comprises: detecting a threshold of the pigment deposit region and the normal skin region in the saturation component distribution using a bordering technique; And detecting the region as the pigmentation region.

According to one embodiment, the saturation-detectable domain may be an HSV domain.

According to an embodiment of the present invention, the operation of detecting the pore region may include converting the domain of the skin image into a domain capable of detecting brightness and detecting a brightness component in the converted domain, Detecting an area of the initial pore region, and detecting an ultimate pore region in the initial pore region excluding an area outside the pore region.

According to one embodiment, the method may further include a pore score according to the pore depth and the number of pores in the final pore region.

According to an embodiment of the present invention, the operation of detecting the initial pore region includes: detecting a threshold value of a pore region and a normal skin region in the brightness component distribution using a bordering technique; To the pore area.

According to one embodiment, the brightness-detectable domain may be an RGB domain.

According to one embodiment of the present invention, the operation of detecting the skin roughness includes: detecting a background skin area excluding the pigment deposition area and the pore area in the skin image; and detecting the skin roughness according to the erection distribution of the background skin area Operation.

7 is a flowchart illustrating an operation of providing skin condition information in an electronic device according to various embodiments. Referring to FIG. 7, the electronic device may receive a skin image taken by the camera 510 in 702 operation.

The electronic device may correct the received skin image in operation 704. According to one embodiment, the skin image of the electronic device can be corrected for brightness using an illumination adjustment method by the image processing module 510.

The electronic device can detect the pigmented area in the corrected image at 706 operation. According to one embodiment, the electronic device converts the domain of the corrected image into a predetermined color space, for example, a hue saturation value (HSV) domain, detects the S (saturation) component in the HSV domain, A threshold value of the pigmented deposition region and the normal skinned region can be detected and a region having a saturation component larger than the threshold value can be detected as the pigmented deposition region. According to one embodiment, the predetermined color space may be any color space as long as it is a color space in which the chroma difference can be confirmed. For example, the color space may be any one of RGB, YUN, and HSV. In the present invention, an example using the HSV domain is described, but the color space may not be limited. According to one embodiment, the threshold values of the pigmented deposition area and the normal skin area can be detected by a line-fitting technique.

The electronic device may detect the pore area in operation 708. [ According to one embodiment, the electronic device converts the domain of the corrected image into RGB (Red Green Blue) domain, detects the B (Blue) component in the RGB domain, The initial pore region can be detected in an area having a B component smaller than the threshold value and the electronic device can detect the final pore region by excluding the pore outside region such as a point in the initial pore region. According to one embodiment, after removing the pore-outside region such as a point in the initial pore region, a final pore region in which noise is removed through a noise technique such as opening can be detected. According to one embodiment, the threshold values of the pore area and the normal skin area can be detected by a line-fitting technique.

The electronic device can estimate the skin roughness using a region excluding the pigmentation region and the pore region in the skin image at 710 operation. According to one embodiment, the electronic device can detect a background skin area excluding a pigmentation area and a pore area in a corrected skin image, and estimate a skin roughness by determining a brightness distribution of a background skin area.

The electronic device can determine the skin condition using the pigmentation area detection result, the pore area detection result, and the skin roughness estimation result in operation 712. According to one embodiment, the electronic device can determine the pigmentation score using the detection result of the pigmentation area. According to one embodiment, the electronic device can determine the pore score using the pore area detection result. According to one embodiment, the electronic device can determine the skin roughness score using the skin roughness estimation result. According to one embodiment, the electronic device can judge whether the skin condition is good or bad by comprehensively judging the pigmentation score, the pore score, and the skin roughness score.

The electronic device may display skin condition information on the display at 714 operation. According to one embodiment, the electronic device may display a pigmentation score, a pore score, and a skin roughness score. According to one embodiment, the electronic device can indicate the degree of good or bad skin condition depending on the pigmentation score, the pore score, and the skin roughness score.

8 is a flow diagram illustrating a pigmentation detection operation in accordance with various embodiments. Referring to FIG. 8, the electronic device may convert the domain of the skin image to a predetermined color space, e. G., A hue saturation value (HSV) domain in 802 operation and an S (saturation) component in the HSV domain at 804 operation. The pigment deposition region may be a region in which the pigment is abnormally deposited in a specific region of the skin, and the skin region has a color different from that of a normal skin region. In the case of skin such as yellow or white except for black skin such as black, the pigmented area may appear more red than normal skin area. Accordingly, according to one embodiment, the domain of the skin image can be converted into a domain capable of distinguishing achromatic and chromatic colors, for example, the HSV domain, and the chroma (S) component can be detected in the HSV domain.

The electronic device can detect a threshold value of the pigmented deposition area and the normal skin area using the distribution of the detected saturation component at operation 806. [ According to one embodiment, the electronic device can detect a threshold value of a pigmented deposition area and a normal skin area using a line-fitting technique in a histogram showing a distribution of detected saturation components.

The electronic device can detect a region having a saturation component larger than the threshold value in the operation 808 as the pigmentation region. According to one embodiment, an electronic device can detect a region having a chroma component lower than a threshold value in a skin image as a pigment region, and a region having a chroma component larger than a threshold value as a normal skin region.

The electronic device can determine the pigmentation score using the ratio and the weight of the detected area of the pigmented area in operation 810. [ According to one embodiment, the electronic device may weight according to the degree of pigmentation (step), i.e., the saturation value of the pigmented area. According to one embodiment, the electronic device can determine the area of the pigment deposition step, and determine the area ratio of the skin pigmentation step in the skin image. The electronic device can calculate the weighted sum of the area ratios for each pigment deposition step and determine the pigmentation score using the following equation (1).

In Equation (1), Ri may be an area ratio of the i-th stage. di may be the weight of the i-th stage. score may be a pigmentation score. N can be determined by experiment or design. Pigmentation scores can be calculated by calculating the weighted sum of the area ratios for each stage of pigmentation from step 1 to step N. According to Equation (1) as described above, the more pigment deposits are thicker and wider, the higher the pigmentation score can be judged.

FIG. 9 is a view showing an example of detection of per-domain pigment deposition according to various embodiments. Referring to Fig. 9, in detecting the pigmentation, the electronic device can detect the saturation component by converting the skin image (a) into a domain in which the saturation difference can be confirmed. For example, the electronic device can detect the saturation component by converting the skin image (a) into the respective domains of RGB, YUN, and HSV. It is possible to detect the B value b by converting the skin image a into the RGB domain and convert the skin image a into the YUN domain to detect the U value c, And the S (d) value can be detected. In the present invention, an example using the HSV domain is described, but the domain may not be limited.

FIG. 10 is a diagram illustrating a histogram for detecting a boundary between a pigmented area and a normal skin area according to various embodiments. Referring to FIG. 10, the electronic device can determine a threshold value T, which is a boundary between a pigmented area and a normal skin area, using a histogram curve 1010 indicating the distribution of S components detected in the HSV domain. The threshold value (T) of the pigmented deposition area and the normal skin area can be detected by a line-fitting technique. For example, the electronic device calculates a point 1040 at which the slope of the two linear approximation functions 1041 and 1042 changes abruptly with respect to the interval between the S component distribution maximum value 1020 and the S component distribution end point 1030 And can be detected at the threshold value (T). For example, the electronic device can calculate the LMSE (Least Mean Square Error) between the linear approximation function of the two left and right portions and the histogram, as shown in Equation 2 and Equation 3, after determining the candidate point of the threshold value to be detected.

Referring to Equations (2) and (3) above, eL, t eR, and t are the candidate points

(L) and right (R) can be indicated on the basis of the reference value. h (x) is the position

, YL, t (x), yR, and t (x) can represent the size of the estimated histogram at position x, respectively. The error of the left (L) and right (R) areas is compared with the standard deviation of each area

,

), And can detect an optimal threshold value t * that minimizes the sum of the normalized LMSEs. The equation (4) for detecting the optimum threshold value t * may be as follows.

Referring to Equation (4), t * can be detected as a point minimizing the error between the estimated linear function and the histogram, and the threshold value T of the pigmented deposition region and the normal skin region can be determined using t * .

11 is a view for explaining the level of pigmentation according to degree of pigmentation according to various embodiments. Referring to FIG. 11, the electronic device can detect the pigmentation area image (a) in the S component image and determine the degree of pigmentation detection in the pigmentation area image (a) to discriminate the degree of pigmentation detection (b) . According to one embodiment, the step of pigmentation can be classified from the first stage to the N stage according to the degree of detection of one pigment. According to one embodiment, the pigmentation step may be divided into three levels of pigmentation, such as first level pigmentation (d1), second level pigmentation (d2), and third level pigmentation (d3). According to one embodiment, a dark pigment deposit region can be displayed brighter. For example, normal skin without pigmentation may be darkest and skin of third level pigmentation 3 may be brightest.

12 is a view for explaining weights according to pigmentation levels according to various embodiments. Referring to FIG. 12, the electronic device can weight according to the S (saturation) value of the pigmented area. According to one embodiment,

), It is possible to assign a large weight to the N-th level in a stepwise manner if the pigmentation is severe from the first level, which is the case where the pigmentation is small. For example, the electronic device can weightfully increase the weights from the first pigment deposition level (d1) with the least pigment deposition to the fifth pigment deposition level (d5) with more pigment deposition.

13 is a flowchart illustrating pore detection operations according to various embodiments. Referring to FIG. 13, the electronic device may convert the domain of the skin image to the RGB domain in operation 1302 and detect the B component in the RGB domain in operation 1304. According to one embodiment, a pore is a space in which a hair grows into a hair orifice, and can be detected as a dark color having a low lightness along with a point and a hair area in a skin image. According to one embodiment, in a domain other than the RGB domain, a domain that can detect a brightness component may be converted to another domain and a component corresponding to a brightness component may be detected.

The electronic device may detect an initial pore region using the detected B component in operation 1306. [ According to one embodiment, the electronic device can detect a low brightness part in the skin image as a pore area by using the B component of the RGB domain. According to an embodiment, the electronic device can detect a threshold value of a pore region and a normal skin region using the distribution of the detected B component, and detect an initial pore region in a region having a B component smaller than a threshold value. According to one embodiment, the threshold values of the pore area and the normal skin area can be detected by a line-fitting technique.

The electronic device may detect the pore-outside region in the initial pore region in operation 1308. [ The area outside the pore may be an area corresponding to a point or a hair other than the actual pore. According to one embodiment, since the electronic device may have a point region or a hair region in the skin image, which is wider than the pore region, an area having a larger area than the predetermined region can be detected as the pore outside region.

The electronic device can detect the final pore area by excluding the pore outside area from the initial pore area in 1310 operation. According to one embodiment, the electronic device can detect the final pore region by excluding the pore outside region which is wider than the pore region in the initial pore region by using a morphology technique.

The electronic device can determine the number of pores and the depth of the pores in the final pore region in operation 1312, and determine the number of pore points according to the number of pores and the depth of the pores. According to one embodiment, the electronic device can remove noise by applying an opening technique to the final pore area detection image, and determine the number of pores and the depth of the pore in the noise-removed final pore area detection image. The number of pores may be the number of pore areas. The depth of the pore can be judged according to the darkness of the pore region. According to one embodiment, the electronic device can weight the brightness value of the pore region according to the depth of the pore. According to one embodiment, the electronic device can calculate the weighted sum of the number of pores according to each pore depth step and determine the pore score using the following equation (5).

In Equation (5), ni may be the number of pores of the i-th stage. di may be the weight of the i-th stage. The score may be a pore score. I can be from 1 to N. N can be determined by experiment or design. The pore score can be calculated as the weighted sum of the number of pores of each pore depth step from step 1 to N step. According to Equation (5), the higher the number of dark pores, the higher the pore score can be determined.

14 and 15 are views for explaining the concept of pore detection according to various embodiments. 14, when the skin image (a) is input, the electronic device can acquire the initial pore region detection image (b), obtain the pore outside region detection image from the initial pore region detection image (b) , The final pore region detection image (d) can be obtained by excluding the pore-outside-region detection image from the initial pore region detection image (b) using the morphology technique.

15, the electronic device obtains a noise-removed final pore area detection image (b) by removing noise using a noise removal technique such as an opening technique on the final pore area detection image (a) It is possible to determine the number of pores (c) in the final pore area detection image (b) and determine whether the pore depth d of each pore is shallow or deep.

16 is a diagram illustrating a histogram for detecting a boundary between an initial pore region and a normal skin region according to various embodiments. Referring to FIG. 16, the electronic device can determine a threshold value T, which is a boundary between an initial pore region and a normal skin region, using a histogram curve 1601 representing the distribution of the B component detected in the RGB domain. The threshold value T of the initial pore area and the normal skin area can be detected by a line-fitting technique. For example, the electronic device calculates a point 1640 at which the slope of the two linear approximation functions 1641 and 1642 changes abruptly with respect to the interval between the maximum value 1620 of the B component distribution and the starting point 1010 of the B component distribution And can be detected at the threshold value (T). For example, the electronic device can calculate the LMSE (Least Mean Square Error) between the linear approximation function of the two left and right portions and the histogram as shown in Equation (6) and Equation (7) after determining the candidate point of the threshold value to be detected.

Referring to Equations (6) and (7), eL, t eR, and t are the candidate points

(L) and right (R) can be indicated on the basis of the reference value. h (x) is the position

, YL, t (x), yR, and t (x) can represent the size of the estimated histogram at position x, respectively. The error of the left (L) and right (R) areas is compared with the standard deviation of each area

,

), And can detect an optimal threshold value t * that minimizes the sum of the normalized LMSEs. The equation (8) for detecting the optimum threshold value t * may be as follows.

Referring to Equation (8), t * can be detected as a point minimizing the error between the estimated linear function and the histogram, and the threshold value T of the pore region and the normal skin region can be determined using t *.

17 is a view for explaining the pore depth level according to the depth of the pores according to various embodiments. 17, the electronic device can obtain the final pore area detection image (a) in the B component image, determine the depth of the pore depth in the final pore area image (a) can do. According to one embodiment, the depth of pore depth can be divided from the first stage to the N stage according to the depth of the pore. According to one embodiment, the pore depth level can be divided into three levels of pore depth levels such as a first level pore depth d1, a second level pore depth d2, and a third level pore depth d3. According to one embodiment, the deep pores can be displayed brighter. For example, normal skin without pores may be displayed darkest, and third-level pore depth d3 may be displayed with the brightest area.

18 is a view for explaining weights according to pore depth levels according to various embodiments. Referring to FIG. 18, the electronic device can assign a weight according to the value B of the pore region. According to one embodiment, the electronic device can assign a weight to each of the steps from the first level, which is shallow when the depth of the pore is shallow, to the N-th level, which is the depth of the pore. For example, the electronic device can provide a weighted value stepwise from the first pore depth level (d1) at which the pore depth is the shallowerest to the fifth pore depth level (d5) at which the pore depth is deeper.

19 is a flowchart showing a skin roughness determination operation according to various embodiments. Referring to FIG. 19, the electronic device can detect a background skin area in the skin area excluding the pigmentation area and the pore area in the operation 1902. FIG.

The electronic device may detect the brightness distribution of the background skin area in operation 1904. According to one embodiment, the brightness histogram distribution of the background skin region can be detected.

The electronic device can determine the skin roughness score according to the brightness histogram distribution result of the background skin area in operation 1906. [ According to one embodiment, the electronic device can determine the skin roughness score using entropy from the brightness histogram distribution of the background skin area. According to one embodiment, the electronic device can determine the skin roughness score according to whether the brightness distribution is concentrated in a certain section or the brightness distribution is spread over a wide section.

20 is a view for explaining the concept of skin roughness estimation according to various embodiments. Referring to Fig. 20, an electronic device acquires an image of a pigmented region (b) from a skin image (a) and acquires an image of a pore region (c) from a skin image (a) can do. The electronic device can obtain the background skin area image (d) by removing the pigmentation area (b) and the pore area (c) in the skin image (a).

Figure 21 is a histogram for estimating skin roughness according to various embodiments. Referring to FIG. 21, the brightness distribution of the background skin region is detected by the electronic device. The brightness histogram distribution of the background skin region can be obtained as shown in FIG. According to one embodiment, the electronic device can determine the skin roughness score using entropy from the brightness histogram distribution of the background skin area. According to one embodiment, the electronic device can determine the skin roughness score according to whether the brightness distribution is concentrated in a predetermined section 2100 or the brightness distribution is spread over a wide range. According to one embodiment, the skin roughness score can be determined using Equation (9).

According to Equation (9), E is entropy and P (zi) is a probability value of the i-th histogram.

FIGS. 22 to 24 are views showing a skin condition detection screen according to various embodiments. 22, the electronic device may display a screen 2200 for skin condition detection on the display. The screen 2200 for skin condition detection includes an original image area in which a skin image is displayed, an illumination removal image area in which illumination is removed from a skin image, a region in which a pore detection image obtained as a result of detection of pores is displayed in a skin image, A region for displaying the acquired pigment deposition detection image may be provided as a result of deposition detection, and a camera 2210, a gallery 2220, and an analyze 2230 button may be displayed. When the user selects the camera 2210 button, the electronic device can photograph the skin image by operating the camera. If the Gallery 2220 button is selected by the user, the electronic device may recall the skin image of the pre-stored images.

Referring to FIG. 23, the electronic device can display a skin image taken by a user or an imported skin image in an original image area. The Analyze 2230 button may be selected by the user while the skin image 2310 is displayed. When the Analyze 2230 button is selected by the user, the electronic device may begin to analyze the skin image 2310 to acquire the illumination removed image, the pore detection image, and the pigmentation detection image, respectively, , The number of pores, the number of pores, the degree of skin roughness, the degree of pigmentation, the number of pores, and the roughness of the skin.

Referring to FIG. 24, the electronic device can display the obtained unlit image 2420 in the unlit image region, display the obtained pore detection image 2430 in the pore detection image region, The obtained pigmentation detection image 2440 can be displayed in the deposition detection image area. The electronic device can display the pore detection result on the skin image 2310 in a superimposed manner when the pore detection image 2430 is displayed and can display the pigment deposition detection result on the skin image 2310 Can be superimposed and displayed. Also, the electronic device can display the number of coloring points, the number of pores, the number of pore points, and the roughness of the skin (2450), and can display the goodness and badness information of the skin in any one of 0 to 10 numbers.

Accordingly, according to various embodiments, when a user photographs a skin photograph through a camera of a portable electronic device such as a smart phone, various skin condition information such as skin pigmentation, pores, and roughness of skin can be provided using the skin photographing image It can be convenient.

Each of the above-described components of the electronic device according to various embodiments of the present invention may be composed of one or more components, and the name of the component may be changed according to the type of the electronic device. The electronic device according to various embodiments of the present invention may be configured to include at least one of the above-described components, and some components may be omitted or further include other additional components. In addition, some of the components of the electronic device according to various embodiments of the present invention may be combined into one entity, so that the functions of the components before being combined can be performed in the same manner.

As used in this document, the term "module" may refer to a unit comprising, for example, one or a combination of two or more of hardware, software or firmware. A "module" may be interchangeably used with terms such as, for example, unit, logic, logical block, component, or circuit. A "module" may be a minimum unit or a portion of an integrally constructed component. A "module" may be a minimum unit or a portion thereof that performs one or more functions. "Modules" may be implemented either mechanically or electronically. For example, a "module" may be an application-specific integrated circuit (ASIC) chip, field-programmable gate arrays (FPGAs) or programmable-logic devices And may include at least one.

At least a portion of a device (e.g., modules or functions thereof) or a method (e.g., operations) according to various embodiments may include, for example, computer-readable storage media in the form of program modules, As shown in FIG. When the instruction is executed by a processor (e.g., processor 120), the one or more processors may perform a function corresponding to the instruction. The computer readable storage medium may be, for example, memory 130. [

The computer readable recording medium may be a hard disk, a floppy disk, a magnetic media (e.g., a magnetic tape), an optical media (e.g., a compact disc read only memory (CD-ROM) digital versatile discs, magneto-optical media such as floptical disks, hardware devices such as read only memory (ROM), random access memory (RAM) Etc. The program instructions may also include machine language code such as those produced by a compiler, as well as high-level language code that may be executed by a computer using an interpreter, etc. The above- May be configured to operate as one or more software modules to perform the operations of the embodiment, and vice versa.

According to various embodiments, there is provided a storage medium storing a program, the program causing the computer to perform the steps of: acquiring a skin image; detecting at least one of a pigmented area, a pore area using the skin image; And an operation of determining a skin condition corresponding to the skin image using the detection result.

Modules or program modules according to various embodiments may include at least one or more of the elements described above, some of which may be omitted, or may further include additional other elements. Operations performed by modules, program modules, or other components in accordance with various embodiments may be performed in a sequential, parallel, iterative, or heuristic manner. Also, some operations may be performed in a different order, omitted, or other operations may be added. And the embodiments disclosed in this document are presented for the purpose of explanation and understanding of the disclosed technology and do not limit the scope of the technology described in this document. Accordingly, the scope of this document should be interpreted to include all modifications based on the technical idea of this document or various other embodiments.

Claims (29)

In an electronic device,
camera;
And a processor for detecting at least one of a pigment deposit area and a pore area using the skin image photographed by the camera and determining a skin condition corresponding to the skin image using the detection result.
The method according to claim 1,
Further comprising a display,
And the processor controls to superimpose the detection result on the skin image to be displayed on the display.
The method according to claim 1,
Wherein the processor is further configured to determine skin roughness using the skin image and the detected pigmentation area and pore area.
The method according to claim 1,
And an image processing module for correcting the brightness of the photographed skin image.
3. The method of claim 2,
The display may display information corresponding to a skin condition including at least one of information on degree of pigment deposition, degree of pore, degree of skin roughness, degree of pigmentation, degree of pore, and degree of skin roughness judged by degree of skin roughness ≪ / RTI >
The electronic device according to claim 1, wherein the processor converts a domain of a skin image into a domain capable of detecting saturation to detect a saturation component, and detects a pigment deposit area using the detected saturation component distribution.
The method according to claim 6,
Wherein the processor determines a pigmentation score using an area ratio and weight of the pigment deposit area.
The method according to claim 6,
Wherein the processor detects a threshold value of a pigmented deposition area and a normal skinned area in the saturation component distribution using a bordering technique and detects a region having a saturation component larger than the threshold value as the pigmented deposition area.
The method according to claim 6,
Wherein the saturation-detectable domain is a hue saturation value (HSV) domain.
The method according to claim 1,
The processor converts the domain of the skin image into a domain capable of brightness detection, detects a brightness component in the converted domain, detects an initial pore region using the detected brightness component, and detects an initial pore region, And detecting the final pore area by excluding the outer pore area.
11. The method of claim 10,
Wherein the processor determines a pore score according to the pore depth and the number of pores in the final pore region.
11. The method of claim 10,
Wherein the processor detects a threshold value of the pore region and the normal skin region in the brightness component distribution using a bordering technique and detects an area having a brightness component lower than the threshold as a pore region.
11. The method of claim 10,
Wherein the brightness-detectable domain is an RGB (Red Green Blue) domain.
The method of claim 3,
Wherein the processor detects a background skin area excluding the pigmentation area and the pore area in the skin image and detects the skin roughness according to the brightness distribution of the background skin area.
A method for detecting a skin condition in an electronic device,
An operation for acquiring a skin image,
Detecting at least one of a pigment deposit area and a pore area using the skin image, and
And determining a skin condition corresponding to the skin image using the detection result.
16. The method of claim 15,
Further comprising superimposing the detection result on the skin image and displaying it on the display.
16. The method of claim 15,
Further comprising determining skin roughness using the skin image and the detected pigment deposition area and pore area.
16. The method of claim 15,
And correcting the brightness of the photographed skin image.
16. The method of claim 15,
Further comprising the step of displaying skin condition information including at least one of good and poor skin information determined using the degree of pigmentation, the degree of pore, the degree of skin roughness, the degree of pigment deposition, the degree of pore, and the degree of skin roughness Way.
16. The method of claim 15,
Wherein the detecting of the pigment deposition comprises:
An operation of converting a domain of the skin image into a domain capable of detecting saturation to detect a saturation component, and
And detecting the pigmented area using the detected saturation component distribution.
21. The method of claim 20,
Determining a pigment deposition score using an area ratio and a weight of the pigment deposit area.
21. The method of claim 20,
Wherein the operation of detecting the pigmented deposition area comprises:
An operation of detecting a threshold value of the pigment deposition area and the normal skin area in the saturation component distribution using a bordering technique, and
Detecting an area having a saturation component greater than the threshold value as the pigmented deposition area.
21. The method of claim 20,
Wherein the saturation-detectable domain is a Hue Saturation Value (HSV) domain.
16. The method of claim 15,
The operation of detecting the pore area may include:
Converting the domain of the skin image into a domain capable of detecting brightness and detecting a brightness component in the converted domain,
Detecting an initial pore region using the detected brightness component, and
Detecting an ultimate pore region by excluding the pore outside region from the initial pore region.
25. The method of claim 24,
Further comprising a pore score according to the pore depth and the number of pores in the final pore area.
25. The method of claim 24,
The operation of detecting the initial pore area may include:
An operation of detecting a threshold value of the pore region and the normal skin region in the brightness component distribution using a bordering technique, and
And detecting an area having a brightness component lower than the threshold value as a pore area.
25. The method of claim 24,
Wherein the brightness-detectable domain is a Red Green Blue (RGB) domain.
18. The method of claim 17,
The skin roughness detecting operation may include:
Detecting a background skin area excluding a pigmentation area and a pore area in the skin image, and
And detecting skin roughness according to an erection distribution of the background skin area.
A storage medium storing a program,
Wherein the program causes, in the electronic device,
An operation for acquiring a skin image,
Detecting at least one of a pigment deposit area and a pore area using the skin image, and
And a skin condition corresponding to the skin image is determined using the detection result.

Images