KR20180109217A - Method for enhancing face image and electronic device for the same - Google Patents

Abstract

The present invention provides an electronic device capable of globally or locally correcting a face. The electronic device of various embodiments includes a memory, a display, and a processor operably connected to the memory or the display. The processor recognizes a face region based on a feature point extracted from an image, sets one or more objects in the face region, sets a reference line in the face region using the objects, divides the face region based on the reference line, calculates a correction ratio for each of the divided face regions, corrects the face regions according to the calculated correction ratio, and displays the corrected image on the display. Other embodiments are also possible.

Description

Technical Field [0001] The present invention relates to a facial image correcting method,

Various embodiments relate to a method and apparatus for facial image correction.

2. Description of the Related Art Various types of electronic devices such as a mobile communication terminal, a personal digital assistant (PDA), an electronic organizer, a smart phone, a personal computer, a wearable device, and the like have been widely used. Such electronic devices are continuously being improved in hardware and / or software portions of electronic devices for function support and enhancement.

On the other hand, while a camera is included in an electronic device, a variety of functions for a camera photographing function are provided, and a correction function for a photographing or photographed image is also developed. For example, for portraits (such as selfie), the electronic device provides a beauty feature that brightens and brightens the face. Beauty features include smoothing to smooth the facial skin, brightening to brighten the face's brightness, slimmer to make the face slimmer, eye enlargement ) Function.

Since the conventional face correction method applies the same effect to the entire face area, blurring may occur in areas such as the eyes, nose, mouth, and the like, or a border line may be generated. Further, since the face is corrected only in the left and right symmetry irrespective of the inclination (e.g., rotation angle) of the face in the past, the face can be corrected unnaturally.

Various embodiments can determine a face slope (or object orientation) based on feature points in a face and correct the face globally or locally based on the determined face slope.

An electronic device according to various embodiments includes a memory, a display, a camera and a processor operatively associated with the memory, the display, or the camera, the processor recognizing the face region based at least on feature points extracted from the image, Setting one or more objects in the face area, setting a reference line in the face area using the one or more objects, dividing the face area based on the reference line, calculating a correction ratio for each of the divided face areas , Correct the face region based on at least the calculated correction ratio, and display the corrected image on the display.

According to various embodiments, an electronic device includes a camera and a processor, wherein the processor is configured to use the camera to obtain an image containing a face, and based on at least the detection method for the face, Determining a method of image processing for the skin based at least on a probability of being at least part of the plurality of guide masks and skin of the face; Based on at least a portion of the skin.

According to various embodiments, an operating method of an electronic device includes: recognizing a face region based at least on feature points extracted from the image; setting one or more objects within the face region; Calculating a correction ratio for each of the divided face regions, correcting the face region based on the calculated correction ratio at least based on the calculated correction ratio, performing an operation of setting an inner reference line, dividing the face region based on the reference line, And displaying the corrected image.

According to various embodiments, the face tilt (or object orientation) may be determined based on the feature points in the face, and the face may be corrected based on the determined face tilt.

According to various embodiments, various face correction masks may be provided based on the face characteristics, and the face may be corrected more effectively by changing the correction ratio of the face correction mask according to the situation required for the face correction.

According to various embodiments, the face image can be corrected more naturally by correcting the face as a whole and correcting the face locally based on the feature points in the face.

According to various embodiments, the face can be corrected by dividing the face region based on the facial feature point and correcting the face by applying different correction ratios to the divided regions, thereby taking into account the overall face ratio.

1 is a diagram illustrating an electronic device in a network environment in accordance with various embodiments.
2 is a block diagram illustrating the configuration of an electronic device according to various embodiments.
3 is a block diagram illustrating a program module in accordance with various embodiments.
4 is a flow chart illustrating a method of operation of an electronic device according to various embodiments.
5 is a flowchart illustrating a face shape correction method according to various embodiments.
6 is a diagram illustrating an example of extracting feature points from an image according to various embodiments.
7 is a diagram illustrating an example of determining a reference line of a face according to various embodiments.
8 is a diagram illustrating an example of calculating a correction ratio for each face region according to various embodiments.
9 is a flow chart illustrating an eye area correction method in accordance with various embodiments.
FIGS. 10A through 10C are diagrams illustrating an example of correcting an eye region based on a baseline slope in accordance with various embodiments.
11 is a flowchart illustrating a correction method using a guide mask according to various embodiments.
12A is a diagram illustrating an example of generating a guide mask and a correction mask according to various embodiments.
12B is a diagram illustrating an example of generating a captured image correction mask according to various embodiments.
12C is a diagram showing an example of generating a preview image correction mask according to various embodiments.
12D is a diagram showing an example of generating a brightness correction mask according to various embodiments.
13 is a flow chart illustrating a method of correcting a face region using correction parameters according to various embodiments.
14 is a diagram illustrating an example of determining correction parameters in accordance with various embodiments.
15 is a diagram showing an example of correcting a face region locally using correction parameters according to various embodiments.
16 is a diagram showing an example of correcting a face area globally or locally according to various embodiments.
17 is a flowchart illustrating a face area correction method according to various embodiments.

Hereinafter, various embodiments of the present document will be described with reference to the accompanying drawings. It is to be understood that the embodiments and terminologies used herein are not intended to limit the invention to the particular embodiments described, but to include various modifications, equivalents, and / or alternatives of the embodiments. In connection with the description of the drawings, like reference numerals may be used for similar components. The singular expressions may include plural expressions unless the context clearly dictates otherwise. In this document, the expressions "A or B" or "at least one of A and / or B" and the like may include all possible combinations of the items listed together. Expressions such as " first, "" second," " first, "or" second, " But is not limited to those components. When it is mentioned that some (e.g., first) component is "(functionally or communicatively) connected" or "connected" to another (second) component, May be connected directly to the component, or may be connected through another component (e.g., a third component).

In this document, the term " configured to (or configured) to "as used herein is intended to encompass all types of hardware, software, , "" Made to "," can do ", or" designed to ". In some situations, the expression "a device configured to" may mean that the device can "do " with other devices or components. For example, a processor configured (or configured) to perform the phrases "A, B, and C" may be implemented by executing one or more software programs stored in a memory device or a dedicated processor (e.g., an embedded processor) , And a general purpose processor (e.g., a CPU or an application processor) capable of performing the corresponding operations.

Electronic devices in accordance with various embodiments of the present document may be used in various applications such as, for example, smart phones, tablet PCs, mobile phones, videophones, electronic book readers, desktop PCs, laptop PCs, netbook computers, workstations, a portable multimedia player, an MP3 player, a medical device, a camera, or a wearable device. Wearable devices may be of the type of accessories (eg, watches, rings, bracelets, braces, necklaces, glasses, contact lenses or head-mounted-devices (HMD) (E.g., a skin pad or tattoo), or a bio-implantable circuit. In some embodiments, the electronic device may be, for example, a television, a digital video disk (Such as Samsung HomeSync ^TM , Apple TV ^TM , or Google TV ^TM ), which are used in home appliances such as home appliances, audio, refrigerators, air conditioners, vacuum cleaners, ovens, microwave ovens, washing machines, air cleaners, set top boxes, home automation control panels, , A game console (e.g., Xbox ^TM , PlayStation ^TM ), an electronic dictionary, an electronic key, a camcorder, or an electronic photo frame.

In an alternative embodiment, the electronic device may be any of a variety of medical devices (e.g., various portable medical measurement devices such as a blood glucose meter, a heart rate meter, a blood pressure meter, or a body temperature meter), magnetic resonance angiography (MRA) A navigation system, a global navigation satellite system (GNSS), an event data recorder (EDR), a flight data recorder (FDR), an automobile infotainment device, a marine electronic equipment (For example, marine navigation systems, gyro compasses, etc.), avionics, security devices, head units for vehicles, industrial or domestic robots, drones, ATMs at financial institutions, of at least one of the following types of devices: a light bulb, a fire detector, a fire alarm, a thermostat, a streetlight, a toaster, a fitness device, a hot water tank, a heater, a boiler, .

According to some embodiments, the electronic device may be a piece of furniture, a building / structure or part of an automobile, an electronic board, an electronic signature receiving device, a projector, or various measuring devices (e.g., Gas, or radio wave measuring instruments, etc.). In various embodiments, the electronic device is flexible or may be a combination of two or more of the various devices described above. The electronic device according to the embodiment of the present document is not limited to the above-described devices. In this document, the term user may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

1 is a diagram illustrating an electronic device 101 in a network environment 100 in accordance with various embodiments.

1, an electronic device 101 may include a bus 110, a processor 120, a memory 130, an input / output interface 150, a display 160, and a communication interface 170. In some embodiments, the electronic device 101 may omit at least one of the components or additionally include other components.

The bus 110 may include circuitry to connect the components 110-170 to one another and to communicate communications (e.g., control messages or data) between the components.

The processor 120 may include one or more of a central processing unit (CPU), an application processor (AP), or a communication processor (CP). The processor 120 may perform computations or data processing related to, for example, control and / or communication of at least one other component of the electronic device 101.

Memory 130 may include volatile and / or non-volatile memory. Memory 130 may store instructions or data related to at least one other component of electronic device 101, for example. According to one embodiment, the memory 130 may store software and / or programs 140. The program 140 may include, for example, a kernel 141, a middleware 143, an application programming interface (API) 145, and / or an application program . At least some of the kernel 141, middleware 143, or API 145 may be referred to as an operating system. The kernel 141 may include system resources used to execute an operation or function implemented in other programs (e.g., middleware 143, API 145, or application program 147) (E.g., bus 110, processor 120, or memory 130). The kernel 141 also provides an interface to control or manage system resources by accessing individual components of the electronic device 101 in the middleware 143, API 145, or application program 147 .

The middleware 143 can perform an intermediary role such that the API 145 or the application program 147 can communicate with the kernel 141 to exchange data. In addition, the middleware 143 may process one or more task requests received from the application program 147 according to the priority order. For example, middleware 143 may use system resources (e.g., bus 110, processor 120, or memory 130, etc.) of electronic device 101 in at least one of application programs 147 Prioritize, and process the one or more task requests. The API 145 is an interface for the application 147 to control the functions provided by the kernel 141 or the middleware 143. The API 145 is an interface for controlling the functions provided by the application 141. For example, An interface or a function (e.g., a command).

Output interface 150 may be configured to communicate commands or data entered from a user or other external device to another component (s) of the electronic device 101, or to another component (s) of the electronic device 101 ) To the user or other external device.

The display 160 may include a display such as, for example, a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, or a microelectromechanical system (MEMS) display, or an electronic paper display . Display 160 may display various content (e.g., text, images, video, icons, and / or symbols, etc.) to a user, for example. Display 160 may include a touch screen and may receive a touch, gesture, proximity, or hovering input using, for example, an electronic pen or a portion of the user's body.

The communication interface 170 establishes communication between the electronic device 101 and an external device (e.g., the first external electronic device 102, the second external electronic device 104, or the server 106) . For example, communication interface 170 may be connected to network 162 via wireless or wired communication to communicate with an external device (e.g., second external electronic device 104 or server 106). The wireless communication may be, for example, LTE, LTE advance, code division multiple access (CDMA), wideband CDMA (WCDMA), universal mobile telecommunications system (UMTS), wireless broadband (WiBro) system for mobile communications, and the like. According to one embodiment, the wireless communication may include wireless fidelity (WiFi), light fidelity (LiFi), bluetooth, bluetooth low energy (BLE), as illustrated in, for example, , Zigbee, near field communication (NFC), magnetic secure transmission, radio frequency (RF), or body area network (BAN).

According to one example, wireless communication may include GNSS. GNSS may be, for example, a global positioning system (GPS), a global navigation satellite system (Glonass), a beidou navigation satellite system (hereinafter "Beidou") or a galileo, the european global satellite-based navigation system. Hereinafter, in this document, "GPS" can be used interchangeably with "GNSS ". The wired communication may include, for example, at least one of a universal serial bus (USB), a high definition multimedia interface (HDMI), a recommended standard 232 (RS-232), a power line communication or a plain old telephone service have. Network 162 may include at least one of a telecommunications network, e.g., a computer network (e.g., LAN or WAN), the Internet, or a telephone network.

Each of the first and second external electronic devices 102, 104 may be the same or a different kind of device as the electronic device 101. According to various embodiments, all or a portion of the operations performed in the electronic device 101 may be performed in one or more other electronic devices (e.g., electronic devices 102, 104, or server 106). According to the present invention, when electronic device 101 is to perform a function or service automatically or on demand, electronic device 101 may perform at least some functions associated therewith instead of, or in addition to, (E.g., electronic device 102, 104, or server 106) may request the other device (e.g., electronic device 102, 104, or server 106) Perform additional functions, and forward the results to the electronic device 101. The electronic device 101 may process the received results as is or additionally to provide the requested functionality or services. For example, Cloud computing, distributed computing, or client-server computing techniques can be used.

2 is a block diagram illustrating the configuration of an electronic device 201 according to various embodiments.

Referring to FIG. 2, the electronic device 201 may include all or part of the electronic device 101 shown in FIG. 1, for example. The electronic device 201 includes one or more processors (e.g., AP) 210, a communication module 220, a memory 230, a sensor module 240, an input device 250, a display 260, an interface 270, An audio module 280, a camera module 291, a power management module 295, a battery 296, an indicator 297, and a motor 298.

The processor 210 may control a plurality of hardware or software components connected to the processor 210, for example, by driving an operating system or an application program, and may perform various data processing and calculations. The processor 210 may be implemented with, for example, a system on chip (SoC). According to one embodiment, the processor 210 may further include a graphics processing unit (GPU) and / or an image signal processor. Processor 210 may include at least some of the components shown in FIG. 2 (e.g., cellular module 221). Processor 210 may load instructions and / or data received from at least one of the other components (e.g., non-volatile memory) into volatile memory) and process the resultant data in non-volatile memory.

The processor 210 in accordance with various embodiments may recognize a face region based on feature points extracted from the image, set at least one object within the face region, and use the at least one object to determine the in- The face area is divided based on the reference line, the correction ratio for each divided area is calculated, the face area is corrected according to the calculated correction ratio, and the corrected image is displayed on the display 260 . ≪ / RTI >

The communication module 220 may have the same or similar configuration as the communication interface 170 of FIG. The communication module 220 may include, for example, a cellular module 221, a WiFi module 223, a Bluetooth module 225, a GNSS module 227, an NFC module 228 and an RF module 229 have. The cellular module 221 can provide voice calls, video calls, text services, or Internet services, for example, over a communication network. According to one embodiment, the cellular module 221 may utilize a subscriber identity module (e.g., a SIM card) 224 to perform the identification and authentication of the electronic device 201 within the communication network. According to one embodiment, the cellular module 221 may perform at least some of the functions that the processor 210 may provide. According to one embodiment, the cellular module 221 may comprise a communications processor (CP).

At least some (e.g., two or more) of the cellular module 221, the WiFi module 223, the Bluetooth module 225, the GNSS module 227, or the NFC module 228, according to some embodiments, (IC) or an IC package. The RF module 229 can, for example, send and receive communication signals (e.g., RF signals). The RF module 229 may include, for example, a transceiver, a power amplifier module (PAM), a frequency filter, a low noise amplifier (LNA), or an antenna. According to another embodiment, at least one of the cellular module 221, the WiFi module 223, the Bluetooth module 225, the GNSS module 227, or the NFC module 228 transmits / receives an RF signal through a separate RF module . The subscriber identification module 224 may include, for example, a card or an embedded SIM containing a subscriber identity module, and may include unique identification information (e.g., ICCID) or subscriber information (e.g., IMSI (international mobile subscriber identity).

Memory 230 (e.g., memory 130) may include, for example, internal memory 232 or external memory 234. Volatile memory (e.g., a DRAM, an SRAM, or an SDRAM), a non-volatile memory (e.g., an OTPROM, a PROM, an EPROM, an EEPROM, a mask ROM, a flash ROM , A flash memory, a hard drive, or a solid state drive (SSD). The external memory 234 may be a flash drive, for example, a compact flash (CF) ), Micro-SD, Mini-SD, extreme digital (xD), multi-media card (MMC), or memory stick, etc. External memory 234 may communicate with electronic device 201, Or may be physically connected.

The sensor module 240 may, for example, measure a physical quantity or sense the operating state of the electronic device 201 to convert the measured or sensed information into an electrical signal. The sensor module 240 includes a gesture sensor 240A, a gyro sensor 240B, an air pressure sensor 240C, a magnetic sensor 240D, an acceleration sensor 240E, a grip sensor 240F, A temperature sensor 240G, a UV sensor 240G, a color sensor 240H (e.g., an RGB (red, green, blue) sensor), a living body sensor 240I, And a sensor 240M. Additionally or alternatively, the sensor module 240 may be configured to perform various functions such as, for example, an e-nose sensor, an electromyography (EMG) sensor, an electroencephalograph (EEG) sensor, an electrocardiogram An infrared (IR) sensor, an iris sensor, and / or a fingerprint sensor. The sensor module 240 may further include a control circuit for controlling at least one or more sensors belonging to the sensor module 240. In some embodiments, the electronic device 201 further includes a processor configured to control the sensor module 240, either as part of the processor 210 or separately, so that while the processor 210 is in a sleep state, The sensor module 240 can be controlled.

The input device 250 may include, for example, a touch panel 252, a (digital) pen sensor 254, a key 256, or an ultrasonic input device 258. As the touch panel 252, for example, at least one of an electrostatic type, a pressure sensitive type, an infrared type, and an ultrasonic type can be used. Further, the touch panel 252 may further include a control circuit. The touch panel 252 may further include a tactile layer to provide a tactile response to the user. (Digital) pen sensor 254 may be part of, for example, a touch panel or may include a separate recognition sheet. Key 256 may include, for example, a physical button, an optical key, or a keypad. The ultrasonic input device 258 can sense the ultrasonic wave generated by the input tool through the microphone (e.g., the microphone 288) and confirm the data corresponding to the ultrasonic wave detected.

Display 260 (e.g., display 160) may include panel 262, hologram device 264, projector 266, and / or control circuitry for controlling them. The panel 262 may be embodied, for example, flexibly, transparently, or wearably. The panel 262 may comprise a touch panel 252 and one or more modules. According to one embodiment, the panel 262 may include a pressure sensor (or force sensor) capable of measuring the intensity of the pressure on the user's touch. The pressure sensor may be integrated with the touch panel 252 or may be implemented by one or more sensors separate from the touch panel 252.

The hologram device 264 can display a stereoscopic image in the air using interference of light. The projector 266 can display an image by projecting light onto a screen. The screen may be located, for example, inside or outside the electronic device 201. The interface 270 may include, for example, an HDMI 272, a USB 274, an optical interface 276, or a D-sub (D-subminiature) 278. The interface 270 may, for example, be included in the communication interface 170 shown in FIG. Additionally or alternatively, the interface 270 may include, for example, a mobile high-definition link (MHL) interface, an SD card / multi-media card (MMC) interface, or an infrared data association have.

The audio module 280 can, for example, convert sound and electrical signals in both directions. At least some of the components of the audio module 280 may be included, for example, in the input / output interface 145 shown in FIG. The audio module 280 may process sound information input or output through, for example, a speaker 282, a receiver 284, an earphone 286, a microphone 288, or the like. The camera module 291 is, for example, a device capable of capturing still images and moving images, and according to one embodiment, one or more image sensors (e.g., a front sensor or a rear sensor), a lens, an image signal processor (ISP) , Or flash (e.g., an LED or xenon lamp, etc.).

The power management module 295 can, for example, manage the power of the electronic device 201. [ According to one embodiment, the power management module 295 may include a power management integrated circuit (PMIC), a charging IC, or a battery or fuel gauge. The PMIC may have a wired and / or wireless charging scheme. The wireless charging scheme may include, for example, a magnetic resonance scheme, a magnetic induction scheme, or an electromagnetic wave scheme, and may further include an additional circuit for wireless charging, for example, a coil loop, a resonant circuit, have. The battery gauge can measure, for example, the remaining amount of the battery 296, the voltage during charging, the current, or the temperature. The battery 296 may include, for example, a rechargeable battery and / or a solar cell.

The indicator 297 may indicate a particular state of the electronic device 201 or a portion thereof (e.g., processor 210), e.g., a boot state, a message state, or a state of charge. The motor 298 can convert the electrical signal to mechanical vibration, and can generate vibration, haptic effects, and the like. Electronic device 201 is, for example, DMB Mobile TV-enabled devices capable of handling media data in accordance with standards such as (digital multimedia broadcasting), DVB (digital video broadcasting), or MediaFLO (mediaFlo ^TM) (for example, : GPU). Each of the components described in this document may be composed of one or more components, and the name of the component may be changed according to the type of the electronic device. In various embodiments, an electronic device (e. G., Electronic device 201) may have some components omitted, further include additional components, or some of the components may be combined into one entity, The functions of the preceding components can be performed in the same manner.

3 is a block diagram of a program module according to various embodiments.

According to one embodiment, program module 310 (e.g., program 140) includes an operating system that controls resources associated with an electronic device (e.g., electronic device 101) and / E.g., an application program 147). The operating system may include, for example, Android ^TM , iOS ^TM , Windows ^TM , Symbian ^TM , Tizen ^TM , or Bada ^TM . 3, program module 310 includes a kernel 320 (e.g., kernel 141), middleware 330 (e.g., middleware 143), API 360 (e.g., API 145) ), And / or an application 370 (e.g., an application program 147). At least a portion of the program module 310 may be preloaded on an electronic device, 102 and 104, a server 106, and the like).

The kernel 320 may include, for example, a system resource manager 321 and / or a device driver 323. The system resource manager 321 can perform control, allocation, or recovery of system resources. According to one embodiment, the system resource manager 321 may include a process manager, a memory manager, or a file system manager. The device driver 323 may include, for example, a display driver, a camera driver, a Bluetooth driver, a shared memory driver, a USB driver, a keypad driver, a WiFi driver, an audio driver, or an inter-process communication .

The middleware 330 may provide various functions through the API 360, for example, to provide functions that are commonly needed by the application 370 or allow the application 370 to use limited system resources within the electronic device. Application 370 as shown in FIG. According to one embodiment, the middleware 330 includes a runtime library 335, an application manager 341, a window manager 342, a multimedia manager 343, a resource manager 344, a power manager 345, a database manager The location manager 350, the graphic manager 351, or the security manager 352. In this case, the service manager 341 may be a service manager, a service manager, a service manager, a package manager 346, a package manager 347, a connectivity manager 348, a notification manager 349,

The runtime library 335 may include, for example, a library module that the compiler uses to add new functionality via a programming language while the application 370 is executing. The runtime library 335 may perform input / output management, memory management, or arithmetic function processing. The application manager 341 can manage the life cycle of the application 370, for example. The window manager 342 can manage GUI resources used in the screen. The multimedia manager 343 can recognize the format required for reproducing the media files and can perform encoding or decoding of the media file using a codec according to the format.

The resource manager 344 can manage the source code of the application 370 or the space of the memory. The power manager 345 may, for example, manage the capacity or power of the battery and provide the power information necessary for operation of the electronic device. According to one embodiment, the power manager 345 may interoperate with a basic input / output system (BIOS). The database manager 346 may create, retrieve, or modify the database to be used in the application 370, for example. The package manager 347 can manage installation or update of an application distributed in the form of a package file.

The connectivity manager 348 may, for example, manage the wireless connection. The notification manager 349 may provide the user with an event such as, for example, an arrival message, an appointment, a proximity notification, and the like. The location manager 350 can manage the location information of the electronic device, for example. The graphic manager 351 may, for example, manage the graphical effects to be presented to the user or a user interface associated therewith. Security manager 352 may provide, for example, system security or user authentication.

According to one embodiment, the middleware 330 may include a telephony manager for managing the voice or video call function of the electronic device, or a middleware module capable of forming a combination of the functions of the above-described components . According to one embodiment, the middleware 330 may provide a module specialized for each type of operating system. Middleware 330 may dynamically delete some existing components or add new ones. The API 360 may be provided in a different configuration depending on the operating system, for example, as a set of API programming functions. For example, for Android or iOS, you can provide a single API set for each platform, and for Tizen, you can provide two or more API sets for each platform.

The application 370 may include a home 371, a dialer 372, an SMS / MMS 373, an instant message 374, a browser 375, a camera 376, an alarm 377, Contact 378, voice dial 379, email 380, calendar 381, media player 382, album 383, watch 384, healthcare (e.g., measuring exercise or blood glucose) , Or environmental information (e.g., air pressure, humidity, or temperature information) application. According to one embodiment, the application 370 may include an information exchange application capable of supporting the exchange of information between the electronic device and the external electronic device. The information exchange application may include, for example, a notification relay application for communicating specific information to an external electronic device, or a device management application for managing an external electronic device.

For example, the notification delivery application can transmit notification information generated in another application of the electronic device to the external electronic device, or receive notification information from the external electronic device and provide the notification information to the user. The device management application may, for example, control the turn-on / turn-off or brightness (or resolution) of an external electronic device in communication with the electronic device (e.g., the external electronic device itself Control), or install, delete, or update an application running on an external electronic device. According to one embodiment, the application 370 may include an application (e.g., a healthcare application of a mobile medical device) designated according to the attributes of the external electronic device. According to one embodiment, the application 370 may include an application received from an external electronic device. At least some of the program modules 310 may be implemented (e.g., executed) in software, firmware, hardware (e.g., processor 210), or a combination of at least two of the same, Program, routine, instruction set or process.

As used herein, the term "module " includes units comprised of hardware, software, or firmware and may be used interchangeably with terms such as, for example, logic, logic blocks, components, or circuits. A "module" may be an integrally constructed component or a minimum unit or part thereof that performs one or more functions. "Module" may be implemented either mechanically or electronically, for example, by application-specific integrated circuit (ASIC) chips, field-programmable gate arrays (FPGAs) And may include programmable logic devices. At least some of the devices (e.g., modules or functions thereof) or methods (e.g., operations) according to various embodiments may be stored in a computer readable storage medium (e.g., memory 130) . ≪ / RTI > When the instruction is executed by a processor (e.g., processor 120), the processor may perform a function corresponding to the instruction.

An electronic device 201 in accordance with various embodiments includes a memory 230, a display 260 and a processor 210 that is operatively coupled to the memory 230 or the display 260, Recognizing a face region based on feature points extracted from the image, setting one or more objects in the face region, setting a reference line in the face region using the one or more objects, Calculate the correction ratio for each of the divided face regions, correct the face region according to the calculated correction ratio, and display the corrected image on the display (260).

The face region is recognized as an ellipse, the reference line is set as a line connecting the elliptical inner axis, and the processor 210 may be configured to determine a slope of the face region based at least on the reference line.

The processor 210 calculates a correction ratio in a major axis direction of the face region based at least on a slope of the face region and calculates a correction ratio in a direction of a short axis of the face region based at least on a slope of the face region Can be set.

The processor 210 may be configured to variably calculate the correction ratio of the divided face regions based on at least the total ratio of the face regions.

The processor 210 forms an elliptical region corresponding to the eye object among the one or more objects, calculates an enlargement ratio based on the gradient of the elliptical region at least based on the calculated enlargement ratio, As shown in FIG.

The processor 210 may be configured to set a reference line connecting the long axes of the elliptical region, determine a slope of the set reference line, and calculate an enlargement ratio based at least on the slope of the reference line.

The processor 210 may be configured to detect a pupil region in the ellipse region and to calculate an enlargement ratio for the pupil region based at least on the size, tilt direction, or position of the pupil region.

The processor 210 may be configured to correct the eye object by applying different magnification ratios to the ellipse region and the pupil region.

The processor (210) is configured to set a correction parameter using information associated with the image, determine a correction priority based on the set correction parameter, and perform a global correction or a local correction based on the correction priority As shown in FIG.

Wherein the processor (210) performs a first process for the correction for the face region using at least one of the global correction or the local correction based at least on the correction parameter, and wherein the global correction or the regional And to perform a second process for the correction on the first processed face area using the remaining one of the correction.

The processor 210 may be configured to perform processing for the global correction when the reflected light is detected in the face region of the image.

An electronic device 201 in accordance with various embodiments includes a camera (e.g., a camera module 291) and a processor 210 that uses the camera 291 to create a face Generating a plurality of guide masks associated with the face based at least in part on a detection method for the face and obtaining at least an image based on at least a portion of the plurality of guide masks, To determine an image processing method for the skin, and to at least partially correct the skin based on the determined image processing method.

The processor 210 may use the guide mask for contour detection for an area corresponding to a plurality of objects constituting the face among at least a part of the image corresponding to the corrected face, The skin area including contours of the objects of the subject.

Wherein the processor (210) is further configured to determine a contour of the plurality of objects by using a filter for an area corresponding to a plurality of objects constituting the face among at least a part of the image corresponding to the corrected face May be set to recognize a skin area having a low probability of being corresponding to the skin area.

The processor 210 may change the combination rule of the plurality of guide masks based on at least the correction type for the image, and change the probability of corresponding to a plurality of objects constituting the face based at least on the changed combination rule And to determine an image processing method for the skin, based at least in part on the determined probability.

4 is a flow chart illustrating a method of operation of an electronic device according to various embodiments.

4, at operation 401, electronic device 201 (e.g., processor 210) may obtain an image (or image). The image may be a preview image obtained in real time from a camera module (e.g., camera module 291) of the electronic device 201 or an image stored in a memory (e.g., memory 230). For example, the processor 210 may execute a camera application at the user's choice, acquire an image from the camera module 291 by the executed camera application and display it on the display (e.g., display 260) of the electronic device 201, ). ≪ / RTI > Alternatively, the processor 210 executes a gallery application (e.g., a photo application) according to the user's selection, selects at least one image from the user in the image list displayed through the executed gallery application, Can be displayed.

The act of acquiring the image may include displaying the image on the display 260. Here, the obtained image may correspond to a portrait including a face. The portrait may include at least one of a face, a face and neck, or an upper body (e.g., a portion from the face to the waist). According to various embodiments, the user may select a correction type for the displayed image when performing an image acquisition operation. The type of correction may be to select which properties (e.g., brightness, color reversal, or color change) the image should be corrected for.

At operation 403, electronic device 201 (e.g., processor 210) may extract feature points from the image. The minutiae can be called 'feature point', 'key point', and 'interesting point' in English. The feature point may be a method used for tracking or recognizing an object. The processor 210 may recognize the face region from the acquired image or extract feature points from the acquired image to analyze objects in the face region (e.g., eye, nose, mouth, face contour). Since the method of extracting feature points from an image corresponds to the conventional art, a detailed description can be omitted.

At operation 405, the electronic device 201 (e.g., processor 210) may recognize the face region based on the extracted feature points. For example, the processor 210 forms an ellipse region using the extracted feature points, and recognizes a portion corresponding to a large ellipse region of the formed ellipse region as a face region. That is, the processor 210 may recognize the face area as an ellipse. For example, the processor 210 may form a large elliptical region by connecting a plurality of minutiae located at the greatest distance from a minutiae located at the very center of the extracted minutiae (e.g., nose minutiae) to one line. Alternatively, the processor 210 may recognize an area corresponding to a small elliptical area among the extracted minutiae as an eye area or a mouth area. The processor 210 may perform an image correction processing operation only within the recognized face area.

At operation 407, the electronic device 201 (e.g., processor 210) may set (or determine) an object in the recognized face area. The object refers to an element constituting a face, and may be at least one of eyebrow, eye, nose, or mouth, for example. The processor 210 can set an object in the face region using the characteristics of the elements constituting the face. Processor 210 may analyze at least one of the attributes (e.g., eyebrows, eyes, nose, mouth), shape (e.g., shape), or position of the object using feature points located within the face region.

For example, the processor 210 may set two types or regions, which are located side by side in the face region, as browse objects using the extracted feature points. Alternatively, the processor 210 may set the two elliptical regions, which are located side by side in the face region, as eye objects using the extracted feature points. If the face is inclined to either side, the eye objects may not be positioned side by side in the same horizontal direction. When two small elliptical regions in the face region are detected, the processor 210 can set eye objects for each elliptical region using the feature points. Alternatively, the processor 210 may set the nose object using feature points in the form of a bar located between the two eye objects below the eye object. Alternatively, the processor 210 may set the mouth object using the minutiae which are located at the lower end of the eyes and nose objects and form a certain region.

In operation 409, the electronic device 201 (e.g., processor 210) may correct the face area using the set object. The processor 210 according to various embodiments may perform at least one of a correction region detection method using a guide mask, a face shape correction method using a gradient, and a face region correction method using an area. For example, the correction region (e.g., skin region) detection method may be a preprocessing process for correcting the face region, for example, to detect an area (or range) to be corrected. Although the face region is recognized at operation 405, the processor 210 may more accurately detect the face region using the correction region detection method. Alternatively, the processor 210 may determine whether there is a portion (e.g., a neck) that needs correction in addition to the face region using the correction region detection method. The part that needs to be corrected may correspond to the skin of the user.

The processor 210 according to various embodiments may perform the face shape correction method and / or the face area correction method after performing the correction region detection method. For example, when the inclination is detected in the detected correction region, the processor 210 may correct the face shape using the face shape correction method. Alternatively, when the inclination is detected in the detected correction region, the processor 210 may perform the face region correction method after correcting the face shape. Alternatively, the processor 210 may perform the face area correction method without performing the face shape correction method when the inclination is not detected in the detected correction area.

At operation 411, the electronic device 201 (e.g., processor 210) may display the corrected image. The processor 120 may perform operations 403 to 409 to correct the image obtained in operation 401 and then display the corrected image on the display 260. [ The user can view the displayed image via the display 260 and optionally store the displayed image. The processor 210 may store the displayed image in the memory 230 if there is a storage request (e.g., a save button selection) from the user.

5 is a flowchart illustrating a face shape correction method according to various embodiments. The flow chart of FIG. 5 details operation 409 of FIG.

Referring to FIG. 5, at operation 501, the electronic device 201 (e.g., processor 210) may determine a reference object. The processor 210 may determine a reference object as a reference object in order to determine a tilt or a rotation angle of a face. The reference object may be at least one object constituting a face, or may be generated using the object. For example, the reference object may be a center point between two eyebrows (e.g., the tail) and a coil. Alternatively, the reference object may be a center point between the eyes and a center point of the mouth. The processor 210 may determine various points in the face region as reference objects to determine the inclination of the face.

At operation 503, the electronic device 201 (e.g., processor 210) may establish a baseline connecting the reference objects. Since the human face is elliptical, the processor 210 can recognize the face area as an ellipse. The processor 210 may set a line connecting the long axis in the ellipse which is the face area as a reference line. The long axis may correspond to a long axis among the line segments connecting both end points of the ellipse. For example, in a human face, the long axis may mean a vertical or vertical direction from the forehead to the jaw. For example, the baseline may be a line extending from the forehead to the middle point between the eyebrows and the tip of the nose. Alternatively, the baseline may be a line extending from the forehead to the midpoint between the eyebrows and the midpoint of the mouth.

If the face of the person is inclined to the left or right rather than the vertical direction perpendicular to the horizon, the reference line may have a slope having a constant angle with respect to a vertical line perpendicular to the horizon. The processor 210 according to various embodiments may set a baseline connecting at least two reference objects in various ways to determine facial tilt. Most of the face of the person is formed in an elliptical shape. When recognizing it as a shape other than an elliptical shape (e.g., a rectangle), a non-face area may be included in a rectangle (e.g. Thus, the processor 210 may recognize the face region as an ellipse using feature points in the image.

At operation 505, the electronic device 201 (e.g., the processor 210) may partition the face area based on the set baseline. For example, the processor 210 may segment the face region from a location above the face (e.g., forehead) to below (e.g., jaw) relative to a baseline. The processor 210 may divide the face region into a first region including a forehead and eye, a second region including a nose and mouth, and a third region including a jaw portion. Alternatively, the processor 210 may include a first region including the forehead, a second region including the eye, a third region including the nose, a fourth region including the mouth, and a fifth region including the jaw portion Can be divided. The method of dividing the area can be variously implemented according to the embodiment, and the present invention is not limited by the following description.

At operation 507, the electronic device 201 (e.g., processor 210) may calculate the segmented per-region correction rate. The face shape correction may mean reducing the contour of the face. The processor 210 may perform correction to reduce the contour of the face in consideration of the overall face ratio. For example, the processor 210 may convert the face region into three regions in the major axis direction (e.g., a first region including forehead and eye, a second region including nose and mouth, a third region including a jaw portion) In the case of division, the correction ratio for the first area, the correction ratio for the second area, and the correction ratio for the third area can be respectively calculated. For example, since the processor 210 does not perform the zoom correction on the head portion, the correction ratio can be calculated so that the first region including the forehead and the eye is smoothly reduced from the forehead to the eye for natural correction. For example, processor 210 may calculate such that the correction ratio for the forehead portion is small (e.g., 0.9), and the correction ratio is increased toward the end of the eye (e.g., 0.5).

In addition, since the processor 210 reduces the amount of correction from the eye portion, the correction ratio (e.g., 0.5) can be uniformly calculated for the second region including the nose and mouth. Alternatively, the processor 210 may calculate the second region so as to have a different correction ratio for each region, by dividing the region including the nose portion and the region including the mouth portion. For example, the processor 210 may calculate a correction ratio for the region including the nose portion to 0.5 and a correction ratio for the region including the mouth portion to 0.6. In addition, since the processor 210 does not perform the reduction correction for the lower jaw portion, the correction ratio can be gently calculated in the downward direction for the third region including the jaw portion. For example, the processor 210 may calculate the correction rate to be smaller (e.g., 0.9) toward the tip of the jaw such that the lower portion of the mouth has a larger correction ratio (e.g., 0.5). The processor 210 according to various embodiments can correct face contours more naturally by applying different correction ratios by dividing the facial contours in the major axis direction by the same ratio without dividing the same.

The processor 210 according to various embodiments may calculate the face contour proportionally to the top / bottom (e.g., major axis direction) and left / right (e.g., minor axis direction) of the face in consideration of the face tilt Can be corrected. The short axis may correspond to a short axis among the line segments connecting both end points of the ellipse. The short axis in a person's face shape may mean a horizontal direction or a horizontal direction leading from the left eye (or right eye) to the right eye (or left eye). For example, the processor 210 may calculate the correction ratio in the major axis direction of the ellipse in consideration of the face shape of the elliptical person, and calculate the correction ratio in the minor axis direction of the ellipse. Conventionally, if the facial contour is corrected to be reduced only in the short axis direction, the present embodiment can solve the disadvantage that the face is lengthened due to the correction by correcting the contour of the face even in the long axis direction. The processor 210 according to various embodiments may set the long axis direction correction ratio to 0.2 times the short axis direction correction ratio in consideration of the fact that the face of the person is elliptical. That is, the processor 210 may calculate the short axis direction correction ratio and then multiply the short axis direction correction ratio by 0.2 to calculate the long axis direction correction ratio. The processor 210 according to various embodiments can determine whether the long axis direction correction ratio is set to several times the short axis direction correction ratio in consideration of the face ratio of the user. For example, in the case of a person with a long face, it may be set to be higher than 0.2 times, and in the case of a person whose face is round, it may be set to be lower than 0.2 times. It is to be understood that the present invention is not limited thereto.

The processor 210 according to various embodiments may determine the correction rate according to the type of face type. For example, if the face type is a horizontally wide type, the processor 210 may increase the short axis direction correction ratio to the short axis direction average correction ratio. Alternatively, when the face type is a vertically long type, the processor 210 may increase the long axis direction correction ratio to be higher than the long axis direction average correction ratio. This enables the processor 210 to learn the percentage of people preferred or to use the big data to determine the correction rate. Alternatively, the processor 210 may determine the correction factor in consideration of the user characteristics of the electronic device 201. [ The processor 210 may analyze the type corresponding to the face type of the user in consideration of the user characteristics and calculate a correction ratio corresponding to the analyzed type.

In operation 509, the electronic device 201 (e.g., processor 210) may correct the face shape according to the calculated correction ratio. The processor 210 can correct the face shape by applying different correction ratios for each divided region. Alternatively, the processor 210 may correct the face shape by applying the long axis direction correction ratio and the short axis direction correction ratio for each divided region, respectively. The processor 210 may correct the face shape by moving the corresponding pixel in accordance with the correction ratio. For example, the processor 210 may fill in the external pixels of the face region by the reduced portion when reducing the face to left and right. Alternatively, the processor 210 may enlarge the face to the left and right and, when zooming up, fill the face boundary with pixels included in the face region by the enlarged portion and fill it with the external pixels of the face region by the reduced portion have.

6 is a diagram illustrating an example of extracting feature points from an image according to various embodiments.

Referring to FIG. 6, an electronic device 201 (e.g., processor 210) may extract feature points from an image. The processor 210 may divide the image into small portions and extract the feature points using the boundary values of the divided regions. For example, the processor 120 may extract, as minutiae, regions that differ from neighboring regions of the divided regions. The processor 210 can recognize the face region 601 following the feature points corresponding to the face outline among the extracted feature points. The processor 210 can predict an area corresponding to the eyes, nose, and mouth in the face area 601 and determine the object using the extracted feature points in the predicted area. Here, the object may include eyebrows, left eye 603-1, right eye 603-2, nose 605, and mouth 607. [ Thus, the processor 210 may obtain at least one of the attributes (e.g., eyebrows, eyes, nose, mouth), shape (e.g., shape) or position of the object using the feature points located in the face region 601 .

7 is a diagram illustrating an example of determining a reference line of a face according to various embodiments.

Referring to FIG. 7, the first reference line 710 may be determined by segmenting a line segment from the top to the bottom of the face region. Since the first reference line 710 is determined in the vertical direction of the image irrespective of the inclination (e.g., the rotation angle) of the face region, the face contour is reduced and corrected so that the left and right sides are symmetrical with respect to the first reference line 710 If the ratio of both sides of the face does not match, the face may look unnatural. For example, if the face contour is corrected by reducing the first contour line based on the first reference line 710, the right contour of the face may be corrected more than the left contour, so that the face may not be symmetrically viewed. The method of correcting the face based on the first reference line 710 may correspond to the prior art.

Electronic device 201 (e.g., processor 210) in accordance with various embodiments may determine a second reference line 720, a third reference line 730, or a fourth reference line 740 using feature points in the face region. have.

The second reference line 720 may be determined by segmenting the line segment from the top to the bottom of the face area in the same manner as the first reference line 710. However, the second reference line 720 divides the face area in left and right symmetry. When the contour of the face is corrected so that the left and right sides are symmetrical with respect to the second reference line 720, Can be similarly reduced and corrected. That is, the processor 210 can correct the left and right face contours in the horizontal direction perpendicular to the second reference line 720. In addition, the processor 210 may correct the contour of the face symmetrically left and right, and calculate the up / down correction ratio based on the left / right correction ratio. For example, since the face 210 can be seen to be long when the face contour is corrected in the left / right symmetrical manner, the processor 210 can correct the contour of the entire face by reducing the contour of the lower jaw. In other words, the processor 210 can reduce and correct the contour of the face in the left / right horizontal direction and reduce the lower jaw portion in the upward direction (e.g., the upward direction arrow) perpendicular to the left / right horizontal direction .

The third reference line 730 may be determined by drawing a line segment from the top to the bottom of the face area so as to be symmetrical to the left and right of the face area. That is, the processor 210 may set (determine) the third reference line 730 so as to be symmetric with respect to the face area using the reference objects of the face area. The processor 210 according to various embodiments may correct the contour of the face in the left / right horizontal direction with respect to the third reference line 730. In addition, the processor 210 corrects the contour of the face in the left / right horizontal direction and corrects the correction ratio in the up / down direction perpendicular to the left / right horizontal direction based on the correction ratio in the left / Can be calculated. The processor 210 corrects the contour of the face in the left / right horizontal direction and corrects the lower jaw portion in an upward direction (e.g., an upward direction arrow) perpendicular to the left / right horizontal direction.

The fourth reference line 740 may be determined by drawing a line segment from the top to the bottom of the face area so as to be symmetrical to the left and right of the face area. The processor 210 according to various embodiments may correct left and right facial contours in a direction perpendicular to the fourth reference line 740. [ In this case, when the processor 210 corrects the facial contour, the facial contour may be corrected left and right symmetrically. In addition, the processor 210 corrects the contour of the face in the direction perpendicular to the fourth reference line 740 and corrects the upper / lower correction ratio based on the correction ratio in the direction perpendicular to the fourth reference line 740 Can be calculated. The processor 210 corrects the contour of the face in the direction perpendicular to the fourth reference line 740 and corrects the face contour in the upward direction (e.g., the fourth reference line 740) at the same slope (or angle) as the fourth reference line 740, And the lower jaw portion can be narrowed down by the upward direction arrow at the same angle as that of the lower jaw.

8 is a diagram illustrating an example of calculating a correction ratio for each face region according to various embodiments.

8, the electronic device 201 (e.g., processor 210) includes a face region 810 with a first region 811 including eyebrows and eyes, a second region 813 including noses and mouths, And a third region 815 including a jaw portion. At this time, the electronic device 201 (for example, the processor 210) divides the face area 810 in the horizontal direction perpendicular to the reference line 817, so that the face area 810 is divided into the first The region 811, the second region 813, and the third region 817 can be divided. The processor 210 according to various embodiments may perform correction to reduce the contour of the face in consideration of the overall face ratio. For example, since the processor 210 does not perform the reduction correction on the head portion, it is possible to calculate the first correction ratio 831 such that the first region 811 is smoothly reduced in the downward direction for natural correction have. In addition, the processor 210 can calculate the second correction ratio 833 so as to have a constant reduction ratio with respect to the second region 813 since the reduction processing is performed from the eye portion. In addition, the processor 210 can calculate the third correction ratio 835 gently in the downward direction with respect to the third region 815, since it does not perform the reduction correction on the lower jaw portion

Referring to the correction curve 830 in the minor axis direction, it can be said that the closer to 0, the higher the correction ratio, and the closer to 1, the lower the correction rate. The higher the correction rate, the more the face contour is reduced, which can affect the overall face ratio. In light of this, the processor 210 may determine a maximum value (e.g., 0.3) of the correction ratio between 0 and 1, and calculate a correction ratio for each region between 1 and the maximum value. The processor 210 may calculate the correction ratio linearly for the first area 811 and the third area 815. [ That is, the processor 210 determines whether the first correction ratio 831 for the first area 811, the second correction ratio 833 for the second area 813, the third correction for the third area 815, The ratio 835 can be determined differently.

The processor 210 may reduce the face area 810 from the lower side to the upper side with the same slope as the reference line 817 because the face may be seen to be long. The reduction direction of the same slope as the reference line 817 may be the major axis direction. The processor 210 according to various embodiments may set the correction ratio in the major axis direction to 0.2 times the minor axis correction ratio in consideration of the point that the face of the person is elliptical. For example, the processor 210 determines the first to third correction ratios 831 to 835 with reference to the correction curve 830 in the short axis direction, and then determines the first to third correction ratios 831 to 835, The fourth region 821, the fifth region 825, and the sixth region 825) by multiplying the ratio 835 by 0.2.

Referring to the correction graph 820 in the long axis direction, the closer to 0, the higher the correction ratio, and the closer to 1, the lower the correction rate. Processor 210 may determine a maximum value (e.g., 0.3) of the correction ratio between 0 and 1, and calculate a regional correction ratio between 1 and the maximum value. The processor 210 may make the correction ratio smaller toward the upper direction of the face area 810 and may calculate the correction rate higher toward the lower direction. For example, the processor 210 may calculate the correction ratio linearly for the fourth region 821 and the sixth region 825. [ The processor 210 determines a fourth correction ratio 821 for the first region 811, a fifth correction ratio 823 for the second region 813, and a sixth correction ratio 812 for the third region 815 825 may be determined differently.

9 is a flow chart illustrating an eye area correction method in accordance with various embodiments. The flowchart of FIG. 9 details operation 409 of FIG. The operation of FIG. 9 may be performed simultaneously with the operation of FIG. 5, or may be performed after or before the operation of FIG.

9, at operation 901, electronic device 201 (e.g., processor 210) may determine an elliptical region of the eye. In the face of a person, two eyes can be placed side by side (eg, horizontally). In consideration of this point, the processor 210 may form two elliptical regions, each of which is positioned in parallel with the feature points included in the face region.

At operation 903, the electronic device 201 (e.g., processor 210) may establish a reference line connecting the major axis of the elliptical region. Here, the long axis corresponds to a long axis among the line segments connecting the two end points of the ellipse. For example, the long axis may mean a horizontal line passing through the center point of the eye when the human eye is viewed from the front.

At operation 905, the electronic device 201 (e.g., processor 210) may determine the slope of the set baseline. If the face of the person is placed in the vertical direction perpendicular to the horizon, the slope of the reference line, which means the horizontal line of the eye, may not be detected. However, when the face of a person is inclined leftward or rightward (biased) in the image, the slope of the reference line can be detected because the horizontal line of the eye is also inclined. The processor 210 may detect a slope when the slope of the reference line has a predetermined angle with respect to the horizontal line. The processor 210 may detect a rotation angle corresponding to the slope of the reference line.

At operation 907, the electronic device 201 (e.g., processor 210) may calculate the magnification ratio according to the baseline slope. The eye correction may mean enlarging and correcting the area of the eye. The processor 210 can determine the enlargement direction of the eye region in consideration of the inclination of the eye and calculate the enlargement ratio according to the enlargement direction. Conventionally, even though the face of the person is inclined to one side and the direction of the eyes is also inclined to one side, there is a disadvantage that the enlargement correction of the eyes looks unnatural by correcting the area of the eyes based on the horizon. That is, conventionally, there is a difference between the position of the actual eye and the position of the corrected eye by enlarging and correcting the eye irrespective of the tilted direction of the eye. The processor 210 may apply different magnification ratios to the elliptical region corresponding to the eyes according to the slope of the reference line.

The processor 210 according to various embodiments may calculate the magnification ratio based on at least one of a magnitude, a gradient direction, or a position of a pupil (or dark brown eyes). The ellipse region may include a white portion corresponding to the background of the eye and a pupil corresponding to the center of the eye. Here, the elliptical region may mean a white region corresponding to the background of the eye. When the eye is enlarged and corrected based on the elliptical region (e.g., a white portion) of the eye, at least one of the size, tilt direction, or position of the pupil can be unnaturally corrected. Thus, the processor 210 may calculate the magnification ratio of the pupil region based on at least one of the magnitude, slope direction, or position of the pupil in the elliptical region of the eye, and apply the magnification ratio to enlarge the pupil region . For reference, the eye is oval, whereas the pupil is circular, so the pupil may look unnatural when the entire area of the eye (for example, the elliptical region and the pupil region) is enlarged at the same ratio. The processor 210 according to various embodiments may calculate the enlargement ratio of the pupil area and the enlargement ratio of the ellipse area (e.g., white part) differently. The processor 210 may calculate the enlargement ratio of the pupil area based on the slope of the reference line, the size of the pupil, and the position of the pupil. The processor 210 may calculate the enlargement ratio of the elliptical region based on the slope of the reference line and the size of the elliptical region.

At operation 909, the electronic device 201 (e.g., processor 210) may magnify the elliptical region based on the calculated magnification ratio. The processor 210 can correct the elliptical area to be larger at a constant enlargement ratio from the boundary of the elliptical area. In addition, the processor 210 can correct the pupil area so that the pupil area is enlarged at a constant enlargement ratio from the boundary of the pupil area. For example, processor 210 may fill an elliptical region of an external pixel (e.g., a skin region) with pixels (e.g., an egg white region) included in the elliptical region when enlarging the elliptical region. Alternatively, when enlarging the pupil region, the processor 210 may fill the pixels (e.g., the whiten region) included in the ellipse region with pixels (e.g., black, brown, green, or blue) included in the pupil region .

The processor 210 according to various embodiments may correct the eye object by applying different magnification ratios to the elliptical region and the pupil region. The processor 210 may magnify the pupil region based on at least one of a size, a gradient direction, or a position of the pupil. For example, if the pupil is biased left or right in the ellipse region, the processor 210 may magnify the pupil region with respect to the biased direction (e.g., tilting direction).

The processor 210 in accordance with various embodiments may correct the pupil region so that the pupil region is always centered in the ellipse region, based on the location of the pupil. When the eye is enlarged and corrected, the white portion of the eye also becomes large, but the pupil can also become large. The processor 210 can always correct the eyes such that the eyes of the user are directed to the front by allowing the position of the eyes to always be located at the center.

FIGS. 10A through 10C are diagrams illustrating an example of correcting an eye region based on a baseline slope in accordance with various embodiments.

10A is a diagram showing an example of correcting the eye region regardless of the slope of the reference line according to the related art.

Referring to FIG. 10A, the eye region 1010 may be inclined at an angle with respect to the reference line 1018 on the face region. Conventionally, the elliptical region 1015 of the actual eye is extracted using the first feature point 1011, the second feature point 1012, the third feature point 1013, or the fourth feature point 1014 among the extracted feature points in the face region. . Further, conventionally, a line that is parallel to the horizon line with respect to the center point 1017 included in the elliptical region 1015 of the actual eye can be determined as the reference line 1018 of the eye. As a result, the ellipse region 1015-1 of the eye used for eye correction can be determined on the basis of the reference line 1018 regardless of the feature point. That is, conventionally, a line connecting the second feature point 1012 and the fourth feature point 1014 of the actual eye ellipse region 1015 is not set as a reference line, and a parallel line passing through the eye center point 1017 is set as a reference line 1018 ), It is possible to enlarge and correct the eyes without considering the inclination of the eyes.

As a result, an enlarged elliptical area 1015-2 can be obtained by applying the enlargement ratio to the elliptical area 1015-1 of the eye based on the center point 1017 and the reference line 1018 of the eye. Referring to the enlargement ratio graph 1020, a constant enlargement ratio 1023 is applied to the elliptical region 1015-1 of the eye, and smooth enlargement ratios 1021, 1025) can be applied. As a result, the enlarged elliptical region 1015-2 may be different from the actual eye region 1015 in its position and shape. In addition, the enlarged pupil region 1016-1 may differ from the actual pupil region 1016 in its position and shape.

FIG. 10B shows an example of correcting the eye region when there is no inclination of the eyes according to the present invention.

10B, the electronic device 201 (e.g., the processor 210) may include a first feature point 1031, a second feature point 1032, a third feature point 1033, The eye object 1030 can be detected using the fourth feature point 1034. [ The processor 210 uses the first feature point 1031, the second feature point 1032, the third feature point 1033 or the fourth feature point 1034 to calculate the elliptical region 1035 corresponding to the eye object 1030 . The processor 210 may set the reference line 1038 connecting the second feature point 1032 and the fourth feature point 1034 to the long axis of the elliptical region 1035 with a line. For example, the processor 210 may convert a line from one feature point (e.g., second feature point 1032) past the midpoint 1037 to another feature point (e.g., the fourth feature point 1034) 1038).

The processor 210 may determine the slope of the reference line 1038 based on whether the reference line 1038 is parallel to the horizon. The processor 210 may determine that the eye region is parallel to the horizon and correct the elliptical region 1035 if no tilt is detected at the reference line 1038. [ For example, referring to the enlargement ratio graph 1040, the processor 210 applies a constant enlargement ratio 1043 to the elliptical region 1035 and calculates a smooth enlargement ratio 1041 from the both ends of the long axis of the elliptical region 1035 , 1045) can be applied. As a result, the enlarged elliptical region 1035-2 is similar in position and shape to the elliptical region 1035 of the actual eye, and can be largely corrected in size. In addition, the processor 210 may apply a constant magnification ratio 1043 to the pupil region 1036 included in the elliptical region 1035. In this case, the enlarged pupil region 1036-1 is similar in position and shape to the pupil region 1036 of the actual eye, and can be largely corrected only in size.

FIG. 10C shows an example of correcting the eye region when there is an inclination of the eye according to the present invention.

10C, the electronic device 201 (e.g., the processor 210) may include a first feature point 1051, a second feature point 1052, a third feature point 1053, The eye object 1050 can be detected using the fourth feature point 1054. [ The processor 210 uses the first feature point 1051, the second feature point 1052, the third feature point 1053 or the fourth feature point 1054 to calculate the elliptical region 1055 corresponding to the eye object 1050 . The processor 210 can set a reference line 1058 connecting the second feature point 1052 and the fourth feature point 1054 to the long axis of the elliptical region 1055 with a line. For example, the processor 210 may generate a line extending from a feature point (e.g., second feature point 1052) through a central point 1057 to another feature point (e.g., fourth feature point 1054) 1058).

The processor 210 may determine the slope of the reference line 1058 based on whether the reference line 1058 is parallel to the horizon. The processor 210 may correct the elliptical region 1055 based on the slope of the reference line 1058 when a tilt is detected at the reference line 1058. [ For example, referring to the magnification ratio graph 1060, the processor 210 applies a constant magnification ratio 1063 to the elliptical region 1055 corresponding to the slope of the reference line 1058, Gentle expansion ratios 1061 and 1065 can be applied from both ends of the major axis. As a result, the enlarged elliptical region 1055-2 is similar in position and shape to the elliptical region 1055 of the actual eye, and can be largely corrected only in size. The processor 210 may also apply a constant magnification ratio 1053 to the pupil region 1056 included in the elliptical region 1055. [ In this case, the enlarged pupil region 1056-1 is similar in position and shape to the pupil region 1056 of the actual eye, and can be largely corrected only in size.

10A and 10C, it can be seen that there is a difference in position, shape, or size between the elliptical region of the actual eye and the elliptical region of the corrected eye.

11 is a flowchart illustrating a correction area detection method using a guide mask according to various embodiments. The flowchart of Fig. 11 can be performed before the operations of Figs. 5 and 9. Fig.

Referring to FIG. 11, at operation 1101, electronic device 201 (e.g., processor 210) may generate at least one guide mask. For example, the processor 210 may generate an object guide mask for detecting an object using feature points in the face region. The processor 210 may generate a shape guide mask for detecting a face shape using feature points in the face region. The processor 210 may generate an edge guide mask for detecting an area distinguished from the surrounding area by using the feature points in the face area. The processor 210 may generate a background guide mask for detecting the background of the image based on the color information constituting the image. The processor 210 may generate a skin guide mask for detecting skin in the face region based on color information corresponding to the skin of the face. In addition, the processor 210 may generate various guide masks using various detection techniques.

At operation 1103, the electronic device 201 (e.g., processor 210) may determine the type of correction. The correction type may be determined depending on what is the attribute of the image to be corrected or what the correction attribute is. For example, if the image acquired in operation 401 is a preview image obtained in real time from a camera module (e.g., camera module 291) of electronic device 201, The correction type can be determined according to whether the image is stored in the memory unit. In addition, when the user requests the image correction, it may be different depending on what the attribute that is desired to be corrected is selected. For example, if you choose to compensate the image for 'brightness', choose to compensate the image for 'color reversal' (eg, black and white, brown) , Warm series) ', depending on whether or not the image is corrected.

At operation 1105, the electronic device 201 (e.g., processor 210) may determine (or set) the combination law of the guide mask based on the correction type. For example, the processor 210 may determine to have different correction values for each pixel using the five guide masks generated in operation 1101 according to the correction type. For example, each pixel value in an image may have a value between 0 and 1, and when the processor 210 applies a plurality of guide masks to an image, a different value may be detected for each guide mask . The processor 210 applies a guide mask to an image to determine (or detect) a detected region as a correction region for a region (e.g., a pixel) close to '1' It may not be judged as a correction region. Here, not judging as the correction region means that the region is not corrected but remains as it is in the original.

For example, when a user requests correction for an image that has already been captured (hereinafter referred to as 'captured image correction'), the processor 210 applies a background guide mask, a skin guide mask, Can be detected. Since the human face region is close to the skin color in the image, the value of the skin guide mask has a value close to " 1 ". In contrast, the value of the background guide mask has a value close to '0'. That is, different values can be detected depending on which guide mask is applied to the same area (e.g., skin area). The processor 210 defines a correction mask value using the difference between the background guide mask and the skin guide mask. For example, the processor 210 may determine that the value (e.g., 0.3) to which the background guide mask is applied is less than or equal to a second threshold value (e.g., 0.3) For example, 0.7), it can be determined as a correction region. The value of the correction mask is set to '1.0' for the skin region determined as the correction region, and the processor 210 can correct the skin region.

Here, the value or the threshold value to which each guide mask is applied may vary depending on the image, and may be adjustable. The numerical values or threshold values applied with the respective guide masks are described for the purpose of helping understanding of the invention, but the invention is not limited by the numerical values described.

In addition, the processor 210 may apply an object guide mask so as to have different values for areas such as eyes, eyebrows, and lips. The object guide mask is applied to the values of the correction mask defined using the skin guide mask and the background guide mask to redefine them. The processor 210 may determine (or limit) the value for the eyebrow area to '0' when correcting for the photographed image. In this case, the value of the correction mask for the eyebrow area becomes '0', and the processor 210 may not correct the eyebrow area. In addition, the processor 210 may determine (or limit) the value of the correction mask for the lip region to 0.35 when correcting for the photographed image. It is possible to determine (limit) the correction value for the lip region to have a value of 0.35 or more. In this case, the processor 210 can correct a little (e.g., by 0.35) for the lip region.

When the user requests the correction of the preview image (hereinafter referred to as 'preview image correction'), the processor 210 applies a skin guide mask, a shape guide mask, an edge guide mask, and an object guide mask to obtain values Can be detected. The processor 210 may limit, based on the skin guide mask, a value that is less than or equal to the value to which the shape guide mask is applied. In addition, the processor 210 may detect a value corresponding to an edge (e.g., boundary portion, contour) region by applying an edge guide mask to the face region in the image. The processor 210 may determine (limit) a value corresponding to the edge region to 0.35 if the detected value (e.g., 0.9) is equal to or greater than the third threshold (e.g., 0.4). In this case, the value of the correction mask corresponding to the edge area becomes 0.35, and the processor 210 can correct a bit (e.g., 0.35) of the edge area. In addition, the processor 210 detects a value corresponding to the eye and eyebrow area by applying an object guide mask to the face area, determines (or restricts) the detected value to be 0.35 or less, The corresponding value can be detected, and the detected value can be determined (limited) to be less than or equal to '0.5'.

When the user requests brightness correction of the image (hereinafter referred to as brightness correction), the processor 210 detects a value for each pixel by applying a background guide mask, a skin guide mask, a shape guide mask, and an object guide mask can do. If the correction mask value obtained by using the skin guide mask background guide mask exceeds the fourth threshold value, the value of the correction mask is determined as 1. If it is less than the fourth threshold value, the value of the brightness correction mask is set to 0 . In addition, the processor 210 may determine a value corresponding to the eye, eyebrow, nose, and mouth area to be 0.7 or more by using the object guide mask.

In addition, when the user requests a color change for an image (hereinafter, referred to as 'color change'), the processor 210 applies a background guide mask, a skin guide mask, a shape guide mask, Can be detected. The processor 210 limits the value to be less than or equal to the value of the shape guide mask based on the captured image correction mask, determines a value corresponding to the eye area of the object guide mask as '0' Can be determined to be 0.5 or more.

At operation 1107, the electronic device 201 (e.g., processor 210) may generate a correction mask having a determined area-specific value (e.g., correction value) using the one or more guide masks. If the value (or the correction value) is '1', it is corrected, and if it is '0', it is not corrected. For example, if the user requests correction for an already photographed image, the processor 210 determines whether the value of the pixel corresponding to the skin region (e.g., correction value) is' 1 'and the value for the eyebrow region is' , And a value for the lip region is 0.35 or more.

In addition, when the user requests correction for the preview image, the processor 210 determines that the value corresponding to the edge area in the face area is 0.35 or less, the value corresponding to the eye and eyebrow area is 0.35 or less, A value corresponding to the area can be generated as a preview image correction mask defined as ' 0.5 '. In addition, when the user requests the brightness correction of the image, the processor 210 may generate the brightness correction mask by applying the shape guide mask and the object guide mask based on the captured image correction mask. In addition, when the user requests the color change for the image, the processor 210 may generate a color correction mask by detecting a value obtained by applying the shape guide mask and the object guide mask to the image based on the captured image correction mask.

The processor 210 according to various embodiments may determine an image processing method for the skin region based at least on a probability that the skin region is at least part of the plurality of guide masks and the face region. The probability that the skin region corresponds to the skin region may be determined by applying a skin guide mask to the image and determining that a region having a threshold value (e.g., 0.7) or more is highly likely to be included in the skin region. If the skin guide mask is applied to the image, it is possible to determine that a region having a value less than a threshold value (e.g., 0.7) is less likely to be included in the skin region. The image processing method may mean the correction mask.

In operation 1109, electronic device 201 (e.g., processor 210) may detect a correction area based on a correction mask. Here, the correction area may mean an area requiring correction. For example, the correction region may be a face region or a skin region including the face region. The skin area may include areas that require correction of the skin, such as the neck, hands, etc., in addition to the face of the user.

Electronic device 201 (e.g., processor 210) in accordance with various embodiments may perform a reflected light detection method to enhance the performance of correction region detection. In the case of a reflected light region represented by light, it may not fall within the range of the color and brightness of a general skin region. Therefore, in the case of the reflected light region, it may be excluded from the detection of the correction region or it may become low probability. The processor 210 may detect a reflected light region from the image and add it to the skin region. Generally, the brightness of a pixel corresponding to a reflected light region can have a value close to 255, and the color of a pixel corresponding to a reflected light region has a value close to about 128. The processor 210 may detect the reflected light region using the average brightness in the image and the average brightness in the face region in the average color or the image.

For example, the processor 210 may detect the detected region as a reflected light region if an area higher than the average brightness of the image multiplied by a constant value of brightness per pixel in the image is detected. This can be expressed by the following equation (1).

Y is the brightness per pixel in the image, and k can be a constant value greater than 1.1.

For example, the processor 210 may detect a reflected light region in consideration of a range of color values per pixel in an image. At this time, the processor 210 may detect the reflected light region by reflecting the error range to the average color value. This can be expressed by the following equation (3).

Margin can indicate the error range.

The processor 210 can detect the area as a reflected light area when an area having a color value 128 or more per pixel is detected when the average color value in the image exceeds 128 and an area smaller than the average color value with the error area is detected. When the average color value in the image is less than or equal to 128, the processor 210 can detect the region as a reflected light region when an area smaller than 128 is detected that is equal to or larger than the average color value in consideration of the error range. The processor 210 according to various embodiments can detect the area detected as the reflected light area as a reflected light area in both Equations (1) and (2).

The processor 210 according to various embodiments may determine whether reflected light is detected for a limited area in the face area. For example, the processor 210 can detect whether there is reflected light only for the forehead and nose regions with respect to the detected face region using the guide mask.

The processor 210 according to various embodiments may be configured to correct at least a portion of the skin using the image processing method. For example, the processor 210 may determine whether or not a guide mask (e.g., a mask) for contour detection is applied to an area corresponding to a plurality of objects constituting the face among the areas corresponding to the corrected face, : Shape guide mask) can be used to recognize a skin area including contours of the plurality of objects. For example, the processor 210 may apply the shape guide mask to the image and recognize an area having a detected value equal to or greater than a threshold (e.g., 0.3) as a skin area.

The processor 210 in accordance with various embodiments may use a filter to filter an area corresponding to a plurality of objects constituting the face among at least a portion of the image corresponding to the corrected face, It is possible to recognize a skin region having a low probability of being associated with the contours of the objects. For example, when the processor 210 processes the preview image correction, since the correction time is short, the correction mask can be generated in a short time using the simple filter. In this case, the processor 210 recognizes a skin region having a low probability of being facial contour using a simple filter, and a skin region (e.g., a boundary portion, a face contour) . For example, a skin area with a low probability of being a contour may be an eye, an eyebrow, a nose, a mouth, a skin, and a skin area likely to correspond to a contour may correspond to a face contour.

The processor 210 in accordance with various embodiments may change the combination rule of the plurality of guide masks based at least in part on the correction type for the image and may include a plurality of objects And determine an image processing method for the skin based at least on the determined probability. For example, if the correction type is a captured image, the processor 210 determines the probabilities corresponding to the plurality of objects constituting the face using a background guide mask, a skin guide mask, and an object guide mask, Based on the determined probability, at least the image processing method for the skin can be determined.

12A is a diagram illustrating an example of generating a guide mask and a correction mask according to various embodiments.

12A, an electronic device 201 (e.g., processor 210) receives an object guide mask 1211, a shape guide mask 1213, an edge guide mask 1215, a background guide mask 1217 < / RTI > or a skin guide mask 1219 can be created. The processor 210 may provide a correction value differentially to the object guide mask 1211, the shape guide mask 1213, the edge guide mask 1215, the background guide mask 1217 or the skin guide mask 1219, A mask 1220, a second correction mask 1230, and a third correction mask 1240. [ The first correction mask 1220 may be used to correct the captured image. The second correction mask 1230 may be one used to correct the preview image. The third correction mask 1240 may be used to correct the brightness for the image.

12B is a diagram illustrating an example of generating a captured image correction mask according to various embodiments.

Referring to FIG. 12B, the processor 210 may generate a first correction mask 1220 using an object guide mask 1211, a background guide mask 1217, and a skin guide mask 1219. The captured image may have more time to correct the image than the preview image, even if the image is not much preprocessed. Thus, the processor 210 can perform a simple image preprocessing process on the captured image. For example, the processor 210 determines that the value detected by the background guide mask 1217 is '0.3' for the area 1221 corresponding to the skin in the face area, and detects it by the skin guide mask 1219 Can be determined as " 0.9 ". In this case, when the temporary correction mask 1220-1 is viewed, the value (e.g., correction value) corresponding to the area 1222 becomes '1.0', and the processor 210 can correct the area 1222. [

The processor 210 also sets the value corresponding to the eyebrow area 1223 to 0 (zero) using the object guide mask 1211 when the value of the provisional correction mask 1220-1 is 0.9 for the eyebrow area 1223, . In this case, in the first correction mask 1220, the correction value of the eyebrow area 1225 becomes '0', and the eyebrow area 1225 may not be corrected. The processor 210 may also use the object guide mask 1211 to set the value corresponding to the lip region 1224 to a value of " 0 ", if the temporary correction mask 1220-1 has a value of 0.1 for the lip region 1224, (Limited) to be 0.35 'or more. In this case, in the first correction mask 1220, the correction value of the lip region 1226 becomes '0.35', and the lip region 1226 can be slightly corrected.

12C is a diagram showing an example of generating a preview image correction mask according to various embodiments.

Referring to FIG. 12C, the processor 210 may generate a second correction mask 1230 if the user has requested correction for the preview image. The second correction mask 1230 may be created using an object guide mask 1211, a shape guide mask 1213, an edge guide mask 1215, and a skin guide mask 1219. The preview image may take less time to correct the image to provide the user with a corrected image in real time than the captured image. However, since the correction for the face image is basically to make the eyes, nose, and mouth of the face sharp and the skin clean and smooth, the processor 210 can not perform the edge area The shape guide mask 1215 may be applied in correspondence with the shape information 1231 to determine the detected value to be 0.35 or less. For example, the processor 210 may apply a shape guide mask 1215 to the edge region 1231 to determine if the detected value (e.g., 0.9) is greater than or equal to a third threshold (e.g., 0.4) The corresponding value can be determined (limited) to '0.35'. In this case, in the second correction mask 1230, the correction value of the edge area 1233 becomes 0.35, and the edge area 1233 can be slightly corrected. In addition, the processor 210 determines (or restricts) a value corresponding to the eye and eyebrow area to be 0.35 or less by applying the object guide mask 1211, and sets the value corresponding to the nose and mouth area to 0.5 (Limited).

12D is a diagram showing an example of generating a brightness correction mask according to various embodiments.

Referring to FIG. 12D, the processor 210 may generate a third correction mask 1240 if the user requested brightness correction for the image. The third correction mask 1240 may be generated by applying a shape guide mask 1213 based on the first correction mask 1220. [ The brightness correction may be a request to correct the brightness correction mainly for the captured image or the preview image. In the case of brightness correction, not only the skin but also the eyes and nose can be expressed brightly. Accordingly, the processor 210 determines the value of the third correction mask 1240 as '1' if the value of the first correction mask 1220 exceeds the fourth threshold value for the image that requested the brightness correction, If it is less than the threshold value, the value of the third correction mask 1240 can be determined as " 0 ". For example, the processor 210 may apply a respective guide mask to the image to create a first correction mask 1220, and if the value of the first correction mask 1220 applies a shape guide mask 1213 to the image The value of the third correction mask 1240 may be changed to the value of the shape guide mask 1213. In this case, For example, since the value corresponding to the background area 1242 in the first correction mask 1220 and the shape guide mask 1213 becomes '0', the value of the third correction mask 1240 also becomes '0' . In the first correction mask 1220 and the shape guide mask 1213, the value corresponding to the eye region 1241 may be '0.3', and the value of the third correction mask 1240 may also be '0.3' .

In addition, the electronic device 201 (e.g., the processor 210) may include an image-specific object guide mask 1211, a shape guide mask 1213, an edge guide mask 1215, a background guide mask 1217, 1219) and various types of correction masks are generated using the object guide mask 1211, the shape guide mask 1213, the edge guide mask 1215, the background guide mask 1217 or the skin guide mask 1219 can do.

13 is a flow chart illustrating a method of correcting a face region using correction parameters according to various embodiments. The flowchart of FIG. 13 describes operation 409 of FIG. 4 in detail. The operation of Fig. 13 may be performed simultaneously with the operations of Figs. 5 and 9, or may be performed after or before the operations of Fig. 5 and Fig.

13, at operation 1301, an electronic device 201 (e.g., processor 210) may obtain information associated with an image. The information associated with the image may include camera hardware information, feature point information, and correction region information. The camera hardware information may be information obtained from the camera (e.g., camera module 291) during the image photographing. For example, the camera hardware information may include at least one of an ISO level, a luminance, an illumination, and a shutter speed. The minutia information may be information about the minutiae extracted from the image, for example, obtained in the operation 403 of Fig. The correction region information may refer to information on the correction region detected by FIG.

In operation 1303, the electronic device 201 (e.g., processor 210) may set correction parameters based on the information. The correction parameter may include at least one of an image correction degree, a skin area setting, a brightness correction degree adjustment, or an eye correction area setting. For example, the processor 210 may set the degree of noise reduction in smoothing using camera hardware information (e.g., ISO level). The degree of noise reduction may mean the degree of image correction. The processor 210 may set the size of the smoothing kernel in the face area using the face size. Here, the smoothing kernel size setting may mean skin area setting. Processor 210 may adjust the degree of brightness correction using camera hardware information (e.g., camera illumination). The processor 210 may estimate the eye region using facial feature points and apply an eye-only correction algorithm to the eye region.

The processor 210 in accordance with various embodiments may analyze user characteristics and utilize the analyzed user characteristics for calibration parameter settings. For example, processor 210 may analyze user characteristics such as sex, age, skin color, and the like in the face region. The preferences may vary depending on gender or age range, and the degree of correction, range or area to be corrected may be different. Since the skin color may be different depending on the white, yellow and black people, the skin color detection method and correction method may be different. Processor 210 may set calibration parameters using a user specification.

In operation 1305, the electronic device 201 (e.g., processor 210) may determine a correction priority based on the set correction parameters. The processor 210 may determine whether to perform the global correction first or the local correction first according to the correction parameter. For example, the global correction means performing a global correction on the face region, and the local correction may mean performing correction on a certain region (e.g., eye, skin) with respect to the face region . Because the correction parameters include at least one of a user's preference, characteristics, or correction requests, the processor 210 determines the priorities of global correction and local correction according to the correction parameters, have.

At operation 1305, the electronic device 201 (e.g., processor 210) may determine whether the determined correction priority is a global correction. The processor 210 may perform an operation 1309 when performing the global correction first and an operation 1321 when performing the local correction first.

If the global correction is performed first, at operation 1305, the electronic device 201 (e.g., processor 210) may perform a global correction on the face region. The processor 210 may perform at least one of smoothing, brightening, and changing skin tone for the face area. For example, the processor 210 may calibrate (e.g., smoothing), brighten (e.g., brighten), and change skin tone (e.g., pink) to make the skin look smooth throughout the face area.

Once the global correction of operation 1305 is complete, at operation 1311, the electronic device 201 (e.g., processor 210) may perform local correction for the face region. The local correction may include at least one of reflected light correction, spot concealing, sharp eye correction, or edge enhancement correction for increasing image sharpness. For example, the processor 210 may divide the face region into regions that need to be corrected, respectively, and may perform correction for the divided regions appropriately by different correction methods. For example, the processor 210 may apply a contrast enhancement algorithm or a sharpening algorithm to the eye region detected using the feature points to correct the eyes to look crisp. The processor 210 determines whether a dark spot is detected with respect to the skin region of the face region, and determines whether the dark spot is detected with the surrounding pixel value of the dark spot detected region, A spot concealing algorithm can be applied to remove any artifacts that are not removed from global smoothing.

The processor 210 in accordance with various embodiments may perform local correction after global correction when reflected light is detected in the image. For example, the processor 210 may change the saturation due to a global correction process such as skin brightening or color change for the reflected light detection area. To compensate for this disadvantage, the processor 210 may perform 'reflected light correction processing' with local correction after global correction if reflected light is detected in the image.

The processor 210 in accordance with various embodiments may perform a first process for the correction for the face region using the global correction and a correction for the first processed face region using the local correction The second process can be performed.

In operation 1321, where electronic localization is first performed, electronic device 201 (e.g., processor 210) may perform local correction for the face region. The description of the regional correction is similar to or the same as the operation 1311, so that detailed description may be omitted. Once the local correction of operation 1321 is complete, at operation 1323, electronic device 201 (e.g., processor 210) may perform a global correction of the face region. The description of the local correction is similar to or the same as the operation 1309, so detailed description can be omitted. The processor 210 in accordance with various embodiments may perform prior to global correction for natural correction for dark-spot convolution and eye magnification correction. For example, processor 210 may perform global correction, such as skin smoothing, upon completion of global spot correction for dark spot confinement and eye enlargement correction. The processor 210 in accordance with various embodiments may perform a first process for the correction for the face region using the local correction and a correction for the first processed face region using the global correction The second process can be performed.

14 is a diagram illustrating an example of determining correction parameters in accordance with various embodiments.

14, an electronic device 201 (e.g., processor 210) may obtain information 1420 associated with an image from an image 1410. For example, The information 1420 associated with the image may include camera hardware (H / W) information 1421, minutia information 1243, and correction area information 1425. Processor 210 may perform image correction processing 1430 using information 1420 associated with the image. The image correction process 1430 can be divided into global correction 1440 and local correction 1450. Global correction 1440 may include skin smoothing 1441, skin brightening 1443, and skin color changing 1445. The local correction 1450 may include reflection compensation 1451, a dark spot concealer 1453, eye enhancement 1455, and edge preserving 1457. The processor 210 may integrate 1460 the global correction 1440 and the local correction 1450 to correct the image. Processor 210 may display the calibrated image 1470 on a display (e.g., display 260).

15 is a diagram showing an example of correcting a face region locally using correction parameters according to various embodiments.

Referring to FIG. 15, an electronic device 201 (e.g., processor 210) may identify a regional area category 1520 for a local area 1510 in an image. For example, the processor 210 may use the correction parameters 1540 to identify the regional area category 1520. [ The regional region category 1520 may include a non-skin region 1521 and a skin region 1524. The non-skin region 1524 includes an eye region 1522 and the skin region 1524 can be divided into an edge region 1523, a reflected light region 1525, and a dark spot region 1526. Processor 210 may perform local correction processing 1530 based on correction parameter 1540 and local area category 1520. [ For example, the processor 210 may apply magnitude enhancement 1531 to the eye region 1522 to significantly magnify the eye. The processor 210 may apply a difference enhancement 1531 and an edge preservation 1533 to the edge region 1523 to correct the edge portion to appear smooth and natural. The processor 210 may apply a reflected light adjustment algorithm 1535 to the reflected light region 1525 to perform correction processing. Processor 210 may apply a spot concealment algorithm 1537 to the dark spot region 1526 for correction processing.

16 is a diagram showing an example of correcting a face area globally or locally according to various embodiments.

16, electronic device 201 (e.g., processor 210) may detect an area requiring local correction from face area 1610. [ Processor 210 may detect areas that require local correction from face area 1610 after global correction is complete or prior to global correction. The processor 210 can detect the eye area 1611 and the dark spot area 1613 in the face area 1610 as areas requiring local correction. The processor 210 may apply a difference enhancement algorithm to the eye region 1611. [ The corrected eye region 1621 may be one that has been corrected to make the eye look louder and clearer. The processor 210 may apply the spot concealment algorithm to the dark spot region 1613. [ The corrected dark spot region 1623 may be corrected to blur the dark spots to look faint. Processor 210 may display the calibrated image 1620 on a display (e.g., display 260).

17 is a flowchart illustrating a face area correction method according to various embodiments.

17, at operation 1701, electronic device 201 (e.g., processor 210) may recognize a face region based at least on feature points extracted from the image. The image may be a preview image obtained in real time from a camera module (e.g., camera module 291) of the electronic device 201 or an image stored in a memory (e.g., memory 230). The processor 210 extracts feature points from the image, forms an ellipse region using the extracted feature points, and recognizes a portion corresponding to a large ellipse region of the formed ellipse region as a face region. For example, the processor 210 may form a large elliptical region by connecting a plurality of minutiae located at the greatest distance from a minutiae located at the very center of the extracted minutiae (e.g., nose minutiae) to one line.

At operation 1703, the electronic device 201 (e.g., processor 210) may set one or more objects within the face area. The object refers to an element constituting a face, and may be at least one of eyebrow, eye, nose, or mouth, for example. The processor 210 can set an object in the face region using the characteristics of the elements constituting the face. Processor 210 may analyze at least one of the attributes (e.g., eyebrows, eyes, nose, mouth), shape (e.g., shape), or position of the object using feature points located within the face region. The operation 1703 is the same as or similar to the operation 407 of FIG. 4, and therefore detailed description may be omitted.

In operation 1705, the electronic device 201 (e.g., processor 210) may set the baseline in the face area using the one or more objects. Since the face of a person is elliptical, the processor 210 can set a line connecting an ellipse and a major axis as a reference line. For example, in a human face, the long axis may mean a vertical or vertical direction from the forehead to the jaw. For example, the baseline may be a line extending from the forehead to the middle point between the eyebrows and the tip of the nose. Alternatively, the baseline may be a line extending from the forehead to the midpoint between the eyebrows and the midpoint of the mouth. The operation 1705 is the same as or similar to the operation 503 of FIG. 5, and therefore detailed description may be omitted.

In operation 1707, the electronic device 201 (e.g., processor 210) may segment the face region based on the baseline. For example, the processor 210 may segment the face region from a location above the face (e.g., forehead) to below (e.g., jaw) relative to a baseline. The processor 210 may divide the face region into a first region including a forehead and eye, a second region including a nose and mouth, and a third region including a jaw portion. The method of dividing the area can be variously implemented according to the embodiment, and the present invention is not limited by the following description.

In operation 1709, the electronic device 201 (e.g., processor 210) may calculate the correction ratio for each of the divided face regions. The face shape (or area) correction may mean to reduce the contour of the face. The processor 210 may perform correction to reduce the contour of the face in consideration of the overall face ratio. For example, when the processor 210 divides the face region into a first region including the forehead and eye, a second region including the nose and mouth, and a third region including the jaw portion, the correction for the first region The correction ratio for the second area, and the correction ratio for the third area, respectively.

At operation 1711, the electronic device 201 (e.g., processor 210) may correct the face region based at least on the calculated correction ratio. The processor 210 may correct the face area by applying different correction ratios for each divided area. Alternatively, the processor 210 may correct the face region by applying the long axis direction correction ratio and the short axis direction correction ratio for each divided region. The processor 210 may correct the face region by moving the corresponding pixel according to the correction ratio. For example, the processor 210 may fill in the external pixels of the face region by the reduced portion when reducing the face to left and right. Alternatively, the processor 210 may enlarge the face to the left and right and, when zooming up, fill the face boundary with pixels included in the face region by the enlarged portion and fill it with the external pixels of the face region by the reduced portion have.

At operation 1713, the electronic device 201 (e.g., processor 210) may display the corrected image. Processor 120 may perform operations 1701 through 1711 to correct the image and then display the corrected image on display 260. [ The user can view the displayed image via the display 260 and optionally store the displayed image. The processor 210 may store the displayed image in the memory 230 if there is a storage request (e.g., a save button selection) from the user.

According to various embodiments, an operating method of an electronic device includes: recognizing a face region based at least on feature points extracted from the image; setting one or more objects within the face region; An operation of dividing the face area based on the reference line, an operation of calculating a correction ratio for each divided face area, an operation of correcting the face area based at least on the calculated correction ratio, And displaying the corrected image.

The face region is recognized as an ellipse, the reference line is set as a line connecting the elliptical inner longitudinal axis, and the method may further include determining an inclination of the face region using the reference line.

Calculating the correction ratio in the major axis direction of the face region based at least on the slope of the face region and calculating the correction ratio in the minor axis direction of the face region based at least on the slope of the face region . ≪ / RTI >

The operation of calculating the enlargement ratio may include an operation of variably calculating the correction ratio of the divided face regions based on at least the total ratio of the face regions.

The method includes: forming an ellipse region corresponding to an eye object among the at least one object; setting a reference line connecting the major axis of the ellipse region; calculating an enlargement ratio based on the slope of the set reference line; And enlarging the ellipse region based on at least the calculated enlargement ratio.

Wherein the operation of calculating the magnification ratio includes an operation of detecting a pupil region in the elliptical region and an operation of calculating an enlargement ratio with respect to the pupil region based on at least one of the size, the gradient direction, or the position of the pupil region can do.

The method may include correcting the eye object by applying different magnification ratios to the elliptical region and the pupil region.

The method includes: generating at least one guide mask using the image; determining an application ratio of the at least one guide mask based on the correction type of the image; determining a correction mask based on the determined application ratio And detecting a correction region including the face region based on the correction mask.

Wherein the operation of detecting the correction region includes the steps of detecting a reflected light region using the average brightness in the image and the average color or the average brightness in the face region and the average color, Operation.

Wherein the act of generating the correction mask comprises: determining a different application ratio to the at least one guide mask based on a correction type of the image; applying the application rate to the at least one guide mask, And generating the different correction mask according to the correction type.

The method comprising: setting a correction parameter using information associated with the image, determining a correction priority based on the set correction parameter, and determining a global correction based on the correction priority, And may further include an operation of performing processing.

Wherein the performing comprises performing a first process for a correction for the face region using either the global correction or the local correction based at least in part on the correction parameter, And performing a second process for correction of the first processed face region using the other of the local correction.

The performing may include performing the local correction after the global correction when reflected light is detected in a face region of the image.

According to various embodiments, a computer-readable medium having computer-executable instructions for performing the steps of: recognizing a face region based on feature points extracted from the image; setting one or more objects within the face region; An operation of dividing the face region based on the reference line, an operation of calculating a correction ratio for each of the divided face regions, an operation of correcting the face region in accordance with the calculated correction ratio, And a program for executing an operation of displaying the corrected image.

The computer-readable recording medium may be a hard disk, a floppy disk, a magnetic medium such as a magnetic tape, an optical recording medium such as a CD-ROM, a DVD, a magnetic-optical medium such as a floppy disk, The instructions may include code that is generated by the compiler or code that may be executed by the interpreter. Modules or program modules according to various embodiments may include at least one or more of the components described above Operations that are performed by modules, program modules, or other components, in accordance with various embodiments, may be performed in a sequential, parallel, iterative, or heuristic manner, or at least in part Some operations may be executed in a different order, omitted, or other operations may be added.

It is to be understood that both the foregoing general description and the following detailed description of the present invention are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. Accordingly, the scope of the present invention should be construed as being included in the scope of the present invention, all changes or modifications derived from the technical idea of the present invention.

101: Electronic device
110: bus 120: processor
130: memory 140: program
150: input / output interface 160: display
170: Communication interface

Claims (20)

In an electronic device,
Memory;
display;
camera; And
And a processor operably coupled to the memory, the display, or the camera,
Recognizing the face region based on at least the feature points extracted from the image, setting one or more objects in the face region, setting the in-face reference line using the one or more objects, To calculate a correction ratio for each of the divided face areas, to correct the face area based at least on the calculated correction ratio, and to display the corrected image on the display. The method according to claim 1,
The face region is recognized as an ellipse,
The reference line is set as a line connecting the elliptical inner longitudinal axis,
Wherein the processor is configured to determine a slope of the face region based at least on the baseline. 3. The apparatus of claim 2,
Calculate a correction ratio in a major axis direction of the face region based at least on a slope of the face region, and calculate a correction ratio in a direction of a short axis of the face region based at least on a slope of the face region. 2. The apparatus of claim 1,
And to calculate the correction ratios of the divided face regions based on at least the total ratio of the face regions. 2. The apparatus of claim 1,
An elliptical region corresponding to the eye object among the at least one object is calculated, an enlargement ratio is calculated based on at least the slope of the elliptical region, and the elliptical region is enlarged based on the calculated enlargement ratio. 6. The apparatus of claim 5,
The reference line connecting the major axis of the elliptical region is set, the slope of the set reference line is determined, and the magnification ratio is calculated based at least on the slope of the reference line. 6. The apparatus of claim 5,
Detect a pupil region in the elliptical region, and calculate an enlargement ratio for the pupil region based at least on the size, tilt direction, or position of the pupil region. 8. The apparatus of claim 7,
And the eye object is corrected by applying different magnification ratios to the ellipse region and the pupil region. 2. The apparatus of claim 1,
Setting a correction parameter using information associated with the image, determining a correction priority based at least on a set correction parameter, and performing a global correction or local correction on the face region based at least on the correction priority . ≪ / RTI > 10. The apparatus of claim 9,
Based on at least the correction parameter,
Performing a first process for the correction on the face area using either the global correction or the local correction,
And perform a second process for the correction for the first processed face region using the other one of the global correction or the local correction. 10. The apparatus of claim 9,
And configured to perform processing for the global correction after performing processing for the global correction when reflected light is detected in the face region of the image. In an electronic device,
camera; And
The processor comprising:
Using the camera to obtain an image including a face;
Generate a plurality of guide masks associated with the face, based at least in part on a detection method for the face;
Determine an image processing method for the skin based at least on a probability that at least a portion of the plurality of guide masks and skin of the face corresponds to the skin; And
And to adjust a portion of the skin based at least in part on the determined image processing method. 13. The system of claim 12,
Using a guide mask for contour detection for an area corresponding to a plurality of objects constituting the face among the areas corresponding to the corrected face of at least a part of the skin of the skin, Wherein the electronic device is configured to recognize a skin area that includes the skin. 13. The system of claim 12,
A probability that a probability corresponding to the contours of the plurality of objects is equal to or greater than a predetermined threshold value is obtained by using a filter for an area corresponding to a plurality of objects constituting the face, An electronic device set to recognize a low skin area. 13. The system of claim 12,
Modifying a combination law of the plurality of guide masks based at least on a correction type for the image,
Determine probabilities corresponding to a plurality of objects constituting the face based at least on the changed combination rule, and
And determine, based at least in part on the determined probability, an image processing method for the skin. A method of operating an electronic device,
Recognizing a face region based at least on feature points extracted from the image;
Setting one or more objects in the face area;
Setting the in-face area baseline using the one or more objects;
Dividing the face region based on the reference line;
Calculating a correction ratio for each of the divided face regions;
Correcting the face region based at least on the calculated correction ratio; And
And displaying the corrected image. 17. The method of claim 16,
The face region is recognized as an ellipse,
The reference line is set as a line connecting the elliptical inner longitudinal axis,
Further comprising determining the slope of the face region using the baseline. 18. The method of claim 17,
Calculating a correction ratio in a major axis direction of the face region based at least on a slope of the face region; And
And calculating a correction ratio in the direction of the short axis of the face region based at least on the slope of the face region. 17. The method of claim 16,
Forming an elliptical region corresponding to a snow object among the at least one object;
Setting a reference line connecting the major axis of the elliptical region;
Calculating an enlargement ratio according to the slope of the set reference line; And
And enlarging the elliptical region based on the calculated enlargement ratio. 20. The method of claim 19,
Detecting a pupil region in the elliptical region; And
Calculating an enlargement ratio for the pupil region based on at least one of a magnitude, a gradient direction, or a position of the pupil region.

Images