WO2022211414A1

WO2022211414A1 - Electronic device for acquiring images including components of light output from light emitting module of electronic device and components of ambient light by using plurality of images, and control method therefor

Info

Publication number: WO2022211414A1
Application number: PCT/KR2022/004323
Authority: WO
Inventors: 전태한; 정지운; 민진홍; 김준호; 오영재; 최형선
Original assignee: 삼성전자 주식회사
Priority date: 2021-03-29
Filing date: 2022-03-28
Publication date: 2022-10-06
Also published as: KR20220134918A

Abstract

Disclosed are an electronic device for acquiring images including components of light output from a light emitting module of the electronic device and components of ambient light by using a plurality of images, and a control method therefor. The electronic device according to an embodiment disclosed herein comprises at least one camera, at least one light emitting module, and at least one processor. The at least one processor can be set to: acquire a plurality of images of a portion of a user's body by using the at least one camera and the at least one light emitting module; recognize face regions in the plurality of acquired images; perform inverse gamma correction on the recognized face regions; and acquire first images and second images on the basis of at least two images among the plurality of images on which the inverse gamma correction has been performed, wherein the first images only include values of RGB components of ambient light, and the second images only include components of light output from the at least one light emitting module of the electronic device.

Description

An electronic device for obtaining images each including a component of light output from a light emitting module of the electronic device and a component of ambient light by using a plurality of images, and a control method therefor

This document relates to an electronic device for obtaining an image each including a component of light output from a light emitting module of the electronic device and a component of ambient light by using a plurality of images, and a method for controlling the same.

Various services and additional functions provided through an electronic device, for example, an electronic device such as a smart phone, are gradually increasing. For example, the user may perform skin diagnosis through an electronic device such as a smart phone. As described above, the continuous increase in demand for using skin diagnosis and beauty devices conveniently at home without visiting a specialized clinic has resulted in a demand for skin analysis technology development.

The light emitting module of the skin analysis device is set to output the most suitable light intensity to analyze the user's skin. Accordingly, a user for skin analysis may measure a skin condition by using a skin analysis device in an environment such as a dark room, for example, in order to block the influence of ambient light. However, such a measurement method may be a constraint on use when a user uses the skin analysis device. Therefore, there is a need to develop a technology for acquiring an image capable of excluding the influence of ambient light for accurate skin analysis.

According to an embodiment of the present document, an image from which an ambient light component is removed may be acquired using a plurality of acquired images, and the accuracy of skin analysis may be improved by performing skin analysis using the acquired image. An electronic device capable of this may be provided.

According to an embodiment of the present document, an electronic device capable of acquiring an image including only an ambient light component by using a plurality of acquired images, and estimating a photographing surrounding environment state using the acquired image may be provided.

According to an embodiment of the present document, an image from which an ambient light component is removed may be acquired using a plurality of acquired images, and the accuracy of skin analysis may be improved by performing skin analysis using the acquired image. A method of controlling an electronic device capable of being used may be provided.

According to an embodiment of the present document, an electronic device capable of acquiring an image including only an ambient light component by using a plurality of acquired images, and estimating a photographing surrounding environment state using the acquired image A control method may be provided.

An electronic device according to an embodiment of the present document includes at least one camera, at least one light emitting module, and at least one processor, wherein the at least one processor includes the at least one camera and the at least one light emitting module. Acquire a plurality of images of a body part of a user by using a module, recognize a face region from the plurality of acquired images, perform inverse gamma correction on the recognized face region, and perform the inverse gamma correction Estimate an intensity of ambient light based on at least two or more images from among a plurality of performed images, and based on the estimated intensity of ambient light, a first image including only a component of the ambient light and the at least one of the electronic device Each of the second images including only the light component output by the light emitting module may be set.

A method of controlling an electronic device according to an embodiment of the present document includes: acquiring a plurality of images of a body part of a user by using at least one camera of the electronic device and at least one light emitting module of the electronic device; An operation of recognizing a face region from a plurality of acquired images, an operation of performing inverse gamma correction on the recognized face region, and an operation of performing inverse gamma correction on ambient light based on at least two or more images among a plurality of images on which inverse gamma correction has been performed Based on the operation of estimating the intensity and the estimated intensity of ambient light, a first image including only a component of the ambient light and a second image including only a light component output by the at least one light emitting module of the electronic device It may include an operation of acquiring each.

Effects according to various embodiments are not limited to the effects described above, and it is apparent to those skilled in the art that various effects are inherent in the present disclosure.

1A and 1B are exemplary views for explaining an electronic device according to an embodiment of the present document.

2 illustrates a first image including only an ambient light component based on an intensity of ambient light estimated by the electronic device according to an embodiment of the present document and a second image including only a light component output from a light emitting module of the electronic device It is an exemplary diagram for explaining each acquired function or operation.

3A, 3B, 3C, and 3D are exemplary views for explaining a function or operation of acquiring a plurality of images including a light component output from a light emitting module of an electronic device and an ambient light component.

4 is an exemplary diagram for explaining a function or operation of estimating an ambient light component according to an embodiment of the present document.

5 is an exemplary view for explaining a function or operation of performing skin analysis using a second image and providing a result of the skin analysis according to an embodiment of the present document.

6 is an exemplary diagram for explaining a function or operation of estimating a photographing environment state (eg, a color temperature and/or a position of lighting) using a first image according to an embodiment of the present document.

7A and 7B are exemplary views for explaining a function or operation of estimating a lighting position using a first image.

8 is an exemplary diagram for explaining that various embodiments of the present document may be applied when a mobile device (eg, a smart phone) according to an embodiment of the present document is used.

1A and 1B , the electronic device 101 according to an embodiment of the present document includes a processor 110 , a camera 120 , a light emitting module 130 , a memory 140 , and a display 150 . may include According to an embodiment of this document, some of these components may be implemented as one integrated circuit. For example, the camera 120 and the light emitting module 130 may be implemented while being embedded in the display 130 . According to an embodiment of this document, at least some of the components of FIG. 1 may be implemented so that the electronic device 101 is not included.

The processor 110 according to an embodiment of the present document executes, for example, software (eg, a program) and at least one other component (eg, hardware or software components) and may perform various data processing or operations. According to an embodiment of the present document, as at least part of data processing or operation, the processor 110 loads instructions or data obtained from another component (eg, the sensor module 176) into the memory 120, A command or data stored in the memory 120 may be processed, and result data may be stored in the memory 120 . According to an embodiment of the present document, the processor 110 is a main processor (eg, a central processing unit or an application processor), and an auxiliary processor (eg, a graphic processing unit, an image signal processor, a sensor hub) that can be operated independently or together processor, or communication processor). Additionally or alternatively, the auxiliary processor may be configured to use less power than the main processor or to specialize in a designated function. The auxiliary processor according to an embodiment of the present document may be implemented separately from or as a part of the main processor. The co-processor may be, for example, on behalf of the main processor while the main processor is in an inactive (eg, sleep) state, or in conjunction with the main processor while the main processor is in an active (eg, running application) state, in an electronic device. It is possible to control at least some of functions or states related to at least one of the components 101 (eg, the display 150 or the camera 120 ). According to an embodiment of the present document, the auxiliary processor (eg, an image signal processor) may be implemented as a part of another functionally related component (eg, the camera 120 ).

At least one camera 120 according to an embodiment of the present document may capture a still image or a moving image. At least one camera 120 according to an embodiment of the present document includes one or more lenses, image sensors (eg, charge coupled device (CCD), CMOS device (CMOS)), image signal processors, or flashes. can do. At least one camera 120 according to an embodiment of this document may receive a camera control signal from the processor 110 . The camera control signal according to the exemplary embodiment of this document may include an electrical signal for photographing at least a part of the user's body currently displayed on the display 150 .

The light emitting module 130 according to an embodiment of this document may include at least one light emitting diode. The light emitting module 130 according to the exemplary embodiment of this document may mean one light emitting diode or a single circuit in which a plurality of light emitting diodes are disposed. The light emitting module 130 according to an embodiment of this document may include a light emitting diode that outputs UV (ultra violet) or various modules. The light emitting module 130 according to an embodiment of this document may control the output of at least some light emitting modules according to the light emitting module control signal received from the processor 110 . The light emitting module 130 according to an embodiment of this document may be disposed on the front side of the electronic device 101 . The light emitting module 130 according to an embodiment of the present document may be provided separately from (eg, separated from) the display 150 as shown in FIG. 1B . The light emitting module 130 according to an embodiment of this document may be implemented while being embedded in a display. Alternatively, the light emitting module 130 according to an embodiment of this document may be provided in such a way that at least a portion of the light emitting module 130 is exposed through a hole provided in the display 150 . The number of light emitting modules 130 according to an embodiment of this document may be variously changed and implemented. The light emitting module 130 according to an embodiment of this document may be controlled by receiving a light emitting module control signal from the processor 110 . The light emitting module 130 according to an embodiment of the present document may be switched from an OFF state to an ON state or from an ON state to an OFF state according to a light emitting module control signal transmitted from the processor 110, , the intensity of the output light amount can be adjusted.

The memory 140 according to an embodiment of the present document may store various data used by at least one component (eg, the processor 110 or the camera 120 ) of the electronic device 101 . The data may include, for example, input data or output data for software (eg, a program) and instructions related thereto. The memory 140 according to the exemplary embodiment of this document may include a volatile memory or a non-volatile memory. A program may be stored as software in a memory, and may include, for example, an operating system, middleware, or an application. The memory 140 according to an embodiment of the present document may at least temporarily store an image acquired by the camera 120 .

The display 150 according to an embodiment of this document may visually provide information to the outside (eg, a user) of the electronic device 101 . The display 150 according to an embodiment of this document may include, for example, a display, a hologram device, or a projector and a control circuit for controlling the corresponding device. According to an embodiment of the present document, the display 150 includes a touch circuitry configured to sense a touch, or a sensor circuitry configured to measure the intensity of a force generated by the touch (eg, a pressure sensor). can do.

FIG. 2 illustrates only a first image including only an ambient light component based on an intensity Ia of ambient light estimated by the electronic device 101 according to an embodiment of the present document and a light component output from a light emitting module of the electronic device It is an exemplary diagram 200 for explaining a function or operation of acquiring each of the included second images. 3A to 3D are exemplary views for explaining a function or operation of acquiring a plurality of images including a light component output from a light emitting module of an electronic device and an ambient light component. 4 is an exemplary diagram for explaining a function or operation of estimating an ambient light component according to an embodiment of the present document. 5 is an exemplary view for explaining a function or operation of performing skin analysis using a second image and providing a result of the skin analysis according to an embodiment of the present document.

Referring to FIG. 2 , in operation 210 , the electronic device 101 according to an embodiment of the present document may acquire a plurality of images using different amounts of light. The electronic device 101 according to an embodiment of the present document may adjust the intensity of light output from the light emitting module 130 to obtain a plurality of images using different amounts of light. The electronic device 101 according to an embodiment of the present document may photograph a plurality of images according to a user's photographing input signal. A photographing signal according to an exemplary embodiment of this document may be input according to a user's touch input to a user interface displayed on the display 150 . The plurality of images according to the exemplary embodiment of this document may include an ambient light component Ia and a light component Id1 , Id2 and/or Id3 output by the light emitting module 130 . For example, referring to FIG. 3A , the image of the user 300 is to be acquired (eg, photographed) while external lighting is in an ON state and light of a first intensity is output from the light emitting module 130 . can In this case, the total light intensity received by one pixel is the ambient light component Ia and the light component Id1 output by the light emitting module 130 as shown in FIG. 3B . A combined amount may be received. In addition, referring to FIG. 3B , the image of the user 300 may be acquired (eg, photographed) while external lighting is in an ON state and light of a second intensity is output from the light emitting module 130 . . In this case, the total light intensity received by one pixel is the ambient light component Ia and the light component Id2 output by the light emitting module 130 as shown in FIG. 3D . A combined amount may be received. In FIG. 3A , an overhead light is shown as a representative example of ambient light, but this is exemplary. According to an embodiment of the present document, ambient light affects the brightness of an image acquired through the camera 120, such as indoor lighting that cannot be controlled by the electronic device 101 or sunlight incident into a house through a window. The state can include all factors.

In operation 220 , the electronic device 101 according to an embodiment of the present document may identify a face region from each of a plurality of acquired images. The electronic device 101 according to an embodiment of the present document may detect a face region from an acquired image through various algorithms for detecting a face. For example, the electronic device 101 according to an embodiment of the present document may include principal component analysis (PCA), linear discriminate analysis (LDA), and elastic bundle graph matching using eigenface. ), the Hidden Markov model, multilinear subspace learning using tensor expressions, or neuronal motivated dynamic link matching. can be detected.

In operation 230 , the electronic device 101 according to an embodiment of the present document may perform inverse gamma correction on the identified face region. Gamma correction may be performed on a plurality of images acquired by the electronic device 101 according to an embodiment of the present document. The electronic device 101 according to an embodiment of the present document performs inverse gamma correction on gamma-corrected data (eg, values of pixels constituting a plurality of images) by using a look-up table for inverse gamma correction. can do. Inverse gamma-corrected data according to an embodiment of the present document may have a linear characteristic. Inverse gamma values enabling inverse gamma-corrected data to have a linear characteristic may be stored in a lookup table according to an embodiment of the present document. For example, in the lookup table according to an embodiment of the present document, a first inverse gamma table for inverse gamma-correcting red (R) data and a second inverse gamma table for inverse gamma-correcting green (G) data and a third inverse gamma table for performing inverse gamma correction of blue (B) data. The lookup table according to the exemplary embodiment of this document may be previously stored in the memory 140 of the electronic device 101 .

In operation 240 , the electronic device 101 according to an exemplary embodiment of the present document estimates values of RGB components of ambient light based on the face region on which inverse gamma correction has been performed, and includes only the values of RGB components of ambient light. 1 An image (eg, an image reflected by ambient light) can be acquired. The electronic device 101 according to an embodiment of the present document may estimate values of RGB components of ambient light using linearity of inverse gamma-corrected data. Referring to FIG. 4 , an electronic device 101 according to an exemplary embodiment of the present document may display a specific part of the acquired image of the user 300 (eg, a plurality of images acquired in operation 210 ). A graph as shown in FIG. 4 may be generated for each of the components (eg, R(Red) component) constituting one pixel 300a. The electronic device 101 according to an embodiment of the present document generates a graph that is substantially straight line based on values of components constituting one pixel 300a (eg, intensity with respect to the R component). can do. The electronic device 101 according to an exemplary embodiment of the present document provides a section between a third image having specific RGB component values and a y-intercept based on linearity (eg, a portion shown by a thick line in FIG. 4 ). estimation may be possible. The electronic device 101 according to an embodiment of the present document may estimate a y-intercept value of the generated graph as a value of a specific component of any one pixel constituting the ambient light. In other words, the electronic device 101 according to an embodiment of the present document may estimate the y-intercept value of the generated graph as values of RGB components of ambient light. The electronic device 101 according to an embodiment of the present document may repeatedly perform operation 240 with respect to other components (eg, a G (green) component and a B (blue) component) constituting a specific pixel. Also, the electronic device 101 according to an embodiment of the present document may repeatedly perform operation 240 with respect to a plurality of pixels constituting a plurality of acquired images. The electronic device 101 according to an embodiment of the present document repeatedly performs operation 240 on each component (eg, RGB components) of a plurality of pixels constituting the plurality of acquired images, thereby composing an ambient light component. An image (eg, the first image) including only (Ia) may be obtained.

In operation 250 , the electronic device 101 according to an exemplary embodiment of the present document performs a second image including only the light component output from the light emitting module of the electronic device 101 based on the estimated RGB component values of the ambient light. can be obtained. The electronic device 101 according to an embodiment of the present document provides a specific component (eg, R component) value of a pixel constituting the obtained third image and a component (eg, R component) of a pixel constituting the obtained first image. An image (eg, a second image) including only a light component (eg, Id1 ) output from the light emitting module 130 of the electronic device 101 for a specific pixel component by repeatedly calculating the difference in component) value for each component ) can be obtained. Similarly, in the electronic device 101 according to an embodiment of the present document, a specific component (eg, R component) value of a pixel constituting the obtained fourth image and a specific component (eg, R component) of a pixel constituting the obtained first image ( For example, by repeatedly calculating the difference in the R component) value for each component, an image including only the light component (eg, Id2) output from the light emitting module 130 of the electronic device 101 can be obtained for a specific pixel component. and iteratively calculates the difference between a value of a specific component (eg, R component) of a pixel constituting the obtained fifth image and a value of a specific component (eg, R component) of a pixel constituting the obtained first image for each component Accordingly, with respect to a specific pixel component, an image including only a light component (eg, Id3) output from the light emitting module 130 of the electronic device 101 may be obtained.

In operation 260 , the electronic device 101 according to an embodiment of the present document may estimate a photographing environment state by using the acquired first image. Operation 260 according to an embodiment of this document will be described in more detail with reference to FIGS. 6, 7A, and 7B.

The electronic device 101 according to an embodiment of the present document performs skin analysis using the acquired second image (eg, an image from which the ambient light component Ia is removed from the fourth image) in operation 270 , , the execution result may be provided through the electronic device 101 . The electronic device 101 according to an embodiment of the present document may display a plurality of images obtained in operation 250 (eg, an image from which an ambient light component is removed from a third image and an image from which an ambient light component is removed from a fourth image). The skin analysis may be performed using any one image (eg, an image from which the ambient light component is removed from the fourth image) among the image and the fifth image from which the ambient light component is removed. The electronic device 101 according to an embodiment of the present document may select any one image according to the purpose of skin analysis (eg, acne distribution analysis, redness distribution analysis, skin tone analysis, etc.). The purpose of skin analysis according to an embodiment of the present document may be selected by a user. Alternatively, the electronic device 101 according to an embodiment of the present document may display a plurality of images obtained in operation 250 (eg, an image from which an ambient light component is removed from the third image, and an ambient light component from the fourth image Any one of the removed image and the image from which the ambient light component is removed from the fifth image (eg, the image from which the ambient light component is removed from the fourth image) may be selected as an image for skin analysis. As shown in FIG. 5 , the electronic device 101 according to an embodiment of the present document may provide a skin analysis result through the display 150 of the electronic device 101 . For example, the electronic device 101 according to an exemplary embodiment of the present document may display a guide message 510 stating, “a portion having severe acne” or “a portion having severe redness” on the display 150 . . The electronic device 101 according to an embodiment of the present document provides a guide message 510 including skin tone information (eg, “skin tone measurement result. L: 65, a: 14, b: 18). The electronic device 101 according to an embodiment of the present document may output a specific image (eg, a fourth image) using a lookup table in which a correspondence relationship between RGB component values and Lab values is defined. ) may be converted into a Lab value, and the converted Lab value may be provided through the electronic device 101 .

The electronic device 101 according to an embodiment of the present document may detect a plurality of feature points in the face region of the user 300 in order to analyze the skin condition. The electronic device 101 according to an embodiment of the present document uses a plurality of detected feature points to determine the position and size of eyes, ears, nose, and mouth, the position and size of eyebrows, the density of eyebrows, and wrinkles At least one of the position, the width of the forehead, the position and size of the chin, the position and size of the cheekbones, the position and size of the forehead, the contour of the face, the size of the face, the position of the hair, or the position and size of the scar may be identified.

6 is an exemplary diagram 600 for explaining a function or operation of estimating a photographing environment state (eg, a color temperature and/or a position of lighting) using a first image according to an embodiment of the present document. 7A and 7B are exemplary views for explaining a function or operation of estimating a lighting position using a first image.

Referring to FIG. 6 , in operation 610 , the electronic device 101 according to an embodiment of the present document may acquire a plurality of images using different amounts of light. The electronic device 101 according to an embodiment of the present document may adjust the intensity of light output from the light emitting module 130 to obtain a plurality of images using different amounts of light. The electronic device 101 according to an embodiment of the present document may photograph a plurality of images according to a user's photographing input signal. A photographing signal according to an exemplary embodiment of this document may be input according to a user's touch input to a user interface displayed on the display 150 . The plurality of images according to the exemplary embodiment of this document may include an ambient light component Ia and a light component Id1 , Id2 and/or Id3 output by the light emitting module 130 . For example, referring to FIG. 3A , the image of the user 300 is to be acquired (eg, photographed) while external lighting is in an ON state and light of a first intensity is output from the light emitting module 130 . can In this case, the total light intensity received by one pixel is the ambient light component Ia and the light component Id1 output by the light emitting module 130 as shown in FIG. 3B . A combined amount may be received.

The electronic device 101 according to an embodiment of the present document may identify a face region from each of the plurality of acquired images in operation 620 . The electronic device 101 according to an embodiment of the present document may detect a face region from an acquired image through various algorithms for detecting a face. For example, the electronic device 101 according to an embodiment of the present document may include principal component analysis (PCA), linear discriminate analysis (LDA), and elastic bundle graph matching using eigenface. ), the Hidden Markov model, multilinear subspace learning using tensor expressions, or neuronal motivated dynamic link matching. can be detected.

The electronic device 101 according to an embodiment of the present document may perform inverse gamma correction on the identified face region in operation 630 . Gamma correction may be performed on a plurality of images acquired by the electronic device 101 according to an embodiment of the present document. The electronic device 101 according to an embodiment of the present document performs inverse gamma correction on gamma-corrected data (eg, values of pixels constituting a plurality of images) by using a look-up table for inverse gamma correction. can do. Inverse gamma-corrected data according to an embodiment of the present document may have a linear characteristic. Inverse gamma values enabling inverse gamma-corrected data to have a linear characteristic may be stored in a lookup table according to an embodiment of the present document. For example, in the lookup table according to an embodiment of the present document, a first inverse gamma table for inverse gamma-correcting red (R) data and a second inverse gamma table for inverse gamma-correcting green (G) data and a third inverse gamma table for performing inverse gamma correction of blue (B) data. The lookup table according to the exemplary embodiment of this document may be previously stored in the memory 140 of the electronic device 101 .

In operation 640 , the electronic device 101 according to an embodiment of the present document estimates values of RGB components of ambient light based on the face region on which inverse gamma correction has been performed, and a second method including only the values of RGB components of ambient light. 1 image can be acquired. The electronic device 101 according to an embodiment of the present document may estimate values of RGB components using linearity of inverse gamma-corrected data. Referring to FIG. 4 , an electronic device 101 according to an exemplary embodiment of the present document may display a specific part of the acquired image of the user 300 (eg, a plurality of images acquired in operation 210 ). A graph as shown in FIG. 4 may be generated for each of the components (eg, R(Red) component) constituting one pixel 300a. The electronic device 101 according to an embodiment of the present document generates a graph that is substantially straight line based on values of components constituting one pixel 300a (eg, intensity with respect to the R component). can do. In other words, the electronic device 101 according to an embodiment of the present document provides a section between the third image having a specific RGB component value and the y-intercept based on linearity (eg, a portion shown by a thick line in FIG. 4 ). ) can be estimated. The electronic device 101 according to an embodiment of the present document may estimate a y-intercept value of the generated graph as a value of a specific component of any one pixel constituting the ambient light. In other words, the electronic device 101 according to an embodiment of the present document may estimate the y-intercept value of the generated graph as values of RGB components of ambient light. The electronic device 101 according to an embodiment of the present document may repeatedly perform operation 640 with respect to other components (eg, a G (green) component and a B (blue) component) constituting a specific pixel. Also, the electronic device 101 according to an embodiment of the present document may repeatedly perform operation 640 on a plurality of pixels constituting a plurality of acquired images. The electronic device 101 according to an embodiment of the present document repeatedly performs operation 640 for each component (eg, RGB components) of a plurality of pixels constituting a plurality of acquired images, thereby composing an ambient light component. An image (eg, the first image) including only (Ia) may be obtained.

In operation 650 , the electronic device 101 according to an embodiment of the present document may estimate a photographing environment state by using the acquired first image. The photographing environment state according to an embodiment of the present document may include, for example, a position of an ambient light and/or a color temperature of the ambient light. Referring to FIG. 7A , the obtained first image has a red (R) pixel value, a green (G) pixel value, and a blue (B) pixel value (or pixel intensity) (eg, an average value or sum of pixel values). It can be an image with the largest pixel value near the upper left (eg, the first region 710) and the smallest pixel value (eg, the average value or sum of the pixel values) in the lower right region (eg, the second region 720). have. In this case, the electronic device 101 according to an embodiment of the present document illuminates near a region (eg, the first region 710 ) having a large value (or intensity) of a pixel value (or pixel intensity). It can be assumed that this is located Similarly, referring to FIG. 7B , the obtained first image has a red (R) pixel value, a green (G) pixel value, and a blue (B) pixel value (or pixel intensity) near the lower right corner (eg, the second region) (720)) (eg, average value or sum of pixel values) is the largest, and the upper left (eg, first region 710 ) has the smallest pixel value (eg, average value or sum of pixel values). can be In this case, the electronic device 101 according to an embodiment of the present document illuminates near a region (eg, the second region 720 ) having a large pixel value (or pixel intensity) value (or intensity). It can be assumed that this is located The electronic device 101 according to an embodiment of the present document calculates a ratio of a red (R) pixel, a green (G) pixel, and a blue (B) pixel constituting a first image, and based on the calculated ratio, The color temperature of ambient light can be estimated. The electronic device 101 according to an embodiment of the present document may estimate the color temperature of ambient light by using a lookup table in which a relationship between a ratio of pixels constituting an image and a color temperature is defined. For example, in the electronic device 101 according to an embodiment of the present document, pixels (eg, a red (R) pixel, a green (G) pixel, and a blue (B) constituting an image (eg, a first image) When the ratio of pixels) is equal to each other as 0.333, it can be estimated that the color temperature is 5471K by referring to the look-up table.

8 is an exemplary diagram for explaining that various embodiments of the present document may be applied when a mobile device (eg, a smart phone) according to an embodiment of the present document is used. Referring to FIG. 8 , the mobile device 810 according to an embodiment of the present document may acquire a plurality of images using different amounts of light. For example, when taking a selfie, the mobile device 810 according to an embodiment of the present document uses a flash on the front surface of the mobile device 810 (eg, a portion where the display of the mobile device 810 is exposed). An image when is operated and an image when the front flash is not operated can be acquired. The mobile device 810 according to an embodiment of the present document may perform operations 220 to 270 using a plurality of images obtained using different amounts of light as described above.

In the electronic device 101 according to an exemplary embodiment of the present document, when at least a partial area of an acquired image (eg, a fifth image) is saturated, another image (eg, a second image) corresponding to the saturated specific area is 4 image), a new image (eg high dynamic range (HDR) image) can be created. The electronic device 101 according to an exemplary embodiment of the present document may perform skin analysis using the generated new image.

The electronic device according to various embodiments disclosed in this document may have various types of devices. The electronic device may include, for example, a portable communication device (eg, a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, a skin analysis device, or a home appliance device. The electronic device according to the embodiment of the present document is not limited to the above-described devices.

Various embodiments of the present document and terms used therein are not intended to limit the technical features described in the present disclosure to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of the embodiments. In connection with the description of the drawings, like reference numerals may be used for similar or related components. The singular form of the noun corresponding to the item may include one or more of the item, unless the relevant context clearly dictates otherwise. In this disclosure, "A or B", "at least one of A and B", "at least one of A or B," "A, B or C," "at least one of A, B and C," and "A , B, or C," each of which may include any one of the items listed together in the corresponding one of the phrases, or all possible combinations thereof. Terms such as "first", "second", or "first" or "second" may simply be used to distinguish an element from other elements in question, and may refer elements to other aspects (e.g., importance or order) is not limited. that one (eg first) component is "coupled" or "connected" to another (eg, second) component with or without the terms "functionally" or "communicatively" When referenced, it means that one component can be connected to the other component directly (eg by wire), wirelessly, or through a third component.

The term “module” used in the present disclosure may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as, for example, logic, logic block, component, or circuit. A module may be an integrally formed part or a minimum unit or a part of the part that performs one or more functions. For example, according to an embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of the present document may be implemented as software including one or more instructions stored in a storage medium readable by a machine (eg, the electronic device 101). For example, the processor (eg, the processor 110 ) of the device (eg, the electronic device 101 ) may call at least one of one or more instructions stored from a storage medium and execute it. This makes it possible for the device to be operated to perform at least one function according to the called at least one command. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory" only means that the storage medium is a tangible device and does not include a signal (eg, electromagnetic wave), and this term refers to the case where data is semi-permanently stored in the storage medium and It does not distinguish between temporary storage cases.

According to an embodiment of the present document, the methods according to various embodiments disclosed in this document may be provided by being included in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a machine-readable storage medium (eg compact disc read only memory (CD-ROM)), or via an application store (eg Play Store ^TM ) or on two user devices ( It can be distributed (eg downloaded or uploaded) directly or online between smartphones (eg: smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium such as a memory of a server of a manufacturer, a server of an application store, or a memory of a relay server.

According to an embodiment of the present document, each component (eg, a module or a program) of the above-described components may include a singular or a plurality of entities. According to various embodiments, one or more components or operations among the above-described corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg, a module or a program) may be integrated into one component. In this case, the integrated component may perform one or more functions of each component of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component are executed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations are executed in a different order, or omitted. , or one or more other operations may be added.

Claims

In an electronic device,

at least one camera,

at least one light emitting module, and

at least one processor, the at least one processor comprising:

Obtaining a plurality of images of a user's body part using the at least one camera and the at least one light emitting module,

Recognizing a face region from the plurality of acquired images,

performing inverse gamma correction on the recognized face region;

Based on at least two or more images among the plurality of images to which the inverse gamma correction has been performed, only a first image including only RGB component values of ambient light and a light component output by the at least one light emitting module of the electronic device The electronic device, characterized in that it is set to acquire each of the second images including the image.
According to claim 1,

The at least one processor is further configured to adjust an output light amount of the at least one light emitting module to obtain the plurality of images.
3. The method of claim 2,

The plurality of images is an electronic device, characterized in that the images are photographed under environments of different output light amounts.
According to claim 1,

The at least one processor is further configured to estimate the intensity of the ambient light based on linearity of the plurality of inverse gamma-corrected images.
According to claim 1,

The at least one processor is further configured to estimate a color temperature of the ambient light using the first image.
According to claim 1,

The at least one processor is further configured to estimate the position of the ambient light using the first image.
According to claim 1,

The at least one processor is further configured to analyze the user's skin condition using the second image and output a result of the analysis through a display of the electronic device.
According to claim 1,

The plurality of acquired images includes both the ambient light component and the light component output by the light emitting module, the electronic device.
According to claim 1,

Any one of the plurality of images is an image obtained in a state in which the output of the light emitting module is OFF,

The at least one processor, the first image and the second image based on an image obtained when the output of the light emitting module is in an OFF state and an image obtained when the output of the light emitting module is in an ON state The electronic device, characterized in that further set to obtain.
According to claim 1,

The at least one processor is further configured to perform an operation of estimating the intensity of the ambient light with respect to one pixel located in the same part of the face region included in the plurality of images. .
A method for controlling an electronic device, comprising:

acquiring a plurality of images of a body part of a user by using at least one camera of the electronic device and at least one light emitting module of the electronic device;

Recognizing a face region from the plurality of acquired images;

performing inverse gamma correction on the recognized face region;

A first image including only RGB component values of ambient light based on at least two or more of the plurality of images on which the inverse gamma correction has been performed, and only a light component output by the at least one light emitting module of the electronic device A method of controlling an electronic device, comprising: acquiring each of the second images.
12. The method of claim 11,

The method of claim 1, further comprising: adjusting an output light amount of the at least one light emitting module to obtain the plurality of images.
13. The method of claim 12,

The method of controlling an electronic device, characterized in that the plurality of images are images photographed under environments of different output light amounts.
According to claim 1,

The estimating of the intensity of the ambient light comprises estimating the intensity of the ambient light based on linearity of the plurality of inverse gamma-corrected images.
12. The method of claim 11,

The method of claim 1, further comprising: estimating a color temperature of the ambient light by using the first image.