KR102707136B1 - Method to obtain outside luminance using camera sensor and electronic device applying the method - Google Patents

Abstract

According to various embodiments, an electronic device is disclosed, including a housing, a display exposed to a front surface of the housing to display a screen, a camera sensor disposed on at least a portion of a front surface of the housing to measure external illuminance, a processor operatively connected to the display and the camera sensor, and a memory operatively connected to the processor. The memory may store instructions that, when executed, cause the processor to detect a turn-on event of the display, turn on the camera sensor to acquire external illuminance using the camera sensor when the display is turned on, acquire preview image information for a specified period of time through the camera sensor, turn off the camera sensor, acquire external illuminance using an exposure time and a brightness value based on the acquired preview image information, and control the brightness of the display based on the external illuminance. In addition to these, various embodiments are possible as understood by the specification.

Description

METHOD TO OBTAIN OUTSIDE LUMINANCE USING CAMERA SENSOR AND ELECTRONIC DEVICE APPLYING THE METHOD

Various embodiments disclosed in this document relate to a method for measuring illuminance using a camera sensor and a technology for implementing an electronic device applying the method.

An electronic device can set the brightness of a screen displayed on a display to correspond to an external environment. For example, the electronic device can measure external illuminance and control the brightness of the screen. The electronic device can include an illuminance sensor that obtains a brightness value to measure the external illuminance. The illuminance sensor can be placed on the front side of the electronic device. The electronic device can obtain a brightness value of a space in front of the electronic device through the illuminance sensor. The electronic device can measure the external illuminance based on the obtained brightness value and control the brightness of the screen according to the external illuminance.

Recently, the bezel, which is the edge area of electronic devices, has been minimized, and the display can be placed practically on the entire front surface of the electronic device. In this case, it may not be easy to place a separate light sensor on the front surface of the electronic device. The electronic device can measure the external illuminance using a camera sensor placed on the front surface of the electronic device without a separate light sensor, and control the brightness of the screen according to the measured external illuminance.

When an electronic device uses a camera sensor to measure external illuminance, a large amount of current may flow to the camera sensor during the period of measuring external illuminance. If the camera sensor is continuously used to measure external illuminance, the battery consumption of the electronic device may increase.

In addition, for a light sensor placed under a display screen, the brightness of the screen may affect the external illuminance value obtained from the light sensor. In particular, when the external illuminance value is lower than a specified illuminance value, the influence of the brightness of the screen increases, making it difficult to accurately measure the external illuminance.

An electronic device according to one embodiment disclosed in the present document includes a housing, a display configured to be exposed to a front surface of the housing and to display a screen, a camera sensor configured to measure external illuminance and disposed on at least a portion of a front surface of the housing, a processor operatively connected to the display and the camera sensor, and a memory operatively connected to the processor, wherein the memory may store instructions that, when executed, cause the processor to detect a turn-on event of the display, turn on the camera sensor to acquire the external illuminance using the camera sensor when the display is turned on, acquire preview image information for a specified period of time through the camera sensor, turn off the camera sensor, acquire the external illuminance using an exposure time and a brightness value based on the acquired preview image information, and control the brightness of the display based on the external illuminance.

In addition, a method for measuring illuminance by using a camera sensor in an electronic device according to an embodiment disclosed in the present document may include an operation of detecting a turn-on event of a display, an operation of turning on the camera sensor to obtain the external illuminance by using the camera sensor when the display is turned on, an operation of obtaining preview image information for a specified time through the camera sensor, an operation of turning off the camera sensor, an operation of obtaining the external illuminance value by using an exposure time and a brightness value based on the obtained preview image information, and an operation of controlling the brightness of the display based on the external illuminance value.

In addition, an electronic device according to another embodiment disclosed in the present document includes a housing, a display exposed to the front of the housing to display a screen, a camera including a camera sensor disposed in an area of the front of the housing excluding an area where the display is disposed and measuring external illuminance toward which the front of the housing faces, an illuminance sensor disposed under the display and measuring the external illuminance, a processor operatively connected to the display, the camera, and the illuminance sensor, and a memory operatively connected to the processor, wherein the memory, when executed, causes the processor to detect a turn-on event of the display, turn on the illuminance sensor when the display is turned on, acquire external illuminance for a specified time through the illuminance sensor, determine whether the external illuminance is less than or equal to a specified first illuminance value, turn on the camera sensor if the external illuminance is less than or equal to the first illuminance value, acquire an exposure time and a brightness value from preview image information acquired by the camera sensor, and use the exposure time and/or the brightness value to determine the Instructions for compensating for external illuminance and controlling the brightness of the display according to the compensated external illuminance can be stored.

According to various embodiments disclosed in this document, the battery consumption of an electronic device that measures external illuminance using a camera sensor can be reduced by reducing the time for which the camera sensor is turned on.

According to various embodiments disclosed in this document, by measuring external illuminance using a camera sensor periodically and for a short period of time, battery consumption can be reduced and changes in the external environment can be responded to.

In addition, according to various embodiments disclosed in this document, when an illuminance sensor arranged under a display screen acquires external illuminance, if the external illuminance value acquired by the illuminance sensor is affected by the brightness of the display screen, the external illuminance value can be determined more accurately using a camera sensor.

In addition, various effects may be provided that are directly or indirectly identified through this document.

FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.
FIG. 2 is a block diagram illustrating a camera module according to various embodiments.
FIG. 3A is a diagram illustrating a housing, a display, and a camera sensor of an electronic device according to one embodiment.
FIG. 3b is a diagram illustrating a housing, a display, and a camera sensor of an electronic device according to another embodiment.
FIG. 4 is a flowchart illustrating a method for an electronic device to obtain external illuminance using a camera sensor according to one embodiment.
FIG. 5A is a graph showing the current consumed by a camera module over time according to one embodiment.
FIG. 5b is a graph showing parameters used for measuring external illuminance according to the brightness of preview image information according to one embodiment.
FIG. 6 is a flowchart illustrating a method in which an electronic device periodically measures external illuminance using a camera sensor and controls the brightness of a screen according to one embodiment.
FIG. 7 is a diagram illustrating a method in which an electronic device according to one embodiment applies different gain values to edge and center regions of preview image information acquired using a camera sensor.
FIG. 8 is a diagram illustrating a method in which an electronic device according to one embodiment applies different gain values to edge and center areas of preview image information acquired using a camera sensor.
FIG. 9 is a flowchart illustrating a method in which an electronic device acquires external illuminance using a camera sensor and displays a screen of a display with a set brightness according to one embodiment.
FIG. 10 is a diagram showing a housing, a display, a camera sensor, a speaker, and a light sensor of an electronic device according to one embodiment.
FIG. 11 is a flowchart illustrating a method in which an electronic device acquires external illuminance using an illuminance sensor and controls the brightness of a display based on the external illuminance acquired using a camera sensor according to one embodiment of the present invention.
In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

Hereinafter, various embodiments of the present invention will be described with reference to the attached drawings. However, this is not intended to limit the present invention to specific embodiments, but should be understood to include various modifications, equivalents, and/or alternatives of the embodiments of the present invention.

FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with the electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with the electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input device (150), an audio output device (155), a display device (160), an audio module (170), a sensor module (176), an interface (177), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the display device (160) or the camera module (180)), or may have one or more other components added. In some embodiments, some of these components may be implemented as a single integrated circuit. For example, a sensor module (176) (e.g., a fingerprint sensor, an iris sensor, or a light sensor) may be implemented embedded in a display device (160) (e.g., a display).

The processor (120) may control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing, for example, software (e.g., a program (140)), and may perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculations, the processor (120) may load a command or data received from another component (e.g., a sensor module (176) or a communication module (190)) into the volatile memory (132), process the command or data stored in the volatile memory (132), and store the resulting data in the nonvolatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor), and a secondary processor (123) (e.g., a graphic processing unit, an image signal processor, a sensor hub processor, or a communication processor) that may operate independently or together therewith. Additionally or alternatively, the auxiliary processor (123) may be configured to use less power than the main processor (121), or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121), or as a part thereof.

The auxiliary processor (123) may control at least a portion of functions or states associated with at least one of the components of the electronic device (101) (e.g., the display device (160), the sensor module (176), or the communication module (190)), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)).

The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or nonvolatile memory (134).

The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

The input device (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input device (150) can include, for example, a microphone, a mouse, a keyboard, or a digital pen (e.g., a stylus pen).

The audio output device (155) can output an audio signal to the outside of the electronic device (101). The audio output device (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes such as multimedia playback or recording playback, and the receiver can be used to receive an incoming call. According to one embodiment, the receiver can be implemented separately from the speaker or as a part thereof.

The display device (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display device (160) can include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display device (160) can include touch circuitry configured to detect a touch, or a sensor circuitry configured to measure a strength of a force generated by the touch (e.g., a pressure sensor).

The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can obtain sound through an input device (150), or output sound through an audio output device (155), or an external electronic device (e.g., an electronic device (102)) (e.g., a speaker or a headphone) directly or wirelessly connected to the electronic device (101).

The sensor module (176) can detect an operating state (e.g., power or temperature) of the electronic device (101) or an external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., the electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that a user can perceive through a tactile or kinesthetic sense. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module (180) can capture still images and moving images. According to one embodiment, the camera module (180) can include one or more lenses, image sensors, image signal processors, or flashes.

The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

The battery (189) can power at least one component of the electronic device (101). In one embodiment, the battery (189) can include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

The communication module (190) may support establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., the electronic device (102), the electronic device (104), or the server (108)), and performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., the application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module or a power line communication module). A corresponding communication module among these communication modules can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi direct, or IrDA (infrared data association)) or a second network (199) (e.g., a long-range communication network such as a cellular network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify and authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). According to one embodiment, the antenna module (197) can include one antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) can include a plurality of antennas. In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), can be selected from the plurality of antennas by, for example, the communication module (190). A signal or power can be transmitted or received between the communication module (190) and the external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, another component (e.g., an RFIC) can be additionally formed as a part of the antenna module (197).

At least some of the above components may be connected to each other and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, a general purpose input and output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI)).

In one embodiment, a command or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102, 104) may be the same or a different type of device as the electronic device (101). In one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of executing the function or service itself or in addition, request one or more external electronic devices to perform at least a part of the function or service. One or more external electronic devices that have received the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a part of a response to the request. For this purpose, for example, cloud computing, distributed computing, or client-server computing technology may be used.

FIG. 2 is a block diagram (200) illustrating a camera module (180) according to various embodiments. Referring to FIG. 2, the camera module (180) may include a lens assembly (210), a flash (220), an image sensor (230), an image stabilizer (240), a memory (250) (e.g., a buffer memory), or an image signal processor (260). The lens assembly (210) may collect light emitted from a subject that is a target of an image capture. The lens assembly (210) may include one or more lenses. According to one embodiment, the camera module (180) may include a plurality of lens assemblies (210). In this case, the camera module (180) may form, for example, a dual camera, a 360-degree camera, or a spherical camera. Some of the plurality of lens assemblies (210) may have the same lens properties (e.g., angle of view, focal length, autofocus, f number, or optical zoom), or at least one lens assembly may have one or more lens properties that are different from the lens properties of the other lens assemblies. The lens assembly (210) may include, for example, a wide-angle lens or a telephoto lens.

The flash (220) can emit light used to enhance light emitted or reflected from a subject. According to one embodiment, the flash (220) can include one or more light-emitting diodes (e.g., red-green-blue (RGB) LEDs, white LEDs, infrared LEDs, or ultraviolet LEDs), or a xenon lamp. The image sensor (230) can acquire an image corresponding to the subject by converting light emitted or reflected from the subject and transmitted through the lens assembly (210) into an electrical signal. According to one embodiment, the image sensor (230) can include one image sensor selected from among image sensors having different properties, such as an RGB sensor, a black and white (BW) sensor, an IR sensor, or a UV sensor, a plurality of image sensors having the same property, or a plurality of image sensors having different properties. Each image sensor included in the image sensor (230) may be implemented using, for example, a CCD (charged coupled device) sensor or a CMOS (complementary metal oxide semiconductor) sensor.

The image stabilizer (240) can move at least one lens or image sensor (230) included in the lens assembly (210) in a specific direction or control the operating characteristics of the image sensor (230) (e.g., adjust the read-out timing) in response to the movement of the camera module (180) or the electronic device (101) including the same. This can compensate for at least some of the negative effects of the movement on the image being captured. In one embodiment, the image stabilizer (240) can detect such movement of the camera module (180) or the electronic device (101) by using a gyro sensor (not shown) or an acceleration sensor (not shown) disposed inside or outside the camera module (180). In one embodiment, the image stabilizer (240) can be implemented as, for example, an optical image stabilizer. The memory (250) can temporarily store at least a portion of an image acquired through the image sensor (230) for the next image processing task. For example, when image acquisition is delayed due to a shutter, or when a plurality of images are acquired at high speed, the acquired original image (e.g., a Bayer-patterned image or a high-resolution image) is stored in the memory (250), and a corresponding copy image (e.g., a low-resolution image) can be previewed through the display device (160). Thereafter, when a specified condition is satisfied (e.g., a user input or a system command), at least a portion of the original image stored in the memory (250) can be acquired and processed by, for example, an image signal processor (260). According to one embodiment, the memory (250) can be configured as at least a portion of the memory (130), or as a separate memory that operates independently therefrom.

The image signal processor (260) can perform one or more image processing operations on an image acquired through the image sensor (230) or an image stored in the memory (250). The one or more image processing operations can include, for example, depth map generation, three-dimensional modeling, panorama generation, feature point extraction, image synthesis, or image compensation (e.g., noise reduction, resolution adjustment, brightness adjustment, blurring, sharpening, or softening). Additionally or alternatively, the image signal processor (260) can perform control (e.g., exposure time control or read-out timing control) on at least one of the components included in the camera module (180) (e.g., the image sensor (230)). An image processed by the image signal processor (260) may be stored back in the memory (250) for further processing or provided to an external component of the camera module (180) (e.g., the memory (130), the display device (160), the electronic device (102), the electronic device (104), or the server (108)). According to one embodiment, the image signal processor (260) may be configured as at least a part of the processor (120) or may be configured as a separate processor that operates independently of the processor (120). When the image signal processor (260) is configured as a separate processor from the processor (120), at least one image processed by the image signal processor (260) may be displayed through the display device (160) as is or after undergoing additional image processing by the processor (120).

According to one embodiment, the electronic device (101) may include a plurality of camera modules (180), each having different properties or functions. In this case, for example, at least one of the plurality of camera modules (180) may be a wide-angle camera, and at least another may be a telephoto camera. Similarly, at least one of the plurality of camera modules (180) may be a front camera, and at least another may be a rear camera.

FIG. 3a is a diagram illustrating a housing (310), a display (320) (e.g., the display device (160) of FIG. 1), and a camera sensor (330) (e.g., the image sensor (230) of FIG. 2) of an electronic device (101) according to one embodiment. FIG. 3b is a diagram illustrating a housing (310), a display (320), and a camera sensor (330) of an electronic device (101) according to another embodiment.

In one embodiment, the housing (310) may form an outer perimeter or frame of the electronic device (101). The housing (310) may include a first side or front surface of the electronic device (101), a second side or back surface facing in an opposite direction from the first side, and a side surface surrounding a space between the front surface and the back surface. The first side of the housing (310) may be formed by a front plate that is at least partially substantially transparent. The front plate may include a substantially rectangular frame or bezel forming the first side of the housing (310). For example, the front plate forming the front surface of the housing (310) may be a glass plate or a polymer plate including various coating layers. The second side of the housing (310) may be formed by a substantially opaque back plate. For example, the back plate forming the back of the housing (310) may be formed by a coated or colored glass, ceramic, polymer, metal (e.g., aluminum, stainless steel (STS), or magnesium), or a combination of at least two of the above materials. The side surfaces of the housing (310) may be formed by a side bezel structure or side member that is joined to the front plate and the back plate and includes a metal and/or a polymer. In some embodiments, the back plate and the side bezel structure of the housing (310) may be formed integrally and include the same material (e.g., a metal material such as aluminum).

In one embodiment, the display (320) can be exposed to the front of the housing (310) to display a screen. The display (320) can be exposed through a significant portion of the front plate. For example, the display (320) can be exposed at least partially through the front plate forming a portion of the first surface and a side surface. The display (320) can display a screen to the outside of the electronic device (101) to provide visual information to the user. For example, the display (320) can display a screen related to information related to an operating state of an application (e.g., an application (146) of FIG. 1) executed by the electronic device (101), information obtained by the electronic device (101) using a communication module (e.g., a communication module (190) of FIG. 1), and/or information processed by the application (146) of the electronic device (101).

In one embodiment, as shown in FIG. 3A, at least some of the corners of the display (320) may be formed in a shape other than a square, which is the shape of the front plate. A part of the front plate arranged in the border area of the display (320) may have a different thickness from the remaining area. For example, if a part of the front plate arranged in the border area of the display (320) is thicker than the remaining area, a part of the front plate forming the part may be formed in the shape of at least one notch (325). The notch (325) may be formed at the upper end of the terminal, such as the upper center, left side, or right side of the electronic device (101). For example, a camera module (180) including a camera sensor (330) may be arranged in the notch (325). As another example, the notch (325) may have a shape including only the camera sensor (330). In this case, at least a part of the camera module (180) may be hidden behind the back of the display (320). In one embodiment, the camera sensor (330) may be placed in an area of the front surface of the housing (310) excluding an area where the display (320) is placed, as shown in FIG. 3A. For example, the camera sensor (330) may be placed in a notch of the front plate. In another embodiment, the camera sensor (330) may be placed in at least a portion of the display (320) as shown in FIG. 3B. For example, the display (320) may have at least one hole formed to expose the camera sensor (330) to the front. For example, the hole may be placed at one edge of the display (320). The camera module (180) may be placed on the back surface of the display (320), and the camera sensor (330) may be viewed externally through the hole on the display (320).

In one embodiment, a plurality of camera modules (180) and/or camera sensors (330) may be arranged. In this case, at least one camera sensor (330) included in at least one camera module (180) among the plurality of camera modules (180) and/or camera sensors (330) may be used for illuminance measurement.

In one embodiment, the camera sensor (330) can measure external illuminance toward the front of the housing (310). For example, the external illuminance can be a value representing the brightness of the environment perceived in the direction toward which the front plate is facing.

In one embodiment, the camera sensor (330) can detect external light incident at an angle within a specified angle range from the front direction of the housing (310) toward the electronic device (101) and measure the external illuminance. For example, the camera sensor (330) can measure the external illuminance by obtaining preview image information. The preview image information may refer to information of a camera image obtained for the purpose of displaying an external environment recognized by the camera sensor (330) on the display (320). For example, the preview image information may refer to initial information of a preview image that is obtained by the camera sensor (330) and transmitted to the internal processor (120) before being output on an actual screen and prepared for display on the display (320). The preview image displayed on the display (320) of the electronic device (101) can be generated and output based on the internally accumulated preview image information. For example, the preview image information may be internal information of the electronic device (101) that is not actually recognizable by the user, and may be distinguished from the preview image output to the display (320). An image signal processor (e.g., image signal processor (260) of FIG. 2) included in the camera module (180) may analyze the preview image information acquired by the camera sensor (330) and acquire parameters necessary for measuring external illuminance.

FIG. 4 is a flowchart (400) illustrating a method for an electronic device (101) to obtain external illuminance using a camera sensor (330) according to one embodiment.

According to an embodiment, an electronic device (101) or a processor (e.g., the processor (120) of FIG. 1) may detect a display (e.g., the display (320) of FIG. 3) turn-on event at operation 410. The turn-on event of the display (320) may include power-on, a key input during standby, reception of an event via a communication module (e.g., the communication module (190) of FIG. 1), or a periodic notification. For example, when the electronic device (101) is turned on, the display (320) may be turned on to display a booting screen of the electronic device (101). As another example, when the electronic device (101) is turned on but the display (320) is turned off (e.g., in a standby state), a key input or a touch input is received by the electronic device (101), and the display (320) may perform a wake-up to turn on the display. Wake-up may be performed when the electronic device (101) receives an input from a user through the display (320) or an input device of the electronic device (101) (e.g., the input device (150) of FIG. 1), when information and/or an event is acquired from the outside through the communication module (190) of the electronic device (101), or when there is a change in the proximity of the electronic device (101) to another object or a change in the angle with respect to the ground. As another example, a turn-on event of the display (320) may occur for a schedule notification stored in a memory of the electronic device (101) (e.g., the memory (130) of FIG. 1) or a schedule notification stored in an application (e.g., the application (146) of FIG. 1).

According to an embodiment, the processor (120) may turn on the camera sensor (330) to obtain external illuminance by using the camera sensor (e.g., the camera sensor (330) of FIG. 3) when the display (320) is turned on, at operation 420. When the camera sensor (330) is used, since the brightness of the surroundings cannot be checked if the camera sensor (330) is turned off or in a sleep (or inactive) state, the processor (120) may be set to turn on the camera sensor (330) to check the brightness of the surroundings when the display (320) is turned on.

In one embodiment, the camera module (e.g., the camera module (180) of FIG. 1) may be turned on to turn on the camera sensor (330). For example, the camera module (180) may be turned on to measure external illuminance through the camera sensor (330). As another example, the processor (120) may selectively turn on and/or turn off only the camera sensor (330).

According to an embodiment, the processor (120) may acquire preview image information for a specified period of time through the camera sensor (330) at operation 430. For example, the processor (120) may turn on the camera sensor (330) for a specified period of time and acquire exposure time and brightness values in an acquisition period after the enable period. For example, the processor (120) may acquire exposure time and brightness values necessary to calculate external illuminance for a specified period of time from preview image information acquired using the camera sensor (330) to check a change in external illuminance. The camera sensor (330) may transmit the exposure time and brightness values to the processor (120) without capturing and/or storing the preview image information.

In one embodiment, the enable period may be a period in which the camera sensor (330) prepares to acquire preview image information after being turned on.

In one embodiment, the acquisition period may be a period during which the camera sensor (330) can acquire information related to external illuminance from an image after the camera sensor (330) is turned on. The camera sensor (330) can acquire and analyze exposure time and brightness values through preview image information during the time set as the acquisition period.

In one embodiment, the exposure time may be the time that the camera sensor (330) is exposed to external light. The exposure time may be an exposure value (EV). The exposure time may be set by the shutter speed of the camera module (180). When the camera module (180) acquires preview image information in a space with low external illumination, the exposure time may increase. When the camera module (180) acquires preview image information in a space with high external illumination, the preview image information may be sufficiently acquired even with a short exposure time, so that the exposure time may decrease. The exposure time may be controlled by the camera module (180). The resolving power of the camera module (180) may be superior when the exposure time is used than when the aperture sensitivity of the camera module (180) or the sensitivity of the image sensor (230) is used. The resolving power may be a numerical value representing the performance of precisely distinguishing external illumination.

In one embodiment, the brightness value may be a value related to brightness information of the surroundings obtained from an image sensor of the camera module (180) (e.g., the image sensor (230) of FIG. 2) and calculated according to an internal algorithm. The brightness value may be a BV (Brightness value).

In one embodiment, the processor (120) may obtain exposure time and/or brightness values from preview image information using the camera sensor (330) when the display (320) is turned on. When the display (320) is turned on, the processor (120) may measure external illuminance to control the brightness of the display (320).

In one embodiment, the processor (120) may turn off the camera sensor (330) at operation 440. The camera sensor (330) may be turned off after performing an operation necessary to obtain external illuminance. For example, the processor (120) may turn on the camera sensor (330) at a specified cycle. When the camera sensor (330) repeats the turn-on and turn-off operations at a specified cycle, the power consumption of the camera sensor (330) may be reduced compared to when the camera sensor (330) is continuously maintained in a turned-on state. According to one embodiment, the processor (120) may variably determine the specified cycle for turning on the camera sensor (330). For example, the processor (120) may adjust the specified cycle based on the obtained external illuminance, as described below with reference to FIG. 6.

According to one embodiment, the processor (120) may obtain external illuminance by processing the exposure time and brightness value at operation 450. The processor (120) may calculate external illuminance by selectively using the exposure time and/or brightness value depending on the brightness of the preview image information.

An electronic device (101) according to another embodiment may perform operations 440 and 450 simultaneously after operation 430. An electronic device (101) according to another embodiment may obtain external illuminance by processing exposure time and brightness values while turning off the camera sensor (330).

According to one embodiment, the processor (120) may control the brightness of the display (320) according to the measured external illuminance at operation 460. The processor (120) may reduce the brightness of the screen of the display (320) in a low illuminance environment to minimize glare to the user due to the brightness of the screen. The processor (120) may increase the brightness of the screen in a high illuminance environment to enable the user to recognize the content displayed on the display (320). For example, the processor (120) may change the brightness of the screen in proportion to the measured external illuminance.

FIG. 5a is a graph (500) showing the current consumed by a camera module (180) over time according to one embodiment.

An electronic device (101) (e.g., processor (120)) according to one embodiment may turn on a camera module (180) and/or a camera sensor (330) to measure external illuminance through the camera sensor (330).

In one embodiment, the processor (120) may turn on the camera sensor (330) at a specified cycle. The camera sensor (330) may be turned on at a first point in time when the first cycle starts and/or at a second point in time when the second cycle starts. The camera sensor (330) may be turned off during the middle of the first cycle and then turned on again at the second point in time. For example, the first cycle and the second cycle may be substantially the same. As another example, the first cycle and the second cycle may be different depending on the state of the electronic device (101) or the surrounding environment. As another example, the first cycle or the second cycle may be changed depending on the state of the electronic device (101) or the surrounding environment.

In one embodiment, the time period during which the camera sensor (330) is turned on may correspond to an enable period and/or an acquisition period. The enable period may be a period during which the camera sensor (330) prepares to acquire preview image information after being turned on. The acquisition period may be a period during which the camera sensor (330) acquires preview image information and acquires exposure time and brightness values from the preview image information.

In one embodiment, the processor (120) may control the camera module (180) to perform a release operation to put the camera sensor (330) into a standby state for a certain period of time after the camera sensor (330) is turned off. For example, the time period during which the camera module (180) performs the release operation may be referred to as a release period.

In one embodiment, a first enable period may start at a time point (e.g., time point 1) when a first cycle starts, which is a time point when the camera sensor (330) is turned on, and a second enable period may start at a time point (e.g., time point 2) when a second cycle starts. The lengths of the first and second enable periods may be about 0.25 seconds or longer and about 0.3 seconds or shorter. However, the lengths of these periods may be set differently depending on the electrical specifications, design, or operating method of the camera.

In one embodiment, a first acquisition interval may start at a time when a first enable interval in a first cycle ends, and a second acquisition interval may start at a time when a second enable interval in a second cycle ends. The lengths of the first and second acquisition intervals may be greater than or equal to about 0.03 seconds and less than or equal to about 0.04 seconds.

In one embodiment, the length of the acquisition interval may be substantially equal to one frame included in the specified time period. If preview image information is recognized during at least one frame interval among a plurality of frames included in the specified time period, the exposure time and brightness value for the specified time period may be acquired. In order to reduce power consumption of the camera module, acquisition of the exposure time and brightness value using the camera sensor may be performed for a time substantially equal to one frame interval (e.g., approximately 1/30 second when there are 30 frames per second). Depending on the situation or depending on the capability supported by the camera sensor (330), the processor (120) may set the turn-on cycle of the camera sensor (330). For example, if the camera sensor (330) can acquire image information of 60 frames per second, the processor (120) may set the cycle to 1/60 second. For another example, if the camera sensor (330) can acquire N frames of image information per second, the processor (120) may set the period to 1/N sec. This may be to enable the camera sensor (330) to perform periodic operations repeatedly by minimizing the power consumed to acquire preview image information once.

In one embodiment, in the acquisition section, a first current (I1) may be supplied to the camera sensor (330). For example, the first current (I1) may be a current used for the camera module (180) and/or the camera sensor (330) to acquire preview image information and perform an operation to acquire an exposure time and a brightness value from the preview image information. In the enable section, a second current (I2) lower than the first current (I1) may flow. The second current (I2) may be a current used for the camera sensor (330) to turn on and prepare to acquire preview image information. The flow of current may mean that current is consumed in the camera module (180) and/or the camera sensor (330). The consumption of current in the camera module (180) and/or the camera sensor (330) may mean that the camera module (180) and/or the camera sensor (330) consumes power.

In one embodiment, when the camera sensor (330) operates periodically to acquire exposure time and brightness values, it may be important to control to keep power for one operation to a minimum. To prevent the camera module (180) from consuming power separately by performing an operation to capture a preview image or store it in memory (130), the processor (120) can process the exposure time and/or brightness values in real time.

In one embodiment, the image signal processor (260) may transmit the exposure time and brightness values to the application (146) in real time when acquiring the exposure time and brightness values from one frame through a separate interface called callback that is internally designed for image quality development purposes. The image signal processor (260) may not require a separate process of capturing and/or storing the preview image information. The processor (120) may receive and utilize the exposure time and brightness values transmitted to the application (146) that processes the preview image information. If the processor (120) directly acquires the exposure time and brightness values among the preview image information from the application (146), the current consumed by the camera module (180) may be reduced. According to one embodiment, the image signal processor (260) may also transmit the exposure time and brightness values to the processor (120).

In one embodiment, the camera sensor (330) may be turned off after the acquisition period ends. A third current (I3) may flow through the camera sensor (330) and/or the camera module (190) during the release period.

FIG. 5b is a graph (550) showing parameters used for measuring external illuminance according to the brightness of preview image information according to one embodiment.

In one embodiment, the processor (120) may measure the external illuminance using an exposure time when the brightness of the preview image information is lower than or equal to a first brightness, measure the external illuminance using a brightness value when the brightness of the preview image information exceeds the first brightness, and measure the external illuminance using a brightness value after using the exposure time when the brightness of the preview image information is lower than or equal to a second brightness that is lower than the first brightness.

In one embodiment, the first luminance may be a luminance that distinguishes the luminance of the preview image information in an indoor environment and the luminance of the preview image information in an outdoor environment. If the luminance of the preview image information is lower than the first luminance, the processor (120) may determine that the camera sensor (330) is affected by lighting, and if it is higher than the first luminance, the processor (120) may determine that the camera sensor (330) is affected by sunlight. For example, the first luminance may be about 3000 lux.

In one embodiment, the second luminance may be a luminance that distinguishes between a general environment where a light source exists and a low-light environment where no light source exists, such as a dark room or a nighttime environment. If the luminance of the preview image information is higher than the second luminance, the processor (120) may determine that the camera sensor (330) is in a general environment, and if it is lower than the second luminance, the processor (120) may determine that the camera sensor (330) is in a low-light environment. For example, the second luminance may be approximately 15 lux.

FIG. 6 is a flowchart (600) illustrating a method for an electronic device (101) to periodically measure illuminance using a camera sensor (330) according to one embodiment.

According to an embodiment, an electronic device (101) or a processor (e.g., the processor (120) of FIG. 1) may determine, in operation 601, whether a display (e.g., the display (320) of FIG. 3) turn-off event has occurred. Since it is necessary to control the brightness of the screen only when the display (320) is turned on, the task of measuring external illuminance using a camera sensor (e.g., the camera sensor (330) of FIG. 3) may be meaningful only when the display (320) is turned on. The processor (120) may terminate the illuminance measurement process when the display (320) is turned off. If there is no turn-off event of the display (320) and the display is turned on, the processor (120) may proceed to operation 610. According to one embodiment, the processor (120) may perform operation 601 after the turn-on of the display (320) is detected, as described above in operation 410 of FIG. 4. For example, the processor (120) may perform operation 601 of FIG. 6 while the display (320) is turned on. For example, operation 420 of FIG. 4 may include operations 601, 610, and 620 of FIG. 6. For example, the processor (120) may perform operations 601, 610, and 620 to adjust the brightness of the display (320).

According to an embodiment, the processor (120) may determine, at operation 610, whether a specified period has arrived. The specified period may be a period in which the camera sensor (330) must acquire preview image information to measure external illuminance and reflect the measured external illuminance in the brightness control of the display (320). For example, if it takes about 3 seconds or more and about 5 seconds or less to control the brightness of the display (320) after measuring the external illuminance in a stationary indoor environment, the specified period may be set to about 4 seconds. As another example, if it takes about 1 second or more and about 2 seconds or less to control the brightness of the display (320) after measuring the external illuminance in a moving situation or an outdoor environment, the specified period may be set to about 1.5 seconds.

In one embodiment, when the camera module (180) is kept turned on, the operating current for operating the camera module (180) may increase. Accordingly, the consumption of the battery (e.g., the battery (189) of FIG. 1) may increase. When the camera module (180) is not used, it is turned off and then temporarily turned on at designated intervals, so that the camera module (180) is not continuously operated, thereby reducing the operating current.

In one embodiment, the processor (120) can change a cycle (e.g., a turn-on cycle of the camera sensor (330)) specified for the camera sensor (330) to obtain an exposure time and brightness value according to an external environment. If the processor (120) increases the specified cycle for turning on the camera sensor (330) to measure external illuminance, the power consumption of the camera sensor (330) can be reduced. If the processor (120) shortens the specified cycle for turning on the camera sensor (330) to measure external illuminance, the camera sensor (330) can quickly detect a change in external illuminance.

In one embodiment, the processor (120) can change the designated cycle according to the surrounding environment. The processor (120) can detect the surrounding environment of the electronic device (101) using the communication module (190). For example, the processor (120) can determine whether the surrounding environment is an indoor environment or an outdoor environment using the reception status of a global positioning system (GPS). According to one embodiment, if the processor (120) determines that the surrounding environment is an indoor environment, it can turn on the camera sensor (330) to increase the designated cycle for measuring external illuminance, thereby reducing the power consumption of the camera sensor (330). According to one embodiment, if the processor (120) determines that the surrounding environment is an outdoor environment, it can turn on the camera sensor (330) to decrease the designated cycle for measuring external illuminance, thereby allowing the camera sensor (330) to quickly detect a change in external illuminance. According to another embodiment, the processor (120) can also set the operating cycles for indoor and/or outdoor to be reversed. For example, after controlling the brightness of the display to be bright based on the judgment of the outdoor environment, the operation cycle can be maintained by judging that it is unlikely to cause discomfort to the user due to the bright screen even after moving indoors while the screen is still bright.

In one embodiment, the processor (120) can change the specified period according to the time information. The processor (120) can receive and/or link the time information indicating the time at the current location of the electronic device (101) by using the communication circuit (190). For example, the processor (120) can receive standard time information (UTC or GMT) by using the communication circuit (190) and receive the current location by GPS to determine whether the time at which the electronic device (101) is operating is daytime or nighttime. If the time at the current location according to the time information is daytime, the processor (120) can decrease the specified period so that the camera sensor (330) can quickly detect a change in external illumination. If the time at the current location according to the time information is nighttime, the processor (120) can increase the specified period so as to reduce the power consumption of the camera sensor (330).

In one embodiment, the processor (120) may change the specified period depending on whether the electronic device (101) moves and/or rotates. The processor (120) may detect whether the electronic device moves and/or rotates by using an acceleration sensor and/or a gyro sensor included in a sensor module (e.g., the sensor module (176) of FIG. 1). The processor (120) may receive information related to the movement and/or rotation of the electronic device (101). According to one embodiment, the processor (120) may decrease the specified period when the electronic device (101) moves and/or rotates, thereby allowing the camera sensor (330) to quickly detect a change in external illuminance. According to one embodiment, the processor (120) may increase the specified period when the electronic device (101) is fixed at a constant location, thereby reducing the power consumption of the camera sensor (330).

In one embodiment, the processor (120) may increase a designated cycle for measuring the external illuminance by turning on the camera sensor (330) when the measured external illuminance remains substantially the same for a predetermined number of times or more and the position of the electronic device (101) is fixed. The processor (120) may determine that the external illuminance will remain substantially the same when the external illuminance measured by the camera sensor (330) for each designated cycle remains substantially the same for a predetermined number of times or more and the position of the electronic device (101) is fixed. When the processor (120) determines that the external illuminance will remain substantially the same, the processor (120) may turn on the camera sensor (330) to increase a designated cycle for measuring the external illuminance, thereby reducing power consumption of the camera sensor (330).

Referring again to FIG. 6, the processor (120) according to one embodiment may, at operation 620, turn on the camera module (180) and then turn on the camera sensor (330) based on the judgment result. For example, the processor (120) may turn on the camera module (180) and measure external illuminance through the camera sensor (330). As another example, the processor (120) may turn on only the camera sensor (330).

According to one embodiment, the processor (120) may acquire the exposure time and brightness value from the preview image information acquired using the camera sensor (330) within the turned-on time at operation 630. The acquisition section may be a section in which the camera sensor (330) may acquire information related to external illuminance from the image after the camera sensor (330) is turned on. In one embodiment, the image signal processor (e.g., the image signal processor (260) of FIG. 2) of the camera module (180) may acquire the exposure time and brightness value from the preview image information. According to one embodiment, the image signal processor (260) may transmit the exposure time and brightness value to an application (e.g., the application (146) of FIG. 1) through an interface. According to another embodiment, the image signal processor (260) may transmit the exposure time and brightness value to a processor (e.g., the processor (120) of FIG. 1).

According to an embodiment, the processor (120) may measure external illuminance using the exposure time and/or the brightness value at operation 640. The exposure time may be advantageous in obtaining a uniform illuminance value regardless of the location of the light source in an indoor lighting environment or a low-light environment. The brightness value may be advantageous in decomposing the illuminance value in a bright image. Alternatively, depending on the situation, the brightness value and the exposure time value may be processed together to decompose the illuminance value, and in a low-light environment, the optimal decomposition capability may be achieved when the actual exposure time value and the brightness value are considered together.

In one embodiment, the processor (120) can obtain the illuminance value of the preview image by using the exposure time in an indoor environment with normal illuminance. The processor (120) can improve the resolution capability in a bright environment by using the brightness value in an outdoor bright environment with high illuminance. The processor (120) can additionally check the brightness value after checking the exposure time in order to secure additional resolution capability of the brightness in a night environment with low illuminance.

According to one embodiment, the processor (120) may control the brightness of the display (320) according to the measured external illuminance at operation 650. The processor (120) may reduce the brightness of the display (320) in a low illuminance environment to minimize glare to the user due to the brightness of the screen of the display (320). In a high illuminance environment, the processor (120) may increase the brightness of the display (320) so that the user can recognize the content displayed on the display (320). For example, the processor (120) may change the brightness of the display (320) in proportion to the measured external illuminance.

FIGS. 7 and 8 are drawings (700, 800) showing a method for applying different gain values to edge areas and center areas (730, 830) of preview image information acquired by an electronic device (101) using a camera sensor (330) according to one embodiment.

In one embodiment, the preview image information acquired by the camera sensor (330) may have a first region (710, 810) which is a region having high external illuminance and a second region (720, 820) which is a region having low external illuminance. For example, the electronic device (101) (e.g., the processor (120)) may determine that the preview image information has higher external illuminance than the actual area when the first region (710) includes the entire central region (730) in the preview image information as illustrated in FIG. 7. As another example, the electronic device (101) (e.g., the processor (120)) may determine that the preview image information has lower external illuminance than the actual area when the second region (820) occupies most of the central region (830) in the preview image information as illustrated in FIG. 8. Even when the average external illuminance of the preview image information (700) of FIG. 7 and the average external illuminance of the preview image information (800) of FIG. 8 are substantially the same, the electronic device (101) (e.g., processor (120)) may determine that the average external illuminance of the preview image information (700) of FIG. 7 is higher than the average external illuminance of the preview image information (800) of FIG. 8.

In one embodiment, the processor (120) may apply a first gain value to the exposure time and brightness values acquired in an edge region of the preview image information acquired by the camera sensor (330), and apply a second gain value to the exposure time and brightness values acquired in a central region (730, 830) of the preview image information. The second gain value may have a smaller value than the first gain value. For example, when calculating external illuminance by applying a filter to the central region (730, 830) of the preview image information, the influence of the exposure time and brightness values acquired in the central region (730, 830) may be reduced. The processor (120) may prevent the camera sensor (330) from distorting the average external illuminance by weighting the central region (730, 830).

In one embodiment, when applying a small gain value to the exposure time and brightness values acquired in the central region (730, 830), the processor (120) may use an algorithm for calculating external illuminance by using the exposure time and brightness values acquired from the preview image information to reduce the deviation according to the position of the light source. In order to reduce the deviation according to the position of the light source, the processor (120) may reduce the reflection ratio of the exposure time and brightness values of the central region (730, 830) among the preview image information acquired from the camera sensor (330), and may use the exposure time and brightness values acquired by being diffused to the edge region.

FIG. 9 is a flowchart (900) illustrating a method in which an electronic device (101) obtains external illuminance using a camera sensor (330) and displays a display screen with a screen brightness of a defined value according to one embodiment.

According to an embodiment, a processor (e.g., processor (120) of FIG. 1) may receive an event for turning on a display (e.g., display (320) of FIG. 3) at operation 910. For example, the electronic device (101) may be turned on by turning on the power. As another example, the electronic device (101) may be in a state where an event for turning on the display (320) of the electronic device (101) is received by a user's input or reception of information from the outside.

According to one embodiment, the processor (120) may initiate a display (320) turn-on at operation 915. For example, the display (320) may initiate operation at minimum brightness. The display (320) may wait in a state where it can receive external illumination from the camera module (180).

According to one embodiment, a processor (120) may perform a camera on sequence for acquiring external illumination at operation 920. The processor (120) may turn on a camera module (e.g., a camera module (180) of FIG. 1) to turn on a camera sensor (e.g., a camera sensor (330) of FIG. 3).

According to an embodiment, the processor (120) may maintain the camera operation for a predetermined period of time at operation 930. The camera sensor (330) may be turned on, may pass through an enable section, and then enter an acquisition section. The processor (120) may be set to allow the camera sensor (330) to maintain the acquisition section for a predetermined period of time. The camera sensor (330) may obtain an exposure time and a brightness value from the preview image information in the acquisition section. The acquisition section may be the same time as at least one frame. For example, if there are 30 frames per second, the processor (120) may be set to allow the camera sensor (330) to maintain the acquisition section for about 0.04 seconds, or practically about 1/30 seconds, taking into account the time within the error range.

According to an embodiment, the processor (120) may perform an off sequence to stop the camera operation of the camera module (180) and switch to a standby state at operation 940. The camera sensor (330) may return to the standby state or be turned off after the acquisition period. The camera sensor (330) may maintain the standby state or the turned-off state except for the acquisition period to reduce the current consumed by the camera module (180). After the acquisition period in which both the camera module (180) and the camera sensor (330) are turned on, the camera sensor (330) may be switched to the turn-off state in the release period, and the camera module (180) may also only consume a small amount of current (e.g., the third current (I3)) to perform a release operation to switch the camera sensor (330) to the standby state for a certain period after the camera sensor (330) is turned off.

In one embodiment, the processor (120) may, at operation 950, analyze the stored values to obtain the external illuminance. For example, the stored values may be exposure time and brightness values. The processor (120) may selectively analyze the exposure time and brightness values to calculate the external illuminance.

According to an embodiment, the electronic device (101) may increase the brightness of the screen based on the acquired external illuminance at operation 960. For example, after the display (320) is turned on at operation 915, the display (320) may start operating at a minimum brightness level. A brightness value of a screen displayed by the display (320) may be defined based on an external illuminance value. The brightness of the screen displayed by the display (320) may sequentially increase until it reaches the defined brightness value. For example, the processor (120) may start the screen displayed by the display (320) at a minimum perceptible brightness that allows a user to identify only whether the screen is turned on, and then sequentially increase the brightness of the screen so that it becomes brighter to the brightness of the screen defined by the external illuminance value.

As another example, the display (320) may start with a default value after being turned on in operation 915. The default value is not limited to the minimum perceived brightness or minimum luminance, but may be a specified luminance value or a luminance value that the display (320) was previously displaying while in operation and was about to be turned off. In this case, the luminance of the display (320) may be increased or decreased to change the luminance value of the display (320) to the brightness of the screen defined by the external illuminance value. As another example, after performing operation 950 of measuring and/or obtaining the external illuminance, the brightness of the display (320) may be determined and the display (320) may be turned on with the defined luminance value. In this case, operation 915 may be performed after performing operations 950 and 960.

According to an embodiment, the electronic device (101) may display a screen of a display with a screen brightness of a defined value at operation 970. The processor (120) may determine that the screen brightness control is completed when the brightness of the screen displayed on the display (320) becomes the same as the screen brightness defined by the set external illuminance value. The processor (120) may repeat operations 920 to 950 at designated intervals while the display (320) maintains a turn-on state. The processor (120) may perform operations 920 to 950 to additionally adjust the brightness of the display (320) based on the newly acquired external illuminance.

FIG. 10 is a diagram illustrating a housing (310), a display (320) (e.g., a display device (160) of FIG. 1), a camera sensor (330) (e.g., an image sensor (230) of FIG. 2), a speaker (1001) (e.g., an audio output device (155) of FIG. 1), and a light sensor (1010) (e.g., a sensor module (176) of FIG. 1) of an electronic device (101) according to one embodiment. The description of the housing (310), the display (320), and the camera sensor (330) of the electronic device (101) according to one embodiment may be substantially the same as the description of the housing (310), the display (320), and the camera sensor (330) described in conjunction with FIGS. 3A and 3B.

In one embodiment, the speaker (1001) may be placed on the front side of the electronic device (101). For example, the speaker (1001) may be placed at the top center of the electronic device (1001). The speaker (1001) may output sound. For example, the speaker (1001) may output the voice of the other party when the electronic device (101) receives a call. As another example, the speaker (1001) may output sound generated according to the execution and/or operation of an application (e.g., application (146) of FIG. 1) of the electronic device (101).

In one embodiment, the light sensor (1010) can measure external illuminance. The light sensor (1010) can transmit the measured external illuminance to a processor (e.g., the processor (120) of FIG. 1). The light sensor (1010) can be positioned so as to face the front side of the electronic device (101) from the bottom of the display (320). The light sensor (1010) can be positioned adjacent to the speaker (1001) of the electronic device (101). For example, the light sensor (1010) can be positioned adjacent to the left and/or right side of the speaker (100). In this case, when a user puts his/her ear to the speaker (1001), it can detect that the external illuminance changes below a specified illuminance and turn off the display (320). However, without being limited thereto, the light sensor (1010) may be positioned at any location within the display (320), at the center of the display (320), and/or within the display (320), from the bottom of the display (320). As another example, the light sensor (1010) may be positioned as a separate module on the outside of the display (320) and/or on the back side of the electronic device (101).

FIG. 11 is a flowchart (1100) illustrating a method for controlling brightness of a display (e.g., a display (320) of FIG. 10) using external illuminance acquired by using an illuminance sensor (e.g., an illuminance sensor (1010) of FIG. 10) of an electronic device according to one embodiment of the present invention, and based on the external illuminance, using a camera sensor (e.g., a camera sensor (330) of FIG. 10) to acquire external illuminance.

In one embodiment, the electronic device (101) or a processor (e.g., the processor (120) of FIG. 1) according to one embodiment may detect a display (320) turn-on event in operation 1110. The turn-on event of the display (320) may include power-on, a key input during standby, reception of an event via a communication module (e.g., the communication module (190) of FIG. 1), or a periodic notification. For example, when the electronic device (101) is turned on, the display (320) may be turned on to display a booting screen of the electronic device (101). As another example, when the electronic device (101) is turned on but the display (320) is turned off (e.g., in a standby state), if a key input or a touch input is received by the electronic device (101), the display (320) may perform a wake-up to turn on. The wake-up may be performed when the electronic device (101) receives an input from a user through the display (320) or an input device of the electronic device (101) (e.g., the input device (150) of FIG. 1), when information and/or events are acquired from the outside through the communication module (190) of the electronic device (101), or when the proximity of the electronic device (101) to another object changes or the angle with respect to the ground changes. As another example, a turn-on event of the display (320) may occur for a schedule notification stored in a memory of the electronic device (101) (e.g., memory (130) of FIG. 1) or an application (e.g., application (146) of FIG. 1).

According to an embodiment, the processor (120) may turn on the light sensor (1010) to obtain external illuminance by using the light sensor (1010) when the display (320) is turned on, at operation 1120. When the light sensor (1030) is used, if the light sensor (1010) is turned off or is in a sleep or inactive state, the brightness of the surroundings cannot be checked, so the processor (120) may be set to turn on the light sensor (1010) to check the brightness of the surroundings when the display (320) is turned on.

In one embodiment, the sensor module (e.g., the sensor module (176) of FIG. 1) may be turned on to turn on the light sensor (1010). For example, the sensor module (176) may be turned on to measure external illuminance through the light sensor (1010). As another example, the processor (120) may selectively turn on and/or turn off only the light sensor (1010).

According to an embodiment, the processor (120) may acquire external illuminance for a specified period of time through the illuminance sensor (1010) at operation 1130. For example, the processor (120) may turn on the illuminance sensor (1010) for a specified period of time and acquire the exposure time of the illuminance sensor (1010) and the brightness value of the external environment in an acquisition period after the enable period. For example, the processor (120) may acquire external illuminance for a specified period of time using the illuminance sensor (1010) to check a change in the external illuminance. The illuminance sensor (1010) may not store the acquired external illuminance but may transmit it to the processor (120).

In one embodiment, the enable period may be a period in which the light sensor (1010) prepares to acquire external illuminance after being turned on.

In one embodiment, the acquisition period may be a period during which the light sensor (1010) can acquire information related to external illuminance from an image after the light sensor (1010) is turned on. The light sensor (1010) can acquire and/or analyze exposure time and external brightness values during the time set as the acquisition period.

In one embodiment, the exposure time may be the time that the light sensor (1010) is exposed to external light. The brightness value may be a value related to brightness information of the surroundings obtained from the light sensor (1010) and calculated according to an internal algorithm.

In one embodiment, the processor (120) may measure external illuminance when the display (320) is turned on. The processor (120) may obtain exposure time and/or external brightness value using the illuminance sensor (1010) when the display (320) is turned on.

According to an embodiment, the processor (120) may determine, at operation 1140, whether the external illuminance is less than or equal to a specified first illuminance value. The first illuminance value may be a threshold value that the illuminance sensor (1010) determines as a low illuminance environment. The first illuminance value may be an external illuminance value in a dark environment, such as at night, dawn, and/or evening. The first illuminance value may be an illuminance value in a dark indoor space, such as a theater. For example, the first illuminance value may be approximately 100 lux. If the external illuminance is less than or equal to the first illuminance value, the external illuminance measured by the illuminance sensor (1010) may be affected by the brightness of the screen of the display (320). If the external illuminance measured by the illuminance sensor (1010) is less than or equal to the first illuminance value, the processor (120) may proceed to operation 1150 to correct the external illuminance to a more accurate value.

In one embodiment, the processor (120) may keep the camera sensor (330) turned off when the external illuminance exceeds the first illuminance value. When the external illuminance exceeds the first illuminance value, the external illuminance measured by the illuminance sensor (1010) may have a specified reliability level or higher. When the camera sensor (330) is kept in the turned-off state when the external illuminance exceeds the first illuminance value, the power consumption used by the camera sensor (330) may be reduced compared to when the camera sensor (330) is continuously kept in the turned-on state.

According to an embodiment, the processor (120) may, at operation 1150, turn on the camera sensor (330) to process the exposure time and brightness value to compensate for external illuminance. The external illuminance measured by the illuminance sensor (1010) under the display (320) in an environment below the first illuminance value may be affected by the brightness of the screen of the display (320). The processor (120) may turn on the camera sensor (330) and acquire the exposure time and brightness value in an acquisition section after the enable section. The camera sensor (330) may transmit the exposure time and brightness value to the processor (120). The enable section may be a section in which the camera sensor (330) prepares to acquire preview image information after being turned on. The acquisition section may be a section in which the camera sensor (330) may acquire information related to external illuminance from an image after being turned on. The camera sensor (330) can obtain and analyze exposure time and brightness values through preview image information during a time set as an acquisition period. The processor (120) can compensate external illuminance to a more accurate value using the camera sensor (330).

According to one embodiment, the processor (120) may control the brightness of the display (320) according to the compensated external illuminance at operation 1160. The processor (120) may reduce the brightness of the screen of the display (320) in a low illuminance environment to minimize glare to the user due to the brightness of the screen. The processor (120) may increase the brightness of the screen in a high illuminance environment to enable the user to recognize the content displayed on the display (320). For example, the processor (120) may change the brightness of the screen in proportion to the measured external illuminance.

An electronic device (101) according to various embodiments includes a housing (310), a display (320) that is exposed to the front of the housing (310) to display a screen, a camera sensor (330) that is arranged on at least a portion of the front of the housing (310) to measure external illuminance toward which the front of the housing (310) faces, a processor (120) that is operatively connected to the display (320) and the camera sensor (330), and a memory (130) that is operatively connected to the processor (120), wherein the memory (130) is configured to cause the processor (120) to detect a turn-on event of the display (320) (e.g., operation 410 of FIG. 4), turn on the camera sensor (330) to obtain the external illuminance by using the camera sensor (330) when the display (320) is turned on (e.g., operation 420 of FIG. 4), and Instructions for obtaining preview image information for a specified period of time through a camera sensor (330) (e.g., operation 430 of FIG. 4), turning off the camera sensor (330) (e.g., operation 440 of FIG. 4), obtaining the external illuminance using the exposure time and the brightness value (e.g., operation 450 of FIG. 4), and controlling the brightness of the display according to the measured external illuminance (e.g., operation 460 of FIG. 4) can be stored.

In one embodiment, the instructions may cause the processor (120) to obtain the exposure time and the brightness value from the preview image information at each specified cycle while the display (320) is turned on.

For example, the specified time may be substantially identical to any one frame included in the specified period.

In one embodiment, the instructions may cause the processor (120) to obtain the external illuminance using the exposure time when the brightness of the preview image information is equal to or lower than a first brightness, to obtain the external illuminance using the brightness value when the brightness of the preview image information exceeds the first brightness, and to obtain the external illuminance using the brightness value after using the exposure time when the brightness of the preview image information is equal to or lower than a second brightness that is lower than the first brightness.

In one embodiment, the electronic device (101) further includes a communication circuit (e.g., a communication module (190) of FIG. 1), and the instructions may cause the processor (120) to change the specified period according to the surrounding environment (e.g., current time information).

In one embodiment, the instructions may cause the processor (120) to vary the specified period depending on whether the electronic device is moved, rotated, and/or positioned.

In one embodiment, the instructions may cause the processor (120) to increase the specified period if the measured external illuminance remains substantially the same a predetermined number of times and the position of the electronic device (101) is fixed.

In one embodiment, the instructions may cause the processor (120) to apply a first gain value to the exposure time and the brightness value acquired in an edge region of the preview image information acquired by the camera sensor (330), and to apply a second gain value to the exposure time and the brightness value acquired in a central region (e.g., the central region (730) of FIG. 7 or the central region (830) of FIG. 8) of the preview image information, and the second gain value may be applied to a value smaller than the first gain value.

A method for measuring illuminance by using a camera sensor (330) in an electronic device (101) according to various embodiments may include an operation of detecting a turn-on event of a display (210) (operation 410 of FIG. 4), an operation of turning on the camera sensor (330) to obtain the external illuminance by using the camera sensor (330) when the display (210) is turned on (operation 420 of FIG. 4), an operation of obtaining preview image information for a specified time through the camera sensor (330) (operation 430 of FIG. 4), an operation of turning off the camera sensor (330) (operation 440 of FIG. 4), and an operation of obtaining the external illuminance by using the exposure time and the brightness value based on the obtained preview image information (operation 450 of FIG. 4), and an operation of controlling the brightness of the display based on the external illuminance value (e.g., operation 460 of FIG. 4).

An electronic device (101) according to various embodiments includes a housing (e.g., a housing (310) of FIG. 3A), a display (e.g., a display (320) of FIG. 3A) that is exposed to the front of the housing (310) to display a screen, a camera (e.g., a camera module (180) of FIG. 1) that includes a camera sensor (e.g., an image sensor (230) of FIG. 2 or a camera sensor (330) of FIG. 3A) that is positioned on at least a portion of the front of the housing (310) to measure external illuminance toward the front of the housing (310), a processor (e.g., a processor (120) of FIG. 1) that is operatively connected to the display (320) and the camera, and a memory (e.g., a memory (130) of FIG. 1) that is operatively connected to the processor (120), wherein, when executed, the processor (120) determines whether the display (320) is in a turn-on state. Instructions can be stored for judging, and turning on the camera sensor (330) at a specified cycle based on the judgment result, acquiring exposure time and brightness value from preview image information acquired by the camera sensor (330) during an acquisition section defined according to the number of frames when the display (320) is turned on, measuring the external illuminance using the exposure time and the brightness value, and controlling the brightness of the screen according to the measured external illuminance.

Electronic devices according to various embodiments disclosed in this document may be devices of various forms. The electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliance devices. Electronic devices according to embodiments of this document are not limited to the above-described devices.

The various embodiments of this document and the terminology used herein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B," "at least one of A and B," "at least one of A or B," "A, B, or C," "at least one of A, B, and C," and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first", "second", or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first) is referred to as "coupled" or "connected" to another (e.g., a second) component, with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

The term "module" as used in this document may include a unit implemented in hardware, software or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integrally configured component or a minimum unit of the component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more commands stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one command among the one or more commands stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the called at least one command. The one or more commands may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, ‘non-transitory’ simply means that the storage medium is a tangible device and does not contain signals (e.g. electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

According to one embodiment, the method according to the various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded between a seller and a buyer as a commodity. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read only memory (CD-ROM)), or may be distributed (e.g., downloaded or uploaded) directly or online via an application store (e.g., Play Store ^™ ) or between two user devices (e.g., smartphones). In the case of online distribution, at least a part of the computer program product may be at least temporarily stored or temporarily generated in a machine-readable storage medium, such as a memory of a manufacturer's server, a server of an application store, or an intermediary server.

According to various embodiments, each component (e.g., a module or a program) of the above-described components may include a single or multiple entities. According to various embodiments, one or more of the components or operations of the above-described components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the components of the plurality of components identically or similarly to those performed by the corresponding component of the plurality of components prior to the integration. According to various embodiments, the operations performed by the module, program or other component may be executed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

101: Electronic Devices 120: Processors
310: Housing 320: Display
330: Camera sensor

Claims (20)

In electronic devices,
housing;
A display that is exposed on the front side of the housing and displays a screen;
A camera sensor positioned at least in a portion of the front surface of the housing to measure external illuminance;
a processor operatively connected to said display and said camera sensor; and
comprising a memory operatively connected to said processor;
The above memory, when executed, causes the processor to:
Detects a turn-on event of the above display,
When the above display is turned on, the camera sensor is turned on to obtain the external illuminance using the camera sensor,
Obtain preview image information for a specified period of time through the above camera sensor,
Turn off the above camera sensor,
Based on the acquired preview image information, exposure time and brightness values are acquired,
If the brightness of the preview image information is lower than the first brightness, the external illuminance is obtained using the exposure time, and if the brightness of the preview image information is higher than the first brightness, the external illuminance is obtained using the brightness value.
An electronic device storing instructions for controlling the brightness of the display based on the external illuminance. In claim 1,
The above instructions cause the processor to:
An electronic device that obtains the exposure time and the brightness value from the preview image information at specified intervals while the display is turned on. In claim 2,
The above instructions cause the processor to:
An electronic device in which the above specified time is substantially identical to any one frame included in the above specified period. In claim 1,
The above instructions cause the processor to:
If the brightness of the above preview image information is lower than the second brightness, the external illuminance is obtained by using the brightness value after using the exposure time,
An electronic device, characterized in that the second luminance is lower than the first luminance. In claim 2,
Further comprising a communication circuit,
The above instructions cause the processor to:
Obtain current time information using the above communication circuit,
An electronic device that changes the specified cycle based on the acquired current time information. In claim 2,
The above instructions cause the processor to:
An electronic device that causes said specified period to vary depending on whether said electronic device is moved, rotated, and/or positioned. In claim 2,
The above instructions cause the processor to:
An electronic device that increases the specified period when the measured external illuminance remains substantially the same for a predetermined number of times or more and the position of the electronic device is fixed. In claim 1,
The above instructions cause the processor to:
Applying a first gain value to the exposure time and brightness value acquired in the edge area of the preview image information acquired by the camera sensor,
Applying a second gain value to the exposure time and brightness value obtained from the central area of the preview image information,
An electronic device that applies a value lower than the first gain value to the second gain value. A method for measuring external illuminance using a camera sensor of an electronic device,
Action to detect a turn-on event of the display;
An action of turning on the camera sensor to obtain the external illuminance by using the camera sensor when the above display is turned on;
An action of acquiring preview image information for a specified period of time through the above camera sensor;
An action of turning off the above camera sensor;
An operation of obtaining exposure time and brightness values based on the acquired preview image information;
An operation of obtaining the external illuminance using the exposure time when the brightness of the preview image information is lower than the first brightness, and obtaining the external illuminance using the brightness value when the brightness of the preview image information is higher than the first brightness; and
A method comprising: controlling brightness of the display based on said external illuminance. In claim 9, the operation of obtaining the external illumination comprises:
A method comprising an operation of obtaining the exposure time and the brightness value from the preview image information at each specified period while the display is turned on. In claim 10,
The above specified time is substantially the same as any one frame included in the above specified period. In claim 9, the operation of obtaining the external illumination comprises:
An operation of obtaining the external illuminance by using the brightness value after using the exposure time when the brightness of the preview image information is lower than the second brightness;
A method characterized in that the second luminance is lower than the first luminance. In claim 10,
An operation of obtaining current time information of a location associated with said electronic device; and
A method further comprising an action of changing the specified period based on the current time information. In claim 10,
A method further comprising changing said designated period depending on whether said electronic device is moved, rotated, and/or positioned. In claim 10,
A method further comprising the action of increasing the specified period when the measured external illuminance remains substantially the same for a predetermined number of times or more and the position of the electronic device is fixed. In electronic devices,
housing;
A display that is exposed on the front side of the housing and displays a screen;
A camera comprising a camera sensor positioned at least in a portion of a front surface of the housing and configured to measure external illuminance directed toward the front surface of the housing;
A light sensor positioned at the bottom of the display to measure the external illuminance;
a processor operatively connected to the display, the camera, and the light sensor; and
comprising a memory operatively connected to said processor;
The above memory, when executed, causes the processor to:
Detects a turn-on event of the above display,
When the above display is turned on, the above light sensor is turned on,
Obtain external illuminance for a specified period of time through the above illuminance sensor,
Determine whether the above external illuminance is less than or equal to the specified first illuminance value,
If the external illuminance is lower than or equal to the first illuminance value, the camera sensor is turned on,
Obtaining exposure time and brightness values from the preview image information acquired by the above camera sensor,
If the brightness of the preview image information is lower than the first brightness, the external illuminance is compensated using the exposure time, and if the brightness of the preview image information is higher than the first brightness, the external illuminance is compensated using the brightness value.
An electronic device storing instructions for controlling the brightness of the display according to the compensated external illuminance. In claim 16,
The above instructions cause the processor to:
In order to obtain the above external illumination, a camera on sequence is performed for the camera so that the camera sensor is turned on and enters the acquisition section after passing through the enable section.
An electronic device that performs an operation to maintain the operation of the above camera for a certain period of time. In claim 17,
The above instructions cause the processor to:
An electronic device that turns on the display and sequentially increases the brightness of the display based on the acquired external illuminance at a minimum level of brightness. In claim 16,
The above instructions cause the processor to:
An electronic device that turns on the camera sensor at a specified cycle to obtain the above external illuminance. In claim 19,
The above instructions cause the processor to:
Applying a first gain value to the exposure time and the brightness value acquired in the edge area of the preview image information acquired by the camera sensor,
Applying a second gain value to the exposure time and brightness value obtained from the central area of the preview image information,
An electronic device that applies a value lower than the first gain value to the second gain value.

Images