WO2017007220A1

WO2017007220A1 - Electronic device and display control method in electronic device

Info

Publication number: WO2017007220A1
Application number: PCT/KR2016/007267
Authority: WO
Inventors: 박현희; 김성오; 김재문; 이용만; 박경민; 박기현; 박대근; 장슬기; 전형주; 최종범; 김광태; 김수형; 염동현; 이기혁
Original assignee: 삼성전자 주식회사
Priority date: 2015-07-06
Filing date: 2016-07-05
Publication date: 2017-01-12

Abstract

Various examples of the present invention provide an electronic device comprising: a display for displaying content in a direction corresponding to a first surface of the electronic device; a sensor for sensing light incident on a second surface of the electronic device; and a processor, wherein the processor determines information of the brightness around the electronic device, at least on the basis of the sensed light, and can be set to adjust at least one attribute of the display or at least one attribute of the content. In addition, other examples besides the various examples of the present invention are possible.

Description

Electronic device and display control method in electronic device

Various embodiments of the present disclosure relate to a method and an apparatus for controlling a property of a display related to display of content in a electronic device or a property of the displayed content.

The electronic device, for example, performs a specified function according to a built-in program such as a home appliance, an electronic notebook, a portable multimedia player, a mobile communication terminal, a tablet PC, a video / audio device, a desktop / laptop computer, or a car navigation system. It can mean a device. For example, such electronic devices may output stored information as sound or an image. As the degree of integration of electronic devices has increased and ultra-high speed and high-capacity wireless communication has become commonplace, in recent years, various functions have been installed in one mobile communication terminal.

For example, not only communication functions, but also entertainment functions such as games, multimedia functions such as music / video playback, communication and security functions for mobile banking, calendar management or electronic wallet, etc. have.

As the functions of the electronic device are diversified, various types of sensors may be provided in the electronic device to implement various functions. For example, the user's visibility may be improved by measuring the ambient brightness through an illuminance sensor provided in the front of the electronic device and adjusting the brightness of the display using the measured value.

When adjusting the brightness of the display in the electronic device, if only the value sensed by the illuminance sensor provided on the front side of the electronic device is used, the visibility of the backlight may not be reflected and thus the visibility of the user may be lowered.

According to various embodiments of the present disclosure, for example, properties of a display or properties of content displayed through the display (for example, luminance or saturation or color) using values sensed by sensors functionally connected to an electronic device. An electronic device for controlling the display and a display control method in the electronic device can be provided.

In addition, various embodiments of the present disclosure may use, for example, an attribute of a display or an attribute of a content displayed through the display using an image sensor provided on the other side of the illumination sensor provided on one side of the electronic device. It is possible to provide an electronic device capable of controlling luminance, saturation, color, etc. and a display control method in the electronic device.

According to an aspect of the present disclosure, there is provided an electronic device including: a display configured to display content in a direction corresponding to a first surface of the electronic device; A sensor for detecting light incident on the second surface of the electronic device; And a processor, wherein the processor is configured to determine brightness information of the periphery of the electronic device based at least on the sensed light, and based at least on the brightness information, at least one attribute of the display or It may be set to adjust at least one property of the content.

Also, a display control method in an electronic device according to one of various embodiments may include displaying content by a display provided on a first surface of the electronic device; Sensing light incident by a sensor provided on a second surface of the electronic device; Determining brightness information around the electronic device based on at least the sensed light; And adjusting at least one property of the display or at least one property of the content based at least on the brightness information.

In addition, the non-transitory computer-readable recording medium that records a program for performing on a computer according to any one of the various embodiments, the program, when executed by the processor, the processor, when the processor displays the content on the display; Sensing light incident by the sensor; Determining brightness information around the electronic device based on at least the sensed light; And an executable command to perform an operation of adjusting at least one property of the display or at least one property of the content based at least on the brightness information.

According to various embodiments of the present disclosure, an electronic device and a display control method in an electronic device may include, for example, a property of a display or a content displayed through the display using a sensor provided on one surface of the electronic device and a sensor provided on the other surface of the electronic device. The visibility of the user can be improved by controlling an attribute (eg, luminance or saturation or color).

In addition, when there is a large difference in brightness due to the surrounding environment of the front and rear with respect to the display on the electronic device, the automatic brightness function that operates only with the front light sensor by using the brightness value of the image sensor on the back together with the light sensor on the front. Can overcome the limitations of

1 is a diagram illustrating a network environment according to an embodiment of the present invention.

2 is a diagram illustrating a configuration example of an electronic device according to an embodiment of the present disclosure.

3 is a flowchart illustrating a procedure of controlling a display in an electronic device according to various embodiments of the present disclosure.

4A is a flowchart illustrating a procedure of controlling luminance or color of a display in an electronic device according to various embodiments of the present disclosure.

4B is a flowchart illustrating a procedure of controlling luminance of a display in an electronic device according to various embodiments of the present disclosure.

5A is a flowchart illustrating a procedure of controlling a color of a display in an electronic device according to various embodiments of the present disclosure.

5B is a flowchart illustrating a procedure of controlling brightness of a display using an image sensor in an electronic device according to various embodiments of the present disclosure.

6A is a flowchart illustrating a procedure of controlling brightness of a display in a backlight environment in an automatic brightness operation state in an electronic device according to various embodiments of the present disclosure.

6B is a flowchart illustrating a procedure of controlling luminance of a display using an illumination sensor and an image sensor in an electronic device according to various embodiments of the present disclosure.

7 is a flowchart illustrating a procedure of controlling a display based on content in an electronic device according to various embodiments of the present disclosure.

8A and 8B are perspective views illustrating an electronic device in which sensors are disposed according to various embodiments of the present disclosure.

9 is a block diagram illustrating a configuration of an image processing apparatus according to various embodiments of the present disclosure.

10 is a diagram illustrating detailed blocks of an image preprocessing module according to various embodiments of the present disclosure.

11 is a diagram illustrating detailed blocks of an image signal processor according to various embodiments of the present disclosure.

12 is a block diagram illustrating a configuration of an image processing apparatus according to various embodiments of the present disclosure.

13 is a block diagram illustrating a configuration of an image processing apparatus according to various embodiments of the present disclosure.

14 is a block diagram illustrating a configuration of an image processing apparatus according to various embodiments of the present disclosure.

15A and 15B are block diagrams illustrating a configuration of an image processing apparatus according to various embodiments of the present disclosure.

16 is a block diagram of an electronic device according to an embodiment of the present disclosure.

17 is a block diagram illustrating a program module according to various embodiments of the present disclosure.

Hereinafter, various embodiments of the present invention will be described with reference to the accompanying drawings. However, it is not intended to limit the present invention to specific embodiments, but it should be understood to include various modifications, equivalents, and / or alternatives of the embodiments of the present invention. In connection with the description of the drawings, similar reference numerals may be used for similar components.

In this document, expressions such as "have", "may have", "include", or "may contain" include the presence of a corresponding feature (e.g., numerical, functional, operational, or component such as a component). Does not exclude the presence of additional features.

In this document, expressions such as "A or B", "at least one of A or / and B", or "one or more of A or / and B" may include all possible combinations of items listed together. . For example, "A or B", "at least one of A and B", or "at least one of A or B" includes (1) at least one A, (2) at least one B, Or (3) both of cases including at least one A and at least one B.

Expressions such as “first,” “second,” “first,” or “second,” used in various embodiments may modify various elements in any order and / or importance, and may modify the elements. It is not limited. The above expressions may be used to distinguish one component from another. For example, the first user device and the second user device may represent different user devices regardless of the order or importance. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may be renamed to the first component.

One component (such as a first component) is "(functionally or communicatively) coupled with / to" to another component (such as a second component) or " When referred to as "connected to", it should be understood that any component may be directly connected to the other component or may be connected through another component (eg, a third component). On the other hand, when a component (e.g., a first component) is said to be "directly connected" or "directly connected" to another component (e.g., a second component), the component and the It may be understood that no other component (eg, a third component) exists between the other components.

The expression "configured to" used in this document is, for example, "suitable for", "having the capacity to" depending on the situation. It may be used interchangeably with "designed to", "adapted to", "made to", or "capable of". The term "configured to" may not necessarily mean only "specifically designed to" in hardware. Instead, in some situations, the expression "device configured to" may mean that the device "can" along with other devices or components. For example, the phrase “processor configured (or set up) to perform A, B, and C” may execute a dedicated processor (eg, an embedded processor) to perform the operation, or one or more software programs stored in a memory device. By doing so, it may mean a general-purpose processor (for example, a CPU or an application processor) capable of performing the corresponding operations.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the scope of other embodiments. Singular expressions may include plural expressions unless the context clearly indicates otherwise. All terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by one of ordinary skill in the art. Commonly used terms are to be interpreted as having the same or similar meaning as the meaning in the context of the related art, and are not to be interpreted as ideal or excessively formal meanings unless clearly defined in this document. . In some cases, even if terms are terms defined in the specification, they may not be interpreted to exclude embodiments of the present disclosure.

In an electronic device according to various embodiments of the present disclosure, for example, the electronic device may be a smartphone, a tablet personal computer, a mobile phone, a video phone, or an e-book reader. reader, desktop personal computer (PC), laptop personal computer (PC), netbook computer, workstation, server, personal digital assistant (PDA), portable multimedia player (PMP), MP3 player , Mobile medical devices, cameras, or wearable devices (e.g. smart glasses, head-mounted-device (HMD)), electronic clothing, electronic bracelets, electronic necklaces, electronic accessories (appcessory), an electronic tattoo, a smart mirror, or a smart watch.

In some embodiments, the electronic device may be a smart home appliance. Smart home appliances are, for example, televisions, digital video disk (DVD) players, audio, refrigerators, air conditioners, vacuum cleaners, ovens, microwave ovens, washing machines, air purifiers, set-top boxes, home automation Home automation control panel, security control panel, TV box (e.g. Samsung HomeSync ™, Apple TV ™, or Google TV ™), game console (e.g. Xbox ™, PlayStation ™), electronics It may include at least one of a dictionary, an electronic key, a camcorder, or an electronic picture frame.

In another embodiment, the electronic device may include various medical devices (eg, various portable medical measuring devices (such as blood glucose meters, heart rate monitors, blood pressure monitors, or body temperature meters), magnetic resonance angiography (MRA), magnetic resonance imaging (MRI), Such as CT (computed tomography, imaging or ultrasound), navigation devices, global positioning system receivers, event data recorders, flight data recorders, automotive infotainment devices, ships Electronic equipment (e.g. ship navigation systems, gyro compasses, etc.), avionics, security devices, vehicle head units, industrial or home robots, automatic teller's machines (financial institutions), shop POS (point of sales) or internet of things (IoT) (e.g. light bulbs, sensors, electrical or gas meters, sprinkler devices, smoke alarms, thermostats, street lights, toas) ter), exercise equipment, hot water tank, heater, boiler, and the like.

According to some embodiments, an electronic device may be a furniture or part of a building / structure, an electronic board, an electronic signature receiving device, a projector, or various measuring devices (eg, Water, electricity, gas, or radio wave measuring instrument). In various embodiments of the present disclosure, the electronic device may be one or a combination of the aforementioned various devices. An electronic device according to an embodiment may be a flexible electronic device. Also, the electronic device according to an embodiment of the present disclosure is not limited to the above-described devices, and may include a new electronic device according to technology development.

According to various embodiments of the present disclosure, an attribute of a display or an attribute (eg, luminance or saturation or color, etc.) of a display is controlled by using values sensed by a plurality of sensors included in the electronic device. An electronic device and a display control method in an electronic device are disclosed. For example, various embodiments of the present disclosure may use attributes of a display or attributes of a content displayed through the display (eg, luminance or saturation) by using an illumination sensor provided on one surface of the electronic device and an image sensor provided on the other surface of the electronic device. An electronic device capable of controlling color) and a display control method in an electronic device are disclosed.

In various embodiments of the present invention described below, 'illuminance' is described as an example of a value corresponding to "brightness (or brightness)." However, various embodiments of the present invention are not limited to the above illuminance. For example, as a value corresponding to the brightness, not only illuminance but also luminance, luminous flux, luminous intensity, and the like may be included. In addition, in various embodiments of the present invention described below, “chroma” may be represented as a number as a main attribute of color, which means a clear and hazy degree of color. In addition, in various embodiments of the present disclosure described below, “color” may be interpreted as a broad concept including saturation. In addition, in various embodiments of the present invention described below, “white balance (WB)” may mean a distribution of colors, and for example, may mean a numerical representation of a distribution of R, G, or B values. . The white balance may be calculated from an RGB histogram sensed by the image sensor.

Hereinafter, an electronic device according to various embodiments of the present disclosure will be described with reference to the accompanying drawings. In this document, the term user may refer to a person who uses an electronic device or a device (eg, an artificial intelligence electronic device) that uses an electronic device.

Referring to FIG. 1, in various embodiments, an electronic device 101 in a network environment 100 is described. The electronic device 101 may include a bus 110, a processor 120, a memory 130, an input / output interface 150, a display 160, a communication interface 170 or a display control module 180, and an illumination sensor 191. And at least one of the image sensor 192. According to an embodiment of the present disclosure, the electronic device 101 may omit at least one of the above components or additionally include other components.

The bus 110 may include, for example, circuits for connecting the components 110 to 192 to each other and transferring communication (eg, control messages and / or data) between the components. .

The processor 120 may include one or more of a central processing unit (CPU), an application processor (AP), or a communication processor (CP). The processor 120 may execute, for example, an operation or data processing related to control and / or communication of at least one other element of the electronic device 101.

The memory 130 may include a volatile and / or nonvolatile memory. The memory 130 may store, for example, commands or data related to at least one other element of the electronic device 101. According to an embodiment of the present disclosure, the memory 130 may store software and / or a program 140. The program 140 may be, for example, a kernel 141, middleware 143, an application programming interface (API) 145, and / or an application program (or “application”) 147. And the like. At least a portion of the kernel 141, the middleware 143, or the API 145 may be referred to as an operating system (OS).

The kernel 141 may be, for example, system resources used to execute an operation or function implemented in other programs (eg, middleware 143, API 145, or application program 147). For example, the bus 110, the processor 120, or the memory 130 may be controlled or managed. In addition, the kernel 141 may control or manage system resources by accessing individual components of the electronic device 101 from the middleware 143, the API 145, or the application program 147. Can provide an interface.

The middleware 143 may serve as an intermediary for allowing the API 145 or the application program 147 to communicate with the kernel 141 to exchange data. In addition, the middleware 143 may be connected to at least one application of the application program 147 in relation to the work requests received from the application program 147. For example, assigning a priority to use the bus 110, the processor 120, or the memory 130, or the like, to control work requests (e.g., scheduling or load balancing). Can be.

The API 145 is, for example, an interface for the application 147 to control functions provided by the kernel 141 or the middleware 143. For example, file control, window control, and image. It may include at least one interface or function (eg, a command) for processing or character control.

The input / output interface 150 may serve as, for example, an interface capable of transferring a command or data input from a user or another external device to other component (s) of the electronic device 101. In addition, the input / output interface 150 may output a command or data received from other component (s) of the electronic device 101 to a user or another external device.

The display 160 is a means for adjusting and displaying an attribute (eg, luminance, saturation, or color) of a screen provided to a user according to various embodiments of the present disclosure. For example, the display 160 may be a liquid crystal display (LCD). , Light emitting diode (LED) displays, organic light emitting diode (OLED) displays, or microelectromechanical systems (MEMS) displays, or electronic paper displays. The display 160 may display, for example, various contents (for example, text, an image, a video, an icon, a symbol, etc.) to a user. The display 160 may include a touch screen. For example, the display 160 may receive a touch, gesture, proximity, or hovering input using an electronic pen or a part of a user's body.

The communication interface 170 may, for example, communicate between the electronic device 101 and an external device (eg, the first external electronic device 102, the second external electronic device 104, or the server 106). Can be set. For example, the communication interface 170 may be connected to the network 162 through wireless or wired communication to communicate with the external device (eg, the second external electronic device 104 or the server 106). . In addition, the communication interface 170 may directly communicate with the external device (eg, the first external electronic device 102) through, for example, wireless communication or wired communication. In the embodiments described below, when the electronic device 101 is a smartphone, the first external electronic device 102 may be a wearable device. For example, according to various embodiments of the present disclosure, the smartphone and the wearable device may communicate with each other to transmit and receive information related to an electronic map.

Wireless communication is, for example, a cellular communication protocol, for example, long-term evolution (LTE), LTE Advance (LTE-A), code division multiple access (CDMA), wideband CDMA (WCDMA), UMTS (universal). At least one of a mobile telecommunications system (WBro), a wireless broadband (WiBro), a global system for mobile communications (GSM), and the like may be used. In addition, wireless communication may include, for example, near field communication 164. The local area communication 164 may include, for example, at least one of wireless fidelity (WiFi), Bluetooth, near field communication (NFC), global navigation satellite system (GNSS), and the like. The GNSS may include, for example, at least one of a Global Positioning System (GPS), a Global Navigation Satellite System (GNA), a Beidou Navigation Satellite System (hereinafter referred to as "Beidou"), or a Galileo, the European global satellite-based navigation system, depending on the region or bandwidth used. In the present document, "GPS" may be interchangeably used with "GNSS." The wired communication may include, for example, a universal serial bus (USB) and an HDMI ( It may include at least one of a high definition multimedia interface (RS-232), a reduced standard 232 (RS-232), a plain old telephone service (POTS), etc. The network 162 may be a telecommunications network, for example, It may include at least one of a computer network (eg, LAN or WAN), the Internet, or a telephone network.

Each of the first and second external electronic devices 102 and 104 may be a device of the same or different type as the electronic device 101. According to an embodiment of the present disclosure, the server 106 may include a group of one or more servers. According to various embodiments of the present disclosure, all or part of operations executed in the electronic device 101 may be executed in another or a plurality of electronic devices (for example, the electronic devices 102 and 104 or the server 106). According to an embodiment of the present disclosure, when the electronic device 101 needs to perform a function or service automatically or by request, the electronic device 101 instead of executing the function or the service by itself or Additionally, at least some functions associated with the request may be requested to another device (eg, the electronic device 102 or 104 or the server 106). The other electronic device (for example, the electronic devices 102 and 104 or the server 106) may execute the requested function or additional function and transmit the result to the electronic device 101. The electronic device 101 may process the received result as it is or additionally to provide the requested function or service. To this end, for example, cloud computing, distributed computing, or client-server computing technology may be used.

In FIG. 1, the electronic device 101 is provided with a communication interface 170 to communicate with an external electronic device 104 or a server 106 through a network 162, but according to various embodiments of the present disclosure, Accordingly, the electronic device 101 may be implemented to operate independently in the electronic device 101 without a separate communication function.

According to an embodiment of the present disclosure, the server 106 drives the electronic device 101 by performing at least one operation (or function) among operations (or functions) implemented in the electronic device 101. Can support For example, the server 106 may include a display control server module (not shown) capable of supporting the display control module 180 implemented in the electronic device 101. For example, the display control server module may include at least one component of the display control module 180 to perform (or at least one of the operations (or functions) the display control module 180 performs. You can also act. According to various embodiments of the present disclosure, the server 106 may be an image editing function providing server capable of providing various image editing related functions to the electronic device 101.

The display control module 180 may be obtained from, for example, other components (eg, the processor 120, the memory 130, the input / output interface 150, the communication interface 170, etc.). At least some of the information may be processed and provided to the user in various ways.

For example, the display control module 180 may display the display 160 based on a value sensed by at least one illuminance sensor 191 or at least one image sensor 192 according to various embodiments of the present disclosure. ) Allows you to adjust or determine the properties of the screen displayed (eg, brightness, saturation or color). Additional information about the display control module 180 is provided through FIG. 2 to be described later.

Although the display control module 180 is shown as a separate module from the processor 120 in FIG. 1, at least a part of the display control module 180 may be a processor 120 or another at least one module (eg, a display ( 160) may be included in the processor 120 or another processor, and the entire function of the display control module 180 may be implemented.

The illuminance sensor 191 is, for example, a sensor capable of sensing a value related to brightness, and is not limited to a sensor of a specific name, but may be of any type capable of determining a value related to the brightness by sensing. The sensor may also be included in the illuminance sensor according to the embodiment of the present invention.

The image sensor 192 is, for example, a sensor capable of detecting light incident to the sensor and sensing a value related to brightness or color for each pixel, and is not limited to a sensor of a specific name, but is incident light Any type of sensor capable of determining a value related to brightness or color for each pixel may be included in an image sensor according to an exemplary embodiment of the present invention. For example, at least a part of the camera module may be included in the image sensor 192.

According to an embodiment, all components of the electronic device 101 (eg, the processor 120 or the display control module 180) are illustrated in a configuration in which the electronic device 101 is included. It is not limited to this. For example, depending on the role, function, or performance of the electronic device 101, at least some of the components of the electronic device 101 may be connected to the electronic device 101 and an external electronic device (eg, the first external electronic device of FIG. 1). The device 102 may be distributed and implemented in the device 102, the second external electronic device 104, or the server 106.

2 is a diagram illustrating a configuration example of an electronic device according to an embodiment of the present disclosure. According to various embodiments of the present disclosure, the electronic device 200 may include a display unit 210, a controller 220, a communication unit 230, a storage unit 240, an input unit 250, and at least one

illuminance sensor

261 and 262. ), And at least one of the at least one

image sensor

271, 272. According to various embodiments of the present disclosure, the controller 220 may include at least one of an adjustment situation determination unit 221, a brightness related value calculation unit 222, a luminance determination unit 223, and a color determination unit 224. It may include.

According to various embodiments of the present disclosure, the first illuminance sensor 261 is disposed on one surface (eg, the front surface) of the electronic device 200, and the second illuminance sensor 262 is the electronic device 200. It may be placed on the other side of the (eg back). In addition, according to various embodiments of the present disclosure, the first image sensor 271 may be disposed on one surface (eg, a front surface or a top surface) of the electronic device 200, and the second image sensor 272 may be It may be disposed on another surface (for example, a rear surface or a bottom surface) of the electronic device 200. The surface on which the display unit 210 of the electronic device 200 is located may be determined as the front surface of the electronic device 200, and the opposite surface of the front surface may be determined as the rear surface of the electronic device 200. One side and the other side are not limited to the opposite side, such as the front side or the rear side, and the front side or the side surface of the electronic device 200 may be determined as the one side and the other side.

Each component of the electronic device 200 of FIG. 2 may include all or part of its functions in at least one element of FIG. 1. For example, at least a part of the controller 220 may be included in the display control module 180 or the processor 120 of FIG. 1. In addition, at least a portion of the storage unit 240 may be included in the memory 130 of FIG. 1, and at least a portion of the display unit 210 may be included in the display 160 of FIG. 1. At least a portion of may be included in the communication interface 170 of FIG.

The storage unit 240 may include, for example, color information 241 or brightness information, luminance information 242, chroma information 243, and display adjustment mapping table 244 information for each pixel. Information stored in the storage 240 may be provided from an electronic device (eg, a server or another electronic device) external to the electronic device 200. In addition, the storage unit 240 may further store various information related to the control of the display.

The adjustment situation determination unit 221 of the control unit 220 may be an attribute (eg, luminance or saturation or color) of the display unit 210 (eg, the display 160), or content displayed through the display unit 210. It is possible to determine whether to adjust the attributes (eg luminance or saturation or color) of the. For example, the adjustment situation determination unit 221 may display an attribute (eg, luminance or saturation or color) of the display 160 or an attribute (eg, luminance or saturation or color) of the content displayed through the display unit 210. It is determined whether a predetermined condition is satisfied in order to adjust the value, and when the condition is satisfied, the luminance determining unit 223 or the color determining unit 224 of the control unit 220 controls the at least one illuminance sensor 261 and 262. ) Or an attribute (eg, luminance or saturation or color) of the display unit 210 using the value sensed by the at least one

image sensor

271, 272, or an attribute (eg, content of the content displayed through the display unit 210). : Luminance or saturation or hue).

The situation for adjusting the brightness or saturation or color of the screen or the content displayed on the screen may be variously set. For example, according to various embodiments of the present disclosure, when the display unit 210 is turned on or when the display unit 210 is turned on, the luminance or saturation or color of the content displayed on the screen or the screen may be adjusted.

In addition, according to various embodiments of the present disclosure, the luminance or saturation or color of the content displayed on the screen or the screen may be periodically adjusted. For example, the second illuminance sensor 262 or the second image sensor 272 disposed on one surface (eg, the rear surface) of the electronic device 200 may be operated at regular time intervals so that one surface (eg, the rear surface) of the electronic device 200 is operated. The brightness information of the can be sensed.

In addition, according to various embodiments of the present disclosure, when the movement of the electronic device 200 occurs, the luminance or saturation or color of the content displayed on the screen or the screen may be adjusted. For example, when a movement of the electronic device 200 is detected from various motion detection sensors (eg, a gyro sensor, an acceleration sensor, etc.) included in the electronic device 200 or the degree of movement is out of a preset threshold. In addition, the brightness or saturation or color of the content displayed on the screen or the screen may be implemented. According to various embodiments of the present disclosure, when a user of the electronic device 200 moves from the inside of the building to the outside or moves from the outside of the building to the inside, a significant change in brightness occurs immediately. It can be implemented to adjust the luminance or saturation or color.

In addition, according to various embodiments of the present disclosure, a sensor (eg, the first illuminance sensor 261 or the first image sensor 271) provided on one surface (eg, the front surface) of the electronic device 200 may include the electronic device 200. When a specific event occurs (eg, when a user presses a power button to check a time or a message) while being covered by a cover of the sensor), a sensor provided on the other side (eg, the rear side) of the electronic device 200. The second illuminance sensor 262 or the second image sensor 272 may be used to adjust luminance or saturation or color of the screen or the content displayed on the screen.

In addition, when the reproduction of the virtual reality content through the electronic device 200 ends, according to various embodiments of the present disclosure, a sensor provided on one surface (eg, a rear surface) before the viewing of the content is terminated. For example, the second illuminance sensor 262 or the second image sensor 272) is used to adjust the brightness or saturation or color of the screen or the content displayed on the screen so that the user responds to a sudden change in ambient brightness. For example, the display unit 210 may control an attribute (eg, luminance or saturation or color) displayed on the screen or the content displayed on the screen.

In addition, according to various embodiments of the present disclosure, properties of the preview screen displayed on the display unit 210 may be changed when the camera is photographed (eg, when the camera is photographed by the at least one image sensor 271 and 272). A camera displaying the preview screen may be selected among the

image sensors

271 and 272.

For example, according to various embodiments of the present disclosure, the brightness or the white balance of the preview screen may be changed according to peripheral illumination or color information determined by the at least one

sensor

261, 262, 271, 272 when the camera is photographed. It can be implemented to change camera settings. In addition, the electronic device including the plurality of cameras may be implemented to display only the one camera by displaying the preview image according to the illumination determined by the sensor.

According to various embodiments of the present disclosure, when the electronic device is a wearable device that provides a virtual reality (VR) function, when the user wears the electronic device, the brightness of the internal screen and the electronic device of the electronic device are worn. The external brightness of may be different. In such a situation, before the user takes off the electronic device, the user may grasp the situation using information obtained from a sensor provided in the front or the rear of the electronic device, and adjust the brightness of the display accordingly. For example, if the user determines that the brightness of the internal screen of the electronic device is dark and the outside of the electronic device is bright based on information obtained from a sensor provided in the front or rear of the electronic device at a preset time before the user finishes watching the VR content. The brightness of the display may be adjusted before the end of viewing of the VR content, thereby preventing glare when the user takes off the electronic device.

For example, as described above, when the electronic device operates in the VR mode and the VR related content is played, the user may be in a relatively dark state because the user is disconnected from the external environment in the VR mode. However, when the VR mode is terminated, the user's eyes may not be able to adapt as the light is instantly switched to the bright state. According to various embodiments of the present disclosure, when the VR-related content is terminated, the rear information of the electronic device may be overlaid and displayed on the screen so as to be adapted to a sudden change in environment. The overlaid backside information may be set to become gradually brighter as the end point of the VR-related content approaches.

As described above, when the adjustment situation determination unit 211 determines that the brightness, saturation, or color of the content displayed on the screen or the screen is to be adjusted, the at least one

illuminance sensor

261 or 262 or at least one image sensor 271 is determined. The brightness or saturation of the content displayed on the screen or the screen may be adjusted by at least one of the brightness-related value calculator 222, the brightness determiner 223, and the color determiner 224 from the value sensed by 272. .

For example, the brightness related value calculator 222 may generate an RGB histogram from values sensed by the first image sensor 271 or the second image sensor 272, and generate the RGB histogram and / or Alternatively, a value related to brightness (for example, illuminance) or white balance may be calculated from the exposure time. Detailed embodiments thereof will be described later. Color information 241 for each pixel sensed by the first image sensor 271 or the second image sensor 272 may be stored in the storage unit 240.

The luminance determiner 223 may detect a value or two or more values detected from at least one of the first illuminance sensor 261, the second illuminance sensor 262, the first image sensor 271, and the second image sensor 272. The luminance of the content displayed on the screen or the screen may be adjusted or determined based on a combination of values sensed from the screen. The determined luminance information 242 may be stored in the storage 240.

For example, according to various embodiments of the present disclosure, a value sensed by the first illuminance sensor 261 provided on one surface (eg, the front surface) of the electronic device 200 and the other surface (eg, the rear surface) of the electronic device 200 may be used. The luminance may be determined based on the value sensed by the second illuminance sensor 262 included in the). In addition, according to various embodiments of the present disclosure, the value sensed by the first illuminance sensor 261 provided on one surface (eg, the front surface) of the electronic device 200 and the other surface (eg, the rear surface) of the electronic device 200 may be used. The luminance may be determined based on the value sensed by the second image sensor 272 included in the). Detailed description of various embodiments thereof will be described later.

The saturation determination unit 224 may detect a value or two or more values detected from at least one of the first illuminance sensor 261, the second illuminance sensor 262, the first image sensor 271, and the second image sensor 272. The luminance of the content displayed on the screen or the screen may be adjusted or determined based on a combination of values sensed from the screen.

For example, according to various embodiments of the present disclosure, the value sensed by the first image sensor 271 provided on one surface (for example, the front surface) of the electronic device 200 and the other surface (for example, the rear surface of the electronic device 200) may be used. ) May determine saturation or color based on the value sensed by the second image sensor 272.

According to various embodiments of the present disclosure, the luminance determiner 223 or the color determiner 224 may include the at least one sensor (a first illuminance sensor 261, a second illuminance sensor 262, and a first image). The brightness or saturation or color to be applied to the screen or the content displayed on the screen in response to the value sensed by the sensor 271 and the second image sensor 272 may be determined through the display adjustment mapping table 244. have.

According to various embodiments of the present disclosure, the controller 220 may perform operations of the electronic device 200 and further process various functions for controlling the operation of the electronic device 200. For example, the controller 220 may be an application processor (AP) or may be a separate processor designed with low power. Alternatively, the controller 220 may be included in a modem processor or may be included in a processor of a separate communication module or a positioning module.

The communication unit 230 may be a device that communicates with other electronic devices or servers other than the electronic device 200 by wire or wirelessly. The other electronic device may be another mobile device or may be a fixed access point (AP), a Bluetooth low energy (BLE) beacon, or the like. Alternatively, the other electronic device may be a base station on a mobile communication network. The input unit 250 may process various types of user inputs for function setting or operation command of the electronic device 200. For example, the input unit 250 may include a touch pad of a touch screen, a hardware button, a user gesture, and the like.

In the various embodiments of the present disclosure, each functional unit or module may mean a functional and structural combination of hardware for performing technical spirit of various embodiments of the present disclosure and software for driving the hardware. For example, each functional unit or module may mean a logical unit of a predetermined code and a hardware resource for performing the predetermined code, and necessarily means a physically connected code or a kind of hardware. Or not can be easily deduced by the average expert in the art of the embodiments of the present invention.

According to various embodiments of the present disclosure, at least a portion of the adjustment situation determination unit 221, the brightness related value calculation unit 222, the luminance determination unit 223, or the color determination unit 224 may be software, firmware, hardware, or the like. It may be implemented in a combination of at least two or more. At least a portion of the adjustment state determination unit 221, the brightness related value calculation unit 222, the luminance determination unit 223, or the color determination unit 224 may be, for example, connected to a processor (eg, the processor 120). May be implemented (eg, executed). At least a portion of the adjustment situation determination unit 221, the brightness related value calculation unit 222, the luminance determination unit 223, or the color determination unit 224 may perform, for example, a module or a program. , Routines, sets of instructions, or process.

An electronic device according to one of various embodiments of the present disclosure may include a display;

A first sensor disposed in front of the electronic device; A second sensor disposed on a rear surface of the electronic device; And a controller configured to determine the brightness of the display based on the value sensed by the first sensor and the value sensed by the second sensor.

According to various embodiments of the present disclosure, the controller may determine the illuminance from the value sensed by the first sensor, determine the color value of each pixel from the value sensed by the second sensor, and determine the illuminance and each pixel. The brightness of the display may be determined based on a color value of.

According to various embodiments of the present disclosure, the first sensor may be an illuminance sensor, and the second sensor may be an image sensor.

According to various embodiments of the present disclosure, the apparatus may further include an image signal processor configured to receive a color value of each pixel sensed by the second sensor to process an image signal, and transmit the image signal processed result to the controller. .

According to various embodiments of the present disclosure, the image signal processor includes a plurality of functional blocks, and if the controller satisfies a preset condition for determining the brightness of the display, at least one functional block among the plurality of functional blocks. Can be turned off.

According to various embodiments of the present disclosure, the image signal processor includes a plurality of functional blocks connected to each other in a pipeline form, and when the controller satisfies a preset condition for determining the brightness of the display, the plurality of functional blocks At least one of the functional blocks may be bypassed.

According to various embodiments of the present disclosure, the apparatus may further include an image preprocessing module provided between the second sensor and the image signal processing unit, and if the control unit satisfies a preset condition for determining the brightness of the display, the image preprocessing module At least one functional block of the plurality of functional blocks included in may be turned off.

According to various embodiments of the present disclosure, the controller may determine whether the controller satisfies a preset condition for determining the brightness of the display, and when the preset condition is satisfied, the controller determines from the first sensor or the second sensor. The received data can be processed.

According to various embodiments of the present disclosure, when the display is in an on state, when the display is switched from an off state to an on state, and when a predetermined period is satisfied, the movement of the electronic device may occur. When it occurs, when the degree of movement of the electronic device is out of a predetermined threshold value, when it is determined that the user of the electronic device moves from the inside of the building to the outside or from the outside of the building to the inside, When a significant change in brightness is expected, when the cover of the electronic device is covered and the first sensor or the second sensor is covered by the cover, and when a predetermined event occurs while the cover is covered It may include any one of.

An electronic device according to one of various embodiments of the present disclosure may include a display for displaying content in a direction corresponding to a first surface of the electronic device; A sensor for detecting light incident on the second surface of the electronic device; And a processor, wherein the processor is configured to determine brightness information of the periphery of the electronic device based at least on the sensed light, and based at least on the brightness information, at least one attribute of the display or It may be set to adjust at least one property of the content.

According to various embodiments of the present disclosure, the apparatus may further include another sensor for detecting light incident on the first surface, and the processor may further include another processor determined based on light detected through the other sensor. When the brightness information belongs to a specified range, it may be set to perform the determining operation.

According to various embodiments of the present disclosure, the apparatus may further include another sensor for detecting light incident on the first surface, and the processor may further include another processor determined based on light detected through the other sensor. The adjusting may be performed based on brightness information.

According to various embodiments of the present disclosure, the sensor may include an image sensor.

According to various embodiments of the present disclosure, the apparatus may further include an image signal processor including a first functional block and a second functional block to process the light obtained from the sensor, and the processor may further include the first functional block. And selecting a function block related to the brightness information from the second function block, and performing the determining operation using the selected function block.

According to various embodiments of the present disclosure, the processor may not use a function block that is not selected among the first function block and the second function block to bypass the operation, or to the second function block. It may be set to turn off the applied power.

According to various embodiments of the present disclosure, the sensor may include a plurality of pixels including a red pixel, a green pixel, or a blue pixel, and the processor may include at least color information corresponding to a designated pixel among the plurality of pixels. On the basis of this, may be set to perform the determining operation.

According to various embodiments of the present disclosure, the processor may be configured to select one or more pixels having the lowest number among the plurality of pixels as the designated pixel.

According to various embodiments of the present disclosure, the processor may be set to determine the brightness information based on color information corresponding to the light.

According to various embodiments of the present disclosure, the processor may be set to determine the brightness information based on the time information of the sensor exposed to the light.

According to various embodiments of the present disclosure, at least one property of the display or at least one property of the content may include brightness, saturation, white balance, color, or a combination thereof.

According to various embodiments of the present disclosure, the electronic device may further include a housing forming at least a portion of an external surface of the electronic device, and the sensor may constitute at least a portion of the housing. .

According to various embodiments of the present disclosure, the sensor may be located between the display and the housing.

According to various embodiments of the present disclosure, the electronic device may further include an image preprocessing module between the sensor and the processor, and when the preset condition for determining the brightness of the display is satisfied by the processor, the electronic device may be configured to the image preprocessing module. At least one functional block of the plurality of included functional blocks may be turned off.

Hereinafter, a display control procedure according to various embodiments of the present disclosure will be described with reference to FIGS. 3 to 8.

3 is a flowchart illustrating a procedure of controlling a display (for example, the display 160) in an electronic device (for example, the processor 120 or the display control module 180) according to various embodiments of the present disclosure.

Referring to FIG. 3, in operation 302, the electronic device (eg, the processor 120) may display content by, for example, a display (eg, the display 160) provided on the first surface of the electronic device. In operation 304, light incident by the sensor provided on the second surface of the electronic device may be detected. In operation 306, the electronic device may determine ambient brightness information of the electronic device based at least on the detected light. In operation 308, the electronic device may determine at least one property of the display of the electronic device or at least one property (eg, luminance or saturation or color) of the content, based at least on the brightness information.

4A is a flowchart illustrating a procedure of controlling luminance or color of a display (eg, display 160) in an electronic device (eg, the processor 120 or the display control module 180) according to various embodiments of the present disclosure. to be.

Referring to FIG. 4A, in operation 402, the electronic device (eg, the processor 120) may determine, for example, whether a display adjustment situation is performed. According to an embodiment of the present disclosure, in operation 402, the electronic device may determine whether the display unit is turned on, the display unit is turned on, or periodically, whether the display is adjusted. As another example, the electronic device is provided in the electronic device when a movement of the electronic device occurs, when a user of the electronic device moves from the inside of the building to the outside or moves from the outside of the building to the inside to cause a momentary change in brightness. When a specific event occurs while the at least one sensor is covered by the cover of the electronic device, when the reproduction of the virtual reality content through the electronic device is ended, it may be determined as a display adjustment situation.

As a result of the comparison or determination in operation 402, when it corresponds to the display adjustment situation, the electronic device (eg, the processor 120) may perform operation 404, for example. According to an embodiment of the present disclosure, in operation 404, the electronic device may determine the illuminance or the white balance from the value sensed by the first sensor.

In operation 402, when the comparison or determination does not correspond to the display adjustment situation, the electronic device (eg, the processor 120) may perform operation 402, for example.

In operation 406, the electronic device (eg, the processor 120) may check or determine the illuminance or the white balance, for example, from the value sensed by the second sensor.

According to various embodiments of the present disclosure, the first sensor and the second sensor may be disposed on the same plane of the electronic device or may be disposed on different planes (eg, front or rear). In addition, the first sensor and the second sensor may be the same type of sensor (eg, the first sensor and the second sensor are an illuminance sensor or an image sensor), and different types (eg, the first sensor are the illuminance sensor and the second sensor). The sensor may be an image sensor or a first sensor may be an image sensor and the second sensor may be a sensor of an illuminance sensor.

In operation 408, the electronic device may display a property of the display (eg, the display 160) or the content displayed through the display based on the illuminance or the white balance determined by the values sensed by the first sensor and the second sensor. Judgment, determination or adjustment can be made. The attribute may include, for example, luminance or saturation or color of a screen. Detailed embodiments thereof will be described later.

Hereinafter, various embodiments of determining luminance, saturation, or color based on a combination of values sensed by the plurality of sensors will be described with reference to FIGS. 4B to 5B.

4B is a flowchart illustrating a procedure of controlling the brightness of a display (eg, the display 160) in an electronic device (eg, the processor 120 or the display control module 180) according to various embodiments of the present disclosure. Referring to FIG. 4B, in operation 412, the electronic device (eg, the processor 120) may determine, for example, whether a display adjustment is performed. According to an embodiment of the present disclosure, in operation 412, the electronic device may determine whether the display unit is turned on, the display unit is turned on, or periodically, in a display adjustment situation. As another example, the electronic device is provided in the electronic device when a movement of the electronic device occurs, when a user of the electronic device moves from the inside of the building to the outside or moves from the outside of the building to the inside to cause a momentary change in brightness. When a specific event occurs while the at least one sensor is covered by the cover of the electronic device, when the reproduction of the virtual reality content through the electronic device is ended, it may be determined as a display adjustment situation.

As a result of the comparison or determination in operation 412, when it corresponds to the display adjustment situation, the electronic device (eg, the processor 120) may perform operation 414, for example. According to an embodiment of the present disclosure, in operation 414, the electronic device may determine the brightness of the display from the value sensed by the first sensor. For example, the electronic device may identify or determine the illuminance or the white balance from the value sensed by the first sensor, and determine or determine the luminance to be applied to the content to be displayed through the display or the display from the determined illuminance or the white balance. In operation 416, the electronic device may determine the brightness of the display from the value sensed by the second sensor. For example, the electronic device may check or determine the illuminance or the white balance from the value sensed by the second sensor, and determine or determine the luminance to be applied to the content to be displayed through the display or the display from the determined illuminance or the white balance.

According to various embodiments of the present disclosure, the first sensor and the second sensor may be disposed on the same plane of the electronic device or may be disposed on different planes (eg, front or rear). Further, the first sensor and the second sensor may be sensors of the same type or may be sensors of different types (eg, the first sensor is an illuminance sensor and the second sensor is an image sensor).

In operation 412, when the comparison or determination does not correspond to the display adjustment situation, the electronic device (eg, the processor 120) may perform operation 412, for example.

In operation 418, the electronic device determines and determines the luminance of the content displayed on the display or the display based on the luminance determined from the value sensed by the first sensor and the luminance determined from the value sensed by the second sensor. Or can be adjusted. Detailed embodiments thereof will be described later.

5A is a flowchart illustrating a procedure of controlling saturation of a display (eg, display 160) in an electronic device (eg, the processor 120 or the display control module 180) according to various embodiments of the present disclosure. Referring to FIG. 5, in operation 502, the electronic device (eg, the processor 120) may determine, for example, whether a display adjustment is in progress. According to an embodiment of the present disclosure, in operation 502, the electronic device may determine whether the display is turned on or turned on, or periodically, whether the display is adjusted. As another example, the electronic device is provided in the electronic device when a movement of the electronic device occurs, when a user of the electronic device moves from the inside of the building to the outside or moves from the outside of the building to the inside to cause a momentary change in brightness. When a specific event occurs while the at least one sensor is covered by the cover of the electronic device, when the reproduction of the virtual reality content through the electronic device is ended, it may be determined as a display adjustment situation.

As a result of the comparison or determination in operation 502, when it corresponds to the display adjustment situation, the electronic device (eg, the processor 120) may perform operation 504, for example. According to an embodiment of the present disclosure, in operation 504, the electronic device may check or determine the white balance from the value sensed by the first sensor. In operation 506, the white balance may be checked or determined from the value sensed by the second sensor.

In operation 502, when the comparison or determination does not correspond to the display adjustment situation, the electronic device (eg, the processor 120) may perform operation 502, for example.

According to various embodiments of the present disclosure, the first sensor and the second sensor may be disposed on the same plane of the electronic device or may be disposed on different planes (eg, front or rear). In addition, the first sensor and the second sensor may be the same type of sensor or different types of sensors. For example, the first sensor and the second sensor may be an image sensor (eg, a sensor constituting the front camera module) provided at the front of the electronic device, and an image sensor (eg, a sensor constituting the rear camera module) provided at the rear of the electronic device. May be).

In operation 508, the electronic device displays a display based on the white balance checked or determined from the value sensed by the first sensor and the white balance checked or determined from the value sensed by the second sensor (eg, the display 160). Alternatively, the saturation or color of the content displayed on the display may be determined, determined, or adjusted.

5B illustrates a procedure of controlling luminance of a display (eg, the display 160) by using an image sensor in an electronic device (eg, the processor 120 or the display control module 180) according to various embodiments of the present disclosure. It is a flow chart showing. Referring to FIG. 5B, in operation 512, the electronic device (eg, the processor 120) may determine, for example, whether a display adjustment is performed.

According to an embodiment of the present disclosure, in operation 512, the electronic device may determine whether the display unit is turned on, the display unit is turned on, or periodically, whether the display is adjusted. As another example, the electronic device is provided in the electronic device when a movement of the electronic device occurs, when a user of the electronic device moves from the inside of the building to the outside or moves from the outside of the building to the inside to cause a momentary change in brightness. When a specific event occurs while the at least one sensor is covered by the cover of the electronic device, when the reproduction of the virtual reality content through the electronic device is ended, it may be determined as a display adjustment situation.

In operation 512, as a result of the comparison or determination, if it corresponds to the display adjustment situation, the electronic device (eg, the processor 120) may perform operation 514, for example. According to an embodiment of the present disclosure, in operation 514, the electronic device may check or determine the illuminance from the value sensed by the illuminance sensor. In operation 516, the electronic device may check or determine the color value of each pixel from the value sensed by the image sensor.

In operation 512, when the comparison or determination does not correspond to the display adjustment situation, the electronic device (eg, the processor 120) may perform operation 512, for example.

According to various embodiments of the present disclosure, the illuminance sensor may be disposed in front of the electronic device, and the image sensor may be at least a part of a rear camera module disposed in the rear of the electronic device.

In operation 518, the electronic device may calculate a brightness related value from the color value of each pixel identified or determined. A method of calculating a brightness-related value from the color value may be implemented using a preset conversion table, which will be described in detail later with reference to FIGS. 16 and 17. In operation 520, the electronic device may determine, determine, or adjust the brightness of the display based on the illuminance checked or determined by the illuminance sensor and the brightness related value checked or determined by the image sensor.

In the foregoing description, various embodiments of determining or adjusting luminance, saturation, or color by a combination of values sensed by the plurality of sensors have been described, but in the following description, a difference between values sensed by the plurality of sensors is described. A process of determining an ambient brightness environment of the electronic device and performing an automatic brightness operation adaptively to the determined ambient brightness environment will be described.

For example, it may be necessary to consider the case where the display brightness is adjusted when there is a large difference in brightness between the front and rear surroundings based on the display. For example, a user may use the electronic device in a backlight environment such that a rear surface of the electronic device faces a window corresponding to a light source that is much brighter than the room in a dark room. In such a backlight environment, the illumination sensor provided on the display surface of the electronic device measures the illumination in the direction toward the user, and the brightness of the display adjusted according to the illumination value is in the state where the user's gaze faces the bright window. May not provide enough brightness to use.

Therefore, it is necessary to adjust the brightness of the display in consideration of a situation where there is a large difference in ambient brightness between the front and rear of the display such as a backlight environment.

6A will be described a process of automatically adjusting the brightness of the display when the brightness of the surrounding environment of the front and rear of the display such as a backlight environment is greater than or equal to the threshold value.

FIG. 6A illustrates a luminance of a display (eg, the display 160) when the electronic device (eg, the processor 120 or the display control module 180) is in a backlight environment in an automatic brightness operation state according to various embodiments of the present disclosure. It is a flowchart showing the procedure for controlling the.

Referring to FIG. 6A, in operation 602, the electronic device may determine whether the display auto brightness is turned on. That is, it is determined whether the automatic brightness function of the electronic device for automatically adjusting the brightness of the display is on. In this case, when the display auto brightness function is not turned on, operation 602 may be performed again. Here, the auto-brightness function is a state in which the brightness of the display is adjusted using an illuminance sensor provided on the front of the electronic device.

On the other hand, when the automatic display brightness is turned on in operation 602, the automatic brightness adjustment of the display using the illumination sensor is being performed, and the electronic device may determine whether the rear camera is turned on in operation 604. Here, the rear camera may be an image sensor disposed on the rear of the electronic device. If the rear camera is turned on, the electronic device may sense a brightness value (BV) in operation 606. For example, the electronic device may obtain an RGB histogram based on the data acquired through the image sensor, and then obtain a brightness value based on the histogram. Subsequently, the electronic device may determine whether the brightness value sensed in operation 608 is equal to or greater than the threshold brightness value. Here, the threshold brightness value may be a predetermined threshold value corresponding to a backlight situation. According to an embodiment of the present disclosure, the backlight situation may be determined based on a brightness value sensed once or a brightness value sensed more than a predetermined number of times.

When the sensed brightness value is greater than or equal to the threshold brightness value, it may be regarded as a backlight situation. If the sensed brightness value is greater than or equal to the threshold brightness value, the electronic device increases the brightness of the display based on the sensed brightness value in operation 610. Can be. As described above, the electronic device automatically adjusts the brightness of the display using the illumination sensor, that is, the brightness of the display is adjusted according to the sensing value of the illumination sensor while the brightness measurement value using the rear camera is higher than a threshold brightness value corresponding to backlight. In addition, after increasing the brightness of the display, an automatic brightness operation of the display may be performed. According to an embodiment of the present disclosure, an increase in brightness of the display may be determined according to a difference between the sensed brightness value and the threshold brightness value.

In FIG. 6A, the case where the camera disposed on the rear side of the electronic device is turned on is described as an example. However, when at least one sensor other than the camera is turned on, the automatic brightness operation is performed by additionally using at least one sensor turned on. It may be implemented.

Meanwhile, in FIG. 6A, the electronic device performs the display brightness using the illuminance sensor and performs the display brightness using the sensing value of the rear camera as an example. The sensing value of the rear camera may also be used, which will be described in detail with reference to FIG. 6B.

6B illustrates luminance of a display (eg, the display 160) by using an illumination sensor and an image sensor in an electronic device (eg, the processor 120 or the display control module 180) according to various embodiments of the present disclosure. A flow chart showing the procedure for controlling.

Referring to FIG. 6B, in operation 612, the electronic device may determine whether the display auto brightness is turned on. In this case, when the display auto brightness function is not turned on, operation 612 may be performed again. If the automatic display brightness is turned on in operation 612, the electronic device may determine whether the rear camera is turned on in operation 614. Here, the rear camera may be an image sensor disposed at the rear of the electronic device. When the rear camera is turned on, the electronic device may sense a brightness value (BV) using the rear camera in operation 616. Subsequently, in operation 618, the electronic device may convert the brightness value into an illuminance value. In operation 620, the electronic device may determine the display brightness to be adjusted using the sensor value of the illuminance sensor provided on the front of the electronic device and the converted illuminance value. have. That is, the electronic device may calculate how much brightness is adjusted by using the sensor value of the illuminance sensor and the sensor value of the rear camera together. In this case, in order to determine how much the brightness is to be adjusted, a predetermined table value, a predetermined function, etc., which maps an illumination value (eg, the converted illumination value and a sensor value of the illumination sensor) and the brightness adjustment range, may be used. Accordingly, in operation 622, the electronic device may set the display luminance according to the display luminance determined in operation 620.

7 is a flowchart illustrating a procedure of controlling a display (eg, the display 160) based on content in an electronic device (eg, the processor 120 or the display control module 180) according to various embodiments of the present disclosure. to be. For example, a flowchart illustrating a procedure of controlling a display in consideration of a reproduction time of content in an electronic device.

Referring to FIG. 7, in operation 702, the electronic device may operate in a virtual reality (VR) mode, and in operation 704, VR-related content may be played. The user may be in a relatively dark state because the user is disconnected from the external environment in the VR mode. However, when the VR mode is terminated, the user's eyes may not be able to adapt as the light is instantly switched to the bright state.

According to various embodiments of the present disclosure, in operation 706, the remaining time until the time at which playback of the VR-related content ends or the ratio of remaining time to total time may be determined. In operation 708, if the remaining time or the ratio of the remaining time to the total time is less than or equal to a preset threshold Tr, in operation 710, the brightness, saturation, or color of the display may be adjusted.

According to various embodiments of the present disclosure, in operation 712, when the VR related content ends, the rear information of the electronic device may be overlaid and displayed on the screen as illustrated in FIG. 18 so as to be adapted to a sudden environmental change. Can be. The overlaid backside information may be set to become gradually brighter as the end point of the VR-related content approaches.

According to various embodiments of the present disclosure, at least one of the operations illustrated in FIGS. 3 to 7 may be omitted, and at least one other operation may be added between the operations. In addition, the operations of FIGS. 3 to 7 may be processed in the order shown, and the execution order of at least one operation may be changed from the execution order of other operations. 3 to 7 may be performed in an electronic device or may be performed in a server. In addition, at least one of the operations illustrated in FIGS. 3 to 7 may be performed in the electronic device, and the remaining operations may be implemented in the server.

According to one of various embodiments of the present disclosure, a method of operating an electronic device may include: displaying content by a display provided on a first surface of the electronic device; Sensing light incident by a sensor provided on a second surface of the electronic device; Determining brightness information around the electronic device based on at least the sensed light; And adjusting at least one property of the display or at least one property of the content based at least on the brightness information.

According to various embodiments of the present disclosure, when the other brightness information determined based on light sensed by another sensor for detecting light incident on the first surface falls within a specified range, the determination may be performed. To perform the operation.

According to various embodiments of the present disclosure, the adjusting may be further performed based on another brightness information determined based on light detected by another sensor for detecting light incident on the first surface. Can be done.

According to various embodiments of the present disclosure, it is determined whether a preset condition is satisfied to adjust the property of the display, and when the preset condition is satisfied, data received from the first sensor or the second sensor is processed. Can be.

According to various embodiments of the present disclosure, the preset condition may include display state information of the electronic device, information related to the movement of the electronic device, information about the surrounding environment of the electronic device, and information related to a cover attached to the electronic device. It may be determined based on at least one of.

8A and 8B are perspective views illustrating an electronic device in which sensors are disposed according to various embodiments of the present disclosure. 8A is a front perspective view of an electronic device according to various embodiments of the present disclosure, and FIG. 8B is a rear perspective view of the electronic device according to various embodiments of the present disclosure.

8A and 8B, the touch screen 890 may be disposed at the center of the front surface of the electronic device 800. The touch screen 890 may be large so as to occupy most of the front surface of the device 800. 8A illustrates an example in which a main home screen is displayed on the touch screen 890. The main home screen may include the first screen displayed on the touch screen 890 when the device 800 is turned on. Also, when the device 800 has different home screens of several pages, the main home screen may be the first home screen of the home pages of the several pages. The home screen may display

shortcut icons

871a, 871b, and 871c, a main menu switch key 871d, a time, and a weather 870 for executing frequently used applications. The main menu switch key 871d displays a menu screen on the touch screen 890. In addition, a status bar may be formed at the top of the touch screen 890 to display the status of the device 800 such as a battery charging status, a received signal strength, and a current time.

A home button 861a, a menu button 861b, and a back button 861c may be formed below the touch screen 890.

The home button 861a displays a main home screen on the touch screen 890. For example, when the home key 861a is touched while a home screen or a menu screen different from the main home screen is displayed on the touch screen 890, the main home screen on the touch screen 890 is displayed. Can be displayed. In addition, when the home button 861a is touched while applications are executed on the touch screen 890, the main home screen illustrated in FIG. 8A may be displayed on the touch screen 890. The home button 861a may also be used to display recently used applications on the touch screen 890 or to display a task manager.

Menu button 861b provides a connection menu that can be used on touch screen 890. The connection menu may include a widget adding menu, a background screen changing menu, a search menu, an editing menu, an environment setting menu, and the like.

The back button 861c may display a screen that was executed immediately before the currently running screen or terminate the most recently used application.

The first camera 866 (eg, the first image sensor), the illuminance sensor 864, and / or the proximity sensor may be disposed at the front edge of the electronic device 800. A second camera 852 (eg, a second image sensor), a flash 853, and a speaker 863 may be disposed on the rear surface 800c of the device 800.

According to various embodiments of the present disclosure, the electronic device 800 may include a housing that forms at least a portion of an external surface of the electronic device, and includes at least one sensor (eg, a first camera 866). ), A second camera 852, an illumination sensor 864, a proximity sensor, etc. may constitute at least a part of the housing.

In addition, the sensor may be located between the display and the housing.

For example, a power / reset button,

volume buttons

861f and 861g, a terrestrial DMB antenna for receiving a broadcast, one or a plurality of microphones 862, and the like may be disposed on the side of the device 800. The DMB antenna may be fixed to the device 800 or may be detachably formed.

In addition, a connector 865 may be formed on the bottom side of the device 800, and an electronic pen 868 may be inserted. The connector 865 is formed with a plurality of electrodes and may be connected to the external device by wire. An earphone connection jack 867 may be formed on the upper side of the device 800. An earphone may be inserted into the earphone connecting jack 867.

8A and 8B, one camera (image sensor) is disposed on the front and the rear of the electronic device 800, and one illuminance sensor is disposed on the front, but according to various embodiments of the present disclosure. Accordingly, the number and / or arrangement position of the image sensor or the illuminance sensor may be variously modified.

According to various embodiments of the present disclosure, the electronic device may have a wrap-around form, a full front display form (for example, a front surface configured as a display and no bezel or minimized form), and a transparent device. It may be implemented in various forms, such as, various embodiments of the present invention is not limited to a specific form of the electronic device.

According to various embodiments of the present disclosure, when the electronic device is a transparent display or a full front display, when the electronic device is placed on the floor, the display color may be changed or adjusted according to the color information of the floor surface.

9 is a block diagram illustrating a configuration of an image processing apparatus according to various embodiments of the present disclosure. 9, an image processing apparatus according to various embodiments of the present disclosure may include an image sensor module 910, an image preprocessing module 920 (eg, a companion chip), and an application processor (AP) 930. It can be configured to include. The image processing apparatus may be implemented to be directly connected to the application processor 930 from the image sensor module 910 without the image preprocessing module 920.

The image sensor module 910 is, for example, a module that senses an image and transmits each pixel value sensed through a mobile industry processor interface (MIPI) line to the image preprocessing module 920 or the application processor 930. Can be. In addition, the image sensor module 910 may transmit and receive various control signals through a serial peripheral interface (SPI) or an inter integrated circuit (I2C). The image sensor module 910 may be implemented by including an image sensor 911 (eg, a CMOS sensor) and control logic 912. The image sensor 911 may be implemented as a complementary metal oxide semiconductor (CMOS), and may sense an image by receiving and outputting a signal in units of pixels. The control logic 912 may perform a function of controlling the driving of the image sensor module 910.

The image preprocessing module 920 may be further configured, for example, to support a specific function of the image sensor. For example, the image preprocessing module 920 may perform preprocessing to improve image quality, a detailed example of which will be described later with reference to FIG. 10.

The application processor 930 may include, for example, an image signal processor 931 (ISP; image signal processor) and a main processor (CPU) 932. The image signal processor 931 may include, for example, a Bayer processor 931a and / or a color processor 931b (Luma / Color). In addition, the Bayer processor 931a or the color processor 931b may have a plurality of processing blocks in the form of a pipeline for each processing function as shown. A detailed implementation example of the image signal processor 931 is illustrated in FIG. 11.

Referring to FIG. 9, according to various embodiments of the present disclosure, data for determining luminance, saturation, or color may be obtained from an image sensor provided on a rear surface of an electronic device. For example, data for determining luminance, saturation or color using data processed by at least a portion of the image signal processor 931 illustrated in FIG. 9 (eg, an RGB histogram or an exposure time). Etc.) can be obtained.

For example, according to various embodiments of the present disclosure, information (eg, RGB histogram, exposure time, etc.) obtained in some blocks of the Bayer processor 931a may be stored in a memory. The processor (eg, the CPU 932) may acquire a value (eg, illuminance value) related to brightness by using the result stored in the memory.

According to various embodiments of the present disclosure, in order to reduce power consumption when obtaining a value related to brightness, only a processing block related to the data acquisition is performed among a plurality of blocks constituting an internal pipeline of the image signal processor ( on, the remaining unrelated processing blocks can be turned off or bypassed on the pipeline. For example, the block indicated by the solid line of the plurality of blocks constituting the Bayer processor 931a may be operated, and the block indicated by the dotted line may be turned off or bypassed. For example, the block indicated by the solid line among the plurality of blocks constituting the color processor 931b may be operated, and the block indicated by the dotted line may be turned off or bypassed.

The value acquired or output by the Bayer processor 931a may include an accumulated pixel value (eg, an RGB pixel accumulation value) or an RGB histogram. The color processor 931b may perform a function of processing brightness or color of the sensed image.

According to various embodiments of the present disclosure, only the processing block related to the data acquisition is turned on among the plurality of blocks constituting the internal pipeline of the image preprocessing module 920 in the image preprocessing module 920. And other unrelated processing blocks can be turned off or bypassed on the pipeline. A detailed example thereof will be described later with reference to FIG. 10.

According to various embodiments, the electronic device (eg, the processor 120 or the controller 220) may transmit high-speed data communication (for example, pixel information) from the image sensor module 910 to the image preprocessing module 920. MIPI) can be turned off and only the control signal line (eg, SPI) can be implemented.

According to various embodiments of the present disclosure, information (eg, RGB histogram, exposure time, etc.) obtained from the ISP 931 of the AP 930 may be stored in a memory (eg, the storage 240). The CPU 932 may calculate backside information using the result stored in the memory. When the information obtained from the ISP 931 of the AP 930 is stored in a memory, only relevant blocks of pipelines inside the ISP 931 are operated to reduce power consumption, and the remaining blocks are turned off or bypassed. Can be.

10 is a diagram illustrating detailed blocks of an image preprocessing module according to various embodiments of the present disclosure. Referring to FIG. 10, the above-described image preprocessing module of FIG. 9 may include a differential pulse code modulation (DPCM) canceling unit 1010, a pixel value adding unit 1020, a cutting unit 1030, a gamma value processing unit 1040, and a binning correction unit. 1050, the DPCM compression unit 1060 may include at least one. As described above, in order to obtain a value related to brightness by information obtained from an image sensor, only at least some blocks of a plurality of blocks of the image preprocessing module may be used.

For example, the DPCM canceling unit 1010 and the pixel value summing unit 1020 are turned on, and the cutting unit 1030, the gamma value processing unit 1040, the binning correction unit 1050, and the DPCM compression unit 1060 are turned off. It can be bypassed.

11 is a diagram illustrating detailed blocks of an image signal processor 931 according to various embodiments of the present disclosure. Referring to FIG. 11, at least some of a plurality of blocks constituting an image signal processing unit (eg, a Defective Pixel Correction unit (DPC), a Color Filter Array Interpolation unit (CFA), and an STATS unit (image statistics unit) for the YCC) Turn on only the remaining blocks (e.g., DPC section, CFA section, CCM (Color Correction Matrix unit), Gamma Correction unit, CSC (Color Space Conversion unit), enhancement section (for Bayer) Noise reduction and Edge Enhancement unit), MANR unit (Motion Adaptive Noise Reduction unit), CR unit (Chroma Resampler unit), CSC unit (Color Space Conversion unit, etc.) can be turned off or bypassed.

For example, the CCM unit is a module for correcting a color variation of an image caused by an optical reason, an illumination variable, a color filter characteristic of a sensor, etc., the gamma unit is a module for correcting a gamma value, and the enhancement unit reduces noise. It is a module to improve edge. In addition, the MANR unit is a module for adaptively reducing noise according to the movement, the CSC unit is a module for converting the color space, the CR unit for converting the YcbCr input to the desired chroma sub-sampling format (sub-sampling) format Module.

The configuration of the image signal module unit illustrated in FIG. 11 is an example of an image signal module unit to which an embodiment of the present invention can be applied, and embodiments of the present invention may be applied to variously configured image signal module units. For example, at least one processing block (eg, a module or a component) of the plurality of detailed processing blocks constituting the image signal module unit may be turned off or bypassed.

Hereinafter, various embodiments in which information related to brightness may be obtained using at least some components of the image processing apparatus described above with reference to FIGS. 12 through 15 will be described.

12 is a block diagram illustrating a configuration of an image processing apparatus according to various embodiments of the present disclosure. Referring to FIG. 12, an image processing apparatus according to various embodiments of the present disclosure includes an image sensor module 1210, an image preprocessing module 1220 (eg, companion chip), and an application processor (AP) 1230. Can be. The image processing apparatus may be implemented to be directly connected to the application processor 1230 from the image sensor module 1210 without the image preprocessing module 1220.

The image sensor module 1210 is a module for sensing an image and may transmit each pixel value sensed through a MIPI line to the image preprocessing module 1220 or the application processor 1230. In addition, the image sensor module 1210 may transmit and receive various control signals through SPI or I2C. The image sensor module 1210 may be implemented to include an image sensor 1211 (eg, a CMOS sensor) and control logic 1212.

The image preprocessing module 1220 may be further configured to support a specific function of the image sensor. For example, the image preprocessing module 1220 may perform preprocessing to improve image quality.

The application processor 1230 may include an image signal processor 1231 (ISP) and a main processor (CPU) 1232. The image signal processor 1231 may include a Bayer processor 1231a and a color processor 1231b (Luma / Color). In addition, the Bayer processor 1231a or the color processor 1231b may have a plurality of processing blocks in a pipeline form for each processing function as shown. A detailed implementation example of the image signal processor 1231 is illustrated in FIG. 11. Since the basic functions of the above components have been described above with reference to FIG. 9, repeated descriptions thereof will be omitted.

Referring to FIG. 12, it is possible to acquire data related to brightness in the image preprocessing module 1220 according to various embodiments of the present disclosure. For example, the image preprocessing module 1220 may obtain data related to the brightness from the Bayer processor 1231a of the application processor (AP) 1230. The image preprocessing module 1220 may extract a Bayer histogram from the Bayer processor 1231a and directly transfer the Bayer histogram to the main processor (CPU) 1232 of the application processor (AP) 1230. According to various embodiments of the present disclosure, the image preprocessing module 1220 extracts a Bayer histogram from the Bayer processing unit 1231a and calculates data related to brightness or white balance using the extracted Bayer histogram. The result may be transferred to the main processor (CPU) 1232 of the application processor (AP) 1230.

According to various embodiments of the present disclosure, when processing the information obtained from the image sensor module 1210, at least some of the blocks included in the ISP 1231 of the AP 1230 may be turned off to reduce power consumption. Can be.

According to various embodiments of the present disclosure, the electronic device (eg, the processor 120 or the controller 220) may transmit pixel information from the image sensor module 1210 to the image preprocessing module 1220. High speed data communication (eg, MIPI) can be turned off, and only control signal lines (eg, SPI) can be implemented to operate.

According to various embodiments of the present disclosure, information (eg, RGB histogram, exposure time, etc.) obtained from the ISP 1231 of the AP 1230 may be stored in the memory (eg, the storage unit 240). The CPU 1232 may calculate backside information using the result stored in the memory. When the information obtained from the ISP 1231 of the AP 1230 is stored in a memory, only relevant blocks of pipelines inside the ISP 1231 are operated to reduce power consumption, and the remaining blocks are turned off or bypassed. Can be.

The main processor (CPU) 1232 of the application processor (AP) 1230 may calculate or process data related to brightness based on information transmitted from the image preprocessing module 1220.

13 is a block diagram illustrating a configuration of an image processing apparatus according to various embodiments of the present disclosure. Referring to FIG. 13, an image processing apparatus according to various embodiments of the present disclosure includes an image sensor module 1310, an image preprocessing module 1320 (eg, companion chip), and an application processor (AP) 1330. Can be. The image processing apparatus may be implemented to be directly connected to the application processor 1330 from the image sensor module 1310 without the image preprocessing module 1320.

The image sensor module 1310 is a module for sensing an image and may transmit each pixel value sensed through a MIPI line to the image preprocessing module 1320 or the application processor 1330. In addition, the image sensor module 1310 may transmit and receive various control signals through SPI or I2C. The image sensor module 1310 may be implemented to include an image sensor 1311 (eg, a CMOS sensor) and control logic 1312.

The image preprocessing module 1320 may be further configured to support a specific function of the image sensor. For example, the image preprocessing module 1320 may perform preprocessing to improve image quality.

The application processor 1330 may include an image signal processor 1331 (ISP) and a main processor (CPU) 1332. The image signal processor 1331 may include a Bayer processor 1331a and a color processor 1331b (Luma / Color). In addition, the Bayer processor 1331a or the color processor 1331b may have a plurality of processing blocks in the form of a pipeline for each processing function as shown. A detailed implementation example of the image signal processor 1331 is illustrated in FIG. 11. Since the basic functions of the above components have been described above with reference to FIGS. 9 and 13, repeated descriptions thereof will be omitted.

According to various embodiments of the present disclosure, in order to reduce power consumption, the image processing apparatus may obtain desired information by operating only at least some functional blocks of the image preprocessing module 1320. For example, the high-speed data communication line (eg, MIPI) for transmitting pixel information between the modules may be turned off, and only the control line (eg, SPI or I2C) may be implemented.

As a specific example, as described above with reference to FIG. 10, among the plurality of functional blocks constituting the image preprocessing module 1320, the DPCM canceling unit 1010 and the pixel value summing unit 1020 are turned on and the cutting unit 1030 is turned on. The gamma value processing unit 1040, the binning correction unit 1050, and the DPCM compression unit 1060 may be turned off or bypassed.

14 is a block diagram illustrating a configuration of an image processing apparatus according to various embodiments of the present disclosure. Referring to FIG. 14, an image processing apparatus according to various embodiments of the present disclosure includes an image sensor module 1410, an image preprocessing module 1420 (eg, a companion chip), and an application processor (AP) 1430. Can be. The image processing apparatus may be implemented to be directly connected to the application processor 1430 from the image sensor module 1410 without the image preprocessing module 1420.

According to various embodiments of the present disclosure, the image sensor module 1410 senses an image by the image sensor 1411 and data related to brightness from information sensed by the image sensor 1411 directly by the control logic 1412. Alternatively, information such as white balance may be generated.

Data or white balance information related to brightness generated by the image sensor module 1410 may be transmitted to the application processor (AP) 1430 through the image preprocessing module 1420.

The application processor (AP) 1430 may include an image signal processor 1431 (ISP) and a main processor (CPU) 1432. The image signal processor 1431 may include a Bayer processor 1431a and a color processor 1431b (Luma / Color). Since the basic functions of the above components have been described above with reference to FIGS. 9 and 13, repeated descriptions thereof will be omitted.

In the embodiment of FIG. 14, since data is directly generated in the image sensor module 1410, it is not necessary to transmit data information for each pixel through a MIPI line. Accordingly, the MIPI line may be turned off and the data generated by the image sensor module 1410 may be transmitted to the main processor (CPU) 1432 of the application processor (AP) 1430 through only the SPI or I2C line.

For example, the image preprocessing module 1420 checks or determines backside information (eg, brightness, white balance, etc.) from information obtained from the image sensor module 1410 (eg, Bayer histogram, RGB cumulative value, etc.), and then the application. Transfer to processor (AP) 1430.

According to various embodiments of the present disclosure, at least some functions of the image preprocessing module 1420 and the ISP 1431 may be turned off to reduce power consumption. Also, for example. Functional blocks other than the image sensor, such as actuators, optical image stabilization (OIS), and the like, of the image sensor module 1410 may be turned off or bypassed. In addition, according to various embodiments of the present disclosure, only some scan lines of the image sensor module 1410 may be driven as shown in FIGS. 15A and 15B to reduce power consumption.

15A and 15B are block diagrams illustrating a configuration of an image processing apparatus according to various embodiments of the present disclosure. 15A and 15B, an image processing apparatus according to various embodiments of the present disclosure may include an image sensor module 1510, an image preprocessing module 1520 (eg, a companion chip), and an application processor (AP) 1530. It can be configured to include. The image processing apparatus may be implemented to be directly connected to the application processor 1530 from the image sensor module 1410 without the image preprocessing module 1520.

According to various embodiments of the present disclosure, the image sensor module 1510 senses an image by the image sensor 1511 and data related to brightness from information sensed by the image sensor 1511 directly by the control logic 1512. Alternatively, information such as white balance may be generated.

Data or white balance information related to the brightness generated by the image sensor module 1510 may be transmitted to the application processor (AP) 1530 through the image preprocessing module 1520.

15A and 15B, since data is generated directly in the image sensor module 1510, it is not necessary to transmit data information for each pixel through a MIPI line. Accordingly, the MIPI line may be turned off, and data generated by the image sensor module 1510 may be transmitted to the main processor (CPU) 1532 of the application processor (AP) 1530 only through the SPI or I2C line.

The application processor (AP) 1530 may include an image signal processor 1531 (ISP) and a main processor (CPU) 1532. The image signal processor 1531 may include a Bayer processor 1531a and a color processor 1531b (Luma / Color). Since the basic functions of the above components have been described above with reference to FIGS. 9 and 13, repeated descriptions thereof will be omitted.

15A and 15B, according to various embodiments of the present disclosure, the image signal processor 1510 directly generates information related to brightness, and since the information on each pixel is not used, the image sensor 1511. Instead of using the entire area of), only some pixel areas may be used to generate brightness-related information.

For example, as illustrated in FIG. 15A, information related to brightness may be generated only by the values sensed in pixels of odd or even lines by alternating with each horizontal or vertical line. In addition, as shown in FIG. 15B, the entire area of the image sensor 1511 is divided into a plurality of

areas

1513a and 1513b, and the brightness is sensed with only a value detected for at least one area 1513a of the plurality of areas. Information related to may also be generated.

According to various embodiments, the electronic device (eg, the processor 120 or the controller 220) may include a plurality of unit pixels (eg, red, green, and / or the like) included in the image sensor module 1510. Alternatively, only some pixels of blue may be used to generate (determine) data or white balance information related to brightness. For example, the electronic device (eg, the controller 220) may include attributes (eg, attributes of the pixels constituting the largest number of unit pixels constituting the image sensor module 1510 (eg, the image sensor 1411). Color information may be used to determine data related to brightness (eg, brightness information) or white balance information. For example, in the image sensor module 1510 (for example, the image sensor 1411), when the unit pixel having the green property is larger than the pixel having the red or blue property, the unit having the green property Brightness or white balance information may be determined using image information (eg, color information) of the pixel. The electronic device (eg, the controller 220) may determine brightness or white balance information while driving some pixels (eg, green pixels) of the control logic 1512 to reduce power consumed by the electronic device. .

According to various embodiments of the present disclosure, the electronic device (eg, the processor 120 or the controller 220) may use data related to brightness (eg, brightness information) by using one or more pixels having the lowest number among a plurality of pixels. Alternatively, the white balance information may be determined. For example, when the power consumed by the electronic device is equal to or greater than a specified value (eg, 80% or more of the total power consumption of the image sensor module 1510), the electronic device ( For example, the controller 220 may determine brightness information by using color information of the pixel having the smallest number of pixels constituting the image sensor module 1510 (eg, the image sensor 1411). For example, when a pixel having a red or blue property is smaller than a unit pixel having a green property, the selected unit pixel is selected by selecting color information of one of the unit pixels having a red or blue property. The brightness information or the white balance information may be determined using the color information of. The electronic device (eg, the controller 220) may drive some pixels (eg, a red pixel or a blue pixel) of the control logic 1512 to reduce brightness or white balance information while reducing power consumed by the electronic device. You can judge.

According to various embodiments of the present disclosure, when the color information of the unit pixel having the smallest number among the plurality of pixels is plural, the color used to determine the brightness information or the white balance information based on the priority set in the electronic device. Information can be determined. For example, the electronic device (for example, the controller 220) may use the red attribute pixel to display the brightness information or the white balance when the red attribute pixel has a high priority among the pixels having the red or blue attribute. Information can be judged. The preset priority may be changed based on the amount of light incident on the image sensor module 1510. For example, the electronic device (eg, the controller 220) compares the amount of light incident on the image sensor module 1510 for each unit pixel, and based on the comparison result, the brightness information or the white balance. The priority used to determine the information can be changed. For example, if the amount of light of a red pixel (e.g., a pixel of red property) is greater than the amount of light of a blue pixel (e.g., the light of the red property is more than the light of a blue property), the priority of the red pixel is recalled. It can be set higher than the priority of blue pixels.

In addition, according to various embodiments of the present disclosure, the illuminance may be determined from the color information by using the illuminance conversion table for the RGB sensor value. For example, the illuminance value corresponding to the color information sensed by the RGB sensor may be determined using a table in which the illuminance value corresponding to the RGB sensor value is set. In addition, when the lighting environment is different from the usual environment such as an incandescent lamp or a fluorescent lamp, the illumination may be adaptively determined according to the current lighting environment through information sensed by the at least one sensor.

16 is a block diagram of an electronic device 1601 according to various embodiments. The electronic device 1601 may include, for example, all or part of the electronic device 101 shown in FIG. 1. The electronic device 1601 may include one or more processors (eg, an application processor (AP)) 1610, a communication module 1620, a subscriber identification module 1624, a memory 1630, a sensor module 1640, and an input device 1650. ), Display 1660, interface 1670, audio module 1680, camera module 1169, power management module 1695, battery 1696, indicator 1697, and motor 1698. have.

The processor 1610 may control, for example, a plurality of hardware or software components connected to the processor 1610 by running an operating system or an application program, and may perform various data processing and operations. The processor 1610 may be implemented with, for example, a system on chip (SoC). According to an embodiment of the present disclosure, the processor 1610 may further include a graphic processing unit (GPU) and / or an image signal processor. The processor 1610 may include at least some of the components illustrated in FIG. 2 (eg, the cellular module 1621). The processor 1610 may load and process instructions or data received from at least one of the other components (eg, nonvolatile memory) into volatile memory, and store various data in the nonvolatile memory. have.

The communication module 1620 may have a configuration that is the same as or similar to that of the communication interface 170 of FIG. 1. The communication module 1620 may be, for example, a cellular module 1621, a WiFi module 1623, a Bluetooth module 1625, a GNSS module 1627 (eg, a GPS module, a Glonass module, a Beidou module, or a Galileo module). It may include an NFC module 1628 and a radio frequency (RF) module 1629.

The cellular module 1621 may provide, for example, a voice call, a video call, a text service, or an internet service through a communication network. According to an embodiment of the present disclosure, the cellular module 1621 may perform identification and authentication of the electronic device 1601 in a communication network using a subscriber identification module (eg, a SIM card) 1624. According to an embodiment of the present disclosure, the cellular module 1621 may perform at least some of the functions that the processor 1610 may provide. According to an embodiment, the cellular module 1621 may include a Communication Processor (CP).

Each of the WiFi module 1623, the Bluetooth module 1625, the GNSS module 1627, or the NFC module 1628 may include, for example, a processor for processing data transmitted and received through a corresponding module. According to some embodiments, at least some (eg, two or more) of the cellular module 1621, the WiFi module 1623, the Bluetooth module 1625, the GNSS module 1627, or the NFC module 1628 may be one integrated chip. (IC) or in an IC package.

The RF module 1629 may transmit / receive, for example, a communication signal (for example, an RF signal). The RF module 1629 may include, for example, a transceiver, a power amp module (PAM), a frequency filter, a low noise amplifier (LNA), an antenna, or the like. According to another embodiment, at least one of the cellular module 1621, the WiFi module 1623, the Bluetooth module 1625, the GNSS module 1627, or the NFC module 1628 may transmit and receive RF signals through a separate RF module. Can be.

Subscriber identification module 1624 may include, for example, a card containing a subscriber identification module and / or an embedded SIM, and may include unique identification information (eg, an integrated circuit card identifier (ICCID)) or It may include subscriber information (eg, international mobile subscriber identity (IMSI)).

The memory 1630 (eg, the memory 130) may include, for example, an internal memory 1632 or an external memory 1634. The internal memory 1632 may be, for example, volatile memory (eg, dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM), etc.) or non-volatile memory (eg, non-volatile memory). One time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash, etc.), It may include at least one of a hard drive or a solid state drive (SSD).

The external memory 1634 may be a flash drive, for example, compact flash (CF), secure digital (SD), micro secure digital (Micro-SD), mini secure digital (Mini-SD), or xD (extreme). It may further include a digital, a multi-media card (MMC) or a memory stick (memory stick). The external memory 1634 may be functionally and / or physically connected to the electronic device 1601 through various interfaces.

For example, the sensor module 1640 may measure a physical quantity or detect an operation state of the electronic device 1601, and may convert the measured or detected information into an electrical signal. The sensor module 1640 may include, for example, a gesture sensor 1640A, a gyro sensor 1640B, an air pressure sensor 1640C, a magnetic sensor 1640D, an acceleration sensor 1640E, a grip sensor 1640F, and a proximity sensor ( 1640G), color sensor 1640H (e.g., red (green, blue) sensor), biometric sensor (1640I), temperature / humidity sensor (1640J), light sensor (1640K), or UV (ultra violet) ) May include at least one of the sensors 1640M. Additionally or alternatively, the sensor module 1640 may include, for example, an olfactory sensor, an electromyography sensor, an electroencephalogram sensor, an electrocardiogram sensor. And infrared (IR) sensors, iris sensors, and / or fingerprint sensors. The sensor module 1640 may further include a control circuit for controlling at least one or more sensors belonging therein. In some embodiments, the electronic device 1601 further includes a processor configured to control the sensor module 1640 as part of or separately from the processor 1610, while the processor 1610 is in a sleep state, The sensor module 1640 may be controlled.

The input device 1650 is, for example, a touch panel 1652, a (digital) pen sensor 1654, a key 1656, or an ultrasonic input device ( 1658). For example, the touch panel 1652 may use at least one of capacitive, resistive, infrared, and ultrasonic methods. The touch panel 1652 may further include a control circuit. The touch panel 1652 may further include a tactile layer to provide a tactile response to the user.

The (digital) pen sensor 1654 may be, for example, part of a touch panel or include a separate sheet for recognition. The key 1656 may include, for example, a physical button, an optical key, or a keypad. The ultrasonic input device 1658 may detect ultrasonic waves generated by an input tool through a microphone (for example, the microphone 1688) and check data corresponding to the detected ultrasonic waves.

Display 1660 (eg, display 160) may include panel 1662, hologram device 1664, or projector 1666. The panel 1662 may include a configuration that is the same as or similar to the display 160 of FIG. 1. The panel 1662 may be implemented to be, for example, flexible, transparent, or wearable. The panel 1662 may be configured as a single module together with the touch panel 1652. The hologram device 1664 may show a stereoscopic image in the air by using interference of light. The projector 1666 may display an image by projecting light onto a screen. For example, the screen may be located inside or outside the electronic device 1601. According to an embodiment, the display 1660 may further include a control circuit for controlling the panel 1662, the hologram device 1664, or the projector 1666.

The interface 1670 may be, for example, a high-definition multimedia interface (HDMI) 1672, a universal serial bus (USB) 1674, an optical interface 1676, or a D-subminiature (D-subminiature). ) 1678. The interface 1670 may be included in, for example, the communication interface 170 shown in FIG. 1. Additionally or alternatively, the interface 1670 may be, for example, a mobile high-definition link (MHL) interface, a secure digital (SD) card / multi-media card (MMC) interface, or an IrDA (infrared). data association) may include a standard interface.

The audio module 1680 may bidirectionally convert, for example, a sound and an electrical signal. At least some components of the audio module 1680 may be included in, for example, the input / output interface 145 illustrated in FIG. 1. The audio module 1680 may process sound information input or output through, for example, a speaker 1802, a receiver 1684, an earphone 1686, a microphone 1688, or the like.

The camera module 1701 is, for example, a device capable of capturing still images and moving images, and according to an embodiment, one or more image sensors (eg, a front sensor or a rear sensor), a lens, an image signal processor (ISP) Or flash (eg, LED or xenon lamp, etc.).

The power management module 1695 may manage power of the electronic device 1601, for example. According to an embodiment, the power management module 1695 may include a power management integrated circuit (PMIC), a charger integrated circuit (ICC), or a battery or fuel gauge. The PMIC may have a wired and / or wireless charging scheme. The wireless charging method may include, for example, a magnetic resonance method, a magnetic induction method, an electromagnetic wave method, or the like, and may further include additional circuits for wireless charging, such as a coil loop, a resonance circuit, a rectifier, and the like. have. The battery gauge may measure, for example, the remaining amount of the battery 1696, the voltage, the current, or the temperature during charging. The battery 1696 may include, for example, a rechargeable battery and / or a solar battery.

The indicator 1697 may display a specific state of the electronic device 1601 or a part thereof (for example, the processor 1610), for example, a booting state, a message state, or a charging state. The motor 1698 may convert an electrical signal into mechanical vibrations, and may generate a vibration or haptic effect. Although not shown, the electronic device 1601 may include a processing device (eg, a GPU) for supporting mobile TV. The processing apparatus for supporting mobile TV may process media data according to a standard such as digital multimedia broadcasting (DMB), digital video broadcasting (DVB), or mediaFlo ^™ .

Each of the components described in this document may be composed of one or more components, and the names of the components may vary according to the type of electronic device. In various embodiments, the electronic device may be configured to include at least one of the components described in this document, and some components may be omitted or further include additional components. In addition, some of the components of the electronic device according to various embodiments of the present disclosure may be combined to form a single entity, and thus may perform the same functions of the corresponding components before being combined.

17 is a block diagram 1700 of a program module 1710 according to various embodiments of the present disclosure. According to an embodiment of the present disclosure, the program module 1710 may include an operating system (OS) for controlling resources related to the electronic device and / or various applications (eg, application programs) running on the operating system. have. The operating system, for example, can be an example, and Android (Android ^{^TM),} iOS ^TM, Windows (Windows ^TM), Symbian (Symbian ^TM), Tizen (Tizen ^TM), or sea (Samsung bada os ^TM) .

The program module 1710 may include a kernel 1720, middleware 1730, an application programming interface (API) 1760, and / or an application 1770. At least a part of the program module 1710 may be preloaded on the electronic device or downloaded from a server.

The kernel 1720 may include, for example, a system resource manager 1721 or a device driver 1723. The system resource manager 1721 may perform control, allocation, or retrieval of system resources. According to an embodiment of the present disclosure, the system resource manager 1721 may include a process manager, a memory manager, or a file system manager. The device driver 1723 may include, for example, a display driver, a camera driver, a Bluetooth driver, a shared memory driver, a USB driver, a keypad driver, a WIFI driver, an audio driver, or an inter-process communication (IPC) driver. have.

The middleware 1730 may provide, for example, a function commonly required by the application 1770 or the API 1760 to enable the application 1770 to efficiently use limited system resources inside the electronic device. Various functions may be provided to the application 1770 through. According to an embodiment of the present disclosure, the middleware 1730 includes a runtime library 1735, an application manager 1741, a window manager 1742, a multimedia manager 1743, and a resource manager. (resource manager) 1744, power manager 1745, database manager 1746, package manager 1747, connectivity manager 1748, notification manager It may include at least one of a notification manager 1749, a location manager 1750, a graphic manager 1751, and a security manager 1702.

The runtime library 1735 may include, for example, a library module used by a compiler to add a new function through a programming language while the application 1770 is running. The runtime library 1735 may perform input / output management, memory management, or a function for an arithmetic function.

The application manager 1741 may manage, for example, a life cycle of at least one of the applications 1770. The window manager 1742 may manage GUI resources used on the screen. The multimedia manager 1743 may identify a format required for playing various media files, and may encode or decode the media file by using a codec suitable for the format. The resource manager 1744 may manage resources such as source code, memory, or storage space of at least one of the applications 1770.

The power manager 1745 may operate together with, for example, a basic input / output system (BIOS) to manage a battery or power, and provide power information necessary for the operation of the electronic device. . The database manager 1746 may generate, search for, or change a database to be used by at least one of the applications 1770. The package manager 1747 may manage installation or update of an application distributed in the form of a package file.

The connection manager 1748 may manage, for example, a wireless connection such as WIFI or Bluetooth. The notification manager 1749 may display or notify an event such as an arrival message, an appointment, a proximity notification, or the like in a manner that does not disturb the user. The location manager 1750 may manage location information of the electronic device. The graphic manager 1751 may manage graphic effects to be provided to the user or a user interface related thereto. The security manager 1702 may provide various security functions required for system security or user authentication. According to an embodiment of the present disclosure, when the electronic device (eg, the electronic device of FIG. 8) includes a telephone function, the middleware 1730 may include a telephone manager for managing a voice or video call function of the electronic device. It may further include.

The middleware 1730 may include a middleware module forming a combination of various functions of the above-described components. The middleware 1730 may provide modules specialized according to types of operating systems in order to provide differentiated functions. In addition, the middleware 1730 may dynamically delete some of the existing components or add new components.

The API 1760 is, for example, a set of API programming functions, and may be provided in a different configuration according to an operating system. For example, in the case of Android or iOS, one API set may be provided for each platform, and in Tizen, two or more API sets may be provided for each platform.

The application 1770 may be, for example, a home 1771, a dialer 1772, an SMS / MMS 1773, an instant message 1774, a browser 1775, a camera 1776, and an alarm 1777. ), Contacts (1778), voice dial (1779), email (1780), calendar (1781), media player (1782), album (1783), or watch (1784), health care (e.g., exercise volume Or one or more applications that may provide functions such as measuring blood sugar, or providing environmental information (eg, providing barometric pressure, humidity, or temperature information).

According to an embodiment of the present disclosure, the application 1770 may be an application that supports information exchange between the electronic device (for example, the electronic device of FIG. 1 or 2) and an external electronic device (hereinafter, for convenience of description, an “information exchange application”). "). The information exchange application may include, for example, a notification relay application for delivering specific information to the external electronic device, or a device management application for managing the external electronic device. .

For example, the notification delivery application may provide a function of delivering notification information generated by another application of the electronic device (eg, an SMS / MMS application, an email application, a health care application, or an environmental information application) to an external electronic device. It may include. In addition, the notification transmission application may receive notification information from an external electronic device and provide the notification information to a user. The device management application may, for example, turn-on / turn-off or brightness of a display of at least one function of an external electronic device (eg, the external electronic device itself (or some component)) in communication with the electronic device. (Or, resolution) adjustment), an application running on the external electronic device or a service (eg, a call service or a message service) provided by the external electronic device may be managed (eg, installed, deleted, or updated).

According to an embodiment of the present disclosure, the application 1770 may include an application (eg, a health care application) designated according to an attribute of the external electronic device (eg, an attribute of the electronic device, and the type of the electronic device is a mobile medical device). Can be. According to an embodiment of the present disclosure, the application 1770 may include an application received from an external electronic device. According to an embodiment of the present disclosure, the application 1770 may include a preloaded application or a third party application downloadable from a server. The names of the components of the program module 1710 according to the shown embodiment may vary depending on the type of operating system.

According to various embodiments of the present disclosure, at least a part of the program module 1710 may be implemented in software, firmware, hardware, or a combination of two or more thereof. At least a part of the program module 1710 may be implemented (for example, executed) by, for example, a processor (for example, the AP 3310). At least a part of the program module 1710 may include, for example, a module, a program, a routine, sets of instructions, or a process for performing one or more functions.

As used herein, the term "module" or "functional unit" may refer to a unit including one or a combination of two or more of hardware, software, or firmware. "Module" or "functional part" is used interchangeably with terms such as, for example, unit, logic, logical block, component, or circuit. Can be The "module" or "functional part" may be a minimum unit or part of an integrally formed part. The module may be a minimum unit or part of performing one or more functions. The "module" or "functional part" can be implemented mechanically or electronically. For example, a “module” or “function unit” may be an application-specific integrated circuit (ASIC) chip, field-programmable gate arrays (FPGAs), or programmable logic devices that perform certain operations, known or developed in the future. -logic device).

According to various embodiments of the present disclosure, at least a portion of an apparatus (eg, modules or functions thereof) or a method (eg, operations) may be, for example, computer-readable storage media in the form of a program module. It can be implemented as a command stored in. When the command is executed by a processor (eg, the controller 220), the one or more processors may perform a function corresponding to the command. The computer-readable storage medium may be, for example, the storage unit 240.

The computer-readable recording medium may include a hard disk, a floppy disk, a magnetic medium (for example, magnetic tape), an optical media (for example, a compact disc read only memory (CD-ROM), a DVD). (digital versatile disc), magneto-optical media (such as floptical disk), hardware devices (such as read only memory, random access memory (RAM), or flash memory Etc. Also, the program instructions may include not only machine code generated by a compiler, but also high-level language code executable by a computer using an interpreter, etc. The above-described hardware device may include It may be configured to operate as one or more software modules to perform the operations of the various embodiments, and vice versa.

Modules or program modules according to various embodiments of the present disclosure may include at least one or more of the above components, some of them may be omitted, or may further include additional components. Operations performed by a module, program module, or other component according to various embodiments of the present disclosure may be executed in a sequential, parallel, repetitive, or heuristic manner. In addition, some operations may be executed in a different order, may be omitted, or other operations may be added.

According to various embodiments of the present disclosure, in a storage medium storing instructions, the instructions are configured to cause the at least one processor to perform at least one operation when executed by the at least one processor. Displaying the content by a display provided on the first side of the electronic device; Sensing light incident by a sensor provided on a second surface of the electronic device; Determining brightness information around the electronic device based on at least the sensed light; And adjusting at least one property of the display or at least one property of the content based at least on the brightness information.

And the embodiments of the present invention invented in the specification and drawings are merely to illustrate the technical contents according to the embodiments of the present invention and to present specific examples only to help the understanding of the embodiments of the present invention, the scope of the embodiments of the present invention It is not intended to be limiting. Therefore, the scope of various embodiments of the present invention should be construed that all changes or modifications derived based on the technical spirit of the various embodiments of the present invention are included in the scope of various embodiments of the present invention in addition to the embodiments of the present invention. .

Claims

In an electronic device,

A display for displaying content in a direction corresponding to the first surface of the electronic device;

A sensor for detecting light incident on the second surface of the electronic device; And

Includes a processor,

The processor,

An electronic device configured to determine brightness information of the periphery of the electronic device based at least on the sensed light and to adjust at least one property of the display or at least one property of the content based at least on the brightness information .
The apparatus of claim 1, further comprising another sensor for sensing light incident on the first surface.

And the processor is configured to perform the determining when another brightness information determined based on light detected by the other sensor falls within a specified range.
The apparatus of claim 1, further comprising another sensor for sensing light incident on the first surface.

And the processor is configured to perform the adjusting operation further based on another brightness information determined based on light detected through the other sensor.
The method of claim 1, wherein the sensor comprises an image sensor,

And an image signal processor including a first functional block and a second functional block for processing the light obtained from the sensor.

The processor is configured to select a function block related to the brightness information among the first function block and the second function block and perform the determining by using the selected function block.
The processor of claim 4, wherein the processor comprises:

The electronic device is configured not to use a function block that is not selected among the first function block and the second function block in the determining operation, or to turn off power applied to the second function block.
The method of claim 1, wherein the sensor comprises a plurality of pixels including a red pixel, a green pixel, or a blue pixel,

The processor is configured to perform the determining operation based at least on color information corresponding to a designated pixel of the plurality of pixels, wherein the one or more pixels having the lowest number among the plurality of pixels are assigned to the designated pixel. An electronic device set up to select.
The processor of claim 1, wherein the processor comprises:

And determine the brightness information based on at least one of color information corresponding to the light and time information when the sensor is exposed to the light.
The electronic device of claim 1, wherein at least one property of the display or at least one property of the content comprises brightness, saturation, white balance, color, or a combination thereof.
The electronic device of claim 1, wherein the electronic device further comprises a housing forming at least a portion of an external surface of the electronic device,

The sensor constitutes at least a portion of the housing and is located between the display and the housing.
The electronic device of claim 1, wherein the electronic device comprises:

An image preprocessing module between the sensor and the processor,

And turning off at least one of the plurality of functional blocks included in the image preprocessing module when the processor satisfies a preset condition of determining the brightness of the display.
In electronic devices,

Displaying content by a display provided on the first side of the electronic device;

Sensing light incident by a sensor provided on a second surface of the electronic device; And

Determining brightness information of the periphery of the electronic device based at least on the sensed light; And

Adjusting at least one attribute of the display or at least one attribute of the content based at least on the brightness information.
In claim 11,

If the other brightness information determined based on light detected through another sensor for detecting light incident on the first surface falls within a specified range, performing the determining operation.
In claim 11,

And performing the adjustment further based on another brightness information determined based on light sensed by another sensor for detecting light incident on the first surface.
The method of claim 11,

Determining whether a preset condition is satisfied to adjust the attribute of the display, and processing data received from the first sensor or the second sensor if the preset condition is satisfied.
The method of claim 14, wherein the preset condition is:

And determined based on at least one of display state information of the electronic device, information related to the movement of the electronic device, information on the surrounding environment of the electronic device, and information related to a cover attached to the electronic device.