KR20230128934A - Electronic device and method for adjusting brightness of display in electronic device - Google Patents

Abstract

An electronic device according to various embodiments includes a display, a camera module, a first sensor disposed on the rear side of the electronic device and operating based on light, and a second sensor disposed on the front side of the electronic device and operating based on light. A sensor module including a, and a processor including a first processor and a second processor, wherein the processor includes data received from the first sensor and data received from the second sensor when the camera module is in an inactive state. The brightness of the display may be adjusted based on data, and the brightness of the display may be adjusted based on data received from the second sensor when the camera module is in an active state.

Description

Electronic device and how to adjust the brightness of the display in the electronic device

Various embodiments relate to a method of adjusting brightness of a display in an electronic device.

Thanks to the remarkable development of information and communication technology and semiconductor technology, the supply and use of various electronic devices are rapidly increasing. In particular, recent electronic devices may carry and communicate, and may include one or more sensors to acquire various surrounding information. A sensor of an electronic device may acquire various types of information, and there may be various types of sensors according to information to be obtained.

Among the sensors of electronic devices, sensors that use light include a camera sensor, UV (ultra violet) sensor, iris sensor, spectral sensor, IR (infrared) (proximity/gesture) sensor, RGB sensor, illumination sensor, ambient light sensor, ALS sensor) and/or a flicker sensor.

In an electronic device, an illuminance sensor disposed on a front side of the electronic device may measure the illuminance of a surrounding light source and adjust the brightness of a display of the electronic device.

However, when the folding angle of the foldable electronic device is changed or the posture of the bar-type electronic device is changed, such as lying down or leaning, the field of view (FOV) of the illuminance sensor disposed on the front of the electronic device ), the ambient light intensity may not be measured.

In particular, as the angle between the ambient light sensor and the ambient light sensor disposed on the front side of the electronic device increases as the folding angle of the electronic device changes or the posture of the electronic device changes, the light source within the FOV (field of view) of the illuminance sensor increases. Since illuminance cannot be measured, a problem in that the brightness of the display can be adjusted to be dark regardless of ambient brightness may occur.

According to various embodiments, a method for controlling brightness of a display in an electronic device is provided.

An electronic device according to various embodiments includes a display, a camera module, a first sensor disposed on the rear side of the electronic device and operating based on light, and a second sensor disposed on the front side of the electronic device and operating based on light. A sensor module including a, and a processor including a first processor and a second processor, wherein the processor includes data received from the first sensor and data received from the second sensor when the camera module is in an inactive state. The brightness of the display may be adjusted based on data, and the brightness of the display may be adjusted based on data received from the second sensor when the camera module is in an active state.

A method for adjusting display brightness in an electronic device according to various embodiments of the present disclosure includes a first method that is disposed on a rear surface of the electronic device and operates based on light when a camera module of the electronic device is in an inactive state by a processor of the electronic device. Adjusting the brightness of the display of the electronic device based on the data received from the first sensor and the data received from the second sensor disposed on the front side of the electronic device and operating based on light, and by the processor , adjusting the brightness of the display based on the data received from the second sensor when the camera module is in an active state.

According to various embodiments of the present disclosure, even if an ambient light source is not detected within a field of view (FOV) of an illuminance sensor disposed on a front side of an electronic device according to a change in the folding angle or posture of the electronic device, low power consumption You can adjust the brightness of the display appropriately.

1 is a block diagram of an electronic device in a network environment according to various embodiments.
2 is a block diagram of a display device according to various embodiments.
3 is a block diagram of an electronic device according to various embodiments.
4 is a block diagram of a sensor included in an electronic device according to various embodiments.
5A to 5C are diagrams for explaining an operation of adjusting the brightness of a display according to a position change of an electronic device according to various embodiments of the present disclosure.
6 is a flowchart illustrating an operation of a first sensor of an electronic device according to various embodiments.
7A to 7B are flowcharts for describing an operation of a first sensor of an electronic device according to various embodiments.
8 is a flowchart illustrating an operation of adjusting brightness of a display of an electronic device according to various embodiments of the present disclosure.

1 is a block diagram of an electronic device 101 within a network environment 100, according to various embodiments. Referring to FIG. 1 , in a network environment 100, an electronic device 101 communicates with an electronic device 102 through a first network 198 (eg, a short-range wireless communication network) or through a second network 199. It may communicate with at least one of the electronic device 104 or the server 108 through (eg, a long-distance wireless communication network). According to one embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108 . According to an embodiment, the electronic device 101 includes a processor 120, a memory 130, an input module 150, an audio output module 155, a display module 160, an audio module 170, a sensor module ( 176), interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196 , or the antenna module 197 may be included. In some embodiments, in the electronic device 101, at least one of these components (eg, the connection terminal 178) may be omitted or one or more other components may be added. In some embodiments, some of these components (eg, sensor module 176, camera module 180, or antenna module 197) are integrated into a single component (eg, display module 160). It can be.

The processor 120, for example, executes software (eg, the program 140) to cause at least one other component (eg, hardware or software component) of the electronic device 101 connected to the processor 120. It can control and perform various data processing or calculations. According to one embodiment, as at least part of data processing or operation, the processor 120 transfers instructions or data received from other components (e.g., sensor module 176 or communication module 190) to volatile memory 132. , processing commands or data stored in the volatile memory 132 , and storing resultant data in the non-volatile memory 134 . According to one embodiment, the processor 120 may include a main processor 121 (eg, a central processing unit or an application processor) or a secondary processor 123 (eg, a graphic processing unit, a neural network processing unit ( NPU: neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, when the electronic device 101 includes the main processor 121 and the auxiliary processor 123, the auxiliary processor 123 may use less power than the main processor 121 or be set to be specialized for a designated function. can The secondary processor 123 may be implemented separately from or as part of the main processor 121 .

The secondary processor 123 may, for example, take the place of the main processor 121 while the main processor 121 is in an inactive (eg, sleep) state, or the main processor 121 is active (eg, running an application). ) state, together with the main processor 121, at least one of the components of the electronic device 101 (eg, the display module 160, the sensor module 176, or the communication module 190) It is possible to control at least some of the related functions or states. According to one embodiment, the auxiliary processor 123 (eg, image signal processor or communication processor) may be implemented as part of other functionally related components (eg, camera module 180 or communication module 190). there is. According to an embodiment, the auxiliary processor 123 (eg, a neural network processing device) may include a hardware structure specialized for processing an artificial intelligence model. AI models can be created through machine learning. Such learning may be performed, for example, in the electronic device 101 itself where the artificial intelligence model is performed, or may be performed through a separate server (eg, the server 108). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning or reinforcement learning, but in the above example Not limited. The artificial intelligence model may include a plurality of artificial neural network layers. Artificial neural networks include deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), It may be one of deep Q-networks or a combination of two or more of the foregoing, but is not limited to the foregoing examples. The artificial intelligence model may include, in addition or alternatively, software structures in addition to hardware structures.

The memory 130 may store various data used by at least one component (eg, the processor 120 or the sensor module 176) of the electronic device 101 . The data may include, for example, input data or output data for software (eg, program 140) and commands related thereto. The memory 130 may include volatile memory 132 or non-volatile memory 134 .

The program 140 may be stored as software in the memory 130 and may include, for example, an operating system 142 , middleware 144 , or an application 146 .

The input module 150 may receive a command or data to be used by a component (eg, the processor 120) of the electronic device 101 from the outside of the electronic device 101 (eg, a user). The input module 150 may include, for example, a microphone, a mouse, a keyboard, a key (eg, a button), or a digital pen (eg, a stylus pen).

The sound output module 155 may output sound signals to the outside of the electronic device 101 . The sound output module 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as multimedia playback or recording playback. A receiver may be used to receive an incoming call. According to one embodiment, the receiver may be implemented separately from the speaker or as part of it.

The display module 160 may visually provide information to the outside of the electronic device 101 (eg, a user). The display module 160 may include, for example, a display, a hologram device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module 160 may include a touch sensor set to detect a touch or a pressure sensor set to measure the intensity of force generated by the touch.

The audio module 170 may convert sound into an electrical signal or vice versa. According to one embodiment, the audio module 170 acquires sound through the input module 150, the sound output module 155, or an external electronic device connected directly or wirelessly to the electronic device 101 (eg: Sound may be output through the electronic device 102 (eg, a speaker or a headphone).

The sensor module 176 detects an operating state (eg, power or temperature) of the electronic device 101 or an external environmental state (eg, a user state), and generates an electrical signal or data value corresponding to the detected state. can do. According to one embodiment, the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a bio sensor, It may include a temperature sensor, humidity sensor, or light sensor.

The interface 177 may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device 101 to an external electronic device (eg, the electronic device 102). According to one embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 may be physically connected to an external electronic device (eg, the electronic device 102). According to one embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 may convert electrical signals into mechanical stimuli (eg, vibration or motion) or electrical stimuli that a user may perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 may capture still images and moving images. According to one embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 may manage power supplied to the electronic device 101 . According to one embodiment, the power management module 188 may be implemented as at least part of a power management integrated circuit (PMIC), for example.

The battery 189 may supply power to at least one component of the electronic device 101 . According to one embodiment, the battery 189 may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.

The communication module 190 is a direct (eg, wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (eg, the electronic device 102, the electronic device 104, or the server 108). Establishment and communication through the established communication channel may be supported. The communication module 190 may include one or more communication processors that operate independently of the processor 120 (eg, an application processor) and support direct (eg, wired) communication or wireless communication. According to one embodiment, the communication module 190 is a wireless communication module 192 (eg, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (eg, : a local area network (LAN) communication module or a power line communication module). Among these communication modules, a corresponding communication module is a first network 198 (eg, a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 199 (eg, a legacy communication module). It may communicate with the external electronic device 104 through a cellular network, a 5G network, a next-generation communication network, the Internet, or a telecommunications network such as a computer network (eg, a LAN or a WAN). These various types of communication modules may be integrated as one component (eg, a single chip) or implemented as a plurality of separate components (eg, multiple chips). The wireless communication module 192 uses subscriber information (eg, International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199. The electronic device 101 may be identified or authenticated.

The wireless communication module 192 may support a 5G network after a 4G network and a next-generation communication technology, for example, NR access technology (new radio access technology). NR access technologies include high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and access of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low latency (URLLC)). -latency communications)) can be supported. The wireless communication module 192 may support a high frequency band (eg, mmWave band) to achieve a high data rate, for example. The wireless communication module 192 uses various technologies for securing performance in a high frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), and full-dimensional multiplexing. Technologies such as input/output (FD-MIMO: full dimensional MIMO), array antenna, analog beam-forming, or large scale antenna may be supported. The wireless communication module 192 may support various requirements defined for the electronic device 101, an external electronic device (eg, the electronic device 104), or a network system (eg, the second network 199). According to one embodiment, the wireless communication module 192 is a peak data rate for eMBB realization (eg, 20 Gbps or more), a loss coverage for mMTC realization (eg, 164 dB or less), or a U-plane latency for URLLC realization (eg, Example: downlink (DL) and uplink (UL) each of 0.5 ms or less, or round trip 1 ms or less) may be supported.

The antenna module 197 may transmit or receive signals or power to the outside (eg, an external electronic device). According to one embodiment, the antenna module 197 may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (eg, PCB). According to one embodiment, the antenna module 197 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is selected from the plurality of antennas by the communication module 190, for example. can be chosen A signal or power may be transmitted or received between the communication module 190 and an external electronic device through the selected at least one antenna. According to some embodiments, other components (eg, a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module 197 in addition to the radiator.

According to various embodiments, the antenna module 197 may form a mmWave antenna module. According to one embodiment, the mmWave antenna module includes a printed circuit board, an RFIC disposed on or adjacent to a first surface (eg, a lower surface) of the printed circuit board and capable of supporting a designated high frequency band (eg, mmWave band); and a plurality of antennas (eg, array antennas) disposed on or adjacent to a second surface (eg, a top surface or a side surface) of the printed circuit board and capable of transmitting or receiving signals of the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (eg, a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and signal ( e.g. commands or data) can be exchanged with each other.

According to an embodiment, commands or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199 . Each of the external electronic devices 102 or 104 may be the same as or different from the electronic device 101 . According to an embodiment, all or part of operations executed in the electronic device 101 may be executed in one or more external electronic devices among the external electronic devices 102 , 104 , or 108 . For example, when the electronic device 101 needs to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device 101 instead of executing the function or service by itself. Alternatively or additionally, one or more external electronic devices may be requested to perform the function or at least part of the service. One or more external electronic devices receiving the request may execute at least a part of the requested function or service or an additional function or service related to the request, and deliver the execution result to the electronic device 101 . The electronic device 101 may provide the result as at least part of a response to the request as it is or additionally processed. To this end, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device 101 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 104 may include an internet of things (IoT) device. Server 108 may be an intelligent server using machine learning and/or neural networks. According to one embodiment, the external electronic device 104 or server 108 may be included in the second network 199 . The electronic device 101 may be applied to intelligent services (eg, smart home, smart city, smart car, or health care) based on 5G communication technology and IoT-related technology.

2 is a block diagram 200 of a display module 160, in accordance with various embodiments.

Referring to FIG. 2 , the display module 160 may include a display 210 and a display driver IC (DDI) 230 for controlling the display 210 . The DDI 230 may include an interface module 231 , a memory 233 (eg, a buffer memory), an image processing module 235 , or a mapping module 237 . The DDI 230 transmits, for example, image data or image information including image control signals corresponding to commands for controlling the image data to other components of the electronic device 101 through the interface module 231. can be received from For example, according to one embodiment, the image information is processed by the processor 120 (eg, the main processor 121 (eg, an application processor) or the auxiliary processor 123 (eg, an auxiliary processor 123 that operates independently of the function of the main processor 121). Example: It may be received from a graphic processing unit)). The DDI 230 may communicate with the touch circuit 250 or the sensor module 276 (eg, the sensor module 176 of FIG. 1 ) through the interface module 231 . Also, the DDI 230 may store at least a portion of the received image information in the memory 233, for example, in units of frames. The image processing module 235 pre-processes or post-processes at least a portion of the image data based on at least the characteristics of the image data or the characteristics of the display 210 (eg, resolution, brightness, or size adjustment). can be performed. The mapping module 237 may generate a voltage value or a current value corresponding to the image data pre-processed or post-processed through the image processing module 235 . According to one embodiment, the generation of the voltage value or current value depends on at least a property of the pixels of the display 210 (eg, an arrangement of pixels (RGB stripe or pentile structure) or a size of each sub-pixel). It can be done on some basis. At least some pixels of the display 210 are driven at least partially based on the voltage value or current value, so that visual information (eg, text, image, or icon) corresponding to the image data is displayed on the display 210. can be displayed through

According to one embodiment, the display module 160 may further include a touch circuit 250. The touch circuit 250 may include a touch sensor 251 and a touch sensor IC 253 for controlling the touch sensor 251 . The touch sensor IC 253 may control the touch sensor 251 to detect, for example, a touch input or a hovering input to a specific location of the display 210 . For example, the touch sensor IC 253 may detect a touch input or a hovering input by measuring a change in a signal (eg, voltage, light amount, resistance, or charge amount) for a specific position of the display 210 . The touch sensor IC 253 may provide information (eg, position, area, pressure, or time) related to the sensed touch input or hovering input to a processor (eg, the processor 120 of FIG. 1 ). According to an exemplary embodiment, at least a part of the touch circuit 250 (eg, the touch sensor IC 253) is disposed as a part of the display driver IC 230, the display 210, or outside the display module 160. It may be included as part of other components (eg, the auxiliary processor 123).

According to an embodiment, the display module 160 may further include at least one sensor (eg, a fingerprint sensor, an iris sensor, a pressure sensor, or an illumination sensor) of the sensor module 276 or a control circuit for the sensor module 276 . In this case, the at least one sensor or a control circuit thereof may be embedded in a part of the display module 160 (eg, the display 210 or the DDI 230) or a part of the touch circuit 250. For example, when the sensor module 276 embedded in the display module 160 includes a biometric sensor (eg, a fingerprint sensor), the biometric sensor is biometric information associated with a touch input through a partial area of the display 210. (e.g. fingerprint image) can be acquired. For another example, if the sensor module 276 embedded in the display module 160 includes a pressure sensor, the pressure sensor may obtain pressure information associated with a touch input through a part or the entire area of the display 210. can According to an embodiment, the touch sensor 251 or the sensor module 276 may be disposed between pixels of a pixel layer of the display 210 or above or below the pixel layer.

3 is a block diagram 300 of an electronic device according to various embodiments.

Referring to FIG. 3 , according to various embodiments, an electronic device 301 (eg, the electronic device 101 of FIG. 1 ) includes a first processor 320, a second processor 350, a sensor module ( 390), a camera module 380 and a display 360,

According to various embodiments, the electronic device 201 includes a processor (eg, the processor 120 of FIG. 1 ), and the processor includes a main processor (eg, the main processor 121 of FIG. 1 ) (eg, the central processor 120 of FIG. 1 ). A first processor 320 representing a processing unit or application processor) and an auxiliary processor (eg, the auxiliary processor 123 of FIG. 1 ) (eg, a sensor hub) operable independently of or together with the first processor 320 It may include a second processor 350 indicating.

According to various embodiments, the electronic device 201 includes a first processor 320 representing a main processor (eg, the main processor 121 of FIG. 1 ) (eg, a central processing unit or an application processor) and an auxiliary processor (eg, the main processor 121 of FIG. 1 ). Second processors 350 (eg, sensor hubs) representing the auxiliary processor 123 of FIG. 1 may be separately included.

According to various embodiments, when the camera module 380 is activated, the first processor 320 receives data (eg: Flicker noise (or band noise) to be generated in an image obtained from the camera module 380 may be removed based on raw data or light source frequency.

According to an embodiment, the first processor 320 activates the camera module 380 by executing a camera application, and the first block included in the first sensor 370 for flicker detection (eg: In order to activate b1) of FIG. 4 , an activation signal of the camera module may be transmitted to the first sensor 370 .

According to an embodiment, the first processor 320 detects the frequency of the light source having 100 Hz or 120 Hz based on the data received from the first sensor 370 and obtains it from the camera module 380 Flicker noise (or band noise) to be generated in the image can be removed.

According to various embodiments, the second processor 350 may include a micro controller unit (MCU) capable of calculating and processing data received from a plurality of sensors included in the sensor module 390 with low power consumption. there is.

According to an embodiment, the second processor 350 may process data received from a plurality of sensors that are always activated and operated with low power consumption.

According to various embodiments, the second processor 350, based on the activation or inactivation state of the camera module 380, the first sensor 370 that operates based on light among the sensors included in the sensor module 390 ) and data (eg, digital raw data) received from at least one of the second sensors 380 operating based on light, the brightness of the display 360 may be controlled.

According to an embodiment, the second processor 350, when the camera module 380 is in an inactive state, the data received from the first sensor 370 disposed on the back of the electronic device 301 (eg : digital raw data) and data (eg, digital raw data) received from the second sensor 380 disposed on the front of the electronic device 301 may be compared. If the data received from the first sensor 370 is greater than the data received from the second sensor 380 as a result of the comparison, the second processor 350 converts the data received from the first sensor 370 to an illumination level. value, and based on the calculated illuminance value, the brightness of the display 360 may be adjusted. The second processor 350, as a result of the comparison, converts the data received from the second sensor 380 into the If it is greater than the data received from the first sensor 370, the data received from the second sensor 380 may be calculated as an illuminance value, and the brightness of the display 360 may be adjusted based on the calculated illuminance value.

According to an embodiment, when the camera module 380 is in an active state, the second processor 350 calculates data received from the second sensor 380 disposed on the front of the electronic device as an illuminance value. and control the brightness of the display 360 based on the calculated illuminance value.

According to an embodiment, when the camera module 380 is in an inactive state, data (eg, digital raw data) received from the first sensor 370 disposed on the rear side of the electronic device 301 and the electronic device Data (e.g., digital raw data) received from the second sensor 380 disposed on the front of the 301 is compared, and based on the comparison result, the data received from the first sensor 370 is calculated. The operation of controlling the brightness of the display using the illuminance value or the illuminance value calculated with the data received from the second sensor 380 may also be performed by the first processor 320 in the same manner.

According to various embodiments, the second processor 350 may use at least one third sensor (eg, an angle sensor, an inclination sensor, or an acceleration sensor) capable of detecting a position of an electronic device among sensors included in the sensor module 390. When a folding state (eg, an open state, a closed state, or an intermediate open state) of the foldable electronic device is detected using at least one sensor among sensors, etc., a first sensor is detected based on the folding state of the foldable electronic device. A signal capable of activating the second block (eg, b2 of FIG. 4 ) included in 370 for detecting ambient light may be transmitted to the first sensor 370. According to an embodiment, the second processor 350 sets the folding angle of the foldable electronic device to a reference angle based on the folding state of the foldable electronic device using the at least one third sensor capable of detecting the folding state of the foldable electronic device. If the above is confirmed, a signal capable of activating the second block (eg, b2 in FIG. 4 ) for detecting ambient light of the first sensor 370 can be transmitted to the first sensor 370 .

According to an embodiment, the second processor includes a hinge connecting two housings on which a display is disposed using at least one third sensor (eg, at least one sensor among an angle sensor, a tilt sensor, and an acceleration sensor). A folding state of may be detected, and a folding angle of the foldable electronic device may be detected according to the folding state.

According to an embodiment, the reference angle determines whether the illuminance of the ambient light source can be measured within the field of view (FOV) of the second sensor 380 disposed on the front surface of the electronic device 301. possible angles can be indicated.

According to an embodiment, the reference position (eg, reference angle) may be determined according to the type of the second sensor 380 disposed on the front side of the electronic device 301 and the mounting structure of the second sensor 38. can

According to an embodiment, the second processor 350 determines the ratio of the data received from the second sensor 380 (eg, R value, G value, B value, C value converted to digital values) When the ratio is 50% or less, it may be determined that the ambient light intensity cannot be measured within a field of view (FOV) range of the second sensor 380 disposed on the front side of the electronic device.

According to an embodiment, the second processor 350, as shown in Table 1 below, according to a change in the folding angle of the electronic device or a change in posture of the electronic device, the second sensor disposed on the front of the electronic device ( 380) and the ambient light source when the angle is 40 degrees, the ratio of the data received from the second sensor 380 (eg R value, G value, B value, C value converted to digital values) is 50 If it is confirmed that the % or less is 49%, the reference position (eg, reference position or reference posture) may be set to “38 degrees” or “39 degrees”.

Angle lift Ratio downward Ratio 0 degree 3191 100% 3191 100% 10 degrees 3047 95% 3094 97% 20 degrees 2704 85% 2749 86% 30 degrees 2125 67% 2175 68% 35 degrees 1874 59% 1927 60% 40 degrees 1564 49% 1553 49% 45 degrees 1286 40% 1262 40% 50 degrees 1005 31% 947 30%

In <Table 1>, the intensity of the incoming light according to the change in the angle between the direction of the 300 lux light source and the direction of the sensor is shown. “Down” indicates the measured illuminance value when the electronic device is tilted downward while rotating while looking at the light source.

In <Table 1>, when the foldable type electronic device is completely unfolded and facing the ambient light source in front or the bar type electronic device is in the front facing the ambient light source, the front side of the electronic device An angle between the disposed second sensor 380 and the surrounding light source may be represented as 0 degrees.

도(예: 10도, 20도, 30도, 35도, 40도, 45도, 또는 50도) 인폴딩된 상태 또는 바 타입의 전자 장치

도(예: 10도, 20도, 30도, 35도, 40도, 45도, 또는 50도)로 자세를 기울인 상태일 때, 전자 장치의 전면부에 배치된 제2 센서(380)와 주변 광원 간의 각도를

도(예: 10도, 20도, 30도, 35도, 40도, 45도, 또는 50도)로 나타낼 수 있다.In Table 1, when the angle between the second sensor 380 disposed on the front side of the electronic device and the ambient light source is 0 degrees, a foldable type electronic device

degree (e.g. 10 degree, 20 degree, 30 degree, 35 degree, 40 degree, 45 degree, or 50 degree) in-fold or bar type electronics

When the posture is inclined at 10 degrees (eg, 10 degrees, 20 degrees, 30 degrees, 35 degrees, 40 degrees, 45 degrees, or 50 degrees), the second sensor 380 disposed on the front of the electronic device and the surroundings the angle between the light sources

It can be expressed in degrees (eg, 10 degrees, 20 degrees, 30 degrees, 35 degrees, 40 degrees, 45 degrees, or 50 degrees).

According to various embodiments, the sensor module 390 (eg, the sensor module 176 of FIG. 1 and/or the sensor module 276 of FIG. 2 ) includes a plurality of sensors, and an electronic device among the plurality of sensors A first sensor 370 disposed on the rear side of the electronic device 301 and operating based on light and a second sensor 380 disposed on the front side of the electronic device 301 and operating based on light may be included.

According to various embodiments, the first sensor 370 may generate data (eg, raw data) for removing flicker noise (or band noise) to be generated in an image obtained from the camera module 380 when the camera module 380 is in an inactive state. data or light source frequency, etc.) is transmitted to the first processor 320, and when the camera module 380 is in an inactive state, data for controlling the brightness of the display 360 (eg, digital raw data) is transmitted. It can be transmitted to the second processor 350.

According to an embodiment, the first sensor 370 may include a flicker sensor.

According to an embodiment, when the first sensor 370 receives an activation signal of the camera module from the first processor 320, the first block for flicker detection included in the first sensor 370 ( Example: b1 of FIG. 4) may be activated, and the second block (eg, b2 of FIG. 4) for ambient light detection included in the first sensor 370 may be deactivated. The first sensor 370 is activated by the first block (eg: Data acquired by the first sensor 379 may be transmitted to the first processor 320 through b1) of FIG. 4 .

According to an embodiment, in the first sensor 370, a second block (eg, b2 in FIG. 4 ) for detecting ambient light included in the first sensor 370 is activated, and the first sensor 370 is activated. While the first block (eg, b1 in FIG. 4 ) for flicker detection included in 370 is deactivated, the first sensor 370 detects the activated second block (eg, b2 in FIG. 4 ). Data obtained from the first sensor 370 may be transmitted to the second processor 350 as illuminance data for adjusting the brightness of the display.

According to an embodiment, in the first sensor 370, a second block (eg, b2 in FIG. 4 ) for detecting ambient light included in the first sensor 370 is activated, and the first sensor 370 is activated. If an activation signal of the camera module is received from the first processor 320 while the first block for flicker detection included in 370 (eg, b1 in FIG. 4 ) is inactive, the first sensor 370 ) is activated, and the second block (eg, b2 in FIG. 4) included in the first sensor 370 for detecting ambient light is activated. can be deactivated. Data obtained from the first sensor 379 may be transmitted to the first processor 320 through the activated first block (eg, b1 of FIG. 4 ).

According to various embodiments, the first sensor 370 may activate a second block (eg, b2 in FIG. 4 ) included in the first sensor 370 for detecting ambient light from the second processor 350 . Upon receiving a signal having a presence, the second block (eg, b2 in FIG. 4) may be activated.

According to an embodiment, in the first sensor 370, the second block (eg, b2 in FIG. 4 ) for ambient light detection included in the first sensor 370 is inactivated, and the first sensor 370 is inactivated. While the first block for flicker detection included in 370 (eg, b1 in FIG. 4 ) is inactivated, the second processor 350 generates a block for detecting ambient light included in the first sensor 370. When a signal capable of activating block 2 (eg, b2 in FIG. 4 ) is received and an activation signal of the camera module is not received from the first processor 320, flicker detection included in the first sensor 370 A first block (for example, b1 of FIG. 4 ) for , may be inactivated, and a second block (for example, b2 of FIG. 4 ) included in the first sensor 370 for detecting ambient light may be activated. The first sensor 370 converts data acquired from the first sensor 370 through the activated second block (eg, b2 of FIG. 4 ) into illuminance data for adjusting brightness of a display to the second processor. (350).

According to an embodiment, in the first sensor 370, the second block (eg, b2 in FIG. 4 ) for ambient light detection included in the first sensor 370 is inactivated, and the first sensor 370 is inactivated. While the first block for flicker detection included in 370 (eg, b1 in FIG. 4 ) is inactivated, the second processor 350 generates a block for detecting ambient light included in the first sensor 370. When a signal capable of activating the second block (eg, b2 in FIG. 4) is received and an activation signal of the camera module is received from the first processor 320, flicker detection included in the first sensor 370 is performed. A first block (eg, b1 in FIG. 4 ) for detecting ambient light included in the first sensor 370 may be deactivated (eg, b2 in FIG. 4 ). The first sensor 370 is activated by the first block (eg, in FIG. Data acquired by the first sensor 379 may be transmitted to the first processor 320 through b1).

According to various embodiments, the second sensor 390 may transmit data for adjusting the brightness of the display 360 (eg, digital raw data) to the second processor 350 .

According to an embodiment, the second sensor 390 may include an illumination sensor.

According to various embodiments, the sensor module 390 may be an angle sensor, an inclination sensor, or an angle sensor capable of detecting a change in the position of the electronic device (eg, a folding angle or/and a posture (or mounting) of the electronic device). at least one of an acceleration sensor, etc.).

According to various embodiments, the display 360 may be implemented substantially the same as or similar to the display module 160 of FIG. 1 .

According to an embodiment, the brightness of the display 360 may be adjusted based on data received from the first sensor 370 and the second sensor 390 .

According to various embodiments, the camera module 380 may be implemented substantially the same as or similar to the camera module 180 of FIG. 1 or/and the camera module 280 of FIG. 2 .

4 is a block diagram 400 of a sensor included in an electronic device according to various embodiments.

Referring to FIG. 4, the first sensor 370 disposed on the rear side of the electronic device (eg, the electronic device 101 of FIG. 1 or/and the electronic device 201 of FIG. 2) includes a light receiving unit 371, It may include an analog to digital converter (ADC) 373, a first block b1 for detecting flicker, and a second block b2 for detecting ambient light.

According to various embodiments, the first sensor 370 may be disposed adjacent to a rear camera module disposed on a rear surface of an electronic device.

According to various embodiments, the light receiving unit 371 may include a 4 X 4 array of 16 photodiodes, and the light incident through the light receiving unit 371 may have an R (red) light amount, a G (green) light amount, An analog electrical signal corresponding to each of the B (blue) light amount or C (clear) light amount may be transmitted to the ADC 373 . The R (red) light amount, the G (green) light amount, the B (blue) light amount, or the C (clear) light amount may be the intensity of light received through the R channel, the G channel, the B channel, or the C channel, respectively.

According to an embodiment, the light receiving unit 371 may include a plurality of channels, for example, an R channel, a G channel, a B channel, and a C channel, or two channels capable of receiving visible light and IR bands ( Example: C channel and IR channel). Channels may include a plurality of light receiving elements capable of receiving light. For example, each of the R channel, G channel, B channel, and C channel may include a plurality of (eg, four) photodiodes capable of receiving external light.

According to various embodiments, the ADC 373 generates an analog electrical signal corresponding to each of the R (red) light amount, G (green) light amount, B (blue) light amount, and C (clear) light amount received from the light receiving unit 371 . can be converted to a digital value.

According to various embodiments, the first block b1 for detecting flicker includes a buffer 375 capable of storing digital values of some sections among digital values (eg, digital raw data) converted through the ADC 373 ( Example: A first communication interface 377 capable of transmitting a first-in first-out (FIFO) buffer and digital values (eg, digital raw data) stored in the buffer 375 to the first processor 320 can include

According to an embodiment, the buffer 375 may include at least one storage length greater than or equal to a flicker period. According to an embodiment, the first processor 320 may store at least a portion of data stored in the buffer The flicker frequency can be calculated using

According to an embodiment, when an activation signal of the camera module is received from the first processor 320, the first block b1 for flicker detection may be activated.

According to an embodiment, when the activation signal of the camera module is not received from the first processor 320, the first block b1 for flicker detection may be deactivated.

According to an embodiment, when the first block b1 for flicker detection is activated by receiving the activation signal of the camera module from the first processor 320, the buffer 375 (eg, FIFO (first-in first-in) out) A digital value of a part of the digital values (e.g., data) received from the ADC 373 for a short period designated for detecting the frequency of the light source having 100 Hz or 120 Hz is stored in the buffer), and the data stored in the buffer may be transmitted to the first processor 320 in order to remove flicker noise (or band noise) to be generated in the image obtained from the camera module.

According to an embodiment, an activation signal of the camera module may be received from the first processor 320 through the first communication interface 377 .

According to an embodiment, data stored in the buffer 375 may be transmitted to the first processor 320 through the first communication interface 377 .

According to an embodiment, the first communication interface 377 uses a digital communication method such as inter-integrated circuit (I2C), serial peripheral interconnect (SPI), or/and mobile industry processor interface (MIPI) to be used by the first processor. Data and/or signals may be transmitted and received between 320 and the first block b1.

According to various embodiments, the second block b2 for detecting ambient light may include a second communication interface 379 .

According to an embodiment, when an activation signal of the camera module is received from the first processor 320, the second block b2 for detecting ambient light may be deactivated.

According to an embodiment, when an activation signal of the camera module is not received from the first processor 320, the second block b2 for detecting ambient light may be activated.

According to an embodiment, when the second block b2 for detecting ambient light is activated because an activation signal of the camera module is not received from the first processor 320, the second block b2 for detecting ambient light is activated through the second communication interface 379. A digital value (eg, data) received from the ADC 373 may be transmitted to the second processor 350 .

According to an embodiment, the second communication interface 379 uses a digital communication method such as inter-integrated circuit (I2C), serial peripheral interconnect (SPI), or/and mobile industry processor interface (MIPI) to communicate with the second communication interface. Data and/or signals may be transmitted and received between the processor 350 and the second block b2.

Referring to FIG. 4, the second sensor 380 disposed on the front of the electronic device (eg, the electronic device 101 of FIG. 1 or/and the electronic device 201 of FIG. 2) is a light receiving unit 381 , an ADC 383, and a third communication interface unit 385.

According to various embodiments, the second sensor 380 may be disposed at the bottom of one area of the display so as to overlap with one area of the display on the front of the electronic device.

According to various embodiments, the light receiving unit 381 is composed of 16 photodiode arrays of 4 X 4, and the light incident through the light receiving unit 381 is R (red) light quantity, G (green) light quantity, B An analog electrical signal corresponding to the amount of (blue) light or the amount of C (clear) light may be transmitted to the ADC 373 . The R (red) light amount, the G (green) light amount, the B (blue) light amount, or the C (clear) light amount may be the intensity of light received through the R channel, the G channel, the B channel, or the C channel, respectively.

According to an embodiment, the light receiving unit 381 may include a plurality of channels, for example, an R channel, a G channel, a B channel, and a C channel, and may include two channels capable of receiving visible light and IR bands ( Example: C channel and IR channel). Channels may include a plurality of light receiving elements capable of receiving light. For example, each of the R channel, G channel, B channel, and C channel may include a plurality of (eg, four) photodiodes capable of receiving external light.

According to various embodiments, the ADC 383 receives an analog electrical signal corresponding to each of the R (red) light amount, G (green) light amount, B (blue) light amount, or C (clear) light amount received from the light receiving unit 371 can be converted to a digital value.

According to various embodiments, digital data received from the ADC 383 through the third communication interface 377? A value (eg, data) may be transmitted to the second processor 350 .

According to an embodiment, the third communication interface 377 uses a digital communication method such as inter-integrated circuit (I2C), serial peripheral interconnect (SPI), or/and mobile industry processor interface (MIPI) to communicate with the second communication interface. Data and/or signals may be transmitted and received between the processor 320 and the second sensor 380 .

Data for removing flicker noise (or band noise) to be generated in the image acquired from the camera module by the first sensor 370 is transmitted to the first processor 320, processed by the first processor 320, and displayed As the data for brightness control is transmitted to the second processor 350 (e.g., sensor hub) capable of processing with low power and processed by the second processor 350, the electronic device consumes little current, and the first sensor 370 The brightness of the display 360 may be adjusted based on data received from the first sensor 380 and the second sensor 380 .

5A to 5C are diagrams 500a to 500c for explaining an operation of adjusting brightness of a display according to a position change of an electronic device according to various embodiments of the present disclosure.

Referring to FIG. 5A, the first block (eg, b1 in FIG. 4 ) included in the first sensor 370 (eg, the first sensor 370 of FIGS. 3 to 4 ) for flicker detection is deactivated. , while the second block (eg, b2 of FIG. 4 ) for detecting ambient light included in the first sensor 370 is activated, the electronic device 301 (eg, the electronic device 101 of FIG. 1 and / or the angle between the second sensor 380 (eg, the second sensor 380 of FIGS. 3 and 4 ) disposed on the front side of the electronic device 301 of FIG. 3 and the ambient light source 530 is 0 degree one When (when the illuminance of the ambient light source is measured within the FOV of the second sensor 380), the second processor (eg, the second processor 350 of FIGS. 3 and 4), for example, the first When data (eg, 50 Lux) is received from the sensor 370 and data (eg, 500 Lux) is received from the second sensor 380, data (eg, 50 Lux) received from the first sensor 370 is higher than The brightness of the display may be adjusted based on data (eg, 500 Lux) received from the second sensor 380 .

Referring to FIG. 5B, the first block (eg, b1 in FIG. 4 ) included in the first sensor 370 (eg, the first sensor 370 of FIGS. 3 and 4 ) for flicker detection is deactivated. , while the second block (eg, b2 of FIG. 4 ) for detecting ambient light included in the first sensor 370 is activated, the electronic device 301 (eg, the electronic device 101 of FIG. 1 and / or the angle between the second sensor 380 disposed on the front side of the electronic device 301 of FIG. 3 (eg, the second sensor 380 of FIGS. 3 and 4) and the ambient light source 530 is 90 degrees. When (when the illuminance of the ambient light source is not measured within the FOV of the second sensor 380), the second processor (eg, the second processor 350 of FIGS. 3 and 4), for example, the first When data (eg, 300 Lux) is received from the sensor 370 and data (eg, 200 Lux) is received from the second sensor 380, the data (eg, 200 Lux) received from the second sensor 380 is larger than the The brightness of the display may be adjusted based on data (eg, 300 Lux) received from the first sensor 370 .

Referring to FIG. 5C , the first block (eg, b1 in FIG. 4 ) included in the first sensor 370 (eg, the first sensor 370 of FIGS. 3 to 4 ) for flicker detection is deactivated. , while the second block (eg, b2 of FIG. 4 ) for detecting ambient light included in the first sensor 370 is activated, the electronic device 301 (eg, the electronic device 101 of FIG. 1 and / or an angle of 135 degrees between the second sensor 380 (eg, the second sensor 380 of FIGS. 3 and 4 ) disposed on the front side of the electronic device 301 of FIG. 3 and the ambient light source 530 When (when ambient light is not measured within the FOV of the second sensor 380), the second processor (eg, the second processor 350 of FIGS. 3 and 4), for example, the first sensor ( 370) and receive data (eg, 100 Lux) from the second sensor 380, the first sensor greater than the data (eg, 100 Lux) received from the second sensor 380. The brightness of the display may be adjusted based on data (eg, 500 Lux) received from the sensor 370 .

According to various embodiments, an electronic device (eg, the electronic device 301 of FIG. 3 ) may include a display (eg, the display 360 of FIG. 3 ), a camera module (eg, the camera module 380 ), and the electronic device A first sensor (e.g., the first sensor 370 of FIG. 4) disposed on the rear side of the electronic device and operating based on light (eg, the first sensor 370 of FIG. 4) and a second sensor (e.g., FIG. A sensor module (eg, the sensor module 390 of FIG. 4) including the second sensor 380 of 4, and a first processor (eg, the first processor 320 of FIG. 4 and the second processor (eg, the sensor module 390 of FIG. 4). and a processor including the second processor 350 of FIG. 3 ), wherein the processor, based on the data received from the first sensor and the data received from the second sensor when the camera module is in an inactive state, The brightness of the display may be adjusted, and the brightness of the display may be adjusted based on data received from the second sensor when the camera module is in an active state.

According to various embodiments, the first processor may represent a main processor, and the second processor may include an auxiliary processor including a sensor hub.

According to various embodiments, the first sensor may include a light receiving unit (eg, the light receiving unit 371 of FIG. 4 ), an ADC (eg, the ADC 373 of FIG. 4 ), and data of a partial section among data received from the ADC. A first communication interface (eg, a first-in first-out (FIFO) buffer) for transmitting the stored buffer (eg, the buffer 375 of FIG. 4 and the first-in first-out (FIFO) buffer) to the first processor (eg, the first-in-first-out (FIFO) buffer) A first block (eg, b1 in FIG. 4) for flicker detection including a first interface 377) and a second communication interface (eg, b1 in FIG. 4) for transmitting data received from the ADC to the second processor (eg, FIG. A second block (eg, b2 in FIG. 4 ) for detecting ambient light including the second communication interface 379 of 4) may be included.

According to various embodiments, when an activation signal of the camera module is not received from the first processor, the first block (eg, b1 of FIG. 4 ) of the first sensor is deactivated, and the second block of the first sensor A block (eg, b2 in FIG. 4 ) may be activated to transmit data received from the ADC to the second processor through the second communication interface.

According to various embodiments, when an activation signal of the camera module is received from the first processor, the first block (eg, b1 of FIG. 4 ) of the first sensor is activated, and the second block of the first sensor (eg, b2 in FIG. 4 ) may be deactivated and set to transmit data stored in the buffer to the first processor through the first communication interface.

According to various embodiments of the present disclosure, the second sensor comprises a light receiving unit (eg, the light receiving unit 381 of FIG. 4 ), an ADC (eg, the ADC 383 of FIG. 4 ), and the received information from the ADC. A third communication interface (eg, the third communication interface 385 of FIG. 4 ) for transmitting data to the second processor may be included.

According to various embodiments, the second processor compares data received from the first sensor with data received from the second sensor when the camera module is in an inactive state, and the data received from the first sensor If it is determined that the brightness of the display is greater than the data received from the second sensor, the brightness of the display may be adjusted using the data received from the first sensor.

According to various embodiments of the present disclosure, when the second processor determines that the data received from the second sensor is greater than the data received from the first sensor, the second processor adjusts the brightness of the display using the data received from the second sensor. can be set to adjust.

According to various embodiments of the present disclosure, the second processor detects the folding state of the electronic device, and if the folding angle of the electronic device is determined to be greater than or equal to the reference angle based on the folding state of the electronic device, the second processor includes the electronic device in the first sensor. A signal capable of activating a second block (eg, b2 in FIG. 4 ) for detecting ambient light may be transmitted to the first sensor.

6 is a flowchart 600 for describing an operation of a first sensor of an electronic device according to various embodiments. Operations of the first sensor may include operations 601 to 617. According to an embodiment, at least one of operations 601 to 617 may be omitted, the order of some operations may be changed, or another operation may be added.

In operation 601, the first block (eg, b1 of FIG. 4 ) included in the first sensor 370 (eg, the first sensor 370 of FIGS. 3 to 5 ) for flicker detection is deactivated, and A second block (eg, b2 of FIG. 4 ) included in the sensor 370 (eg, the first sensor 370 of FIGS. 3 to 5 ) for ambient light detection may be activated.

According to an embodiment, a first block (eg, b1 in FIG. 4 ) for flicker detection included in the first sensor is deactivated, and a second block (eg, b2 in FIG. 4 ) for detecting ambient light is activated. It can be.

In operation 603, the first sensor 370 (eg, the first sensor 370 of FIGS. 3 to 5 ) detects the ambient light through a second block (eg, b2 of FIG. 4 ). Sensor data (eg, digital raw data) may be transmitted to a second processor (eg, the second processor 350 of FIGS. 3 and 4 ).

According to an embodiment, according to the activation of the second block (eg, b2 of FIG. 4), digital values (eg, digital Raw data) may be transmitted to the second processor 350 through the second communication interface 379 of the first sensor to adjust the brightness of the display.

In operation 605, the second processor 350 (eg, the second processor 350 of FIGS. 3 to 4) transmits the data of the first sensor 370 and the second sensor (eg, the second processor 350 of FIGS. 3 to 4). The brightness of the display can be adjusted based on data from the sensor).

In operation 607, the first processor 320 (eg, the first processor 320 of FIGS. 3 to 4 ) may check activation of the camera module.

According to an embodiment, the first processor may activate the camera module 380 by executing a camera application.

In operation 609, the first processor 320 (eg, the first processor 320 of FIGS. 3 to 4) sends the first sensor 370 (eg, the first sensor 370 of FIGS. 3 to 5). An activation signal of the camera module may be transmitted.

According to an embodiment, the first processor 320 transmits an activation signal of a camera module capable of requesting data for removing flicker noise (or band noise) to be generated from an image acquired from the camera module to the first sensor (370).

In operation 611, the first block (eg, b1 of FIG. 4 ) included in the first sensor 370 (eg, the first sensor 370 of FIGS. 3 to 5 ) for flicker detection is activated, and A second block (eg, b2 of FIG. 4 ) included in the sensor 370 (eg, the first sensor 370 of FIGS. 3 to 5 ) for detecting ambient light may be deactivated.

According to an embodiment, based on the activation signal of the camera module received from the first processor 320, a first block (eg, b1 in FIG. 4 ) for flicker detection included in the first sensor is activated, , the second block for ambient light detection (eg, b2 in FIG. 4 ) may be inactivated.

In operation 613, the first sensor 370 (eg, the first sensor 370 of FIGS. 3 to 5 ) through the first block (eg, b1 of FIG. 4 ) included in the flicker detection Data (eg, raw data or light source frequency) may be transmitted to the first processor (eg, the first processor 320 of FIGS. 3 and 4 ).

According to an embodiment, upon activation of the first block (eg, b1 of FIG. 4 ), a buffer included in the first block (eg, b1 of FIG. 4 ) (eg, the buffer 375 of FIG. 4 , A digital value of a part of the digital values (eg, digital raw data) converted through the ADC (eg, ADC 373 of FIG. 3) of the first sensor is stored in a first-in first-out (FIFO) buffer) and transmit the data stored in the buffer to the first processor 320.

In operation 615, the first processor 320 (eg, the first processor 320 of FIGS. 3 to 4 ), the first sensor 370 (eg, the first sensor 370 of FIGS. 3 to 5 ) Flicker noise (or band noise) to be generated in an image obtained from a camera module (eg, the camera module 380 of FIG. 3 ) may be removed based on the data received from .

According to an embodiment, the first processor 320 detects the frequency of the light source having 100 Hz or 120 Hz based on the data received from the first sensor 370, and generates the frequency in the image obtained from the camera module. Flicker noise (or band noise) can be removed.

In operation 617, if an activation signal of the camera module is not received from the first processor 320 (eg, the first processor 320 of FIGS. 3 to 4), the first sensor 370 (eg, the first processor 320 of FIGS. 3 to 4) The first block (eg, b1 in FIG. 4 ) for flicker detection included in the first sensor 370 in FIG. 5 is deactivated, and the second block (eg, b2 in FIG. 4 ) for detecting ambient light is activated. and the data (eg, digital raw data) of the first sensor is transferred to a second processor (eg, the second processor of FIGS. 3 to 4 ) through the second block (eg, b2 of FIG. 4 ) of the first sensor. Operations 601 to 603 that can be transmitted to (350)) can be performed.

In operation 617, when an activation signal of the camera module is received from one processor 320 (eg, the first processor 320 of FIGS. 3 to 4), the first sensor 370 (eg, the first processor 320 of FIGS. 3 to 5) The first block (eg, b1 in FIG. 4 ) for flicker detection included in the first sensor 370 of the is activated, and the second block (eg, b2 in FIG. 4 ) for ambient light detection is deactivated, Data (eg, raw data or light source frequency) of the first sensor is transferred to a first processor (eg, FIGS. 3 to 4 ) through the first block (eg, b1 of FIG. 4 ) of the first sensor. Operation 611 to operation 613 that can be transmitted to the first processor 320) may be performed.

7A to 7B are flowcharts 700a to 700b for describing an operation of a first sensor of an electronic device according to various embodiments. Operations of the first sensor may include operations 701 to 723. According to an embodiment, at least one of operations 701 to 723 may be omitted, the order of some operations may be changed, or another operation may be added.

In operation 701, the first block (eg, b1 of FIG. 4 ) included in the first sensor 370 (eg, the first sensor 370 of FIGS. 3 to 5 ) for flicker detection is deactivated, and A second block (eg, b2 of FIG. 4 ) included in the sensor 370 (eg, the first sensor 370 of FIGS. 3 to 5 ) for detecting ambient light may be deactivated.

According to an embodiment, a first block (eg, b1 in FIG. 4 ) for flicker detection included in the first sensor is disabled, and a second block (eg, b2 in FIG. 4 ) for detecting ambient light is disabled. It can be.

In operation 703, the second processor 350 (eg, the second processor 350 of FIGS. 3 to 4) selects an electronic device among sensors included in a sensor module (eg, the sensor module 390 of FIG. 3). The folding state of the electronic device may be detected using at least one sensor capable of detecting the position of the electronic device (eg, at least one third sensor among an angle sensor, a tilt sensor, and an acceleration sensor).

According to an embodiment, the second processor 350 may determine that the position value of the electronic device is equal to or greater than the reference position value by using the at least one sensor capable of detecting the position of the electronic device.

According to one embodiment, the second processor 350 may determine that the folding angle of the electronic device is greater than or equal to the reference angle by using the at least one sensor capable of detecting the folding state of the electronic device.

In operation 705, the second processor 350 (eg, the second processor 350 of FIGS. 3 to 4) performs a second block (eg, the second processor 350 of FIG. 4 ) for detecting ambient light of the first sensor 370. A signal capable of activating b2) may be transmitted to the first sensor 370 .

In operation 707, the first block (eg, b1 of FIG. 4 ) included in the first sensor 370 (eg, the first sensor 370 of FIGS. 3 to 5 ) for flicker detection is deactivated, and A second block (eg, b2 of FIG. 4 ) included in the sensor 370 (eg, the first sensor 370 of FIGS. 3 to 5 ) for ambient light detection may be activated.

According to an embodiment, the first sensor 370 may activate the second block (eg, b2 of FIG. 4 ) for detecting ambient light received from the second processor 350 to detect the ambient light. A second block (eg, b2 in FIG. 4 ) for ambient light detection may be activated.

In operation 709, the first sensor 370 (eg, the first sensor 370 of FIGS. 3 to 5 ) detects the ambient light through a second block (eg, b2 of FIG. 4 ). Sensor data (eg, digital raw data) may be transmitted to a second processor (eg, the second processor 350 of FIGS. 3 and 4 ).

According to an embodiment, according to the activation of the second block (eg, b2 of FIG. 4), digital values (eg, digital Raw data) may be transmitted to the second processor 350 through the second communication interface 379 of the first sensor to adjust the brightness of the display.

In operation 711, the second processor 350 (eg, the second processor 350 of FIGS. 3 to 4) transmits the data of the first sensor 370 and the second sensor (eg, the second processor 350 of FIGS. 3 to 4). The brightness of the display can be adjusted based on data from the sensor).

In operation 713, the first processor 320 (eg, the first processor 320 of FIGS. 3 to 4 ) may check activation of the camera module.

According to an embodiment, the first processor may activate the camera module 380 by executing a camera application.

In operation 715, the first processor 320 (eg, the first processor 320 of FIGS. 3 to 4) sends the first sensor 370 (eg, the first sensor 370 of FIGS. 3 to 5). An activation signal of the camera module may be transmitted.

According to an embodiment, the first processor 320 transmits an activation signal of a camera module capable of requesting data for removing flicker noise (or band noise) to be generated from an image acquired from the camera module to the first sensor (370).

In operation 717, a first block (eg, b1 in FIG. 4 ) included in the first sensor 370 (eg, the first sensor 370 of FIGS. 3 to 5 ) for flicker detection is activated, and A second block (eg, b2 of FIG. 4 ) included in the sensor 370 (eg, the first sensor 370 of FIGS. 3 to 5 ) for detecting ambient light may be deactivated.

According to an embodiment, based on the activation signal of the camera module received from the first processor 320, a first block (eg, b1 in FIG. 4 ) for flicker detection included in the first sensor is activated, , the second block for ambient light detection (eg, b2 in FIG. 4 ) may be inactivated.

In operation 719, the first sensor 370 (eg, the first sensor 370 of FIGS. 3 to 5 ) through the first block (eg, b1 of FIG. 4 ) included in the flicker detection Data (eg, raw data or light source frequency) may be transmitted to the first processor (eg, the first processor 320 of FIGS. 3 and 4 ).

According to an embodiment, upon activation of the first block (eg, b1 of FIG. 4 ), a buffer included in the first block (eg, b1 of FIG. 4 ) (eg, the buffer 375 of FIG. 4 , A digital value of a part of the digital values (eg, digital raw data) converted through the ADC (eg, ADC 373 of FIG. 3) of the first sensor is stored in a first-in first-out (FIFO) buffer) and transmit the data stored in the buffer to the first processor 320.

In operation 721, the first processor 320 (eg, the first processor 320 of FIGS. 3 to 4 ) sends the first sensor 370 (eg, the first sensor 370 of FIGS. 3 to 5 ). Flicker noise (or band noise) to be generated in an image obtained from a camera module (eg, the camera module 380 of FIG. 3 ) may be removed based on the data received from .

According to an embodiment, the first processor 320 detects the frequency of the light source having 100 Hz or 120 Hz based on the data received from the first sensor 370, and generates the frequency in the image obtained from the camera module. Flicker noise (or band noise) can be removed.

In operation 723, if an activation signal of the camera module is not received from the first processor 320 (eg, the first processor 320 of FIGS. 3 to 4), the first sensor 370 (eg, the first processor 320 of FIGS. 3 to 4) The first block (eg, b1 in FIG. 4 ) for flicker detection included in the first sensor 370 in FIG. 5 is deactivated, and the second block (eg, b2 in FIG. 4 ) for detecting ambient light is activated. and the data (eg, digital raw data) of the first sensor is transferred to a second processor (eg, the second processor of FIGS. 3 to 4 ) through the second block (eg, b2 of FIG. 4 ) of the first sensor. Operation 707 to operation 709 that can be transmitted to (350)) can be performed.

In operation 723, when an activation signal of the camera module is received from one processor 320 (eg, the first processor 320 of FIGS. 3 to 4), the first sensor 370 (eg, the first processor 320 of FIGS. 3 to 5) The first block (eg, b1 in FIG. 4 ) for flicker detection included in the first sensor 370 of the is activated, and the second block (eg, b2 in FIG. 4 ) for ambient light detection is deactivated, Data (eg, raw data or light source frequency) of the first sensor is transferred to a first processor (eg, FIGS. 3 to 4 ) through the first block (eg, b1 of FIG. 4 ) of the first sensor. Operation 717 to operation 719 that can be transmitted to the first processor 320) may be performed.

8 is a flowchart 800 for explaining a brightness control operation of a display of an electronic device according to various embodiments. Operations for adjusting the brightness of the display may include operations 801 to 809. According to an embodiment, at least one of operations 801 to 809 may be omitted, the order of some operations may be changed, or another operation may be added. The brightness control operation of the display may be performed by the electronic device 101 of FIG. 1 , the processor 120 of FIG. 1 , the electronic device 301 of FIG. 3 , or the processor of FIG. 3 (eg, the first processor 320 of FIG. 3 ). or/and the second processor 350).

In operation 801, the electronic device 301 (eg, the second processor 350 of FIGS. 3 to 4) receives information from the first sensor 370 (eg, the first sensor 370 of FIGS. 3 to 5). You can check whether the data has been received.

In operation 801, if the electronic device 301 (eg, the second processor 350 of FIGS. 3 and 4) fails to confirm reception of data from the first sensor, in operation 809, the second sensor (eg, the second processor 350) : The brightness of the display may be adjusted based on data (eg, an illuminance value) received from the second sensor 380 of FIGS. 3 and 4 .

In operation 801, when the electronic device 301 (eg, the second processor 350 of FIGS. 3 and 4) confirms reception of data from the first sensor, in operation 803, the electronic device 301 receives data from the first sensor. The received data and the data received from the second sensor (the second sensor 380 of FIGS. 3 and 4 ) may be compared.

According to an embodiment, the electronic device 301 may include the first sensor (eg, the first processor 320 of FIGS. 3 to 4 ) receiving a camera activation signal from a first processor (eg, the first processor 320 of FIGS. 3 to 4 ). In the first sensor 370 of FIG. 4 , the first block (eg, b1 of FIG. 4 ) for flicker detection included in the first sensor is deactivated, and the block for detecting ambient light included in the first sensor As the second block (eg, b2 of FIG. 4 ) is activated, data usable as illuminance data may be received through the second block (eg, b2 of FIG. 4 ) included in the first sensor.

According to an embodiment, the electronic device 301 may receive data usable as illuminance data from a second sensor (eg, the second sensor 380 of FIGS. 3 and 4 ).

In operation 805, the electronic device 301 (eg, the second processor 350 of FIGS. 3 to 4) receives information from the first sensor 370 (eg, the first sensor 370 of FIGS. 3 to 5). If it is determined that the data (eg, illuminance value) is greater than the data (eg, illuminance value) received from the second sensor (eg, the second sensor 380 of FIGS. 3 and 4 ), in operation 807, the first sensor You can adjust the brightness of the display based on data from (e.g., illuminance value).

In operation 805, the electronic device 301 (eg, the second processor 350 of FIGS. 3 and 4 ) receives data from a second sensor (eg, the second sensor 380 of FIGS. 3 and 4 ). If it is determined that (eg, the illuminance value) is greater than data (eg, the illuminance value) from the first sensor 370 (eg, the first sensor 370 of FIGS. 3 to 5 ), in operation 809, the second The brightness of the display can be adjusted based on data received from the sensor (e.g., illuminance value).

According to various embodiments of the present disclosure, a method of controlling display brightness in an electronic device may include, by a processor of the electronic device, when a camera module of the electronic device is in an inactive state, disposed on a rear surface of the electronic device and operated based on light. Adjusting the brightness of the display of the electronic device based on the data received from the first sensor and the data received from the second sensor disposed on the front side of the electronic device and operating based on light, and to the processor and adjusting the brightness of the display based on the data received from the second sensor when the camera module is in an active state.

According to various embodiments, the processor may include a first processor and a second processor, the first processor may represent a main processor, and the second processor may include an auxiliary processor including a sensor hub.

According to various? h embodiments, in the first sensor that does not receive the activation signal of the camera module from the first processor of the processor, the first block for flicker detection included in the first sensor (eg, in FIG. 4 When b1) is deactivated and a second block (eg, b2 in FIG. 4 ) for detecting ambient light included in the first sensor is activated, transmitting data of the first sensor to a second processor of the processor. may further include.

According to various embodiments, according to the activation of the second block (eg, b2 of FIG. 4) included in the first sensor, through the second communication interface of the second block (eg, b2 of FIG. 4) , transmitting the data received from the ADC of the first sensor to the second processor of the processor.

According to various embodiments, in the first sensor receiving the activation signal of the camera module from the first processor of the processor, the first block for flicker detection included in the first sensor (eg, b1 of FIG. 4) ) is activated and the second block (eg, b2 in FIG. 4 ) for ambient light detection included in the first sensor is deactivated, an operation of transmitting the data of the first sensor to the first processor of the processor is performed. can include more.

According to various embodiments, the first block (eg, b1 of FIG. 4 ) stored in the buffer of the first block (eg, b1 of FIG. 4 ) is activated according to the activation of the first block (eg, b1 of FIG. 4 ) included in the first sensor. The method may further include transmitting data of a partial section among sensor data to the first processor of the processor through the first communication interface of the first block (eg, b1 of FIG. 4 ).

According to various embodiments, the operation of comparing the data received from the first sensor with the data received from the second sensor when the camera module is in an inactive state, and the data received from the first sensor If it is determined that the brightness is higher than the data received from the second sensor, an operation of adjusting the brightness of the display using the data received from the first sensor may be further included.

According to various embodiments, when it is determined that the data received from the second sensor is greater than the data received from the first sensor, an operation of adjusting the brightness of the display using the data received from the second sensor is performed. can include more.

According to various embodiments, when the operation of detecting the folding state of the electronic device and the folding angle of the electronic device are determined to be equal to or greater than the reference angle based on the folding state of the electronic device, the ambient light of the first sensor is detected. An operation of transmitting a signal capable of activating a second block for detection (eg, b2 in FIG. 4 ) to the first sensor may be further included.

According to various? h embodiments, a signal capable of activating the second block (eg, b2 of FIG. 4 ) of the first sensor is received from a second processor of the processor, and the camera module is activated from the first processor In the first sensor that does not receive a signal, the first block (eg, b1 in FIG. 4) for flicker detection included in the first sensor is deactivated, and the first block for detecting ambient light included in the first sensor When block 2 (eg, b2 in FIG. 4 ) is activated, the data received from the ADC of the first sensor through the second communication interface of the second block (eg, b2 in FIG. 4 ) is transmitted to the second processor of the processor. An operation of transmitting to may be further included.

Electronic devices according to various embodiments disclosed in this document may be devices of various types. The electronic device may include, for example, a portable communication device (eg, a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance. An electronic device according to an embodiment of the present document is not limited to the aforementioned devices.

Various embodiments of this document and terms used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, like reference numerals may be used for like or related elements. The singular form of a noun corresponding to an item may include one item or a plurality of items, unless the relevant context clearly dictates otherwise. In this document, "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "A Each of the phrases such as "at least one of , B, or C" may include any one of the items listed together in that phrase, or all possible combinations thereof. Terms such as "first", "second", or "first" or "secondary" may simply be used to distinguish a given component from other corresponding components, and may be used to refer to a given component in another aspect (eg, importance or order) is not limited. A (e.g., first) component is said to be “coupled” or “connected” to another (e.g., second) component, with or without the terms “functionally” or “communicatively.” When mentioned, it means that the certain component may be connected to the other component directly (eg by wire), wirelessly, or through a third component.

The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and is interchangeable with terms such as, for example, logic, logical blocks, parts, or circuits. can be used as A module may be an integrally constructed component or a minimal unit of components or a portion thereof that performs one or more functions. For example, according to one embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of this document are a storage medium (eg, internal memory 136 or external memory 138) readable by a machine (eg, the electronic device 101 or the electronic device 301). It may be implemented as software (eg, program 140) including one or more instructions stored in ). For example, a processor (eg, the processor 520) of a device (eg, the electronic device 301) may call at least one command among one or more instructions stored from a storage medium and execute it. This enables the device to be operated to perform at least one function according to the at least one command invoked. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' only means that the storage medium is a tangible device and does not contain a signal (e.g. electromagnetic wave), and this term refers to the case where data is stored semi-permanently in the storage medium. It does not discriminate when it is temporarily stored.

According to one embodiment, the method according to various embodiments disclosed in this document may be included and provided in a computer program product. Computer program products may be traded between sellers and buyers as commodities. A computer program product is distributed in the form of a device-readable storage medium (eg compact disc read only memory (CD-ROM)), or through an application store (eg Play Store ^TM ) or on two user devices ( It can be distributed (eg downloaded or uploaded) online, directly between smart phones. In the case of online distribution, at least part of the computer program product may be temporarily stored or temporarily created in a device-readable storage medium such as a manufacturer's server, an application store server, or a relay server's memory.

According to various embodiments, each component (eg, module or program) of the above-described components may include a single object or a plurality of entities, and some of the plurality of entities may be separately disposed in other components. there is. According to various embodiments, one or more components or operations among the aforementioned corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg modules or programs) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by a corresponding component of the plurality of components prior to the integration. . According to various embodiments, the actions performed by a module, program, or other component are executed sequentially, in parallel, iteratively, or heuristically, or one or more of the actions are executed in a different order, or omitted. or one or more other actions may be added.

Claims (20)

In electronic devices,
display;
camera module;
a sensor module including a first sensor disposed on a rear surface of the electronic device and operating based on light and a second sensor disposed on a front surface of the electronic device and operating based on light; and
A processor comprising a first processor and a second processor;
The processor controls the brightness of the display based on data received from the first sensor and data received from the second sensor when the camera module is in an inactive state,
An electronic device configured to adjust the brightness of the display based on data received from the second sensor when the camera module is in an active state.
According to claim 1,
the first processor represents a main processor;
The second processor includes a co-processor including a sensor hub.
The method of claim 1, wherein the first sensor,
light receiving unit;
ADC;
a first block for flicker detection including a buffer storing data of a partial section among data received from the ADC and a first communication interface for transmitting the data stored in the buffer to the first processor; and
and a second block for ambient light detection including a second communication interface for transmitting data received from the ADC to the second processor.
According to claim 3,
When the activation signal of the camera module is not received from the first processor, the first block of the first sensor is deactivated, and the second block of the first sensor is activated, and the ADC through the second communication interface An electronic device configured to transmit data received from the second processor.
According to claim 3,
When an activation signal of the camera module is received from the first processor, the first block of the first sensor is activated, the second block of the first sensor is deactivated, and stored in the buffer through the first communication interface. An electronic device configured to transmit stored data to the first processor.
The method of claim 1, wherein the second sensor,
light receiving unit;
ADC; and
and a third communication interface for transmitting data received from the ADC to the second processor.
The method of claim 1, wherein the second processor,
Comparing data received from the first sensor with data received from the second sensor when the camera module is in an inactive state;
The electronic device configured to adjust the brightness of the display using the data received from the first sensor when it is determined that the data received from the first sensor is greater than the data received from the second sensor.
The method of claim 7, wherein the second processor,
The electronic device set to adjust the brightness of the display using the data received from the second sensor when it is determined that the data received from the second sensor is greater than the data received from the first sensor.
The method of claim 1, wherein the second processor,
detecting the folding state of the electronic device;
When the folding angle of the electronic device is determined to be greater than or equal to the reference angle based on the folding state of the electronic device, a signal capable of activating a second block included in the first sensor for detecting ambient light is transmitted to the first sensor. An electronic device set up to transmit.
The method of claim 9, wherein the first sensor,
When receiving a signal capable of activating the second block of the first sensor from the second processor and not receiving an activation signal of the camera module from the first processor, for detecting flicker included in the first sensor The electronic device configured to transmit data of the first sensor to the second processor by inactivating the first block and activating the second block included in the first sensor.
A method for controlling display brightness in an electronic device,
When the camera module of the electronic device is in an inactive state by a processor of the electronic device, data received from a first sensor that is disposed on the rear side of the electronic device and operates based on light and is disposed on the front side of the electronic device, adjusting the brightness of the display of the electronic device based on data received from a second sensor that operates based on light; and
and controlling, by the processor, brightness of the display based on data received from the second sensor when the camera module is in an active state.
According to claim 11,
The processor includes a first processor and a second processor,
the first processor represents a main processor;
wherein the second processor comprises a co-processor comprising a sensor hub.
According to claim 11,
In the first sensor that has not received the activation signal of the camera module from the first processor of the processor, a first block for flicker detection included in the first sensor is deactivated, and ambient light detection included in the first sensor and transmitting the data of the first sensor to a second processor of the processor when the second block for is activated.
According to claim 13,
An operation of transmitting data received from the ADC of the first sensor to the second processor of the processor through a second communication interface of the second block according to activation of the second block included in the first sensor; How to include more.
According to claim 11,
In the first sensor that receives the activation signal of the camera module from the first processor of the processor, a first block for flicker detection included in the first sensor is activated, and ambient light detection included in the first sensor is activated. and transmitting the data of the first sensor to a first processor of the processor when the second block for the first sensor is inactivated.
According to claim 15,
Upon activation of the first block included in the first sensor, data of a partial section among the data of the first sensor stored in the buffer of the first block is transmitted through the first communication interface of the first block to the processor. The method further comprising transmitting to the first processor.
According to claim 11,
comparing data received from the first sensor with data received from the second sensor when the camera module is in an inactive state; and
and adjusting the brightness of the display using the data received from the first sensor when it is determined that the data received from the first sensor is greater than the data received from the second sensor.
According to claim 17,
and adjusting the brightness of the display using the data received from the second sensor when it is confirmed that the data received from the second sensor is greater than the data received from the first sensor.
According to claim 11,
detecting a folding state of the electronic device; and
Transmitting a signal capable of activating a second block for detecting ambient light of the first sensor to the first sensor when the folding angle of the electronic device is determined to be greater than or equal to the reference angle based on the folding state of the electronic device How to include more action.
According to claim 19,
In the first sensor that receives a signal capable of activating the second block of the first sensor from the second processor of the processor and does not receive the activation signal of the camera module from the first processor, to the first sensor When the included 1st block for flicker detection is inactivated and the 2nd block for ambient light detection included in the 1st sensor is activated, from the ADC of the 1st sensor through the 2nd communication interface of the 2nd block The method further comprising transmitting the received data to a second processor of the processor.

Images