WO2024101879A1

WO2024101879A1 - Electronic device, and method of controlling image frame synchronization in electronic device

Info

Publication number: WO2024101879A1
Application number: PCT/KR2023/017846
Authority: WO
Inventors: 김승진; 김정배; 김민우; 박준영; 조광래
Original assignee: 삼성전자 주식회사
Priority date: 2022-11-08
Filing date: 2023-11-08
Publication date: 2024-05-16

Abstract

According to various embodiments, an electronic device comprises: a display; a memory; and at least one processor, wherein the at least one processor may be configured to: as a first image frame is updated in a graphics buffer area of the memory on the basis of a first synchronizing signal, control the display to await reception of the first image frame during a designated standby time from a time point when a second synchronizing signal is received; if the first image frame is received within the designated standby time, control the received first image frame to be displayed; and as a second image frame is not updated in the graphics buffer area on the basis of the second synchronizing signal, control the display to continuously display the first image frame without a standby time, at a time point when a third synchronizing signal is received, and other embodiments can be possible.

Description

Electronic devices and methods for controlling image frame synchronization in electronic devices

Various embodiments relate to image frame synchronization control in an electronic device.

As electronic, information, and communication technologies develop, various functions are being integrated into a single portable communication device or electronic device. For example, an electronic device (eg, a smart phone, a wearable electronic device, or other portable electronic device) can perform a variety of functions through the installation and execution of various applications in addition to communication functions. Additionally, the electronic device may include a display capable of displaying information related to the execution of various functions. For example, the display may include various display devices such as a liquid crystal display (LCD), an organic light emitting diode display (OLED display), or an electrophoretic display.

When executing an application, the electronic device may perform rendering on image information (or display data) using at least one processor and display the rendered image frame through a display. Electronic devices may require synchronization between rendering image frames and displaying image frames.

For example, the electronic device may perform a vertical synchronization (eg, vsync (vertical synchronization)) function that transmits image frames rendered by at least one processor to the display according to the screen refresh rate of the display. When performing the vsync function, the electronic device matches the starting point of scanning one field on the transmitting side (e.g., at least one processor) that transmits the rendered image frame and the receiving side (e.g., display) that receives the rendered image frame. For this purpose, a vertical synchronizing signal (e.g. vsync signal) can be used.

When using the vsync function, the electronic device uses at least one processor to transmit rendered image frames to the display every specified time unit (e.g., vsync time unit) based on the vsync signal, and the display sends the received rendered image frames to the display using vsync signals. It can be played (or output) at specified time units based on the signal. If at least one processor fails to complete rendering within a specified time unit while using the vsync function, the electronic device may repeatedly play the previous image frame displayed on the display in the first time period in the second time period. In this way, when an electronic device reproduces one image frame in two time periods, the image frame may be displayed unnaturally.

Electronic devices can use the adaptive sync function to prevent one image frame from being played in two vsync time sections. For example, when using the adaptive sync function, the electronic device controls the display to wait for an image frame to be received for a certain waiting time (e.g. 2ms), and if an image frame is received within the waiting time, it plays the received image frame. Even if an image frame is not received immediately upon receiving a vsync signal (e.g., the starting point of a specified time period), delayedly received image frames can be displayed after a slight delay.

When using the adaptive sync function, the electronic device includes waiting time in each designated time section (e.g., vsync time section), so the time length of each vsync time section may be increased compared to when the adaptive sync function is not used. For example, if the frame rate is 60hz, the time length of the vsync section may be 16.6ms, and if the waiting time for an image frame due to the adaptive sync function is 3.4ms, the time length of the vsync section may increase to a maximum of 20ms. You can.

The adaptive sync feature can be useful to compensate for delays in receiving image frames from a display, but the display may have to wait up to the maximum latency to receive image frames even when there are no image frames to be received. For example, even if image frames are not rendered continuously and there are no rendered image frames in between, the display may have unnecessary waiting time in the vsync time section, which unnecessarily increases the time in the vsync section. The next image frame may not be played quickly.

According to one embodiment, when synchronizing an image frame between at least one processor and a display in an electronic device, if there is no image frame rendered by at least one processor, a waiting time for the image frame may not be applied in the corresponding synchronization time section. You can.

An electronic device according to one embodiment may include a display, memory, and at least one processor. The at least one processor according to an embodiment is configured to update the first image frame in the graphics buffer area of the memory based on the first synchronization signal, so that the display displays the first image frame during a designated waiting time from the point of receiving the second synchronization signal. Control can be performed to wait for image frame reception, and when the first image frame is received within the designated waiting time, control can be performed to display the received first image frame. According to one embodiment, the at least one processor is configured to display the first image frame without waiting time at the point of receiving the third synchronization signal as the second image frame is not updated in the graphic buffer area based on the second synchronization signal. You can control the continuous display of image frames.

A method of controlling image frame synchronization in an electronic device according to an embodiment includes updating the first image frame in the graphic buffer area of the memory of the electronic device based on the first synchronization signal, thereby causing the display to change to a specified state from the time of receiving the second synchronization signal. It may include controlling the display to wait for reception of the first image frame during the waiting time, and controlling to display the received first image frame when the first image frame is received within the designated waiting time. The method according to one embodiment is such that the second image frame is not updated in the graphic buffer area based on the second synchronization signal, so that the display displays the first image frame without waiting time at the time of receiving the third synchronization signal. It may include an operation to control continuous display.

A non-volatile storage medium storing instructions, wherein the instructions are set to cause the electronic device to perform at least one operation when executed by the electronic device, wherein the at least one operation is based on a first synchronization signal. As the first image frame is updated in the graphics buffer area of the memory of the electronic device, the display is controlled to wait for reception of the first image frame for a designated waiting time from the point of receiving the second synchronization signal, and the display is controlled to wait for reception of the first image frame within the designated waiting time. An operation of controlling to display a received first image frame when one image frame is received, and when the second image frame is not updated in the graphic buffer area based on the second synchronization signal, the display displays a third synchronization signal It may include a control operation to continuously display the first image frame without waiting time at the point of reception.

According to one embodiment, the electronic device identifies whether to render an image frame and/or update an image frame by referring to the graphics buffer status and/or the operation status information of at least one processor, and displays and/or displays based on the identification result. By controlling the DPU's adaptive sync function to be activated (e.g. on) or deactivated (e.g. off), unnecessary frame waiting time in the no frame section can be reduced. This can prevent the next new frame update time from being delayed.

1 is a block diagram of an electronic device in a network environment according to various embodiments.

Figure 2 is a block diagram of a display device according to various embodiments.

Figure 3 is a configuration diagram of an electronic device according to an embodiment.

FIG. 4 is a diagram for explaining the operation of a CPU in an electronic device according to an embodiment.

Figure 5 is a flowchart showing an image frame synchronization control operation in an electronic device according to an embodiment.

FIG. 6 is a flowchart illustrating a display operation based on whether an image frame is updated in a graphics buffer area in an electronic device according to an embodiment.

FIG. 7 is a diagram illustrating image frames when adaptive sync is applied in an electronic device according to an embodiment.

FIG. 8 is a diagram illustrating a case in which an image frame is not updated in the graphics buffer area while adaptive sync is applied in an electronic device according to an embodiment.

FIG. 9 is a diagram illustrating not applying adaptive sync when an image frame is not updated in the graphics buffer area while applying adaptive sync in an electronic device according to an embodiment.

FIG. 10 is a diagram illustrating not applying adaptive sync when the next section is no frame while the vsync section of the display is increased based on the first frame delay in an electronic device according to an embodiment.

In relation to the description of the drawings, identical or similar reference numerals may be used for identical or similar components.

Terms used in this document are merely used to describe specific embodiments and may not be intended to limit the scope of other embodiments. Singular expressions may include plural expressions, unless the context clearly indicates otherwise. All terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by a person of ordinary skill in the technical field of the present invention. Terms defined in commonly used dictionaries can be interpreted as having the same or similar meaning as the meaning they have in the context of the related technology, and unless clearly defined in this document, they are not interpreted in an ideal or excessively formal sense. . In some cases, even terms defined in this document cannot be interpreted to exclude embodiments of the present invention.

FIG. 1 is a block diagram of an electronic device 101 in a network environment 100 according to various embodiments.

Referring to FIG. 1, in the network environment 100, the electronic device 101 communicates with the electronic device 102 through a first network 198 (e.g., a short-range wireless communication network) or a second network 199. It is possible to communicate with the electronic device 104 or the server 108 through (e.g., a long-distance wireless communication network). According to one embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108. According to one embodiment, the electronic device 101 includes a processor 120, a memory 130, an input module 150, an audio output module 155, a display module 160, an audio module 170, and a sensor module ( 176), interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196 , or may include an antenna module 197. In some embodiments, at least one of these components (eg, the connection terminal 178) may be omitted or one or more other components may be added to the electronic device 101. In some embodiments, some of these components (e.g., sensor module 176, camera module 180, or antenna module 197) are integrated into one component (e.g., display module 160). It can be.

The processor 120, for example, executes software (e.g., program 140) to operate at least one other component (e.g., hardware or software component) of the electronic device 101 connected to the processor 120. It can be controlled and various data processing or calculations can be performed. According to one embodiment, as at least part of data processing or computation, the processor 120 stores commands or data received from another component (e.g., sensor module 176 or communication module 190) in volatile memory 132. The commands or data stored in the volatile memory 132 can be processed, and the resulting data can be stored in the non-volatile memory 134. According to one embodiment, the processor 120 includes a main processor 121 (e.g., a central processing unit or an application processor) or an auxiliary processor 123 that can operate independently or together (e.g., a graphics processing unit, a neural network processing unit ( It may include a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device 101 includes a main processor 121 and a secondary processor 123, the secondary processor 123 may be set to use lower power than the main processor 121 or be specialized for a designated function. You can. The auxiliary processor 123 may be implemented separately from the main processor 121 or as part of it.

The auxiliary processor 123 may, for example, act on behalf of the main processor 121 while the main processor 121 is in an inactive (e.g., sleep) state, or while the main processor 121 is in an active (e.g., application execution) state. ), together with the main processor 121, at least one of the components of the electronic device 101 (e.g., the display module 160, the sensor module 176, or the communication module 190) At least some of the functions or states related to can be controlled. According to one embodiment, co-processor 123 (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module 180 or communication module 190). there is. According to one embodiment, the auxiliary processor 123 (eg, neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. Artificial intelligence models can be created through machine learning. For example, such learning may be performed in the electronic device 101 itself, where artificial intelligence is performed, or may be performed through a separate server (e.g., server 108). Learning algorithms may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but It is not limited. An artificial intelligence model may include multiple artificial neural network layers. Artificial neural networks include deep neural network (DNN), convolutional neural network (CNN), recurrent neural network (RNN), restricted boltzmann machine (RBM), belief deep network (DBN), bidirectional recurrent deep neural network (BRDNN), It may be one of deep Q-networks or a combination of two or more of the above, but is not limited to the examples described above. In addition to hardware structures, artificial intelligence models may additionally or alternatively include software structures.

The memory 130 may store various data used by at least one component (eg, the processor 120 or the sensor module 176) of the electronic device 101. Data may include, for example, input data or output data for software (e.g., program 140) and instructions related thereto. Memory 130 may include volatile memory 132 or non-volatile memory 134.

The program 140 may be stored as software in the memory 130 and may include, for example, an operating system 142, middleware 144, or application 146.

The input module 150 may receive commands or data to be used in a component of the electronic device 101 (e.g., the processor 120) from outside the electronic device 101 (e.g., a user). The input module 150 may include, for example, a microphone, mouse, keyboard, keys (eg, buttons), or digital pen (eg, stylus pen).

The sound output module 155 may output sound signals to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker or a receiver. Speakers can be used for general purposes such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part of it.

The display module 160 can visually provide information to the outside of the electronic device 101 (eg, a user). The display module 160 may include, for example, a display, a hologram device, or a projector, and a control circuit for controlling the device. According to one embodiment, the display module 160 may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of force generated by the touch.

The audio module 170 can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module 170 acquires sound through the input module 150, the sound output module 155, or an external electronic device (e.g., directly or wirelessly connected to the electronic device 101). Sound may be output through the electronic device 102 (e.g., speaker or headphone).

The sensor module 176 detects the operating state (e.g., power or temperature) of the electronic device 101 or the external environmental state (e.g., user state) and generates an electrical signal or data value corresponding to the detected state. can do. According to one embodiment, the sensor module 176 includes, for example, a gesture sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, humidity sensor, or light sensor.

The interface 177 may support one or more designated protocols that can be used to connect the electronic device 101 directly or wirelessly with an external electronic device (eg, the electronic device 102). According to one embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102). According to one embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 can convert electrical signals into mechanical stimulation (e.g., vibration or movement) or electrical stimulation that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 can capture still images and moving images. According to one embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 can manage power supplied to the electronic device 101. According to one embodiment, the power management module 188 may be implemented as at least a part of, for example, a power management integrated circuit (PMIC).

The battery 189 may supply power to at least one component of the electronic device 101. According to one embodiment, the battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

Communication module 190 is configured to provide a direct (e.g., wired) communication channel or wireless communication channel between electronic device 101 and an external electronic device (e.g., electronic device 102, electronic device 104, or server 108). It can support establishment and communication through established communication channels. Communication module 190 operates independently of processor 120 (e.g., an application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module 190 is a wireless communication module 192 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (e.g., : LAN (local area network) communication module, or power line communication module) may be included. Among these communication modules, the corresponding communication module is a first network 198 (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 199 (e.g., legacy It may communicate with an external electronic device 104 through a telecommunication network such as a cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., LAN or WAN). These various types of communication modules may be integrated into one component (e.g., a single chip) or may be implemented as a plurality of separate components (e.g., multiple chips). The wireless communication module 192 uses subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199. The electronic device 101 can be confirmed or authenticated.

The wireless communication module 192 may support 5G networks after 4G networks and next-generation communication technologies, for example, NR access technology (new radio access technology). NR access technology provides high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and access to multiple terminals (massive machine type communications (mMTC)), or ultra-reliable and low-latency (URLLC). -latency communications)) can be supported. The wireless communication module 192 may support high frequency bands (eg, mmWave bands), for example, to achieve high data rates. The wireless communication module 192 uses various technologies to secure performance in high frequency bands, for example, beamforming, massive array multiple-input and multiple-output (MIMO), and full-dimensional multiplexing. It can support technologies such as input/output (FD-MIMO: full dimensional MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module 192 may support various requirements specified in the electronic device 101, an external electronic device (e.g., electronic device 104), or a network system (e.g., second network 199). According to one embodiment, the wireless communication module 192 supports Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mmTC, or U-plane latency (e.g., 164 dB or less) for realizing URLLC. Example: Downlink (DL) and uplink (UL) each of 0.5 ms or less, or round trip 1 ms or less) can be supported.

The antenna module 197 may transmit or receive signals or power to or from the outside (eg, an external electronic device). According to one embodiment, the antenna module 197 may include an antenna including a radiator made of a conductor or a conductive pattern formed on a substrate (eg, PCB). According to one embodiment, the antenna module 197 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is connected to the plurality of antennas by, for example, the communication module 190. can be selected. Signals or power may be transmitted or received between the communication module 190 and an external electronic device through the at least one selected antenna. According to some embodiments, in addition to the radiator, other components (eg, radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module 197.

According to various embodiments, the antenna module 197 may form a mmWave antenna module. According to one embodiment, a mmWave antenna module includes a printed circuit board, an RFIC disposed on or adjacent to a first side (e.g., bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., mmWave band); And a plurality of antennas (e.g., array antennas) disposed on or adjacent to the second side (e.g., top or side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (e.g., bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and signal ( (e.g. commands or data) can be exchanged with each other.

According to one embodiment, commands or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199. Each of the external

electronic devices

102 or 104 may be of the same or different type as the electronic device 101. According to one embodiment, all or part of the operations performed in the electronic device 101 may be executed in one or more of the external

electronic devices

102, 104, or 108. For example, when the electronic device 101 must perform a certain function or service automatically or in response to a request from a user or another device, the electronic device 101 may perform the function or service instead of executing the function or service on its own. Alternatively, or additionally, one or more external electronic devices may be requested to perform at least part of the function or service. One or more external electronic devices that have received the request may execute at least part of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device 101. The electronic device 101 may process the result as is or additionally and provide it as at least part of a response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology can be used. The electronic device 101 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 104 may include an Internet of Things (IoT) device. Server 108 may be an intelligent server using machine learning and/or neural networks. According to one embodiment, the external electronic device 104 or server 108 may be included in the second network 199. The electronic device 101 may be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

FIG. 2 is a block diagram 200 of the display device 160 according to various embodiments.

Referring to FIG. 2 , the display device 160 may include a display 210 and a display driver integrated circuit (DDI) (or display driving circuit) 230 for controlling the display 210 . The DDI 230 may include an interface module 231, a memory 233 (eg, buffer memory), an image processing module 235, or a mapping module 237. For example, the DDI 230 transmits image information, including image data or an image control signal corresponding to a command for controlling the image data, to an electronic device (e.g., the electronic device of FIG. 1) through the interface module 231. may receive from other components of the device 101). For example, according to one embodiment, the image information is stored in the processor 120 (e.g., the main processor 121 (e.g., an application processor) or the auxiliary processor 123 ( For example, the DDI 230 may communicate with the touch circuit 250 or the sensor module 176 through the interface module 231. According to one embodiment, at least some of the received image information may be stored in the memory 233, for example, on a frame basis, and the memory 233 may include a register (not shown). , the register may store setting values for performing functions of the electronic device 101. For example, the image processing module 235 may store at least a portion of the image data or the display 210. The mapping module 237 may perform pre-processing or post-processing (e.g., resolution, brightness, or size adjustment) based on at least the characteristics of the image data pre-processed or post-processed through the image processing module 235. According to one embodiment, the generation of the voltage value or current value may be generated by, for example, an attribute of the pixels of the display 210 (e.g., an array of pixels (RGB stripe or At least some pixels of the display 210 may be driven, for example, based at least in part on the voltage value or the current value. Visual information (eg, text, image, or icon) corresponding to the image data may be displayed through the display 210.

According to one embodiment, the display device 160 may further include a touch circuit 250. The touch circuit 250 may include a touch sensor 251 and a touch sensor IC 253 for controlling the touch sensor 251. The touch sensor IC (or touch sensor circuit) 253 may control the touch sensor 251 to, for example, detect a touch input or hovering input for a specific position on the display 210. For example, the touch sensor IC 253 may detect a touch input or a hovering input by measuring a change in a signal (eg, voltage, light amount, resistance, or charge amount) for a specific position of the display 210. The touch sensor IC 253 may provide information (e.g., location, area, pressure, or time) about the detected touch input or hovering input to the processor 120. According to one embodiment, at least a portion of the touch circuit 250 (e.g., touch sensor IC 253) is disposed as part of the display driver IC 230, the display 210, or outside the display device 160. It may be included as part of other components (e.g., auxiliary processor 123).

According to one embodiment, the display device 160 may further include at least one sensor (eg, a fingerprint sensor, an iris sensor, a pressure sensor, or an illumination sensor) of the sensor module 176, or a control circuit therefor. In this case, the at least one sensor or a control circuit therefor may be embedded in a part of the display device 160 (eg, the display 210 or the DDI 230) or a part of the touch circuit 250. For example, when the sensor module 176 embedded in the display device 160 includes a biometric sensor (e.g., a fingerprint sensor), the biometric sensor transmits biometric information associated with a touch input through a portion of the display 210. (e.g. fingerprint image) can be obtained. For another example, when the sensor module 176 embedded in the display device 160 includes a pressure sensor, the pressure sensor may acquire pressure information associated with a touch input through part or the entire area of the display 210. You can. According to one embodiment, the touch sensor 251 or the sensor module 176 may be disposed between pixels of a pixel layer of the display 210, or above or below the pixel layer. According to one embodiment, the sensor module 176 may further include a pen sensor (eg, a pen sensor circuit, a pen sensor IC, or a digitizer controller).

Referring to FIG. 3, the electronic device 301 according to one embodiment may include at least one processor 320, a memory 330, and a display 360, and may further include other components. .

At least one processor 320 according to an embodiment renders image information and renders the rendered image frame in a specified time unit (or vertical synchronization signal (e.g., vsync signal (vertical synchronization signal))) based on a specified synchronization signal (or vertical synchronization signal (vertical synchronization signal)). Example: vsync time unit) can be transmitted to the display 360. For example, the at least one processor 320 stores the first image frame of the first time period rendered based on the first vsync signal in the graphics buffer area of the memory 330 (or a separate buffer memory (not shown)). The first image frame stored in the graphic buffer area can be controlled to be transmitted to the display 360 based on the second vsync signal following the first vsync signal. The display 360 according to one embodiment may output (or play) the first image frame transmitted in the second time period based on the second vsync signal.

At least one processor 320 according to an embodiment applies adaptive sync (e.g., activates the adaptive sync function) or does not apply adaptive sync (e.g., activates the adaptive sync function) when the display 360 receives an image frame from the at least one processor 320 and outputs it. : deactivating the adaptive sync function), at least one processor 320 according to an embodiment may control whether an image frame is updated in the graphics buffer area (e.g., whether a newly rendered image frame is stored in the graphics buffer area). According to one embodiment, the adaptive sync function of the display 360 may be controlled to be activated or deactivated based on whether an image frame to be transmitted from the graphic buffer area to the display 360 exists. According to one embodiment, when an image frame is updated in the graphic buffer area, at least one processor 320 may be controlled to activate the adaptive sync function of the display 360. According to one embodiment, the adaptive sync function of the display 360 may be controlled to be disabled if the adaptive sync function is activated so that the next image frame is not updated in the graphic buffer area. In this case, you can control the adaptive sync function to be disabled.

According to one embodiment, the at least one processor 320 stores the first image frame of the first time period rendered based on the first vsync signal in the graphics buffer area of the memory 330, and stores the next image frame of the first vsync signal in the graphics buffer area of the memory 330. When the first image frame is transmitted to the display 360 based on the second vsync signal, the display 360 may be controlled (or set) to perform adaptive sync (e.g., activate the adaptive sync function). With the adaptive sync function activated, the display 360 waits to receive an image frame for a specified waiting time (e.g., 2 ms) from the moment of receiving the second vsync signal, and the first image frame is received through the graphics buffer area within the specified waiting time. Once received, the received first image frame may be displayed.

According to one embodiment, the at least one processor 320 performs adaptive sync when there is no image frame rendered based on the second vsync signal following the first vsync signal or when the image frame is not updated in the graphics buffer area. You can control (or configure) it not to perform (e.g. disabling the adaptive sync feature). The display 360 may continue to display the first image frame, which is the previously received image frame, without waiting time at the time of receiving the second vsync signal while the adaptive sync function is disabled.

According to one embodiment, the at least one processor 320 displays (360) when there is a second image frame rendered based on the third vsync signal following the second vsync signal or when the second image frame is updated in the graphics buffer area. ) can be controlled (or set) to perform adaptive sync (e.g., activating the adaptive sync function). With the adaptive sync function activated, the display 360 waits to receive the second image frame for a specified waiting time (e.g., 2 ms) from the point of receiving the third vsync signal, and displays the second image frame through the graphic buffer area within the specified waiting time. When a frame is received, the received third image frame may be displayed.

At least one processor 320 according to one embodiment may include a central processing unit (CPU) 322, a graphic processing unit (GPU) 324, and/or a data processing unit (DPU) 326. . The CPU 322 according to one embodiment may perform overall control operations for the components of the electronic device 301. For example, the CPU 322 can execute instructions and processing necessary for the operating system. The GPU 324 according to one embodiment may perform image processing on image information (eg, data to be displayed). For example, the GPU 324 may perform rendering on image information and obtain a rendered image frame. The DPU 326 according to one embodiment may parse and process image frames obtained through processing through the CPU 322 and GPU 324 and transmit them to the display 360.

In the above description, the case where at least one processor 320 includes the CPU 322, GPU 324, and DPU 326 has been described as an example, but at least one processor 320 includes the CPU 322. , GPU 324, and may include one application processor (AP) that performs the functions of DPU 326.

The memory 330 according to one embodiment may store various data and information used by at least one processor 320. Data may include, for example, input data or output data for software (e.g., a program) and instructions related thereto. Memory 330 may include volatile memory or non-volatile memory. The memory 330 according to one embodiment may store instructions for processing operations performed by each of the at least one processor 320.

The display 360 according to one embodiment may visually provide information to the outside of the electronic device 301 (eg, a user). For example, the display 360 may display an image corresponding to an image frame processed and transmitted by at least one processor 330.

An electronic device (e.g., the electronic device 101 of FIG. 1 or the electronic device 301 of FIG. 3) according to an embodiment includes a display (e.g., 360 of FIG. 3), a memory (e.g., 330 of FIG. 3), And it may include at least one processor (eg, 320 in FIG. 3). The at least one processor according to an embodiment updates the first image frame in the graphics buffer area of the memory based on the first synchronization signal, so that the display displays the first image for a designated waiting time from the point of receiving the second synchronization signal. Control can be performed to wait for frame reception, and when the first image frame is received within the designated waiting time, control can be performed to display the received first image frame. According to one embodiment, the at least one processor is configured to display the first image frame without waiting time at the point of receiving the third synchronization signal as the second image frame is not updated in the graphic buffer area based on the second synchronization signal. You can control the continuous display of image frames.

The at least one processor according to an embodiment updates the third image frame in the graphic buffer area based on the third synchronization signal, so that the display displays the third image frame during a designated waiting time from the time of receiving the fourth synchronization signal. It can be controlled to wait for reception, and if the third image frame is received within the designated waiting time, it can be controlled to display the received third image frame.

The at least one processor according to an embodiment may include at least one of a central processing unit (CPU), a graphic processing unit (GPU), and a data processing unit (DPU).

The at least one processor according to an embodiment may be set to render image information and store the rendered image frame in the graphics buffer area for a designated time period based on a synchronization signal.

The at least one processor according to one embodiment may be set to activate the adaptive sync function of the display as the rendered image frame is updated in the graphic buffer area.

The at least one processor according to one embodiment may be set to disable the adaptive sync function of the display as the rendered image frame is not updated in the graphic buffer area.

The frame rate of the image frame according to one embodiment may be 60hz, and the designated time interval based on the synchronization signal may be 16.6ms.

The waiting time according to one embodiment may be 3.4ms.

Referring to FIG. 4, the CPU (e.g., the processor 120 of FIG. 1 or the electronic device 301 of FIG. 3) of an electronic device (e.g., the electronic device 101 of FIG. 1 or the electronic device 301 of FIG. 3) according to an embodiment. The CPU 322) and/or the GPU (e.g., the processor 120 in FIG. 1 or the GPU 324 in FIG. 3) include an application control module (APP) 410 and a surfaceflinger module (surfaceflinger) 420. , and may include a display driver module (display driver) 430. In Figure 4, an example in which the CPU 322 performs the functions of the GPU 324 is explained.

The application control module 410 according to an embodiment renders image information, stores the rendered image frames in each of the graphics buffers (e.g., image frame storage areas of the memory) of the memory (e.g., 330), and then Information (e.g., storage area address) of each graphic buffer in which image frames are stored may be transmitted (e.g., enqueue) to the surfaceflinger module 420.

The surfaceflinger module 420 according to one embodiment may composite and commit the frame of image frames stored in the graphic buffers using information on each of the graphic buffers. For example, the surfaceflinger module 420 provides information on each of the graphics buffers (e.g., each graphics buffer area address information (e.g., memory address information), each graphic buffer module 430 through a graphics HAL (hardware abstraction layer). Memory size information of the graphic buffer area, image frame sequence information stored in each graphic buffer area, image frame data information stored in each graphic buffer area, and/or resolution information (e.g., horizontal and vertical sizes) of image frames stored in the graphic buffer area. ) can be transmitted.

The display driver module 430 according to one embodiment may include an adaptive sync control module 432 and a frame status monitoring module 434. The frame status monitoring module 434 according to one embodiment may monitor status information of graphics buffer areas. For example, the state of the graphic buffer areas may include at least one of the enqueued state, dequeued state, acquired state, and free state. According to one embodiment, the dequeued state may be a state in which an image frame is being saved. For example, in the dequeued state, the application control module 410 may perform an operation to create an image frame and store it in the graphics buffer area (e.g., if the application control module 410 has acquired ownership of the graphics buffer area) situation). According to one embodiment, the enqueued state may be a state in which an image frame is stored. For example, after the application control module 410 completes storing the generated image frame in the graphics server area, it can wait for the DPU to take the image frame in an enqueued state. The acquired state according to one embodiment may be a state in which an image frame is read. For example, the acquired state may be a state in which the DPU takes ownership to read an image frame from the graphics buffer area. The free state according to one embodiment may be a state in which image frame consumption (e.g., reading task) by the DPU has been completed. For example, the DPU can read an image frame from the graphics buffer area and transmit it to the display 360, and after this read operation is completed, the acquired state can be changed from the acquired state to the free state. For example, in the free state, the stored image frame may be delivered to the display 360 and no newly updated image frame may exist.

According to one embodiment, the graphics buffer area may repeat the enqueued state, dequeued state, acquired state, and free state. If all graphic buffer areas (e.g., 3) are in the free state, no rendering operation is performed. can be identified.

The frame status monitoring module 434 according to one embodiment determines that, when all of the graphic buffer areas are in the free state, the state of the graphic buffer area is set to an image frame update state (e.g., a state in which a newly rendered image frame is stored in the graphic buffer area, or It can be identified as not being in a state where an image frame to be output exists in the graphics buffer area. According to one embodiment, the frame state monitoring module 434 identifies whether the state of the GPU 324 is in a running state or an idle state, and when the state of the GPU 324 is in an idle state, the state of the graphics buffer area is in the image frame update state. It may be identified as not being the case (e.g., a newly rendered image frame is stored in the graphics buffer area, or an image frame to be output exists in the graphics buffer area).

The adaptive sync control module 432 according to one embodiment determines the state of the graphic buffer area in an image frame update state (e.g., a newly rendered image frame is stored in the graphic buffer area, or an image frame to be output exists in the graphic buffer area). If it is identified as a state), the adaptive sync function can be activated. The adaptive sync control module 432 according to one embodiment determines the state of the graphic buffer area in an image frame update state (e.g., a newly rendered image frame is stored in the graphic buffer area, or an image frame to be output exists in the graphic buffer area). If it is identified as not being in a state), the adaptive sync feature can be disabled.

The adaptive sync control module 432 according to one embodiment may transmit information (or a command) notifying activation of the adaptive sync function to the DPU 326 and the display 360 to activate the adaptive sync function. The adaptive sync control module 432 according to one embodiment may transmit information (or a command) indicating deactivation of the adaptive sync function to the DPU 326 and the display 360 in order to deactivate the adaptive sync function.

When the DPU 326 according to one embodiment receives new graphics buffer information within a specified time from the point of receiving a synchronization signal (e.g., TE signal) (or vsync signal) from the display 360, the DPU 326 updates the image frame stored in the graphics buffer area. Post-processing operations such as hardware (HW) blending and/or display stream compression (DSC) may be performed and image frames may be transmitted to the display 360 through a mobile industry processor interface (MIPI) lane.

If there is no image frame transfer command at the time of receiving the first synchronization signal (e.g., first vsync signal) corresponding to the first time interval while the adaptive sync function is activated, the DPU 326 according to one embodiment may perform a designated waiting time. Wait for the image frame transfer command for (e.g. timeout time), and if the image frame transfer command is received within the waiting time, the image frame is delivered to the display 360, and the image frame transfer command is received when the waiting time is exceeded. Then, an image frame can be transmitted to the display 360 at the time of receiving the second synchronization signal (eg, the second vsync signal) after the first vsync signal. If there is an image frame transfer command at the time of receiving the first vsync signal corresponding to the first time interval while the adaptive sync function is activated, the DPU 326 according to one embodiment may generate a second signal corresponding to the first time interval without waiting time. 1 image frame can be transmitted to the display 360.

The display 360 according to one embodiment may store the image frame received from the DPU 326 in a memory associated with the display 360 (eg, memory 233 or GRAM in FIG. 2).

Since updated image frames are not received when the adaptive sync function is disabled, the display 360 according to one embodiment may be controlled to continuously display previously output image frames.

The display 360 according to one embodiment waits for a specified waiting time with the adaptive sync function activated, and when an image frame is delivered within the specified waiting time, the display 360 scans the image frame stored in the GRAM and responds to the image frame through the display panel. A screen can be displayed. The display 360 according to one embodiment waits for a specified waiting time with the adaptive sync function activated, and when an image frame is delivered while the specified waiting time is exceeded, the display 360 scans the image frame stored in GRAM in the next time period and displays it. The screen corresponding to the image frame can be displayed through the panel.

Referring to FIG. 5, at least one processor (e.g., the processor 120 of FIG. 1 or the electronic device 301 of FIG. 3) of an electronic device (e.g., the electronic device of FIG. 1 or the electronic device 301 of FIG. 3) according to an embodiment. At least one processor 320 may perform at least one of operations 510 to 540.

In operation 510, at least one processor 320 according to an embodiment may render image information and store the rendered image frame in the graphics buffer area of the memory 330 (or a separate buffer memory (not shown)). . At least one processor 320 according to an embodiment stores the rendered image frames in the memory 330 at designated time units (e.g., vsync time interval) using a vertical synchronization signal (e.g., vsync signal). It can be stored in the graphics buffer area of .

In operation 520, at least one processor 320 according to an embodiment may identify activation or deactivation of the adaptive sync function of the display for the second time period based on whether the image frame is updated during the first time period. For example, the at least one processor 320 determines whether an image frame is updated in the graphics buffer area during the first time interval (e.g., whether a newly rendered image frame is stored in the graphics buffer area or an image to be output to the graphics buffer area). Activation or deactivation of the adaptive sync function for the second time period of the display 360 may be identified based on whether a frame exists or not. According to one embodiment, at least one processor 320 may control to activate an adaptive sync function for the second time period of the display 360 when an image frame is updated in the graphic buffer area during the first time period. According to one embodiment, at least one processor 320 may control to disable the adaptive sync function for the second time period of the display 360 when the image frame is not updated in the graphic buffer area during the first time period. .

In operation 530, the at least one processor 320 according to an embodiment controls the display 360 to wait to receive an image frame for a specified waiting time in a second time period based on activation of the adaptive sync function and waits for the specified wait time. If an image frame is received within the time (e.g. 2ms), the display of the received image frame can be controlled.

In operation 540, the at least one processor 320 according to an embodiment controls the display 360 to continue displaying the image frame displayed in the first time period in the second time period based on deactivation of the adaptive sync function. can do.

Referring to FIG. 6, at least one processor (e.g., the processor 120 of FIG. 1 or the electronic device 301 of FIG. 3) of an electronic device (e.g., the electronic device of FIG. 1 or the electronic device 301 of FIG. 3) according to an embodiment. At least one processor 320 may perform at least one of operations 610 to 630.

In operation 610, at least one processor 320 according to an embodiment performs adaptive operation of the display 360 as the first image frame is updated in the graphics buffer area based on the first synchronization signal (e.g., the first vsync signal). Activate the sync function, control the display 360 to wait to receive the first image frame for a specified waiting time (e.g., 2 ms) from the point of receiving the second synchronization signal (e.g., the second vsync signal), and within the specified waiting time. When the first image frame is received, control can be made to display the received first image frame.

In operation 620, the at least one processor 320 according to an embodiment uses the adaptive sync function of the display 360 as the image frame is not updated in the graphics buffer area based on the second vsync signal following the first vsync signal. can be deactivated and controlled so that the display continues to display the first image frame, which is the previously received image frame, without waiting time at the time of receiving the second vsync signal.

In operation 630, the at least one processor 320 according to an embodiment updates the second image frame in the graphics buffer area based on a third synchronization signal (e.g., a third vsync signal) following the second vsync signal. Activate the adaptive sync function of the display 360, control the display 360 to wait to receive the second image frame for a specified waiting time (e.g., 2 ms) from the point of receiving the third vsync signal, and display the second image frame within the specified waiting time. When an image frame is received, control can be made to display the received second image frame.

The method according to one embodiment is such that the display waits to receive the third image frame for a specified waiting time from the time of receiving the fourth synchronization signal as the third image frame is updated in the graphic buffer area based on the third synchronization signal. and, if the third image frame is received within the designated waiting time, to display the received third image frame.

The method according to one embodiment may include rendering image information and storing the rendered image frame in the graphics buffer area for a designated time period based on a synchronization signal.

The method according to one embodiment may include activating an adaptive sync function of the display as the rendered image frame is updated in the graphic buffer area.

The method according to one embodiment may include disabling the adaptive sync function of the display when the rendered image frame is not updated in the graphic buffer area.

In the method according to one embodiment, the frame rate of the image frame may be 60hz, and the designated time interval based on the synchronization signal may be 16.6ms.

In the method according to one embodiment, the waiting time may be 3.4ms.

Referring to FIG. 7, reference number 710 according to one embodiment is rendered by the CPU (e.g., CPU 322 in FIG. 3) and/or GPU (e.g., GPU 324 in FIG. 3) and stored in the graphics buffer. It can represent image frames that are Reference numeral 720 according to one embodiment may represent image frames delivered to the display 360 by the DPU 326. Reference number 730 according to one embodiment may represent image frames played by the display 360.

The CPU 322 and DPU 326 according to one embodiment may operate based on the vsync signal at specified time intervals (e.g., 16.6 ms), and the specified time intervals may be adaptively adjusted according to the frame delay time.

In the first time interval T1 (e.g., first vsync interval) based on the first vsync signal according to one embodiment, the CPU 322 performs rendering on the image information and stores the rendered first image frame in memory ( It can be stored in the graphics buffer area of 320, and the DPU 326 and display 360 can be in a standby state without any received image frames.

If the rendering time is delayed in the second time interval T2 (e.g., the second vsync interval) based on the second vsync signal, the CPU 322 according to one embodiment may display the rendered second image frame in the second time interval (T2). It can be stored in the graphic buffer area of the memory 320 in the third time interval (T3) when T2) is exceeded. The DPU 326 according to one embodiment fails to deliver the second image frame to the display 360 at the start of the third time interval T3 based on the third vsync signal, and after a predetermined time delay, the second image frame stored in the graphics buffer area 2 Image frames can be transmitted to the display 360. The display 360 according to one embodiment may display the received second image frame when a time-delayed second image frame is received while waiting to receive an image frame based on applying adaptive sync (or activating the adaptive sync function).

According to one embodiment, according to the adaptive sync function, the time length of the third time section (T3) corresponding to the third vsync signal is increased by the frame transmission delay time (735), and then the time section interval is normalized from the fourth vsync signal. You can return to a specified time.

Referring to FIG. 8, reference numeral 810 according to one embodiment is a number that is rendered by a CPU (e.g., CPU 322 in FIG. 3) and/or GPU (e.g., GPU 324 in FIG. 3) and stored in a graphics buffer. Reference numeral 820 according to an embodiment may represent image frames transmitted to the display 360 by a DPU (eg, DPU 326 in FIG. 3). Number 830 may represent image frames reproduced by display 360.

The CPU 322 and DPU 326 according to one embodiment may operate based on the vsync signal at specified time intervals (eg, 16.6 ms). The specified time interval can be adaptively adjusted depending on the frame delay time.

The CPU 322 according to one embodiment performs rendering on image information and stores the rendered first image frame corresponding to the first time interval T1 (e.g., first vsync interval) based on the first vsync signal. It can be stored in the graphics buffer area of (320). The DPU 326 according to one embodiment processes the first image frame stored in the graphics buffer area in a second time interval T2 (e.g., a second vsync interval) based on the second vsync signal and delivers it to the display 360. You can. When applying adaptive sync (or activating the adaptive sync function), the display 360 according to one embodiment receives the first image frame for a designated waiting time (e.g., 2 ms) in the second time interval (T2) based on the second vsync signal. It waits, and if the first image frame is received within the designated waiting time, the received first image frame may be displayed.

The CPU 322 according to one embodiment may not perform rendering in the second time interval T2 based on the second vsync signal after the first vsync signal, or may not perform an update of the image frame rendered in the graphics buffer area. there is. The DPU 326 according to one embodiment displays the display 360 in the third time interval T3 when there is no image frame in the graphic buffer area or the image frame is not updated (NO frame) in the second time interval T2. Image frames may not be transmitted (NO frame). As the image frame is not delivered from the DPU 326 in the third time period T3, the display 324 according to one embodiment waits to receive the image frame until the maximum waiting time 835, and then transfers the image frame after the maximum waiting time. The received first image frame may be displayed. According to FIG. 8, when image frames are not rendered continuously and there is no rendered image frame in the middle, the display 360 may have an unnecessary waiting time (e.g., 835) in the corresponding time section, which is Unnecessarily increasing the time can prevent the next image frame from being played quickly.

FIG. 9 is a diagram illustrating that adaptive sync is not applied when an image frame is not updated in the graphics buffer area while applying adaptive sync in an electronic device according to an embodiment.

Referring to FIG. 9, reference numeral 910 according to an embodiment is a number that is rendered by a CPU (e.g., CPU 322 in FIG. 3) and/or GPU (e.g., GPU 324 in FIG. 3) and stored in a graphics buffer. Reference numeral 920 according to an embodiment may represent image frames transmitted to the display 360 by a DPU (eg, DPU 326 in FIG. 3). Number 930 may represent image frames reproduced by display 360.

The CPU 322 according to one embodiment performs rendering on image information and stores the rendered first image frame corresponding to the first time interval T1 (e.g., first vsync interval) based on the first vsync signal. It can be stored in the graphics buffer area of (320). The DPU 326 according to an embodiment processes the first image frame stored in the graphics buffer area in a second time period T2 (e.g., a second vsync period) based on the second vsync signal and delivers it to the display 360. You can. When applying adaptive sync (or activating the adaptive sync function), the display 360 according to one embodiment displays a designated waiting time (e.g., 2 ms) in a second time interval (T2) (e.g., a second vsync interval) based on the second vsync signal. ), and when the first image frame is received within the designated waiting time, the received first image frame can be displayed.

The CPU 322 according to one embodiment does not perform rendering in the second time interval T2 (e.g., the second vsync interval) based on the second vsync signal after the first vsync signal, or does not perform rendering on the image rendered in the graphics buffer area. Frame updates may not be performed. The CPU 322 according to an embodiment may identify that rendering is not performed in the second time period T2 or that the image frame rendered in the graphic buffer area is not updated. If rendering is not performed in the second time interval T2 or the image frame rendered in the graphics buffer area is not updated, the CPU 322 according to one embodiment may operate with the DPU 326 in the third time interval T3. The display 326 can be controlled not to apply adaptive sync (or disable the adaptive sync function).

The DPU 326 according to one embodiment may deliver the first image frame to the display 360 in the second time interval T3 and provide setting information for not applying adaptive sync (or deactivating the adaptive sync function) (925). You can.

The display 360 according to an embodiment displays the first image frame received in the second time interval T3 and displays the first image frame in the third time interval T3 according to setting information for not applying adaptive sync (or deactivating the adaptive sync function). You can set it to have no waiting time.

The DPU 326 according to one embodiment may not transmit an image frame to the display 360 in the third time interval T3, and the display 360 may be based on non-application of adaptive sync in the third time interval T3. Thus, the previously received first image frame can be continuously displayed without waiting time.

Referring to FIG. 10, reference numeral 1010 according to an embodiment is a block that is rendered by a CPU (e.g., CPU 322 in FIG. 3) and/or GPU (e.g., GPU 324 in FIG. 3) and stored in a graphics buffer. Reference number 1020 according to an embodiment may represent image frames transmitted to the display 360 by a DPU (eg, DPU 326 in FIG. 3). Number 1030 may represent image frames reproduced by display 360.

The CPU 322 according to one embodiment performs rendering on image information and stores the rendered first image frame corresponding to the first time interval T1 (e.g., first vsync interval) based on the first vsync signal. It can be stored in the graphics buffer area of (320). If the rendering time of the first image frame is delayed, storage of the first image frame may be completed in the second vsync period.

The DPU 326 according to an embodiment may process the delayed first image frame in a second time interval T2 (e.g., a second vsync interval) based on the second vsync signal and transmit it to the display 360 with delay. there is. The display 360 according to one embodiment waits to receive the first image frame until the maximum waiting time in the second time interval T2 based on the second vsync signal based on applying adaptive sync (or activating the adaptive sync function), If the first image frame is received within the maximum waiting time (1035) (e.g., 3.4 ms (1035)), the first image frame may be displayed. Due to an increase in waiting time according to one embodiment, the second vsync section may be increased to 20ms.

The CPU 322 according to one embodiment may not perform rendering in the second time interval T2 based on the second vsync signal after the first vsync signal, or may not perform an update of the image frame rendered in the graphics buffer area. there is. The DPU 326 according to one embodiment identifies (1021) when there is no image frame in the graphics buffer area or the image frame is not updated (NO frame) in the second time interval (T2) and the display 360 performs adaptive sync. Control not to apply (or disable the adaptive sync function) (or transmit a command to the display 360 to cause the display 360 to not apply adaptive sync (or deactivate the adaptive sync function)) (1023), and a third time interval (T3) ), the image frame may not be transmitted to the display 360 (NO frame).

The display 324 according to one embodiment does not apply adaptive sync (or deactivates the adaptive sync function) according to a control or command from the DPU 326 in the third time interval T3, so that a specified waiting time (e.g. : 2ms), and if the first image frame is not received within the specified waiting time (e.g., 2ms), the previously received first image frame may be displayed. In the third time interval T3 according to one embodiment, the designated time interval (e.g., 16.6 ms) may not be unnecessarily increased due to non-application of adaptive sync (or deactivation of the adaptive sync function). Accordingly, the next second image frame may be displayed faster than when adaptive sync is applied.

The CPU 322 according to one embodiment may store the second image frame rendered in the third time interval T3 (e.g., the third vsync interval) based on the third vsync signal in the graphics buffer area of the memory 320. there is. If the rendering time of the second image frame is delayed, storage of the second image frame may be completed in the fourth vsync section.

The DPU 326 according to one embodiment may process the delayed second image frame in the fourth time interval T4 (e.g., the fourth vsync interval) based on the fourth vsync signal and transmit it to the display 360 with delay. there is. According to one embodiment, the display 360 receives a fourth vsync signal in a state in which adaptive sync is applied (or the adaptive sync function is activated) as a command to not apply adaptive sync (or disable the adaptive sync function) is not received from the DPU 326. Wait for reception of a second image frame until the maximum waiting time in a fourth time interval (T4) based on According to one embodiment, an electronic device (e.g., the electronic device 101 of FIG. 1 and the electronic device 301 of FIG. 3) renders an image frame in advance by referring to the graphics buffer status and/or GPU status information. By identifying whether and/or whether the image frame is updated and controlling the adaptive sync function of the display 360 and/or DPU 325 to be activated (e.g. on) or deactivated (e.g. off) based on the identification result, Unnecessary frame waiting time in the no frame section can be reduced. This can prevent the next new frame update time from being delayed.

Electronic devices according to various embodiments disclosed in this document may be of various types. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to embodiments of this document are not limited to the above-described devices.

The various embodiments of this document and the terms used herein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various changes, equivalents, or replacements of the embodiments. In connection with the description of the drawings, similar reference numbers may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the above items, unless the relevant context clearly indicates otherwise. In this document, “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B, or C,” “at least one of A, B, and C,” and “A. Each of phrases such as “at least one of , B, or C” may include any one of the items listed together in the corresponding phrase, or any possible combination thereof. Terms such as "first", "second", or "first" or "second" may be used simply to distinguish one component from another, and to refer to those components in other respects (e.g., importance or order) is not limited. One (e.g., first) component is said to be “coupled” or “connected” to another (e.g., second) component, with or without the terms “functionally” or “communicatively.” When mentioned, it means that any of the components can be connected to the other components directly (e.g. wired), wirelessly, or through a third component.

The term “module” used in this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be an integrated part or a minimum unit of the parts or a part thereof that performs one or more functions. For example, according to one embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of this document are one or more stored in a storage medium (e.g., built-in memory 136 or external memory 138) that can be read by a machine (e.g., electronic device 101). It may be implemented as software (e.g., program 140) including instructions. For example, a processor (e.g., processor 120) of a device (e.g., electronic device 101) may call at least one command among one or more commands stored from a storage medium and execute it. This allows the device to be operated to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. Device-readable storage media may be provided in the form of non-transitory storage media. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain signals (e.g. electromagnetic waves). This term refers to cases where data is stored semi-permanently in the storage medium. There is no distinction between temporary storage cases.

According to one embodiment, methods according to various embodiments disclosed in this document may be provided and included in a computer program product. Computer program products are commodities and can be traded between sellers and buyers. The computer program product may be distributed in the form of a machine-readable storage medium (e.g. compact disc read only memory (CD-ROM)) or via an application store (e.g. Play StoreTM) or on two user devices (e.g. It can be distributed (e.g. downloaded or uploaded) directly between smartphones) or online. In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

According to various embodiments, each component (eg, module or program) of the above-described components may include a single entity or a plurality of entities. According to various embodiments, one or more of the components or operations described above may be omitted, or one or more other components or operations may be added. Alternatively or additionally, multiple components (eg, modules or programs) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each component of the plurality of components identically or similarly to those performed by the corresponding component of the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or , or one or more other operations may be added.

According to one embodiment, in a non-volatile storage medium storing instructions, the instructions are set to cause the electronic device to perform at least one operation when executed by the electronic device, the at least one operation being: As the first image frame is updated in the graphics buffer area of the memory of the electronic device based on the first synchronization signal, the display is controlled to wait for reception of the first image frame for a specified waiting time from the time of receiving the second synchronization signal, Controlling to display the received first image frame when the first image frame is received within the designated waiting time, and when the second image frame is not updated in the graphic buffer area based on the second synchronization signal The method may include controlling the display to continuously display the first image frame without waiting time when a third synchronization signal is received.

In addition, the embodiments of the present disclosure invented in the specification and drawings are merely presented as specific examples to easily explain the technical content according to the embodiments of the present disclosure and to aid understanding of the embodiments of the present disclosure, and the scope of the embodiments of the present disclosure It is not intended to limit it. Therefore, the scope of the various embodiments of the present disclosure should be interpreted as including all changes or modified forms derived based on the technical idea of the various embodiments of the present disclosure in addition to the embodiments invented herein. do.

Claims

In the electronic device (301 in FIG. 3),

Display (360 in Figure 3);

a memory for storing instructions (330 in FIG. 3); and

Comprising a processor (320 in FIG. 3) operatively connected to the display and the memory,

The instructions, when executed by the processor, cause the electronic device to:

As the first image frame is updated in the graphics buffer area of the memory based on the first synchronization signal, the display is controlled to wait for reception of the first image frame for a specified waiting time from the point of receiving the second synchronization signal, and the specified wait time If the first image frame is received within time, control to display the received first image frame,

An electronic device configured to control the display to continue displaying the first image frame without waiting time at the point of receiving the third synchronization signal as the second image frame is not updated in the graphic buffer area based on the second synchronization signal. .
According to paragraph 1,

When executed by the processor, the instructions cause the electronic device to update the third image frame in the graphic buffer area based on the third synchronization signal, thereby causing the display to change to a specified point from the point of receiving the fourth synchronization signal. An electronic device configured to control the electronic device to wait for reception of a third image frame during the waiting time, and to display the received third image frame when the third image frame is received within the designated waiting time.
According to claim 1 or 2,

The processor is an electronic device including at least one of a central processing unit (CPU), a graphic processing unit (GPU), and a data processing unit (DPU).
According to any one of claims 1 to 3,

The instructions, when executed by the processor, cause the electronic device to:

An electronic device configured to render image information and store rendered image frames in the graphics buffer area for a specified time interval based on a synchronization signal.
According to any one of claims 1 to 4,

The instructions, when executed by the processor, cause the electronic device to activate an adaptive sync function of the display as the rendered image frame in the graphic buffer area is updated.
According to any one of claims 1 to 5,

The instructions, when executed by the processor, cause the electronic device to disable the adaptive sync function of the display as the rendered image frame is not updated in the graphic buffer area.
According to any one of claims 1 to 6,

The electronic device wherein the frame rate of the image frame is 60hz, and the designated time interval based on the synchronization signal is 16.6ms.
According to any one of claims 1 to 7,

An electronic device wherein the latency is 3.4 ms.
Method for controlling image frame synchronization in electronic device

As the first image frame is updated in the graphics buffer area of the memory of the electronic device based on the first synchronization signal, the display is controlled to wait for reception of the first image frame for a specified waiting time from the time of receiving the second synchronization signal, Controlling to display the received first image frame when the first image frame is received within the designated waiting time; and

An operation of controlling the display to continue displaying the first image frame without waiting time at the point of receiving the third synchronization signal as the second image frame is not updated in the graphic buffer area based on the second synchronization signal. How to.
According to clause 9,

As the third image frame is updated in the graphic buffer area based on the third synchronization signal, the display is controlled to wait for reception of the third image frame for a designated waiting time from the point of receiving the fourth synchronization signal, and the designated waiting time The method further includes controlling to display the received third image frame when the third image frame is received.
According to claim 9 or 10,

The method further includes rendering image information and storing the rendered image frame in the graphics buffer area for a specified time period based on a synchronization signal.
According to any one of claims 9 to 11,

The method further includes activating an adaptive sync function of the display as the rendered image frame is updated in the graphic buffer area.
According to any one of claims 9 to 12,

The method further includes disabling the adaptive sync function of the display when the rendered image frame is not updated in the graphic buffer area.
According to any one of claims 9 to 13,

The frame rate of the image frame is 60hz, and the designated time interval based on the synchronization signal is 16.6ms,

A method wherein the waiting time is 3.4ms.
A non-volatile storage medium storing instructions, wherein the instructions are set to cause the electronic device to perform at least one operation when executed by the electronic device, the at least one operation being:

As the first image frame is updated in the graphics buffer area of the memory of the electronic device based on the first synchronization signal, the display is controlled to wait for reception of the first image frame for a specified waiting time from the time of receiving the second synchronization signal, Controlling to display the received first image frame when the first image frame is received within the designated waiting time; and

An operation of controlling the display to continue displaying the first image frame without waiting time at the point of receiving the third synchronization signal as the second image frame is not updated in the graphic buffer area based on the second synchronization signal. storage media.