KR102577184B1 - Electronic apparatus and operating method thereof - Google Patents

Abstract

An electronic device according to various embodiments includes a graphics buffer accessed by at least two objects; display; and a processor functionally connected to the graphics buffer and the display, wherein the processor analyzes the graphics processing conditions, sets the number of graphics buffers, and displays graphics data recorded in the graphics buffer on the display. It can be processed as much as possible.
Various other embodiments are possible.

Description

Electronic device and operating method thereof {ELECTRONIC APPARATUS AND OPERATING METHOD THEREOF}

Various embodiments of the present invention relate to a memory management method of an electronic device.

With the recent development of digital technology, mobile communication terminals, smart phones, tablet PCs (Personal Computers), PDA (Personal Digital Assistants), electronic notebooks, laptops, or wearable devices, etc. Various types of electronic devices are widely used. Electronic devices are reaching a stage of mobile convergence that includes the functions of other devices.

Electronic devices can process content containing high-resolution images and run various applications simultaneously.

As electronic devices change to an environment where various applications run simultaneously, memory capacity may increase. For example, when an electronic device using the Android operating system (OS) displays an application screen, the space occupied by the graphics buffer may gradually increase, which may cause memory shortage and deteriorate performance. there is.

Electronic devices according to various embodiments of the present invention can provide an apparatus and method for managing memory by setting a minimum number of graphics buffers appropriate for the situation of each application when executing an application.

Electronic devices according to various embodiments of the present invention can provide an apparatus and method that can dynamically allocate the number of buffers within a range that does not degrade performance when executing an application.

An electronic device according to various embodiments of the present invention includes a graphics buffer accessed by at least two objects; display; and a processor functionally connected to the graphics buffer and display. The processor may analyze the graphics processing conditions, set the number of graphics buffers, and process graphics data recorded in the graphics buffer to be displayed on the display.

A method of operating an electronic device according to various embodiments of the present invention includes executing an application that performs a graphic display function through a graphic buffer; Analyzing graphics processing conditions of the executing application and setting the number of graphics buffers; and an operation of processing graphic data recorded in the graphic buffer and displaying it on a display.

According to various embodiments, an electronic device can prevent graphics performance from being degraded due to frame drops by dynamically adjusting the number of graphics buffers while processing graphics data, and can prevent graphics performance from being degraded by reducing the graphics performance to a minimum level without degrading graphics performance. Memory can be saved by calculating the number of graphics buffers. For example, when allocating graphics buffers to system memory in mobile systems such as Android, where performance bottlenecks frequently occur due to lack of system memory, unnecessary graphics buffer allocation can be avoided, thereby improving performance.

1 shows a block diagram of a network environment system according to various embodiments.
Figure 2 shows a block diagram of an electronic device according to various embodiments.
3 shows a block diagram of a program module according to various embodiments.
FIG. 4 is a diagram illustrating the configuration of an electronic device according to various embodiments of the present invention.
Figure 5 is a block diagram of an electronic device according to various embodiments of the present invention.
FIG. 6 is a diagram illustrating the basic operation of double buffering for processing graphics data in a graphics system.
Figure 7 is a diagram explaining the operation of double buffering between an application and a window compositor.
Figure 8 is a diagram illustrating double buffering operation between a window compositor and a display.
Figure 9 is a diagram showing double buffering operation in the Android OS system.
FIGS. 10A to 10C are diagrams for explaining the operation of a graphics buffer consisting of a triple buffer in an electronic device.
Figure 11 is a diagram to explain the delay time as the graphics buffer increases.
FIG. 12 is a diagram illustrating the configuration of an electronic device using the Android OS according to various embodiments of the present invention.
FIG. 13 is a diagram illustrating an example operation in which the possession time of a graphic buffer of an application is constant in each frame in an electronic device according to various embodiments of the present invention.
FIG. 14 is a diagram illustrating another operation in which the possession time of the graphic buffer of the Surface Flinger is constant in each frame in an electronic device according to various embodiments of the present invention.
FIG. 15 is a diagram illustrating another operation in an example in which the possession time of the graphic buffer of the Surface Flinger is not constant in an electronic device according to various embodiments of the present invention.
FIG. 16 is a diagram illustrating an example of operation according to a change in composition type in an electronic device according to various embodiments of the present invention.
Figure 17 is a flowchart showing an operation of processing graphic data in an electronic device according to an embodiment of the present invention.
FIG. 18 is a flowchart illustrating an operation of an electronic device processing graphic data according to various embodiments of the present invention.

Hereinafter, various embodiments of this document are described with reference to the attached drawings. The examples and terms used herein are not intended to limit the technology described in this document to specific embodiments, and should be understood to include various modifications, equivalents, and/or substitutes for the examples. In connection with the description of the drawings, similar reference numbers may be used for similar components. Singular expressions may include plural expressions, unless the context clearly indicates otherwise. In this document, expressions such as “A or B” or “at least one of A and/or B” may include all possible combinations of the items listed together. Expressions such as “first,” “second,” “first,” or “second,” can modify the corresponding components regardless of order or importance, and are used to distinguish one component from another. It is only used and does not limit the corresponding components. When a component (e.g., a first) component is said to be "connected (functionally or communicatively)" or "connected" to another (e.g., second) component, it means that the component is connected to the other component. It may be connected directly to a component or may be connected through another component (e.g., a third component).

In this document, “configured to” means “suitable for,” “having the ability to,” or “changed to,” depending on the situation, for example, in terms of hardware or software. ," can be used interchangeably with "made to," "capable of," or "designed to." In some contexts, the expression “a device configured to” may mean that the device is “capable of” working with other devices or components. For example, the phrase "processor configured (or set) to perform A, B, and C" refers to a processor dedicated to performing the operations (e.g., an embedded processor), or by executing one or more software programs stored on a memory device. , may refer to a general-purpose processor (e.g., CPU or application processor) capable of performing the corresponding operations.

Electronic devices according to various embodiments of the present document include, for example, smartphones, tablet PCs, mobile phones, video phones, e-book readers, desktop PCs, laptop PCs, netbook computers, workstations, servers, PDAs, and PMPs. (portable multimedia player), MP3 player, medical device, camera, or wearable device. Wearable devices may be accessory (e.g., watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, or head-mounted-device (HMD)), fabric or clothing-integrated (e.g., electronic clothing), The electronic device may include at least one of body attached (e.g., skin pad or tattoo) or bioimplantable circuitry. In some embodiments, the electronic device may include, for example, a television, a digital video disk (DVD) player, Audio, refrigerator, air conditioner, vacuum cleaner, oven, microwave, washing machine, air purifier, set-top box, home automation control panel, security control panel, media box (e.g. Samsung HomeSyncTM, Apple TVTM, or Google TVTM), game console It may include at least one of (e.g. XboxTM, PlayStationTM), an electronic dictionary, an electronic key, a camcorder, or an electronic picture frame.

In another embodiment, the electronic device may include various medical devices (e.g., various portable medical measurement devices (such as blood sugar monitors, heart rate monitors, blood pressure monitors, or body temperature monitors), magnetic resonance angiography (MRA), magnetic resonance imaging (MRI), CT (computed tomography), radiography, or ultrasound, etc.), navigation devices, satellite navigation systems (GNSS (global navigation satellite system)), EDR (event data recorder), FDR (flight data recorder), automobile infotainment devices, marine electronic equipment (e.g. marine navigation devices, gyro compasses, etc.), avionics, security devices, head units for vehicles, industrial or home robots, drones, ATMs at financial institutions, point-of-sale (POS) at stores. of sales), or Internet of Things devices (e.g., light bulbs, various sensors, sprinkler devices, fire alarms, thermostats, street lights, toasters, exercise equipment, hot water tanks, heaters, boilers, etc.). According to some embodiments, the electronic device may be a piece of furniture, a building/structure or a vehicle, an electronic board, an electronic signature receiving device, a projector, or various measuring devices (e.g. water, electrical, It may include at least one of gas, radio wave measuring equipment, etc.). In various embodiments, the electronic device may be flexible, or may be a combination of two or more of the various devices described above. Electronic devices according to embodiments of this document are not limited to the above-described devices. In this document, the term user may refer to a person using an electronic device or a device (e.g., an artificial intelligence electronic device) using an electronic device.

1, an electronic device 101 within a network environment 100, in various embodiments, is described. The electronic device 101 may include a bus 110, a processor 120, a memory 130, an input/output interface 150, a display 160, and a communication interface 170. In some embodiments, the electronic device 101 may omit at least one of the components or may additionally include another component. The bus 110 connects the components 110 to 170 to each other and may include circuitry that transfers communication (eg, control messages or data) between the components. The processor 120 may include one or more of a central processing unit, an application processor, or a communication processor (CP). The processor 120 may, for example, perform operations or data processing related to control and/or communication of at least one other component of the electronic device 101.

Memory 130 may include volatile and/or non-volatile memory. For example, the memory 130 may store commands or data related to at least one other component of the electronic device 101. According to one embodiment, memory 130 may store software and/or program 140. Program 140 may include, for example, a kernel 141, middleware 143, an application programming interface (API) 145, and/or an application program (or “application”) 147, etc. . At least a portion of the kernel 141, middleware 143, or API 145 may be referred to as an operating system. Kernel 141 may, for example, provide system resources (e.g., middleware 143, API 145, or application program 147) used to execute operations or functions implemented in other programs (e.g., : Bus 110, processor 120, or memory 130, etc.) can be controlled or managed. In addition, the kernel 141 provides an interface for controlling or managing system resources by accessing individual components of the electronic device 101 in the middleware 143, API 145, or application program 147. You can.

The middleware 143 may, for example, perform an intermediary role so that the API 145 or the application program 147 can communicate with the kernel 141 to exchange data. Additionally, the middleware 143 may process one or more work requests received from the application program 147 according to priority. For example, the middleware 143 may use system resources (e.g., bus 110, processor 120, or memory 130, etc.) of the electronic device 101 for at least one of the application programs 147. Priority may be assigned and the one or more work requests may be processed. The API 145 is an interface for the application 147 to control functions provided by the kernel 141 or middleware 143, for example, at least for file control, window control, image processing, or character control. Can contain one interface or function (e.g. command). The input/output interface 150, for example, transmits commands or data input from a user or other external device to other component(s) of the electronic device 101, or to other component(s) of the electronic device 101 ( Commands or data received from (fields) can be output to the user or other external device.

Display 160 may be, for example, a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting diode (OLED) display, a microelectromechanical system (MEMS) display, or an electronic paper display. It can be included. For example, the display 160 may display various contents (e.g., text, images, videos, icons, and/or symbols, etc.) to the user. The display 160 may include a touch screen and may receive, for example, a touch, gesture, proximity, or hovering input using an electronic pen or a part of the user's body. The communication interface 170, for example, establishes communication between the electronic device 101 and an external device (e.g., the first external electronic device 102, the second external electronic device 104, or the server 106). You can. For example, the communication interface 170 may be connected to the network 162 through wireless or wired communication and communicate with an external device (eg, the second external electronic device 104 or the server 106).

Wireless communications include, for example, LTE, LTE Advance (LTE-A), code division multiple access (CDMA), wideband CDMA (WCDMA), universal mobile telecommunications system (UMTS), Wireless Broadband (WiBro), or Global GSM (GSM). It may include cellular communication using at least one of the System for Mobile Communications). According to one embodiment, wireless communication includes, for example, wireless fidelity (WiFi), Bluetooth, Bluetooth Low Energy (BLE), Zigbee, near field communication (NFC), Magnetic Secure Transmission, and radio. It may include at least one of frequency (RF) or body area network (BAN). According to one embodiment, wireless communications may include GNSS. GNSS may be, for example, Global Positioning System (GPS), Global Navigation Satellite System (Glonass), Beidou Navigation Satellite System (hereinafter “Beidou”), or Galileo, the European global satellite-based navigation system. Hereinafter, in this document, “GPS” may be used interchangeably with “GNSS.” Wired communication may include, for example, at least one of universal serial bus (USB), high definition multimedia interface (HDMI), recommended standard 232 (RS-232), power line communication, or plain old telephone service (POTS). there is. Network 162 may include at least one of a telecommunications network, for example, a computer network (e.g., a LAN or WAN), the Internet, or a telephone network.

Each of the first and second external electronic devices 102 and 104 may be of the same or different type as the electronic device 101. According to various embodiments, all or part of the operations performed on the electronic device 101 may be executed on one or more electronic devices (e.g., the electronic devices 102 and 104, or the server 106). One embodiment According to this, when the electronic device 101 is to perform a certain function or service automatically or upon request, the electronic device 101 performs at least part of the function or service associated therewith instead of or in addition to executing the function or service on its own. A function may be requested from another device (e.g., electronic device 102, 104, or server 106). The other electronic device (e.g., electronic device 102, 104, or server 106) may request the requested function. Alternatively, an additional function may be executed and the result may be transmitted to the electronic device 101. The electronic device 101 may process the received result as is or additionally to provide the requested function or service. For this purpose, e.g. For example, cloud computing, distributed computing, or client-server computing technologies may be used.

Figure 2 is a block diagram of an electronic device 201 according to various embodiments. The electronic device 201 may include, for example, all or part of the electronic device 101 shown in FIG. 1 . The electronic device 201 includes one or more processors (e.g., AP) 210, a communication module 220, a subscriber identification module 224, a memory 230, a sensor module 240, an input device 250, and a display ( 260), an interface 270, an audio module 280, a camera module 291, a power management module 295, a battery 296, an indicator 297, and a motor 298. The processor 210, for example, can run an operating system or application program to control a number of hardware or software components connected to the processor 210, and can perform various data processing and calculations. The processor 210 may be implemented, for example, as a system on chip (SoC). According to one embodiment, the processor 210 may further include a graphic processing unit (GPU) and/or an image signal processor. The processor 210 may include at least some of the components shown in FIG. 2 (eg, the cellular module 221). The processor 210 may load commands or data received from at least one of the other components (eg, non-volatile memory) into the volatile memory, process the commands, and store the resulting data in the non-volatile memory.

For example, the communication module 220 may have the same or similar configuration as the communication interface 170. The communication module 220 may include, for example, a cellular module 221, a WiFi module 223, a Bluetooth module 225, a GNSS module 227, an NFC module 228, and an RF module 229. there is. For example, the cellular module 221 may provide voice calls, video calls, text services, or Internet services through a communication network. According to one embodiment, the cellular module 221 may use the subscriber identification module (eg, SIM card) 224 to distinguish and authenticate the electronic device 201 within the communication network. According to one embodiment, the cellular module 221 may perform at least some of the functions that the processor 210 can provide. According to one embodiment, the cellular module 221 may include a communication processor (CP). According to some embodiments, at least some (e.g., two or more) of the cellular module 221, WiFi module 223, Bluetooth module 225, GNSS module 227, or NFC module 228 are one integrated chip. (IC) or may be included within an IC package. For example, the RF module 229 may transmit and receive communication signals (eg, RF signals). The RF module 229 may include, for example, a transceiver, a power amp module (PAM), a frequency filter, a low noise amplifier (LNA), or an antenna. According to another embodiment, at least one of the cellular module 221, WiFi module 223, Bluetooth module 225, GNSS module 227, or NFC module 228 transmits and receives RF signals through a separate RF module. You can. Subscriber identity module 224 may include, for example, a card or embedded SIM that includes a subscriber identity module and may include unique identification information (e.g., integrated circuit card identifier (ICCID)) or subscriber information (e.g., IMSI). (international mobile subscriber identity)).

The memory 230 (eg, memory 130) may include, for example, an internal memory 232 or an external memory 234. The built-in memory 232 may include, for example, volatile memory (e.g., DRAM, SRAM, or SDRAM, etc.), non-volatile memory (e.g., one time programmable ROM (OTPROM), PROM, EPROM, EEPROM, mask ROM, flash ROM, etc. , may include at least one of a flash memory, a hard drive, or a solid state drive (SSD). The external memory 234 may include a flash drive, for example, compact flash (CF), or secure digital (SD). ), Micro-SD, Mini-SD, xD (extreme digital), MMC (multi-media card), or memory stick, etc. The external memory 234 is functionally connected to the electronic device 201 through various interfaces. It can be connected physically or physically.

For example, the sensor module 240 may measure a physical quantity or sense the operating state of the electronic device 201 and convert the measured or sensed information into an electrical signal. The sensor module 240 includes, for example, a gesture sensor 240A, a gyro sensor 240B, an atmospheric pressure sensor 240C, a magnetic sensor 240D, an acceleration sensor 240E, a grip sensor 240F, and a proximity sensor ( 240G), color sensor (240H) (e.g. RGB (red, green, blue) sensor), biometric sensor (240I), temperature/humidity sensor (240J), illuminance sensor (240K), or UV (ultra violet) ) It may include at least one of the sensors 240M. Additionally or alternatively, the sensor module 240 may include, for example, an olfactory (e-nose) sensor, an electromyography (EMG) sensor, an electroencephalogram (EEG) sensor, an electrocardiogram (ECG) sensor, It may include an infrared (IR) sensor, an iris sensor, and/or a fingerprint sensor. The sensor module 240 may further include a control circuit for controlling at least one sensor included therein. In some embodiments, the electronic device 201 further includes a processor configured to control the sensor module 240, either as part of the processor 210 or separately, while the processor 210 is in a sleep state, The sensor module 240 can be controlled.

The input device 250 may include, for example, a touch panel 252, a (digital) pen sensor 254, a key 256, or an ultrasonic input device 258. The touch panel 252 may use at least one of, for example, a capacitive type, a resistive type, an infrared type, or an ultrasonic type. Additionally, the touch panel 252 may further include a control circuit. The touch panel 252 may further include a tactile layer to provide a tactile response to the user. The (digital) pen sensor 254 may be, for example, part of a touch panel or may include a separate recognition sheet. Keys 256 may include, for example, physical buttons, optical keys, or keypads. The ultrasonic input device 258 can detect ultrasonic waves generated from an input tool through a microphone (e.g., microphone 288) and check data corresponding to the detected ultrasonic waves.

Display 260 (e.g., display 160) may include a panel 262, a holographic device 264, a projector 266, and/or control circuitry for controlling them. Panel 262 may be implemented as flexible, transparent, or wearable, for example. The panel 262 may be composed of a touch panel 252 and one or more modules. The hologram device 264 can display a three-dimensional image in the air using light interference. The projector 266 can display an image by projecting light onto the screen. The screen may be located, for example, inside or outside the electronic device 201. The interface 270 may include, for example, HDMI 272, USB 274, optical interface 276, or D-subminiature (D-sub) 278. Interface 270 may be included in, for example, communication interface 170 shown in FIG. 1 . Additionally or alternatively, the interface 270 may include, for example, a mobile high-definition link (MHL) interface, a SD card/multi-media card (MMC) interface, or an infrared data association (IrDA) standard interface. there is.

The audio module 280 can, for example, convert sound and electrical signals in two directions. At least some components of the audio module 280 may be included in, for example, the input/output interface 145 shown in FIG. 1 . The audio module 280 may process sound information input or output through, for example, a speaker 282, a receiver 284, an earphone 286, or a microphone 288. The camera module 291 is, for example, a device capable of shooting still images and moving images, and according to one embodiment, it includes one or more image sensors (e.g., a front sensor or a rear sensor), a lens, and an image signal processor (ISP). , or may include a flash (e.g., LED or xenon lamp, etc.). The power management module 295 may manage power of the electronic device 201, for example. According to one embodiment, the power management module 295 may include a power management integrated circuit (PMIC), a charging IC, or a battery or fuel gauge. PMIC may have wired and/or wireless charging methods. The wireless charging method includes, for example, a magnetic resonance method, a magnetic induction method, or an electromagnetic wave method, and may further include an additional circuit for wireless charging, for example, a coil loop, a resonance circuit, or a rectifier. there is. The battery gauge may measure, for example, the remaining amount of the battery 296, voltage, current, or temperature during charging. Battery 296 may include, for example, rechargeable cells and/or solar cells.

The indicator 297 may display a specific state of the electronic device 201 or a part thereof (eg, the processor 210), such as a booting state, a message state, or a charging state. The motor 298 can convert electrical signals into mechanical vibration and generate vibration or haptic effects. The electronic device 201 is, for example, a mobile TV support device capable of processing media data according to standards such as digital multimedia broadcasting (DMB), digital video broadcasting (DVB), or mediaFloTM (e.g., GPU) may be included. Each of the components described in this document may be composed of one or more components, and the names of the components may vary depending on the type of electronic device. In various embodiments, an electronic device (e.g., the electronic device 201) omits some components, further includes additional components, or combines some of the components to form a single entity. The functions of the corresponding components before combination can be performed identically.

3 is a block diagram of a program module according to various embodiments. According to one embodiment, the program module 310 (e.g., program 140) is an operating system that controls resources related to an electronic device (e.g., electronic device 101) and/or various applications running on the operating system ( Example: may include an application program (147). The operating system may include, for example, AndroidTM, iOSTM, WindowsTM, SymbianTM, TizenTM, or BadaTM. Referring to Figure 3, the program module 310 includes a kernel 320 (e.g. kernel 141), middleware 330 (e.g. middleware 143), (API 360) (e.g. API 145) ), and/or an application 370 (e.g., an application program 147). At least a portion of the program module 310 is preloaded on an electronic device or an external electronic device (e.g., an electronic device ( 102, 104), server 106, etc.).

The kernel 320 may include, for example, a system resource manager 321 and/or a device driver 323. The system resource manager 321 may control, allocate, or retrieve system resources. According to one embodiment, the system resource manager 321 may include a process management unit, a memory management unit, or a file system management unit. The device driver 323 may include, for example, a display driver, a camera driver, a Bluetooth driver, a shared memory driver, a USB driver, a keypad driver, a WiFi driver, an audio driver, or an inter-process communication (IPC) driver. . For example, the middleware 330 provides functions commonly required by the application 370 or provides various functions through the API 360 so that the application 370 can use limited system resources inside the electronic device. It can be provided as an application (370). According to one embodiment, the middleware 330 includes a runtime library 335, an application manager 341, a window manager 342, a multimedia manager 343, a resource manager 344, a power manager 345, and a database manager ( 346), a package manager 347, a connectivity manager 348, a notification manager 349, a location manager 350, a graphics manager 351, or a security manager 352.

The runtime library 335 may include, for example, a library module used by a compiler to add new functions through a programming language while the application 370 is running. The runtime library 335 may perform input/output management, memory management, or arithmetic function processing. The application manager 341 may, for example, manage the life cycle of the application 370. The window manager 342 can manage GUI resources used on the screen. The multimedia manager 343 can determine the format required to play media files and encode or decode the media files using a codec suitable for the format. The resource manager 344 may manage the source code or memory space of the application 370. The power manager 345 may, for example, manage battery capacity or power and provide power information necessary for the operation of the electronic device. According to one embodiment, the power manager 345 may be linked to a basic input/output system (BIOS). The database manager 346 may create, search, or change a database to be used in the application 370, for example. The package manager 347 can manage the installation or update of applications distributed in the form of package files.

Connectivity manager 348 may manage wireless connections, for example. The notification manager 349 may provide events such as arrival messages, appointments, and proximity notifications to the user, for example. The location manager 350 may, for example, manage location information of the electronic device. The graphics manager 351 may, for example, manage graphic effects to be provided to users or user interfaces related thereto. Security manager 352 may provide, for example, system security or user authentication. According to one embodiment, the middleware 330 may include a telephony manager for managing the voice or video call function of the electronic device or a middleware module that can form a combination of the functions of the above-described components. . According to one embodiment, the middleware 330 may provide specialized modules for each type of operating system. The middleware 330 may dynamically delete some existing components or add new components. The API 360 is, for example, a set of API programming functions and may be provided in different configurations depending on the operating system. For example, in the case of Android or iOS, one API set can be provided for each platform, and in the case of Tizen, two or more API sets can be provided for each platform.

The application 370 includes, for example, home 371, dialer 372, SMS/MMS (373), instant message (IM) 374, browser 375, camera 376, and alarm 377. , contact (378), voice dial (379), email (380), calendar (381), media player (382), album (383), watch (384), health care (e.g., measuring the amount of exercise or blood sugar, etc.) , or may include an application that provides environmental information (e.g., atmospheric pressure, humidity, or temperature information). According to one embodiment, the application 370 may include an information exchange application that can support information exchange between an electronic device and an external electronic device. The information exchange application may include, for example, a notification relay application for delivering specific information to an external electronic device, or a device management application for managing the external electronic device. For example, a notification delivery application may deliver notification information generated by another application of an electronic device to an external electronic device, or may receive notification information from an external electronic device and provide it to the user. A device management application may, for example, control the functions of an external electronic device that communicates with the electronic device, such as turning on/off the external electronic device itself (or some of its components) or the brightness (or resolution) of the display. control), or you can install, delete, or update applications running on an external electronic device. According to one embodiment, the application 370 may include an application designated according to the properties of the external electronic device (eg, a health management application for a mobile medical device). According to one embodiment, the application 370 may include an application received from an external electronic device. At least a portion of the program module 310 may be implemented (e.g., executed) in software, firmware, hardware (e.g., processor 210), or a combination of at least two of these, and may be a module for performing one or more functions; May contain programs, routines, instruction sets, or processes.

The term “module” used in this document includes a unit comprised of hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A “module” may be an integrated part, a minimum unit that performs one or more functions, or a part thereof. A “module” may be implemented mechanically or electronically, for example, in application-specific integrated circuit (ASIC) chips, field-programmable gate arrays (FPGAs), known or hereafter developed, that perform certain operations. May include programmable logic devices.

At least a portion of the device (e.g., modules or functions) or method (e.g., operations) according to various embodiments is stored in a computer-readable storage medium (e.g., memory 130) in the form of a program module. It can be implemented with commands. When the instruction is executed by a processor (eg, processor 120), the processor may perform the function corresponding to the instruction. Computer-readable recording media include hard disks, floppy disks, magnetic media (e.g. magnetic tape), optical recording media (e.g. CD-ROM, DVD, magneto-optical media (e.g. floptical disks), built-in memory, etc. An instruction may include code generated by a compiler or code that can be executed by an interpreter.

A module or program module according to various embodiments may include at least one of the above-described components, some of them may be omitted, or may further include other components. According to various embodiments, operations performed by a module, program module, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be performed sequentially, in parallel, iteratively, or heuristically. can be added

FIG. 4 is a diagram illustrating the configuration of an electronic device according to various embodiments of the present invention.

Referring to FIG. 4, the electronic device 400 (e.g., the electronic device 101 of FIG. 1) includes at least one application (e.g., a first application 404 and a second application 405) and a plurality of applications. Layers (e.g., first layer 401, second layer 402, and third layer 403, etc.), window compositor 410, display controller 420, and display panel ( 430) may be included. According to various embodiments of the present invention, the first layer 401 may be included in the first application 404 or may be created corresponding to the first application 404, and the second layer 402 and the third The layer 403 may be included in the second application 405 or may be created corresponding to the second application 405. According to one embodiment, the layers (e.g., the first layer 401 to the third layer 403) are external to the application (e.g., the first application 404 and the second application 405) (e.g., It may also be included in the window compositor 410). In addition, the window compositor 410 includes a buffer management module 412, a layer manager module 414, a composition module 416, and a graphic buffer management module. module) (418). According to various embodiments of the present invention, the plurality of applications 404 and 405, the plurality of layers 401, 402, and 403, the window compositor 410, the display controller 420, and the display panel 430 It may be operated by a processor (eg, processor 120 of FIG. 1).

According to various embodiments of the present invention, depending on the type of framework (or operating system), the window compositor 410 may be used as a term to mean the composition module 416. For example, buffer management module 412, or graphics buffer management module 418 may be placed external to window compositor 410. According to various embodiments, the plurality of modules 410, 412, 414, 416, and 418 may be implemented by being integrated or divided into one or more modules. For example, the composition module 416 and the layer management module 418 may be implemented as one module (eg, surface flinger).

When an application (e.g., the first application 404 or the second application 405) is executed by an electronic device, the application creates a screen for drawing the application image on the framework (e.g., the window compositor 410). You can request some areas. The framework (eg, window compositor 410) may allocate a partial area of the screen in response to requests from each application and manage information (eg, layer) about the allocated area.

A layer is information that manages the screen area (e.g. layer area or surface area) corresponding to the application. For example, a layer is an object or instance corresponding to a specified data structure and can be created and deleted by application request. In an embodiment, a layer may include the address of a graphics buffer from which an application will draw an image (or graphics data), and frame area (position occupied by the layer in the entire area of the frame buffer) information. The term layer used in this specification is not limited to a specific term, and other terms (e.g., surface, window, etc.) may be used instead of layer. FIG. 4 shows, as an example, a first layer 401 created corresponding to a first application, and a second layer 402 and a third layer 403 created corresponding to a second application. In FIG. 4, each layer is placed inside the applications, but according to various embodiments, the actual layer may be located in another module, such as the window compositor 410.

The layer management module 418 (eg, a processor) may manage at least one layer created by applications. For example, the layer management module 418 can create or delete layers according to requests from applications and manage at least one layer as a list.

According to one embodiment, when an application is executed, the layer management module 418 (eg, the processor) may create at least one layer corresponding to the application. According to one embodiment, when the application changes the screen configuration, the layer management module 418 may change the layer information in response to the change in the screen configuration. For example, in the case of a map application, the layer management module 418 can create a layer that provides map data and a menu layer used in the map application. Additionally, the layer management module 418 can create a status bar layer displayed on the display panel 430 together with the map application.

According to various embodiments of the present invention, the application may include not only an application executed through an application icon, but also a home screen application (or a launcher application) including an application icon. During execution of the home screen application, the layer management module 418 may create a launcher layer, a wallpaper layer, and a status bar layer. Additionally, when a notification occurs during application execution, the layer management module 418 may further create a layer for the notification.

According to one embodiment, the buffer management module 412 can create and manage a graphics buffer. An application or layer may request (dequeue) a buffer from the buffer management module 412. The buffer management module 412 can create (or allocate) a graphic buffer to an application or layer for which a buffer has been requested. The graphic buffer may be allocated to the buffer management module 412 from the kernel (eg, kernel 320 in FIG. 3). The graphics buffer may be allocated for each layer. For example, the graphic buffer corresponding to the second layer 402 of the second application 405 and the graphic buffer corresponding to the third layer 403 of the second application 405 may be allocated separately. The buffer management module 412 may allocate a plurality of graphic buffers to one layer according to multiple buffering (eg, double buffer, triple buffer) method. And depending on the vsync cycle, for example, in a flip method, multiple buffers allocated to one layer can be used alternately.

Applications 404 and 405 may draw images in buffers 401, 402, and 403 allocated to the layer. Drawing, for example, may be an operation in which an image (or graphic data) is transferred (or written) to a buffer. The image may be an image that the application wishes to provide through the display panel 430. According to one embodiment, the image may be drawn in the graphics buffer according to the coordinates of the area occupied by the layer among the entire frame area. For example, if the frame area of a specific layer has coordinates of x start = 0, x end = 800, y start = 25, y end = 1280 based on a resolution of 800 * 1280, the application You can draw the final image to be displayed at the corresponding coordinates in the buffer. The application (or layer) may transmit information (e.g., coordinate information) about the graphic buffer into which the image is drawn to the buffer management module 412. For example, an application may request output from the buffer management module 412 for a graphics buffer on which drawing has been completed. The buffer management module 412 may manage the graphic buffer in a synthesis standby state in response to the graphic buffer output request.

When the frame rate is 60Hz, the compositing module 416 is triggered by vsync generated every 16.6ms, and can check whether there is a graphics buffer in the buffer management module 412 waiting for compositing. If there is no graphics buffer in a compositing standby state, the compositing module 416 enters a sleep state until the next vsync, and if there is a graphics buffer in a compositing standby state, compositing can be performed. The compositing module 416 may retrieve a graphics buffer in a compositing standby state.

According to one embodiment, the compositing module 416 may perform compositing using the graphics library 440. The graphics library may include, for example, OpenGL ES, Direct3D, and Stage3D. For example, a graphics library may use the GPU to perform compositing.

According to various embodiments of the present invention, the composition module 416 may manage a layer list array containing information about the executing layer (or executing application). For example, when executing an application (or layer), the application may transmit information such as frame section, source crop, and pixel format to the compositing module 416. Additionally, if information such as frame section, source crop, or pixel format changes during execution, the application may notify the composition module 416 of this.

The synthesis module 416 may synthesize graphic data using a first synthesis type or a second synthesis type. The first synthesis type may be a type synthesized in software. The first compositing type can perform compositing using a graphics library. Additionally, the first synthesis type may perform synthesis using a GPU. For example, the first composite type could be an open GL (GLES) type. In the following description, the first composition type will be described interchangeably with the GLES type. The second composition type may be a hardware composition (HWC) type. The second synthesis type can be synthesized using the blender 424 included in the display controller 420. In the following description, the second composition type will be described interchangeably with the HWC type.

The graphic buffers of the layers of the first synthesis type (e.g., GLES type) are composited into one frame buffer, and information (information such as memory start address, width, height, stride, etc.) of the composite frame buffer is stored in a display controller ( It can be transmitted to the first plane 421 of 420). For example, the composition module 416 may transmit information about a frame buffer for which composition has been completed to the display controller 420 through the kernel.

If it is the second synthesis type (HWC type), the synthesis module 416 can hold the display controller 420 to synthesize graphic data. Since the first composition type has two objects (for example, the application 400 and the window compositor 410), the graphic buffer can be configured as a double buffer. However, since the second composition type has three objects (e.g., the application 400, the window compositor 410, and the display controller 420), the graphics buffer may require a triple buffer. The composition module 416 may synthesize graphic data based on the composition type or hold the graphic data so that the display controller 420 can perform hardware composition.

The graphic buffer management module 418 dynamically monitors the number of objects owning the graphic buffer, the volatility of the time each object owns the graphic buffer, and/or whether the composition type has changed, etc. to manage the graphic buffer. The number can be managed to an appropriate number.

According to one embodiment, the graphics buffer management module 418 may check the currently used composition type from the window compositor 410. The graphic buffer management module 418 may set the graphic buffer to a triple buffer if the composition type is a second composition type (eg, HWC type, the number of objects that can access the graphic buffer is 3 or more). The graphic buffer management module 418 can set the graphic buffer to a double buffer if the composition type is the first composition type (e.g., GLES type, the number of objects that can access the graphic buffer is 2 or less). .

According to one embodiment, the graphics buffer management module 418 may dynamically manage the number of graphics buffers based on the variability of the time each object owns the graphics buffer if it is a first composite type. The electronic device can manage the graphic buffers of each application 400 and the window compositor 410 using the buffer management module 412. When each application 400 and the window compositor 410 need to acquire a graphics buffer, they request the buffer management module 412, and when the use of the graphics buffer ends, they can transmit the release of use of the graphics buffer to the buffer management module 412. there is. When the application 400 (e.g., layer) requests graphic buffer allocation (e.g., buffer request (dequeueBuffer)), the buffer management module 412 allocates the graphic buffer and delivers it to the requesting side, and the application 400 generates the graphic buffer. If you reply (e.g. output request (queueBuffer)), the corresponding graphics buffer can be managed in a composition standby state. A buffer request (dequeueBuffer) can be a request to allocate a graphics buffer (for the application to draw) to the application layer. , the output request (queueBuffer) may be passing the graphics buffer drawn by the application to the composition module (e.g., allowing the synthesis module to synthesize).

The graphic buffer management module 418 calculates the time when each object owns a graphic buffer from the information (time when dequeueBuffer and queueBuffer occurred) requested by the buffer management module 412 from each object (App, SurfaceFlinger in the case of Android OS). (time_queueBuffer - time_dequeueBuffer) can be measured. In the case of the first composite type (e.g., a GLES type application with two graphic buffer owning objects), the graphic buffer management module 418 can set the graphic buffer as a double buffer if the graphic buffer ownership time is constant, and the graphic buffer can be set to a double buffer. If the possession time is not constant, the graphics buffer can be applied as a triple buffer.

According to one embodiment, the graphics buffer management module 418 may dynamically adjust the number of graphics buffers when the composition type changes while performing a graphics processing operation. For example, it can be seen that the window compositor 410 changes from the first composition type (GLES type) to the second composition type (HWC type). When the composition type is changed, the graphics buffer management module 418 can change the number of graphics buffers at an appropriate timing based on the changed composition type without deteriorating graphics performance.

According to one embodiment, the graphics buffer management module 428 may dynamically manage the number of graphics buffers based on the possession time of the graphics buffer while processing GLES composite type graphics. If the graphic possession time of an object changes to an irregular state while processing GLES composite type graphics, the graphic buffer management module 418 can manage the graphic buffer by changing it from a double buffer to a triple buffer. If the graphic possession time of an object changes to a constant state while processing GLES composite type graphics, the graphic buffer management module 418 can manage the graphic buffer by changing it from a triple buffer to a double buffer.

According to various embodiments of the present invention, the display controller 420 uses not only double buffering technology using two buffers, but also triple buffering technology using three buffers, or technology using four or more buffers. You can also use .

The first plane 421 can fetch a frame buffer in which images of the first layer 401 and the second layer 402 are combined using direct memory access (DMA). Additionally, similarly, the second plane 422 can fetch the graphics buffer of the third layer 403 through DMA.

The blender 424 can synthesize each frame buffer and graphic buffer fetched from the first plane 421 and the second plane 422, and store the final image resulting from the synthesis in the internal memory 426.

The blender 424 can perform the above synthesis every vsync. For example, the blender 424 combines the frame buffer of the first plane 421 and the graphics buffer of the second plane 422 in the first vsync, and the frame buffer of the first plane 421 and the graphics buffer in the second vsync. The graphics buffer of the second plane 422 can be synthesized.

However, according to various embodiments of the present invention, when the images of the first layer 401 and the second layer 402 are not updated in the second vsync, and only the image of the third layer 403 is updated, the synthesis module 416 may not perform synthesis of the images of the first layer 401 and the second layer 402. Additionally, in this case, the blender 424 recycles the frame buffer (composite image of the first layer 401 and the second layer 402) from the first vsync to display the image of the third layer 403 with the updated image. Graphics buffers and compositing can be performed.

The display controller 420 may transmit the image in the internal memory 426 to the physically connected display panel 430 every vsync. Interfaces that physically connect the display controller 420 and the display panel 430 may include, for example, MIPI-DSI, HDMI, and eDP (embedded display port). The display controller 420 can package the image in the frame buffer according to the protocol of each interface and transmit it to the display panel 430.

Electronic devices according to various embodiments of the present invention may dynamically manage the number of graphics buffers based on composition type and/or graphics buffer possession time while processing an application.

Figure 5 is a block diagram of an electronic device according to various embodiments of the present invention.

Referring to FIG. 5, an application processor (AP) 500 (e.g., processor 120 of FIG. 1) includes a program module 510, a first synthesizer 520, and a second synthesizer 530. can do. The AP 500 is composed of a system on chip (SoC) and can mount the first synthesizer 520 and the second synthesizer 530.

According to various embodiments of the present invention, the first synthesizer 520 may be a graphics processing unit (GPU). Additionally, the second synthesizer 530 may be configured in the display controller. Alternatively, the second synthesizer 530 may be configured as an I/O controller (input/output controller). According to various embodiments of the present invention, the second synthesizer 530 may be implemented on the AP 500 as an independent IP.

Electronic devices according to various embodiments of the present invention can efficiently use memory while preventing performance degradation by dynamically controlling the number of buffers depending on the situation. When processing graphics data, electronic devices according to various embodiments of the present invention can dynamically control the number of graphics buffers for buffering graphics data.

According to one embodiment, the electronic device may dynamically apply the graphics buffer as a double buffer or triple buffer based on the composition type of the application.

According to one embodiment, the electronic device analyzes the possession time of the graphics buffer when processing graphics data based on double buffering, and may switch to a triple buffer if the possession time of the graphics buffer is not constant.

According to one embodiment, the electronic device processes graphics data by allocating buffers based on the composition type, and when the composition type changes while processing graphics data, the number of graphics buffers can be adjusted based on the changed composition type. .

FIG. 6 is a diagram illustrating the basic operation of double buffering for processing graphics data in a graphics system.

Electronic devices may use a double buffering method when processing graphic data. The double buffering method is used in the field of computer science when the consumer 640 reads the data written by the producer 610 to the buffer 620. It can be a buffering method that supports the 610 and the consumer 640 to operate simultaneously. The buffer 620 may include a double buffer of a first buffer 623 and a second buffer 625. For example, when the producer 610 records graphic data in the first buffer (back buffer) 623, as in 600, the consumer 630 can read the data in the second buffer (front buffer) 625. . When the producer 610 finishes writing to the first buffer 623, the consumer 630 can complete reading the data written to the second buffer 625. When the producer 610 completes the recording operation in the first buffer 623, the producer 610 and the consumer 630 own the first buffer 623 and the second buffer 625, as shown at 650. can be exchanged (flip). When the roles of the buffer 620 are flipped, the producer 610 writes data to the second buffer (back buffer) 625, and at the same time, the consumer 630 writes data to the first buffer (front buffer) 623. You can read the data recorded in .

Figure 7 is a diagram explaining the operation of double buffering between an application and a window compositor.

Referring to FIG. 7, in the graphics system of a general OS, a plurality of applications and a windows compositor can process graphics data while executing a double buffering operation. A plurality of applications 710 and 715 may render graphic images expressed as one layer or multiple layers. In FIG. 7, application A 710 and application B 715 can express a graphic image in one layer and can each be operated as a producer.

At 700 in FIG. 7, the A application 710 writes graphic data to the back buffer 721, and the window compositor 730 can read the graphic data written to the buffer (front buffer) 723. . Additionally, at 700 in FIG. 7, the B application 715 writes graphic data to the back buffer 725, and the window compositor 730 can read the graphic data written to the buffer (front buffer) 723. there is. Graphic data recorded in the buffer 723 and buffer 727, which are front buffers of the window compositor 730, can be read and composed.

When the A application 710 and the B application 715 complete the task of recording data in the buffer 721 and the buffer 725, respectively, as shown at 750 in FIG. 7, the A application 710 and the B application ( 715) and the window compositor 730 can flip the roles of the buffer 720. When the role of the buffer 720 is flipped, the A application 710 writes graphic data to the back buffer 723, as shown in 750 of FIG. 7, and the B application 715 writes the buffer (back buffer) ( Graphic data can be written to 727, and the window compositor 730 can read graphic data recorded in the buffer 723 and buffer 725. The window compositor 730 can write buffer 721, which is the front buffer. ) and the graphic data recorded in the buffer 725 can be read and composed.

When processing graphic data of the GLES composition type, the window compositor 730 can play the role of combining and displaying all windows overlapping on the screen on one screen. For example, if you have two web browsers overlapping and select one web browser, the selected web browser can come to the front and cover the web browser behind it. Conversely, if you select the web browser in the back, the web browser in the back can come to the front and cover the web browser that was previously in front. The window compositor 730 can use the priorities of all windows to create a result screen as if they were stacked on one screen.

Figure 8 is a diagram illustrating a double buffering operation between a window compositor and a display.

Referring to FIG. 8, in the OS graphics system, the window compositor 810 and the display 830 perform a double buffering operation using the frame buffer 820 consisting of a first buffer 823 and a second buffer 825. can do. The window compositor 810 may be the window compositor 730 of FIG. 7 . As shown at 800 in FIG. 8, when the window compositor 810 writes data to the first frame buffer (back buffer) 823, the display 830 writes data to the second frame buffer (front buffer) 820. You can read recorded data. After performing the buffer flip operation, as shown at 850 in FIG. 8, data is recorded in the second frame buffer (back buffer) 825 of the window compositor 810, and the display 830 records the first frame buffer (back buffer) 825. The data recorded in the front buffer (823) can be read. As shown at 800 and 850 in FIG. 8, the window compositor 810 and the display 830 can access the frame buffer 820 using a double buffering method. At this time, the window compositor 810 performs a consumer function for the graphics buffer (e.g., the graphics buffer 720 in FIG. 7) as shown in FIG. 7, but in FIG. 8, the window compositor 810 performs a consumer function for the frame buffer 820 in FIG. Can perform producer function.

In the following description, the compositing module (e.g., the compositing module 416 in FIG. 4) of the window compositor (e.g., the window compositor 410 in FIG. 4) will be explained by taking SurfaceFlinger as an example. . SurfaceFlinger can perform the function of combining multiple layers (or windows or surfaces) and outputting them on a single screen (for example, writing to a frame buffer). For example, Surface Flinger can perform the function of combining the first surface of an image and the second surface of other content (e.g. GUI) into one screen.

The compositing module of the Windows compositor (eg, Surface Flinger) can compose graphic data using a first compositing type or a second compositing type. The first synthesis type may be a type synthesized in software. The first compositing type can perform compositing using a graphics library. Additionally, the first synthesis type may perform synthesis using a GPU. The second composition type may be a hardware composition (HWC) type. The second synthesis type can be synthesized using the blender 424 included in the display controller 420.

In the following description, the first composition type will be explained using the open GL (GLES) type as an example, and the second composition type will be explained using the HWC type as an example.

Figure 9 is a diagram showing double buffering operation in the Android OS system.

Referring to FIG. 9, the application 910 and SurfaceFlinger 930 can access graphic data by performing a double buffering operation. The surface linger 930 can read data written to the graphics buffer 920 in at least one application. The surface linger 930 can write data to the frame buffer 940, and the display 950 can read data written to the frame buffer 940.

As shown in FIG. 9, when the electronic device is a GLES composition type, the graphic buffer 920 and the frame buffer 940 can be operated using the double buffering method. As shown at 900 in FIG. 9, the first graphic buffer and the first frame buffer can be operated as a back buffer, and the second graphic buffer and the second frame buffer can be operated as a front buffer. . And when the graphics data is written to the back buffer and the read operation of the graphics data written to the front buffer is completed, the electronic device flips the roles of the graphics buffer 920 and the frame buffer 950 as shown at 960 in FIG. 9. You can do it. Then, the second graphic buffer and the second frame buffer can be operated as a back buffer, and the first graphic buffer and the first frame buffer can be operated as a front buffer.

In FIG. 9, the surface linger 930 may operate as a consumer that reads data written to the graphic buffer 920, and may operate as a producer that records data to the frame buffer 950.

FIGS. 10A to 10C are diagrams for explaining the operation of a graphics buffer consisting of a triple buffer in an electronic device. In Android OS, the graphics buffer can be used using double buffering or triple buffering methods. For example, Android OS used the graphics buffer as a double buffering method until version 4.0, and triple buffering can be used starting from version 4.1 (Jellybean version applies Project Butter). If there are two objects (App, SurfaceFlinger) that own the graphic buffer, it can be effective to implement the graphic buffer as a double buffer. When HWC type synthesis is supported, there are three objects that own the graphic buffer (App, SurfaceFlinger, Display panel), so it can be efficient for electronic devices to configure the graphic buffer as a triple buffer.

Referring to FIG. 10A, in the first cycle (e.g., Vsync 1), the application 1010 draws (draws, writes) graphics data (surface, layer) in the first graphics buffer 1043 and The ringer 1020 can hold the graphic data written in the second graphic buffer 1045, and the display 1030 reads the graphic data written in the third graphic buffer 1047 and processes it (for example, For example, blending) and display. At this time, the surface linger 1020 can transmit the meta information of the graphic data recorded in the second graphic buffer 1045 to the display 1030. The display 1030 may include a display controller and a display panel.

Referring to FIG. 10B, in the second cycle (e.g., Vsync 2), the application 1010 records graphics data in the second graphics buffer 1045, and the surface linger 1020 records graphics data in the third graphics buffer (1045). The graphic data recorded in 1047 can be held, and the display 1030 can read, process, and display the graphic data recorded in the first graphic buffer 1043. At this time, the surface linger 1020 can transmit the meta information of the graphic data recorded in the third graphic buffer 1047 to the display 1030.

Referring to FIG. 10C, in the third cycle (e.g., Vsync 3), the application 1010 records graphics data in the third graphics buffer 1047, and the surface linger 1020 records graphics data in the first graphics buffer (1047). The graphic data recorded in the second graphic buffer 1043 can be held, and the display 1030 can read, process, and display the graphic data recorded in the second graphic buffer 1045. At this time, the surface linger 1020 can transmit the meta information of the graphic data recorded in the first graphic buffer 1041 to the display 1030.

Figure 11 is a diagram to explain the delay time as the graphics buffer increases. The surface flinger in FIG. 11 may be an example of the compositing module 416 of FIG. 4. In the following description, the synthesis module will be explained as an example of a surface flinger.

Referring to FIG. 11, the graphics buffer is configured as a double buffer (e.g., GB1, GB2), the application records graphics data in the graphics buffer as in 1110, and the Surface Flinger records the graphics recorded in the graphics buffer as in 1120. You can lead the data. In this case, it takes a delay time (1115) until the graphics buffer GB1, where the application recorded graphics data, is read by the surface flinger. The longer this delay time (1115) is, the more the input lag increases. However, if the graphic buffer is configured as a triple buffer, the application can record graphic data in the graphic buffer, such as 1150, and the Surface Flinger can read the graphic data written in the graphic buffer, such as 1160. In this case, it takes a delay time (1155) until the graphics buffer GB3, where the application recorded graphics data, is read by the Surface Slinger, and the delay time is longer than the delay time (1115) that occurs when using a double buffer. You can see that input lag increases.

Therefore, when the electronic device increases the number of graphics buffers, memory shortage and input lag may increase. Electronic devices using the Android OS may be transformed into an environment in which high-resolution data and various applications are executed simultaneously, and as a result, the space occupied by the graphics buffer in memory may gradually increase. For example, when running the web on a Galaxy Tab S (WQXGA 2560x1600), the electronic device may allocate a total of 270MB of memory to the graphics buffer (occupying about 9% of 3GB RAM). In this case, the electronic device may run out of memory, causing performance deterioration. Electronic devices according to various embodiments of the present invention can be dynamically set so that each application can use the minimum number of graphics buffers within a range that does not degrade graphics performance.

The electronic device can dynamically allocate a graphics buffer when a surface (surface or layer) is created. An application can request the use of a graphics buffer (buffer request, dequeueBuffer) and output request (queueBuffer). When a request for a graphics buffer occurs in an application, the window compositor can dynamically set the graphics buffer based on the composition type and the volatility of buffer usage time.

FIG. 12 is a diagram illustrating the configuration of an electronic device using the Android OS according to various embodiments of the present invention. The surface flinger in FIG. 12 may be an example of the compositing module 416 of FIG. 4. In the following description, the synthesis module will be explained as an example of a surface flinger.

Referring to FIG. 12 , the electronic device may include an application 1210, a window compositor 1220, and a display controller 1230. The window compositor 1220 may include a service flinger 1223, a buffer management module 1225, and a graphics buffer management module 1227. According to various embodiments of the present invention, the application 1210, the window compositor 1220, and the display controller 1230 may be operated by a processor (eg, the processor 120 of FIG. 1).

The application 1210 may be a plurality of applications, and each application may create at least one layer (layer, surface). The application 1210 can write (write, draw) pixel data to the graphics buffer of the created layer.

The window compositor 1220 can calculate the number of graphic buffers by dynamically monitoring the number of objects owning graphic buffers, the volatility of the time each object owns a graphic buffer, or a change in composition type. The window compositor 1220 may hold data recorded in the graphic buffer based on the composition type, or read the data and record it in a composition standby state in the frame buffer.

Looking at the operation of the window compositor 1220, the buffer management module 1225 can create and manage a graphics buffer. An application (or layer) can request a graphic buffer (dequeueBuffer) from the buffer management module 1225, and the buffer management module 1225 can allocate a graphic buffer to the corresponding application (or layer). The application (or layer) can make an output request (queueBuffer) to the buffer management module 1225 for a graphic buffer for which drawing has been completed, and the buffer management module 1225 can manage the graphic buffer requested for output in a synthesis standby state. there is. For example, the buffer management module 1225 may be the buffer management module 412 of FIG. 4.

Surface Flinger 1223 can manage all graphic information including composition types of all running applications 1210. The surface linger 1223 may have a configuration including the synthesis module 416 of FIG. 4 . If the surface flinger 1223 needs to acquire a graphics buffer, it can request the buffer management module 1225 and transfer the used graphics buffer back to the buffer management module 1225.

The graphics buffer management module 1227 may calculate the number of graphics buffers based on the composite type transmitted from the surface linger 1223 and/or the dequeueBuffer and queueBuffer of the buffer management module 1225.

In one embodiment, the graphics buffer management module 1227 may calculate the number of graphics buffers based on composition type information transmitted from the surface linger 1223. The composition type information transmitted from the surface flinger 1223 may be the first composition type or the second composition type. For example, the first synthesis type can be a GLES type, and the second synthesis type can be a HWC type. As shown in FIG. 9, the GRES type can have two or fewer objects (application, surfaceflinger). As shown in FIG. 10, the HWC type may have three or more objects (application, surfaceflinger, display controller). As shown in Table 1 below, the graphic buffer management module 1227 can manage to apply a triple buffer if the composition type is the HWC type, and to apply a double buffer if the composition type is the GLES type.

Composition type number of objects Graphic buffer policy GLES type 2 Double buffer applies to HWC type 3 Triple buffer application target

In one embodiment, the graphics buffer management module 1227 may measure the volatility of the time each object owns the graphics buffer based on the dequeueBuffer and queueBuffer passed from the buffer management module 1225. When the graphic buffer management module 1227 allocates a double buffer to process graphic data (combining graphic data in the GLES type), the graphic buffer of the object is stored using the times of the dequeueBuffer and queueBuffer as shown in Equation 1 below. Possession time can be measured. The graphic buffer management module 1227 can manage to apply a double buffer when the graphic buffer ownership time of an object is constant, and to apply a triple buffer when the graphic buffer ownership time of an object is not constant.

Electronic devices according to various embodiments of the present invention can adjust the number of graphics buffers according to the number of objects owned. In the Android OS, the electronic device can know the number of objects that own the corresponding graphic buffer as shown in <Table 1> above based on the composition type. The surface flinger 1223 of the window compositor 1220 can manage all graphic information including the composition types of all running applications 1210. And the graphics buffer management module 1227 can obtain composition type information from the surface flinger 1223. The graphics buffer management module 1227 can manage the number of graphics buffers based on the number of objects. For example, in the case of the HWC type, there are 3 objects that own the graphic buffer, so the graphic buffer management module 1227 can apply the graphic buffer as a triple buffer, and in the case of the GLES type, there are 2 objects that own the graphic buffer. Therefore, the graphic buffer management module can manage the graphic buffer as a candidate for application of the double buffer.

If the graphic buffer management module 1227 is of the GLES type, it can monitor the graphic buffer possession time of each object and manage the number of graphics buffers as double buffers or triple buffers. The electronic device can manage the graphic buffer of each application 1210 and the surface linger 1223 using the buffer management module 1225. When each application 1210 or surface linger 1223 needs to obtain a graphic buffer, it requests (dequeueBuffer) the buffer management module 1225, and transfers the used graphic buffer back to the buffer management module 1225 ( queueBuffer). The graphic buffer management module 1227 remembers the time each object owns the graphic buffer based on the request (dequeueBuffer, queueBuffer) of each object (application 1210, surface flinger 1225). It can be measured as shown in <Equation 1>.

If the electronic device is a GLES type with two graphic buffer-owning objects, a double buffer can be applied by default. However, even when there are two owning objects, if the time to own the graphics buffer is not constant, graphics performance may deteriorate due to frame drop. When the electronic device according to various embodiments of the present invention is a GLES type with two graphic buffer-owning objects, the graphic buffer can be applied as a double buffer if the graphic buffer ownership time of the object is constant, and the graphic buffer ownership time of the object is constant. If it is not constant, the graphics buffer can be applied as a triple buffer.

FIG. 13 is a diagram illustrating an example operation in which the possession time of a graphic buffer of an application is constant in each frame in an electronic device according to various embodiments of the present invention. The surface flinger in Figure 13 may be an example of the compositing module 416 of Figure 4. In the following description, the synthesis module will be explained as an example of a surface flinger.

In Figure 13, numbers 1310-1330 show examples of the operation of an application, Surface Flinger, and a display when the graphic buffer is applied as a double buffer, and 1360-1380 show an example of the operation of the Surface Flinger and a display when the graphic buffer is applied as a triple buffer. An example of operation of the ringer and display is shown.

Referring to FIG. 13, when applying the graphic buffer as a double buffer, the application can record graphic data in the graphic buffer as shown at 1310 in FIG. 13, and the Surface Flinger records the graphic data recorded in the graphic buffer as 1320. You can lead. When graphic data is recorded in a graphics buffer as shown in 1310, frame drops such as 1343 and 1345 in FIG. 13 may occur in the graphic data displayed on the display as 1330 in FIG. 13.

When applying the graphic buffer as a triple buffer, the application can write data to the graphic buffer as shown at 1350 in FIG. 13, and Surface Flinger can read data written to the graphic buffer as shown at 1370. When data is recorded in a graphics buffer, such as 1360, frame drops such as 1393 and 1395 may occur in graphic data displayed on the display, such as 1380 in FIG. 13.

As shown in Figure 13, when the graphic buffer possession time of the application and the surface flinger is constant, it can be seen that the same frame drop occurs even if the graphic buffer is applied as a double buffer or a triple buffer.

FIG. 14 is a diagram illustrating another operation in an electronic device according to various embodiments of the present invention, where the possession time of the graphic buffer of the Surface Flinger is constant in each frame. In Figure 14, numbers 1410-1430 show examples of the operation of an application, Surface Flinger, and display when the graphic buffer is applied as a double buffer, and 1460-1480 show examples of the operation of the application, Surface Plus, when the graphic buffer is applied as a triple buffer. An example of operation of the ringer and display is shown. The surface flinger in Figure 14 may be an example of the compositing module 416 of Figure 4. In the following description, the synthesis module will be explained as an example of a surface flinger.

Referring to FIG. 14, when applying the graphic buffer as a double buffer, the application can record graphic data in the graphic buffer as shown at 1410 in FIG. 14, and the Surface Flinger records the graphic data recorded in the graphic buffer as 1420. You can lead. 1410 in FIG. 14 shows an example of the application recording graphics data while waiting at 1443, 1445, and 1447 when recording data for each frame to the graphics buffer. When data is recorded in the graphics buffer as shown at 1410 in FIG. 14, frame drops such as 1453, 1455, and 1457 in FIG. 14 may occur in the graphic data displayed on the display as at 1430 in FIG. 14.

When applying the graphic buffer as a triple buffer, the application can write graphic data to the graphic buffer as shown at 1450 in FIG. 14, and Surface Flinger can read the graphic data recorded in the graphic buffer as shown at 1470. At 1460 in FIG. 14, an example is shown where the application records data with waiting times equal to 1494 and 1496 when recording data for each frame to the graphics buffer. When data is recorded in the graphics buffer as shown at 1460 in FIG. 14, frame drops such as 1493, 1495, and 1497 may occur in the graphic data displayed on the display as shown at 1480 in FIG. 14.

As shown in Figure 14, if the application and Surface Flinger have a certain buffer possession time and apply the graphic buffer as a triple buffer, the application requests the next graphic buffer as 1460 in Figure 14 and creates a new buffer (e.g. , GB3) can be allocated. However, even if the application records data in the graphics buffer as shown in 1460 of FIG. 14, the graphic data displayed on the display may have the same frame drop as when applying a double buffer as shown in 1480 of FIG. 14.

Figures 13 and 14 may be diagrams illustrating the operation when the composition type is the GLES type and the graphic buffer possession time of the application and the surface flinger is kept constant in each frame. If the possession time of the graphic buffer of an object that can access the graphic buffer is constant, frame drops cannot be improved even if the graphic buffer is changed from a double buffer to a triple buffer. Therefore, if the composition type is the GLES type and the possession time of the graphics buffer is constant, configuring the graphics buffer as a double buffer can be a way to prevent memory waste and use it efficiently.

FIG. 15 is a diagram illustrating another operation in an example in which the possession time of the graphic buffer of the Surface Flinger is not constant in an electronic device according to various embodiments of the present invention. In Figure 15, 1510-1530 show an example of the operation of the Surface Flinger and display when the graphic buffer is applied as a double buffer, and 1560-1580 show an application when the graphic buffer is applied as a triple buffer, Surface Play. An example of operation of the ringer and display is shown.

Referring to FIG. 15, as shown at 1510 in FIG. 15, the time an application holds a graphics buffer for recording data may vary. For example, at 1510 of FIG. 15, the application owns the first buffer's possession time (e.g., writing time to graphics buffer GB1) and the second buffer's possession time (e.g., writing time to graphics buffer GB2). This may be different. If the application records data in the graphic buffer as in 1510, Surface Flinger can read the data written in the graphic buffer as in 1520, and the graphic data displayed on the display as 1530 in FIG. 15 is 1543 and 1543 in FIG. 15. Frame drops such as 1545 may occur.

If the composition type is the GLES type and a triple buffer is applied, the application can record data in the graphics buffer as shown at 1560 in FIG. 15. The application can record data in the GB1 buffer - GB3 buffer, which is composed of triple buffers, as shown at 1560 in FIG. 15. At this time, the time that the application owns the GB1 buffer - GB3 buffer to record data may be different. However, the Surface Flinger and the display can process data recorded in the graphics buffer, which is a triple buffer, as shown at 1570 and 1580 in FIG. 15, respectively. Then, frame drops that occur as shown at 1545 in FIG. 15 can be improved.

As shown in FIG. 15, if the time that the application and/or the surface lingerer owns the graphics buffer is not constant, the electronic device adds more graphics buffers (e.g., changes the graphics buffer from a double buffer to a triple buffer). configuration) to reduce frame drops.

FIG. 16 is a diagram illustrating an example of operation according to a change in composition type in an electronic device according to various embodiments of the present invention. Figure 16 shows an example of changing the composition type from the GLES type to the HWC type.

Referring to FIG. 16, if the buffer ownership time is constant in the GLES composition type, the application and Surface Flinger can write and read data to the graphics buffer using the double buffering method, as shown at 1610 and 1620 in FIG. 16, respectively. At this time, if the composition type is changed from GLES type to HWC type as shown at 1640 in FIG. 16, Surface Flinger can recognize the change in composition type and change the graphics buffer from double buffer to triple buffer. Then, the application and the surface slinger can write and read data to the graphics buffer to which the triple buffer is applied, as shown at 1610 and 1620 in FIG. 16, respectively, after time point 1640 in FIG. 16. And the display can display data according to the change in composition type, as shown at 1630 in FIG. 16.

If the composite type changes (from GLES type to HWC type or HWC type to GLES type) while processing graphics data, the number of buffer-owning objects may change. As shown in FIG. 16, when initially operating as the GLES type, the electronic device can process graphics data by operating the graphics buffer in a double buffering method. Afterwards, when the GLES type is changed to the HWC type, the electronic device can change the graphics buffer to the triple buffering method. As shown in FIG. 16, if the composite type is changed at time 1640, the time when a new graphics buffer (GB2) is needed may be when the application attempts to record (drawing) the next graphics data after changing to the HWC type. . Therefore, the electronic device (e.g., graphics buffer management module) allocates a new graphics buffer (GB2) (changes the graphics buffer to a triple buffer) at the time when the GLES type is changed to the HWC type (at 1610 in FIG. 16) and waits. You can.

When the composition type changes from the GLES type to the HWC type (for example, after point 1640 in FIG. 16), the display controller reads the graphics data written to the graphics buffers (GB1 - GB3) as shown at 1630 in FIG. 16. It can be synthesized and displayed as a hardware composition. As a result, the application does not need to wait to acquire the graphics buffer, preventing a decrease in graphics performance.

In one embodiment, when the composition type changes from the GLES type to the HWC type while processing graphics data, the electronic device can process the graphics data without performance degradation by changing the graphics buffer to the triple buffering method. According to one embodiment, when the composition type changes from the HWC type to the GLES type while processing graphics data, the electronic device can process the graphics data while efficiently using memory by changing the graphics buffer to the double buffering method.

An electronic device according to various embodiments of the present invention includes a graphics buffer accessed by at least two objects; display; and a processor functionally connected to the graphics buffer and display. The processor may analyze the graphics processing conditions, set the number of graphics buffers, and process graphics data written to the graphics buffer to be displayed on the display.

The processor may set the graphic buffer as a triple buffer if the number of objects owning the graphic buffer is three or more.

The processor may set the graphic buffer as a double buffer if the number of objects owning the graphic buffer is two or less.

The processor monitors the buffer ownership time of the object that owns the graphic buffer, and if the buffer ownership time is not constant, the graphic buffer can be set to a triple buffer.

The processor may calculate the buffer possession time by subtracting the buffer request time from the output request time of the buffer management module.

The processor sets the graphic buffer as a triple buffer if the number of objects owning the graphic buffer is 3 or more, and monitors the buffer possession time of the object owning the graphic buffer if the number of objects is 2 or less. If the buffer ownership time is not constant, the graphic buffer can be set as a triple buffer, and if the buffer ownership time is constant, the graphic buffer can be set as a double buffer.

The processor may include a window compositor. The window compositor is

a buffer management module that allocates the graphic buffer in response to a buffer request of the object and deallocates the graphic buffer in response to an output request; a composition module that manages graphic information including composition types of running applications; and

It may include a graphic buffer management module that adjusts the number of graphic buffers by the possession time of the object calculated based on the composition type of the composition module and the buffer request time and output request time of the buffer management module.

The compositing type may include a first compositing type using a graphics library and a second compositing type using a hardware compositing type on a display controller.

The window compositor may set the graphic buffer to a triple buffer if the composition type is the second composition type.

The window compositor monitors the buffer possession time of the object if the composition type is the first composition type, sets the graphic buffer to a triple buffer if the buffer possession time is not constant, and sets the graphic buffer to a triple buffer if the buffer possession time is constant. The graphic buffer can be set to a double buffer.

The window compositor sets the graphic buffer to a triple buffer if the composition type is the second composition type, and monitors the buffer possession time of the object owning the graphic buffer if the composition type is the first composition type, If the buffer ownership time is not constant, the graphic buffer can be set as a triple buffer, and if the buffer ownership time is constant, the graphic buffer can be set as a double buffer.

FIG. 17 is a flowchart illustrating an operation of processing graphic data in an electronic device according to an embodiment of the present invention.

Referring to FIG. 17 , an electronic device (e.g., the processor 120 of FIG. 1 or the application processor 210 of FIG. 2) may monitor the number of objects owning a graphics buffer in operation 1711. The electronic device can adjust the number of graphics buffers depending on the number of objects that own the graphics buffer. For example, an electronic device using the Android OS can know the number of objects that own a graphic buffer based on the composition type, as shown in Table 1 above. The window compositor of the electronic device can manage all graphic information including the composition type of all running applications. For example, the graphics buffer buffer module of the Windows compositor can obtain composition type information from Surface Flinger.

If the number of objects owning a graphic buffer is three or more, the electronic device may recognize this in operation 1713 and apply the graphic buffer as a triple buffer in operation 1721. If the graphic buffer is applied as a triple buffer, the electronic device can synthesize and display graphic data while writing and reading graphic data to the graphic buffer in the same manner as shown in FIGS. 10A to 14C.

If the number of objects owning a graphics buffer is two or less, the electronic device may recognize this in operation 1713. The electronic device can be a candidate for application as a double buffer if the number of objects is 2 or less (for example, if the composition type is GLES type). If the number of objects owning the graphic buffer is 2 or less, the electronic device The device may monitor the graphics buffer possession time of each object (e.g., an application and a window compositor (e.g., a Surface Flinger)) in operation 1715. The electronic device may use the buffer management module of the window compositor to display each You can manage the graphic buffers of applications and Surface Flinger. If each application or Surface Flinger needs to acquire a graphic buffer, it can request the use of a graphic buffer (dequeueBuffer) to the buffer management module. When finished, the end of use of the graphic buffer (queueBuffer) can be passed back to the buffer management module. Electronic devices (for example, the graphic buffer management module of Windows Composer) have the buffer management module request ( Based on (dequeueBuffer, queueBuffer), the time that each object owns the graphic buffer can be calculated as in <Equation 1> above.

After monitoring the possession time of the graphics buffer, the electronic device may analyze the possession time of the graphics buffer held by the object in operation 1717. If the electronic device is a GLES type application with two graphic buffer objects, the graphic buffer can be basically applied as a double buffer. However, even when there are two owning objects, if the time for which the objects own the graphics buffer is not constant, a frame drop may occur as shown in 1510-1530 of FIG. 15, which may cause graphics performance to deteriorate. Electronic devices according to various embodiments of the present invention prevent frame drops by applying the graphics buffer as a triple buffer, as shown in numbers 1560-1580 of FIG. 15, when the time that an object owns the graphics buffer is not constant in the GLES type. can do.

If the possession time held by the object is constant, the electronic device may apply the double buffering method to the graphics buffer in operation 1717. If the possession time held by the object is not constant, the electronic device may apply the triple buffering method to the graphic buffer in operation 1721.

In one embodiment, the electronic device may set the number of graphics buffers by analyzing whether the possession time of the graphics buffer is less than a certain threshold time. If the buffer possession time is less than the set threshold time, the electronic device may not need to allocate a large number of graphics buffers. The electronic device may monitor the possession time of the graphics buffer in operation 1715, and may compare and analyze the possession time of the graphics buffer with a certain threshold time in operation 1717. If the possession time of the graphics buffer is less than or equal to the threshold time, the electronic device may set the number of graphics buffers to a double buffer in operation 1719. If the possession time of the graphics buffer is more than the threshold time, the electronic device may set the number of graphics buffers to a triple buffer in operation 1721.

In one embodiment, the electronic device may set the number of graphics buffers based on the possession time of the graphics buffers. In this case, operations 1711 and 1713 in FIG. 17 (e.g., setting the number of graphics buffers based on the number of objects owning the graphics buffers) may be omitted. When setting the number of graphics buffers, the electronic device monitors the graphics buffer possession time of an object (e.g., a layer) in operation 1715, and determines in operation 1717 that the graphics buffer ownership time is constant (or the graphics buffer ownership time is set to a certain threshold time). Below) can be analyzed at leisure. If the graphics buffer possession time is constant (or less than or equal to a certain threshold time), the electronic device sets the number of graphics buffers to a double buffer in operation 1719. If the buffer ownership time is not constant (or more than a certain threshold time), the electronic device sets the number of graphics buffers to double buffer in operation 1721. The number of buffers can be set to triple buffer.

For example, if the screen update frequency (graphic buffer update frequency) of the application is high, the electronic device can allocate a triple buffer (for example, when allocating the buffer later), and if the screen update frequency is low, a double buffer can be applied. You can.

In one embodiment, the electronic device may store monitored information (number of primarily used buffers) for each application in a memory (for example, memory 130 in FIG. 1). Thereafter, the electronic device may be used by allocating graphic buffers to the specified number of buffers stored in memory when the corresponding application is running.

FIG. 18 is a flowchart illustrating an operation of an electronic device processing graphic data according to various embodiments of the present invention.

Referring to FIG. 18, the electronic device may process graphic data by executing an application in operation 1811. In one embodiment, when an application is executed, the electronic device analyzes the composition type and applies the graphic buffer as a triple buffer if it is a HWC type, and applies the graphic buffer as a double buffer if it is a GLES type. In one embodiment, when an application is executed, the electronic device may apply a graphics buffer as a triple buffer to stabilize graphics processing operations.

The electronic device can dynamically adjust the number of graphics buffers by monitoring the composition type and possession time of the graphics buffer while performing graphics processing operations. The electronics may analyze the composite type in operation 1813. If the composition type is the HWC type, the electronic device may recognize this in operation 1815 and determine the graphics buffer as a triple buffer in operation 1817.

If the composition type is a GLES type, the electronic device may recognize this in operation 1815 and analyze the graphics buffer possession time of each object in operation 1821. If the possession time of the graphics buffer is constant, the electronic device may set the graphics buffer to a double buffer in operation 1825. If the possession time of the graphics buffer is not constant, the electronic device may set the graphics buffer to a triple buffer in operation 1817.

By setting the number of graphic buffers, each object can write or read data to the graphic buffer and display the data through the display. Additionally, when the composition type or graphics ownership time for processing graphics data changes, the electronic device can dynamically adjust the number of graphics buffers.

In one embodiment, when the composition type is changed, the electronic device may adjust the number of graphics buffers based on the changed composition type. For example, electronics that process graphics data may monitor the composite type in operation 1813. When the composition type is changed from the GLES type to the HWC type, the electronic device recognizes this in operation 1815 and can set the graphics buffer by changing it from a double buffer to a triple buffer in operation 1817. When the composition type is changed from the HWC type to the GLES type, the electronic device may recognize this in operation 1815 and determine that the graphics buffer is subject to double buffer application. Afterwards, the electronic device can monitor the buffer possession time of the object and adjust the number of graphics buffers.

In one embodiment, the electronic device may adjust the number of graphics buffers according to changes in buffer ownership time in the GLES type. For example, an electronic device that processes graphics in the GLES type may monitor buffer possession times of objects in operation 1821. If the buffer ownership time of an object is not constant while using the graphics buffer as a double buffer, the electronic device recognizes this in operation 1823 and adjusts the graphics buffer to a triple buffer in operation 1817 to prevent graphics performance from being degraded. there is. If the buffer ownership time of an object is constant while using the graphic buffer as a triple buffer, the electronic device may recognize this in operation 1823 and adjust the graphic buffer to a double buffer in operation 1825 to prevent memory waste.

If the electronic device has two graphic buffer-owning objects (for example, GLES type) and the graphic buffer ownership time is constant, the graphic buffer can be set to a double buffer. The electronic device can apply the graphic buffer as a triple buffer when there are three objects that own the graphic buffer (for example, HWC type) or when the graphic buffer ownership time is not constant. When the application is terminated, the electronic device may recognize this in operation 1827 and terminate execution of the application.

Electronic devices according to various embodiments of the present invention can save memory by calculating the minimum number of graphics buffers within a range that does not degrade graphics performance due to frame drops. For example, in mobile systems such as Android, where performance bottlenecks frequently occur due to lack of system memory, graphics performance can be improved by avoiding unnecessary graphics buffer allocation when allocating graphics buffers to system memory. <Table 2> below can be the graphic buffer size of one graphic buffer according to resolution.

resolution number of pixels Graphic buffer size [KB] WXGA 1280X800 4000 FHD 1290X1080 8100 WQXGA 2560X1600 16000

<Table 3> below can be graphic buffer allocation information in the web execution environment of Galaxy Tab S (resolution 2560x1600), and <Table 4> can be graphic buffer allocation information in the home screen of Galaxy Tab E (1280x800). . In the optimal situation, if each graphics buffer can be changed from triple/quadruple to double buffer, a total of 61.29MB of memory in the Galaxy Tab S and 15.85MB in the Galaxy Tab E can be saved without reducing graphics performance. You can.

Layer graphic buffer
Size [KB] graphic buffer
Count Total graphics buffer
Size [KB] SurfaceView id=8 16000 3 -> 2 48000 -> 32000 SurfaceView id=127 14020 3 -> 2 42060 -> 28040 sbrowser 16000 3 -> 2 48000 -> 32000 StatusBar 500 3 -> 2 1500 -> 1000 MultiWindowTrayService
/CenterBar 47.25 3 -> 2 141.75 -> 94.5 MultiWindowTrayService
/AppListWindow 200 3 -> 2 600 -> 400 FocusedStackFrame 16000 3 -> 2 48000 -> 32000 Total size of graphics buffer allocated by app 188301.75 -> 125534.5

<Graphics buffer allocation information on the home screen of Galaxy Tab E (1280x800)> Layer graphic buffer
Size [KB] graphic buffer
Count Total graphics buffer
Size [KB] Launcher 4000 4 -> 2 16000 -> 8000 StatusBar 103.12 4 -> 2 412.48 -> 206.24 FocusedStackFrame 4000 3 -> 2 12000 -> 8000 StrictModeFlash 4000 3 -> 2 12000 -> 8000 MultiWindowTrayService
/CenterBar 13.83 4 -> 2 55.32 -> 27.66 Total size of graphics buffer allocated by app 46867.8 -> 30633.9

As the mobile market is gradually moving toward a high-resolution display environment, the size of the graphics buffer is also increasing accordingly. Electronic devices according to various embodiments of the present invention can save system memory space by setting memory allocation for unnecessary graphics buffers to a minimum, thereby improving overall system performance.

A method of operating an electronic device according to various embodiments of the present invention includes executing an application that performs a graphic display function through a graphic buffer; Analyzing graphics processing conditions of the executing application and setting the number of graphics buffers; and an operation of processing graphic data recorded in the graphic buffer and displaying it on a display.

The operation of setting the number of graphic buffers can set the graphic buffer as a triple buffer if the number of objects owning the graphic buffer is three or more.

The operation of setting the number of graphic buffers may set the graphic buffer as a double buffer if the number of objects owning the graphic buffer is two or less.

Setting the number of graphic buffers may include monitoring a buffer possession time of an object that owns the graphic buffer; And if the buffer possession time is not constant, the operation of setting the graphic buffer to a triple buffer may be further included.

Monitoring the buffer possession time may include calculating the buffer possession time by subtracting the buffer request time from the output request time of the graphic buffer.

Setting the number of graphic buffers may include setting the graphic buffer as a triple buffer if the number of objects owning the graphic buffer is 3 or more; Monitoring the buffer ownership time of an object owning the graphic buffer if the number of objects is two or less; setting the graphic buffer as a triple buffer if the buffer possession time is not constant; And, if the buffer possession time is constant, it may include setting the graphic buffer as a double buffer.

The operation of setting the number of graphic buffers includes: monitoring the composition type of a running application; setting the graphics buffer as a triple buffer if the composition type is a second composition type in which information uses a hardware compositor of a display controller; and, if the composition type is a first composition type using a graphics library, setting the graphics buffer as a double buffer.

The operation of setting the number of graphic buffers may include: monitoring the buffer possession time of the object if the composition type is a first composition type; And if the buffer possession time is not constant, the operation of setting the graphic buffer to a triple buffer may be further included.

Monitoring the buffer possession time of the object may include calculating the buffer possession time by subtracting the buffer request time from the output request time of the graphic buffer.

Claims (20)

In electronic devices,
A graphics buffer accessed by at least two objects;
display; and
A processor functionally connected to the graphics buffer and the display,
The processor,
Identify the composite type of the application being executed,
If the composition type is identified as a first composition type synthesized using software, set the graphics buffer to a double buffer,
If the synthesis type is identified as a second synthesis type synthesized using hardware, set the graphics buffer as a triple buffer,
Displaying graphic data recorded in the graphic buffer through the display,
When the composition type is the first composition type and the graphics buffer is set to the double buffer, monitor the time that objects owning the graphics buffer own the graphics buffer,
If it is determined that the time to own the graphic buffer is not constant, set the graphic buffer to the triple buffer,
An electronic device configured to set the graphics buffer to the double buffer when the composition type is the first composition type, and the graphics buffer is set to the triple buffer, and the possession time of the graphics buffer is determined to be constant. .
delete delete delete According to claim 1,
The processor,
An electronic device configured to calculate the time to own the graphics buffer based on the time the objects request use of the graphics buffer and the time the objects communicate termination of use of the graphics buffer.
delete According to claim 1,
The processor,
With the graphic buffer set to the double buffer, if it is determined that the composition type has changed from the first composition type to the second composition type, setting the graphic buffer to the triple buffer,
When the graphic buffer is set to the triple buffer and it is determined that the composition type has changed from the second composition type to the first composition type, the electronic device is configured to set the graphics buffer to the double buffer.
According to claim 1,
The first composition type is a type that uses a graphics library,
The second synthesis type is an electronic device that uses a hardware synthesizer on a display controller.
delete delete delete In a method of operating an electronic device,
An operation of executing an application that performs a graphic display function through a graphic buffer;
Identifying a composite type of the executing application;
If the composition type is identified as a first composition type synthesized using software, setting the graphics buffer as a double buffer;
If the composition type is identified as a second composition type synthesized using hardware, setting the graphics buffer as a triple buffer;
An operation of combining graphic data recorded in the graphic buffer and displaying it on a display;
When the composition type is the first composition type and the graphics buffer is set to the double buffer, monitoring the time that objects owning the graphics buffer owns the graphics buffer;
If it is determined that the possession time of the graphic buffer is not constant, setting the graphic buffer to the triple buffer; and
When the composition type is the first composition type, and the graphics buffer is set to the triple buffer, and the time for owning the graphics buffer is determined to be constant, setting the graphics buffer to the double buffer. A method of operating an electronic device.
delete delete delete According to claim 12,
The operation of monitoring the time to own the graphics buffer is:
A method of operating an electronic device comprising calculating a time to own the graphics buffer based on a time when the objects request use of the graphics buffer and a time when the objects signal an end of use of the graphics buffer.
delete According to claim 12,
When the graphic buffer is set to the double buffer and it is determined that the composition type has changed from the first composition type to the second composition type, setting the graphic buffer to the triple buffer; and
When the graphic buffer is set to the triple buffer and it is determined that the composition type has changed from the second composition type to the first composition type, the electronic device further includes setting the graphics buffer to the double buffer. How it works. delete delete

Images