EP2672480B1 - Composition de fenêtre d'affichage à carreaux - Google Patents

Composition de fenêtre d'affichage à carreaux Download PDF

Info

Publication number
EP2672480B1
EP2672480B1 EP12171363.0A EP12171363A EP2672480B1 EP 2672480 B1 EP2672480 B1 EP 2672480B1 EP 12171363 A EP12171363 A EP 12171363A EP 2672480 B1 EP2672480 B1 EP 2672480B1
Authority
EP
European Patent Office
Prior art keywords
buffer
application image
viewport
tiles
tile
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP12171363.0A
Other languages
German (de)
English (en)
Other versions
EP2672480A1 (fr
Inventor
Etienne Belanger
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
2236008 Ontario Inc
Original Assignee
2236008 Ontario Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 2236008 Ontario Inc filed Critical 2236008 Ontario Inc
Priority to EP16179806.1A priority Critical patent/EP3098807B1/fr
Priority to EP12171363.0A priority patent/EP2672480B1/fr
Priority to CN201310208305.7A priority patent/CN103489150B/zh
Priority to CA2817492A priority patent/CA2817492C/fr
Publication of EP2672480A1 publication Critical patent/EP2672480A1/fr
Application granted granted Critical
Publication of EP2672480B1 publication Critical patent/EP2672480B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/39—Control of the bit-mapped memory
    • G09G5/393—Arrangements for updating the contents of the bit-mapped memory
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/39—Control of the bit-mapped memory
    • G09G5/395—Arrangements specially adapted for transferring the contents of the bit-mapped memory to the screen
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/39—Control of the bit-mapped memory
    • G09G5/399—Control of the bit-mapped memory using two or more bit-mapped memories, the operations of which are switched in time, e.g. ping-pong buffers

Definitions

  • This disclosure relates to graphics and, in particular, to graphics buffers.
  • GUI Graphical user interfaces
  • PiP Picture in Picture
  • other types of graphical images may be composited from multiple independently rendered images.
  • a GUI may contain a background image and one or more rendered graphic layers on top of the background image.
  • the graphic layers may comprise applications, task bars, or other type of graphic elements.
  • Each of the graphic layers may be rendered in an application buffer independently of the other graphic layers by individual software applications.
  • the graphic layers may be composited together into a single composite image in a display buffer, which may be presented on a display device, transmitted over a network, or otherwise processed.
  • the application buffer may represent an image that is larger than the corresponding graphic layer composited into the single composite image.
  • EP 0 312 720 A2 describes a graphic display system comprising a video display controller including two similar frame buffer memories for alternatively receiving and storing incoming pixel data and for periodically refreshing a display on a screen selectively in accordance with pixel data stored by either one of the two frame buffer memories. While the video display controller periodically refreshes the screen display in accordance with the pixel data stored in a first of the frame buffer memories, incoming pixel data is stored in the second frame buffer memory. The video display controller begins periodically refreshing the screen display in accordance with the pixel data stored in the second frame buffer memory. Updated pixel data stored in the second frame buffer memory is then copied into the first frame buffer memory.
  • WO2005/069271 A1 describes a secondary frame buffer for use by classic applications designed to paint directly to a frame buffer.
  • US5877762 describes methods and apparatus for capturing images on screens which display multiple windows.
  • US20040179019 A1 describes tile buffers in a graphics processing system that are managed using "copy-on-write" semantics, in which tile data stored in a memory location is not transferred to another location until the tile data for one of the buffers is modified.
  • US 2010/0281402 relates to software-based asynchronous backing stores.
  • the system for buffering an application image includes a composition manager and a memory comprising a front buffer and a back buffer.
  • the application image is logically divided or separated into tiles. Each of the tiles identifies a corresponding area of the application image.
  • the composition manager determines which of the tiles are overlapped by a viewport.
  • the viewport identifies a visible portion of the application image. The visible portion is smaller than the application image. For example, the visible portion may be a portion of a map that is displayed on a display device in a navigation system.
  • the composition manager (or an application that renders the application image) double buffers (or more generally, multiple buffer) the tiles that the viewport overlaps.
  • the composition manager (or the application) may not double buffer (or multiple buffer) the tiles that the viewport does not overlap. Reducing the amount of memory that is multiple buffered may reduce bandwidth requirements for accessing the memory.
  • Double buffering may prevent visual tearing artifacts caused when one process writes to a buffer while another process reads from the same buffer.
  • the application may render the application image in the back buffer while another process, such as the composition manager, reads the application image from the front buffer.
  • a buffer that the system writes to in order to construct a frame of the application image is the back buffer.
  • a buffer that the system reads the completed frame from is the front buffer.
  • the system may include and use multiple back buffers.
  • the front buffer and the back buffer are swapped such that the back buffer becomes the front buffer, and the front buffer becomes the back buffer.
  • the buffer swap may also be referred to as a buffer flip.
  • the application renders a new frame in the back buffer. The process of rendering, flipping, and displaying may be repeated.
  • the composition manager and/or the application skips double buffering (or multiple buffering) of the tiles that the viewport does not overlap.
  • the composition manager and/or the application buffer the tiles that are overlapped by the viewport in both the front and back buffers, and buffer the tiles that are not overlapped by the viewport in the back buffer but not in the front buffer.
  • Each of the tiles is buffered in a corresponding tile buffer.
  • the tile buffers may be noncontiguously arranged in memory. Two buffers may be noncontiguous if the buffers are not adjacent to each other in the memory address space of the memory that the buffers are in.
  • the composition manager may copy a buffered visible portion of the application image directly from the noncontiguous tile buffers based on the composition manager having knowledge of the viewport and the tile buffers.
  • the application does not need to create an intermediate buffer that contains the buffered visible portion of application image in contiguous memory so that the composition manager may copy the visible portion from the contiguous memory. Avoiding the creation of the intermediate buffer reduces the bandwidth requirements for accessing memory.
  • the composition manager may generate a composite image from the visible portion of the application image and one or more additional application images.
  • FIG. 1 illustrates a tiled viewport composition system 100.
  • the system 100 may include an application 102, application buffers 104, a display buffer 106, and a composition manager 108.
  • the tiled viewport composition system 100 is a system for buffering an application image 110.
  • the application 102 may render the application image 110.
  • the application image 110 may include a window generated by a windows based operating system, such an operating system for a mobile electronic device, a desktop computer, or a server.
  • windows based operating system may include MICROSOFT WINDOWS®, which is a registered trademark of Microsoft Corporation of Redmond, Washington; LINUX®, which is a registered trademark of Linus Torvalds of Finland; and ANDROIDTM, which is a trademark of Google, Inc. of Mountain View, California.
  • the application image 110 may include a Graphical User Interface (GUI), a Picture in Picture (PiP), a task bar, a background picture, a frame in a sequence of frames that may be part of a video, or any other type of digital image.
  • GUI Graphical User Interface
  • PiP Picture in Picture
  • a task bar a background picture
  • background picture a frame in a sequence of frames that may be part of a video
  • any other type of digital image may be any other type of digital image.
  • Examples of the application 102 may include a word processing program, a mail program, a web browser, a mobile application, or other types of software program or hardware component that may render the application image 110.
  • the application buffers 104 may be one or more areas of memory that represent the application image 110 and/or portions thereof.
  • the memory may include contiguous and/or noncontiguous areas of memory.
  • the display buffer 106 may comprise an area of memory that represents a composite image 112 assembled from individual images generated by one or more applications, such as the application 102 illustrated in FIG. 1 .
  • the composite image 112 may be presented on a display device, transmitted over a network, or locally or remotely processed.
  • the composition manager 108 may composite or direct the composition of the individual images into the composite image 112.
  • the individual images may include all or a portion of application images.
  • Examples of the composition manager 108 include a hardware compositor, a window manager, or any other component that composites individual images into the composite image 112.
  • the composition manager 108 may render the composite image 112 from a visible portion of the application image 110 identified by a viewport 114.
  • the visible portion identified by the viewport 114 may be smaller than the entire application image 110.
  • the composition manager 108 may render the composite image 112 from the visible portion of the application image 110 and from one or more additional images.
  • the application 102 may move the viewport 114 around the application image 110 as the visible portion of the application image 110 changes over time.
  • the application image 110 may be a map image, and the viewport 114 identifies the portion of the map image that is to be displayed in a navigation system. As the location of the navigation system changes, the portion of the map image displayed in the navigation system may also change.
  • the application image 110 may be an image of a level in a video game, and the viewport 114 may identify the portion of an image of the level that is to be displayed on a screen. As a game character moves around the level, the portion of the image of the level displayed on the screen changes.
  • a webpage viewer may render more content than may be shown on the display.
  • the composite image 112 may update more quickly, or produce a visibly smoother transition appearance, when the application 102 uses the viewport 114 to update the visible portion of the application image 110.
  • the application 102 may logically divide or partition the application image 110 into tiles 116.
  • Each of the tiles 116 may identify a corresponding area of the application image 110.
  • the tiles 116 may be uniform in size and shape throughout the application image 110. Alternatively, the size and the shape of the tiles 116 may vary throughout the application image 110. The size and shape of the tiles 116 may be different in different examples.
  • the tiles 116 may be represented in corresponding tile buffers 118.
  • the tiles 116 that are individually designated T1-T4 in FIG. 1 may be represented in the tile buffers 118 that are individually designated A1-A4.
  • the tile buffers 118 may be included in the application buffers 104.
  • the composition manager 108 may determine which of the tiles 116 are overlapped by the viewport 114. Each of the tiles 116 that are overlapped or intersected by the viewport 114 may contain at least part 120 of the visible portion of the application image 110. In contrast, the tiles 116 that are not overlapped by the viewport 114 may not contain any visible part 120 of the application image 110.
  • the composition manager 108 may double buffer the tiles 116 that are overlapped by the viewport 114, but not the remaining tiles 116.
  • a front buffer 122 may include the tile buffers 118 (A3 and A4) representing the tiles 116, (T3 and T4) overlapped by the viewport 114. Accordingly, the tile buffers 118 (A3 and A4) in the front buffer 122 include a buffered visible portion 124 of the application image 110.
  • a back buffer 126 may include the tile buffers 118 (A1 and A2) representing the tiles 116 (T1 and T2) not overlapped by the viewport 114.
  • the back buffer 126 may include the tile buffers 118 (B3 and B4) that correspond to the tile buffers 118 (A3 and A4) that are in the front buffer 122 and that represent the tiles 116 overlapped by the viewport 114.
  • the composition manager 108 may extract the buffered visible portion 124 of the application image 110 from the tile buffers 118 (A3 and A4) that are in the front buffer 122.
  • the buffered visible portion 124 may be of frame n of the application image 110.
  • the application 102 may render frame n+1 of the application image 110 in the back buffer 126 by modifying the tile buffers 118 (A1, A2, B3, and B4) that are in the back buffer 126.
  • the composition manager 108 changes an association between the tile buffers 118 and the front and back buffers 122 and 126.
  • the composition manager 108 may track, in a data structure, which of the tile buffers 118 are associated with the front buffer 122, and which of the tile buffers 118 are associated with the back buffer 126.
  • the associations between the tile buffers 118 and the front and back buffers 122 and 126 indicate which of the tile buffers 118 are included in each of the front and back buffers 122 and 126.
  • the composition manager 108 may disassociate the subset of the tile buffers 118 (B3 and B4) from the back buffer 126, and associate the subset of the tile buffers 118 (B3 and B4) with the front buffer 122.
  • the composition manager 108 may change the tile buffers 118 (A3 and A4) that were originally associated with the front buffer 122 to be associated with the back buffer 126.
  • the composition manager 108 in response to the buffer flip, may swap a first set of the tile buffers 118 associated with the front buffer 122 with a second set of the tile buffers 118 associated with the back buffer 126, where the tile buffers 118 that are swapped represent the tiles 116 that the viewport 114 overlaps but not the tiles 116 that the viewport 114 does not overlap.
  • the tile buffers 118 swapped into the back buffer 126 may represent an older frame, such as frame n.
  • the tile buffers 118 swapped into the front buffer 126 may represent frame n+1.
  • the application 102 may render a third frame, such as frame n+2, in the back buffer 126.
  • the composition manager 108 may double buffer the newly overlapped tile 116. Conversely, when the viewport 114 moves and no longer overlaps one of the tiles 116, then the composition manager 108 may stop double buffering the previously overlapped tile 116.
  • the composition manager 108 adjusts which of the tile buffers 118 are in the front buffer 122 so that the tile buffers 118 in the front buffer 122 correspond to the tiles 116 overlapped by the viewport 114. Similarly, when the viewport 114 moves, the composition manager 108 may adjust which of the tile buffers 118 are in the back buffer 126.
  • the tile buffer 118 in the back buffer 126 that corresponds to the newly overlapped tile 116 are included in the front buffer 122.
  • the tile buffer 118 added to the front buffer does not need to be completely re-rendered because the application 102 may have already rendered the tile buffer 118 in the back buffer 126.
  • the tile buffers 118 may be dynamically allocated and freed as the viewport 114 moves. For example, if the viewport 114 moves and overlaps four of the tiles 116 (T1-T4) illustrated in FIG. 1 instead of two of the tiles 116 (T3 and T4), then the composition manager 108 may dynamically allocate two tile buffers 118 (B1 and B2, which are not shown in FIG. 1 ). The front buffer 122 may then include four of the tile buffers 118 (A1-A4), and the back buffer 126 may include the four corresponding tile buffers 118 (B1-B4).
  • the tile buffers 118 may be allocated in advance. For example, upon an initialization requested by the application 102, the composition manager 108 may allocate enough of the tile buffers 118 to fully represent the application image 110 in both the back buffer 126 and the front buffer 122. The composition manager 108 may dynamically associate the pre-allocated tile buffers 118 to the front and back buffers 122 and 126.
  • the application 102 and the composition manager 108 may have different features than those described.
  • the composition manager 108 (instead of the application 102) may divide or partition the application image 110 into tiles 116.
  • the application 102 may invoke an API (Application Programming Interface) to indicate to the composition manager 108 the size of the application image 110.
  • the composition manager 108 may divide the application image 110 into the tiles 116 and allocate the corresponding tile buffers 118.
  • the application 102 (instead of the composition manager 108) may double buffer the tiles 116 that are overlapped by the viewport 114, but not the remaining tiles 116.
  • one or more components other than the application 102 and the composition manager 108 may execute described functions of the application 102 and the composition manager 108.
  • the composition manager 108 may render the composite image 112 from the visible portion of the application image 110 identified by the viewport 114.
  • the composition manager 108 having knowledge of the tile buffers 118 and the viewport 114, may copy the buffered visible portion 124 directly from the tile buffers 118 in the front buffer 122 to the display buffer 106 even if the buffered visible portion 124 is spread across multiple noncontiguous tile buffers 118. Copying the buffered visible portion 124 from the tile buffers 118 in the front buffer to an intermediate buffer, and then copying from the intermediate buffer to the display buffer 106 may be unnecessary.
  • the application 102 may provide the composition manager 108 with information about the viewport 114 and about the tile buffers 118 in the front buffer 122.
  • the composition manager 108 may use the information provided by the application 102 to copy the buffered visible portion 124 from the tile buffers 118 in the front buffer 122 to the display buffer 106. Alternatively, the composition manager 108 may render the composite image 112 from an intermediate buffer. The intermediate buffer may be populated by the application 102 by copying from the buffered visible portion 124 in the front buffer 122.
  • the frame of the application image 110 represented in the front buffer 122 may be frame n or any other frame.
  • the frame of the application image 110 represented in the back buffer 126 may be frame n+1 or any other frame.
  • the composition manager 108, the application 102, and/or some other component may double buffer the tiles 116 that are overlapped or lie or extend and cover part of by the viewport 114.
  • the composition manager 108 and/or the application 102 may apply multiple buffering to the tiles 116 that are overlapped by the viewport 114, and not to the tiles 116 that are not overlapped by the viewport 114.
  • Multiple buffering may include double buffering, triple buffering, or other buffering processes and/or hardware that includes storing one or more frames of the application image 110 in one or more front buffers for extraction of the application image 110, and storing one or more frames of the application image 110 in one or more back buffers for rendering of the application image 110.
  • FIG. 2 is a hardware diagram of the tiled viewport composition system 100.
  • the tiled viewport composition system 100 may include a processor 204, a memory interface 206, and a memory 208.
  • the tiled viewport composition system 100 may be referred to simply as the system 100.
  • the processor 204 may be hardware that executes computer executable instructions or computer code embodied in the memory 208 or in other memory to perform one or more features of the tiled viewport composition system 100.
  • the processor 204 may include a general processor, a central processing unit, a graphics processing unit, an application specific integrated circuit (ASIC), a digital signal processor, a field programmable gate array (FPGA), a digital circuit, an analog circuit, a microcontroller, any other type of processor, or any combination thereof.
  • ASIC application specific integrated circuit
  • FPGA field programmable gate array
  • the memory interface 206 may manage the transportation of data going to and from the memory 208, such as between the processor 204 and the memory 208.
  • the memory interface 206 may include any memory controller, such as a Memory Chip Controller (MCC), a Double Data Rate2 (DDR2) memory controller used to drive DDR2 SDRAM (double data rate synchronous dynamic random-access memory), a Peripheral Component Interconnect (PCI) bus controller, or any other type of memory controller.
  • MCC Memory Chip Controller
  • DDR2 Double Data Rate2
  • PCI Peripheral Component Interconnect
  • the memory interface 206 may communicate with the memory 208 over a bus 214, such as a 64 bit DDR2 bus operating at 400 Megahertz or any other type of bus.
  • the memory 208 may comprise a device for storing and retrieving data or any combination thereof.
  • the memory 208 may include non-volatile and/or volatile memory, such as a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a flash memory.
  • RAM random access memory
  • ROM read-only memory
  • EPROM erasable programmable read-only memory
  • flash memory Alternatively or in addition, the memory 208 may include an optical, magnetic (hard-drive) or any other form of data storage device.
  • the memory 208 may store computer code, such as the composition manager 108, the application 102, and/or any other application.
  • the computer code may include instructions executable with the processor 204.
  • the computer code may be written in any computer language, such as C, C++, assembly language, shader code, channel program code, and/or any combination of computer languages.
  • the memory 208 may include buffers, such as the application buffers 104 and the display buffer 106.
  • the application buffers 104 may include the tile buffers 118.
  • the system 100 may include more, fewer, or different components than illustrated in FIGS. 1 and 2 .
  • the system 100 may also include a display controller 210 and a display device 212.
  • the display controller 210 may read the composite image 112 from the display buffer 106 through the memory interface 206 and cause the composite image 112 to be displayed in the display device 212.
  • the processor 204 may be in communication with the memory 208 via the memory interface 206.
  • the processor 204 may be in communication with other components, such as the display controller 210 and/or a network controller (not shown).
  • each one of the components of system 100 may include more, fewer, or different elements than is illustrated in FIGS. 1 and 2 .
  • the memory 208 may include more, fewer, or different modules, graphics buffers, and applications.
  • the application 102 and/or the composition manager 108 may include various modules. Flags, data, databases, tables, entities, and other data structures may be separately stored and managed, may be incorporated into a single memory or database, may be distributed, or may be logically and physically organized in many different ways.
  • the components may operate independently or be part of a same program or hardware.
  • the components may be resident on separate hardware, such as separate removable circuit boards, or share common hardware, such as a same memory and processor for implementing instructions from the memory. Programs may be parts of a single program, separate programs, or distributed across several memories and processors.
  • one or more of the components of the system 100 may be included in a System on a Chip (SOC) 216.
  • the SOC 216 may include the processor 204, the memory interface 206 and the display controller 210.
  • the SOC 216 may include additional components, such as memory.
  • the system 100 may be implemented in many processes and/or systems. For example, although some features are shown stored in computer-readable memories as logic implemented as computer-executable instructions or as data structures in memory, portions of the system 100 and its logic and data structures may be stored on, distributed across, or read from any other machine-readable storage media. Examples of the storage media may include memories, a cache, a buffer, RAM, a removable media, a hard drive, a hard disk, a floppy disk, a CD-ROM, or any other type of machine-readable or computer-readable storage medium. The media may include any non-transitory computer-readable storage media, such as volatile or non-volatile memory, RAM, ROM, CD-ROM, any other suitable storage device or any combination thereof. Alternatively or in addition, features and/or modules described as logic implemented as computer-executable instructions or as data structures in memory may be implemented in hardware or in a combination of hardware and software, such as in a field programmable gate array (FPGA).
  • FPGA field programmable gate array
  • the functions, acts or tasks illustrated in the figures or described may be executed in response to one or more sets of logic or instructions stored in or on computer readable media.
  • the functions, acts or tasks are independent of the particular type of instructions set, storage media, processor or processing strategy and may be performed by software, hardware, integrated circuits, firmware, micro code and the like, operating alone or in combination.
  • processing strategies may include multiprocessing, multitasking, parallel processing, distributed processing, and/or any other type of processing.
  • the instructions are stored on a removable media device for reading by local or remote systems.
  • the logic or instructions are stored in a remote location for transfer through a computer network or over telephone lines.
  • the logic or instructions may be stored within a given computer, central processing unit (“CPU"), graphics processing unit (“GPU”), or system.
  • the processing capability of the system 100 may be distributed among multiple entities, such as among multiple processors and memories, optionally including multiple distributed processing systems. Parameters and other data structures may be separately stored and managed, may be incorporated into a single memory or database, may be logically and physically organized in many different ways, and may be implemented with different types of data structures such as linked lists, hash tables, or implicit storage mechanisms. Logic, such as programs or circuitry, may be combined or split among multiple programs, distributed across several memories and processors.
  • FIG. 3 is a flow diagram of the logic of the tiled viewport composition system 100.
  • the application image 110 may be provided as logically divided into the tiles 116.
  • the tile buffers 118 may be in the memory 208 and represent the tiles 116.
  • the logic may determine (310) which of the tiles 116 are overlapped by the viewport 114.
  • the viewport 114 may identify the visible portion of the application image 110, and the visible portion may be smaller than the application image 110.
  • the tiles 116 that are overlapped by the viewport 114 may be buffered (320) in the front buffer 122 and the back buffer 126, where the visible portion of the application image 110 is extracted from the front buffer 122, and the application image 110 is modified in the back buffer 126.
  • the front buffer 122 may buffer frame n
  • the back buffer 126 may buffer frame n+1.
  • the tiles 116 not overlapped by the viewport 114 may be buffered (330) in the back buffer 126 in which the application image 119 is modified but not in the front buffer 122 from which the visible portion of the application image 110 is extracted.
  • the logic may end, for example, by the composition manager 108 extracting the buffered visible portion 124 of the application image 110 from the front buffer 122.
  • the logic may include additional, different, or fewer operations.
  • the logic may include dividing the application image 110 into the tiles 116 by allocating the tile buffers 118.
  • the operations may be executed in a different order than illustrated in FIG. 3 .
  • the phrases "at least one of ⁇ A>, ⁇ B>, ... and ⁇ N>” or “at least one of ⁇ A>, ⁇ B>, ... ⁇ N>, or combinations thereof" or " ⁇ A>, ⁇ B>, ... and/or ⁇ N>” are defined by the Applicant in the broadest sense, superseding any other implied definitions herebefore or hereinafter unless expressly asserted by the Applicant to the contrary, to mean one or more elements selected from the group comprising A, B, ... and N, that is to say, any combination of one or more of the elements A, B, ... or N including any one element alone or in combination with one or more of the other elements which may also include, in combination, additional elements not listed.

Claims (6)

  1. Système (100) pour mettre en mémoire tampon une image d'application (110) à afficher sur un dispositif d'affichage (212) ;
    le système comprenant :
    un processeur (204) pouvant être connecté au dispositif d'affichage (212) et agencé pour fournir l'image d'application (110) séparée en une pluralité de carreaux (116), où chacun des carreaux (116) identifie une zone correspondante de l'image d'application (110) ; et
    une mémoire lisible par ordinateur (208), comprenant :
    un ensemble de tampons de carreaux (118) ;
    un gestionnaire de composition (108) agencé pour associer chaque tampon de carreau (118) soit à un tampon avant (122) ou à un tampon arrière (126) indiquant ainsi lesquels des tampons de carreaux (118) sont inclus dans chacun des tampons avant et arrière (122, 126) ;
    le gestionnaire de composition (108) étant en outre agencé pour déterminer lesquels des carreaux (116) de l'image d'application (110) sont recouverts par une fenêtre d'affichage (114), laquelle fenêtre d'affichage (114) identifie une partie visible de l'image d'application (110), la partie visible étant plus petite que l'image d'application,
    dans lequel le gestionnaire de composition est agencé pour attribuer le tampon arrière (126) pour être adapté pour stocker les carreaux (116) de l'image d'application (110) qui sont recouverts par la fenêtre d'affichage (114) et des carreaux (116) de l'image d'application (110) qui ne sont pas recouverts par la fenêtre d'affichage (114) dans les tampons de carreaux (118) associés au tampon arrière (126) et de sorte que chaque tampon de carreau (118) associé au tampon arrière (126) soit adapté pour stocker un carreau (116) correspondant de l'image d'application (110) ; et
    dans lequel le gestionnaire de composition est agencé pour attribuer le tampon avant (122) de sorte que chaque tampon de carreau (118) associé au tampon avant (112) soit adapté pour stocker un carreau (116) correspondant de l'image d'application (110) et de sorte que le tampon avant (122) soit adapté pour stocker uniquement les carreaux (116) correspondants de l'image d'application (110) qui sont recouverts par la fenêtre d'affichage (114) ;
    et dans lequel ledit système est en outre configuré pour lire ladite partie visible à partir du tampon avant (122) pour l'affichage ;
    le gestionnaire de composition (108) étant en outre configuré pour échanger, en réponse à une commutation de tampons, l'ensemble de tampons de carreaux (118) associés au tampon avant (122) avec certains tampons de carreaux (118) de l'ensemble de tampons de carreaux (118) associés au tampon arrière (126), dans lequel les tampons de carreaux (118) qui sont échangés stockent les carreaux (116) de l'image d'application (110) recouverts par la fenêtre d'affichage (114) mais pas les carreaux (116) de l'application d'image (110) non recouverts par la fenêtre d'affichage (114) ;
    le gestionnaire de composition (108) étant en outre configuré pour ajuster, lorsque la fenêtre d'affichage (114) se déplace, lesquels d'une pluralité de tampons de carreaux (118) se trouvent dans le tampon avant (122) en incluant dans le tampon avant (122) ces tampons de carreaux (118) associés au tampon arrière (126) qui correspondent aux carreaux (116) de l'image d'application (110) nouvellement recouverts par la fenêtre d'affichage (114) ;
    dans lequel le système est agencé pour rendre l'image d'application dans le tampon arrière (126) avant chaque échange de tampons.
  2. Système de la revendication 1, dans lequel le gestionnaire de composition est en outre configuré pour déterminer qu'au moins deux tampons de carreaux non contigus (118) associés au tampon avant coupent la fenêtre d'affichage, et le gestionnaire de composition est en outre configuré pour lire la partie visible de la première image d'application directement à partir des au moins deux tampons de carreaux non contigus, où les au moins deux tampons de carreaux non contigus (118) ne sont pas adjacents l'un à l'autre dans un espace d'adresse mémoire de la mémoire lisible par ordinateur.
  3. Système des revendications 1 ou 2, dans lequel le gestionnaire de composition (108) est en outre configuré pour séparer l'image d'application en carreaux (116) en réponse à une demande d'une application.
  4. Procédé de fonctionnement du système selon la revendication 1, le procédé comprenant les étapes consistant :
    à fournir l'image d'application (110) par le processeur séparée en une pluralité de carreaux (116), où chacun des carreaux (116) identifie une zone correspondante de l'image d'application (110) ;
    à rendre l'image d'application (110) dans le tampon arrière (126) par le système ;
    à déterminer, par le gestionnaire de composition (108) lesquels des carreaux (116) de l'image d'application (110) sont recouverts par une fenêtre d'affichage (114) ;
    à échanger, par le biais du gestionnaire de composition (108), en réponse à une commutation de tampons, l'ensemble de tampons de carreaux (118) associés au tampon avant (122) avec certains tampons de carreaux (118) de l'ensemble de tampons de carreaux (118) associés au tampon arrière (126), où les tampons de carreaux (118) qui sont échangés stockent les carreaux (116) de l'image d'application (110) recouverts par la fenêtre d'affichage (114) mais pas les carreaux (116) de l'image d'application (110) non recouverts par la fenêtre d'affichage (114) ; et après l'échange de tampons, à rendre une nouvelle trame de l'image d'application dans le tampon arrière (126) par le système et à lire la partie visible de l'image d'application mise en mémoire tampon à partir du tampon avant (122) ; et
    à ajuster, lorsque la fenêtre d'affichage (114) se déplace, par le biais du gestionnaire de composition (108), lesquels d'une pluralité de tampons de carreaux (118) se trouvent dans le tampon avant (122) en incluant dans le tampon avant (122) ces tampons de carreaux (118) associés au tampon arrière (126) qui correspondent aux carreaux (116) de l'image d'application (110) nouvellement recouverts par la fenêtre d'affichage (114).
  5. Procédé de la revendication 4, comprenant en outre le fait de lire, avec un gestionnaire de composition (108) ayant connaissance de la fenêtre d'affichage et au moins deux tampons de carreaux non contigus (118) associés au tampon avant, une partie visible mise en mémoire tampon (124) de l'image d'application directement à partir d'au moins deux tampons de carreaux non contigus, où les au moins deux tampons de carreaux non contigus (118) ne sont pas adjacents l'un à l'autre dans un espace d'adresse mémoire de la mémoire lisible par ordinateur.
  6. Support de stockage lisible par ordinateur codé avec des instructions exécutables par ordinateur, les instructions exécutables par ordinateur pouvant être exécutées avec un processeur (204) pour mettre en mémoire tampon une image d'application (110), le support de stockage lisible par ordinateur comprenant :
    des instructions pouvant être exécutées pour mettre en oeuvre le procédé de la revendication 4 ou 5.
EP12171363.0A 2012-06-08 2012-06-08 Composition de fenêtre d'affichage à carreaux Active EP2672480B1 (fr)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP16179806.1A EP3098807B1 (fr) 2012-06-08 2012-06-08 Composition de fenetre d'affichage a carreaux
EP12171363.0A EP2672480B1 (fr) 2012-06-08 2012-06-08 Composition de fenêtre d'affichage à carreaux
CN201310208305.7A CN103489150B (zh) 2012-06-08 2013-05-30 平铺视区合成
CA2817492A CA2817492C (fr) 2012-06-08 2013-05-31 Composition de cloture a pave

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP12171363.0A EP2672480B1 (fr) 2012-06-08 2012-06-08 Composition de fenêtre d'affichage à carreaux

Related Child Applications (2)

Application Number Title Priority Date Filing Date
EP16179806.1A Division EP3098807B1 (fr) 2012-06-08 2012-06-08 Composition de fenetre d'affichage a carreaux
EP16179806.1A Division-Into EP3098807B1 (fr) 2012-06-08 2012-06-08 Composition de fenetre d'affichage a carreaux

Publications (2)

Publication Number Publication Date
EP2672480A1 EP2672480A1 (fr) 2013-12-11
EP2672480B1 true EP2672480B1 (fr) 2016-11-16

Family

ID=46208373

Family Applications (2)

Application Number Title Priority Date Filing Date
EP12171363.0A Active EP2672480B1 (fr) 2012-06-08 2012-06-08 Composition de fenêtre d'affichage à carreaux
EP16179806.1A Active EP3098807B1 (fr) 2012-06-08 2012-06-08 Composition de fenetre d'affichage a carreaux

Family Applications After (1)

Application Number Title Priority Date Filing Date
EP16179806.1A Active EP3098807B1 (fr) 2012-06-08 2012-06-08 Composition de fenetre d'affichage a carreaux

Country Status (3)

Country Link
EP (2) EP2672480B1 (fr)
CN (1) CN103489150B (fr)
CA (1) CA2817492C (fr)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2672480B1 (fr) * 2012-06-08 2016-11-16 2236008 Ontario Inc. Composition de fenêtre d'affichage à carreaux
KR102275712B1 (ko) 2014-10-31 2021-07-09 삼성전자주식회사 렌더링 방법, 렌더링 장치 및 전자 장치
KR101717355B1 (ko) * 2015-07-29 2017-03-16 엘에스산전 주식회사 에너지 관리 시스템의 디스플레이 장치 및 방법

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5877762A (en) * 1995-02-27 1999-03-02 Apple Computer, Inc. System and method for capturing images of screens which display multiple windows
US5933155A (en) * 1996-11-06 1999-08-03 Silicon Graphics, Inc. System and method for buffering multiple frames while controlling latency
US20040179019A1 (en) * 2003-03-12 2004-09-16 Nvidia Corporation Double-buffering of pixel data using copy-on-write semantics
WO2005069271A1 (fr) * 2003-12-18 2005-07-28 Apple Computer, Inc Graphismes composites rendus a l'aide de tampons multi-trames
US20100281402A1 (en) * 2009-04-29 2010-11-04 Torch Mobile Inc. Software-based asynchronous tiled backingstore

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4839828A (en) * 1986-01-21 1989-06-13 International Business Machines Corporation Memory read/write control system for color graphic display
EP0312720A3 (fr) * 1987-10-20 1990-06-13 Tektronix Inc. Système de dessein graphique à double mémoire-tampon
EP2672480B1 (fr) * 2012-06-08 2016-11-16 2236008 Ontario Inc. Composition de fenêtre d'affichage à carreaux

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5877762A (en) * 1995-02-27 1999-03-02 Apple Computer, Inc. System and method for capturing images of screens which display multiple windows
US5933155A (en) * 1996-11-06 1999-08-03 Silicon Graphics, Inc. System and method for buffering multiple frames while controlling latency
US20040179019A1 (en) * 2003-03-12 2004-09-16 Nvidia Corporation Double-buffering of pixel data using copy-on-write semantics
WO2005069271A1 (fr) * 2003-12-18 2005-07-28 Apple Computer, Inc Graphismes composites rendus a l'aide de tampons multi-trames
US20100281402A1 (en) * 2009-04-29 2010-11-04 Torch Mobile Inc. Software-based asynchronous tiled backingstore

Also Published As

Publication number Publication date
EP3098807A1 (fr) 2016-11-30
CN103489150A (zh) 2014-01-01
CA2817492A1 (fr) 2013-12-08
CN103489150B (zh) 2017-03-01
CA2817492C (fr) 2017-03-14
EP2672480A1 (fr) 2013-12-11
EP3098807B1 (fr) 2018-01-17

Similar Documents

Publication Publication Date Title
US9251555B2 (en) Tiled viewport composition
US7889202B2 (en) Transparent multi-buffering in multi-GPU graphics subsystem
US9070211B1 (en) Webview tag for a sandboxed multiprocess browser
US9292950B2 (en) Cell-based composited windowing system
US9582849B2 (en) Method and system to virtualize graphic processing services
US9129581B2 (en) Method and apparatus for displaying images
CN108476306B (zh) 一种图像显示的方法及终端设备
US20160328871A1 (en) Graphics system and associated method for displaying blended image having overlay image layers
US9786256B2 (en) Method and device for generating graphical user interface (GUI) for displaying
US7817166B2 (en) Stereo windowing system with translucent window support
AU2005262676A1 (en) Display updates in a windowing system using a programmable graphics processing unit.
CN104765595A (zh) 一种显示图形用户界面的方法及装置
US6714218B1 (en) Scaling images
EP2672480B1 (fr) Composition de fenêtre d'affichage à carreaux
US8847970B2 (en) Updating graphical content based on dirty display buffers
JP2005339443A (ja) 解像度が異なる複数のウィンドウを表示するためのシステム、方法およびプログラム
US9324299B2 (en) Atlasing and virtual surfaces
JP2007241738A (ja) 画面合成装置
US10140680B2 (en) Hardware-accelerated resource tiling
CN109416828B (zh) 将帧缓冲映射到逻辑显示器的装置及方法
US20120154411A1 (en) Multiple display frame rendering method and apparatus
US10789913B2 (en) Arbitrary block rendering and display frame reconstruction
CA2817307A1 (fr) Systeme de fenetrage a composition a base de cellules
US9904665B2 (en) Partial rasterization of web page tiles
KR20080031595A (ko) 오프스크린 버퍼링 관리 장치 및 방법

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20120608

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

17Q First examination report despatched

Effective date: 20140724

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: 2236008 ONTARIO INC.

GRAJ Information related to disapproval of communication of intention to grant by the applicant or resumption of examination proceedings by the epo deleted

Free format text: ORIGINAL CODE: EPIDOSDIGR1

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20160527

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 846574

Country of ref document: AT

Kind code of ref document: T

Effective date: 20161215

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602012025346

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161116

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20161116

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 846574

Country of ref document: AT

Kind code of ref document: T

Effective date: 20161116

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161116

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161116

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170216

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170217

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161116

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161116

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161116

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161116

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161116

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161116

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170316

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161116

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 6

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161116

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161116

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161116

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161116

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161116

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602012025346

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161116

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161116

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170216

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161116

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20170817

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161116

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161116

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170608

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170630

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170608

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170630

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 7

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170608

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20120608

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161116

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161116

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161116

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 602012025346

Country of ref document: DE

Representative=s name: MERH-IP MATIAS ERNY REICHL HOFFMANN PATENTANWA, DE

Ref country code: DE

Ref legal event code: R081

Ref document number: 602012025346

Country of ref document: DE

Owner name: BLACKBERRY LIMITED, WATERLOO, CA

Free format text: FORMER OWNER: 2236008 ONTARIO INC., WATERLOO, ONTARIO, CA

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161116

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170316

REG Reference to a national code

Ref country code: GB

Ref legal event code: 732E

Free format text: REGISTERED BETWEEN 20200730 AND 20200805

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230518

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20230626

Year of fee payment: 12

Ref country code: DE

Payment date: 20230626

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20230627

Year of fee payment: 12