GB2556743A - Management of memory for storing display data - Google Patents

Management of memory for storing display data Download PDF

Info

Publication number
GB2556743A
GB2556743A GB1801346.6A GB201801346A GB2556743A GB 2556743 A GB2556743 A GB 2556743A GB 201801346 A GB201801346 A GB 201801346A GB 2556743 A GB2556743 A GB 2556743A
Authority
GB
United Kingdom
Prior art keywords
display
memory
data
control device
display data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
GB1801346.6A
Other versions
GB201801346D0 (en
GB2556743B (en
Inventor
Skinner Colin
Mark Edmonds Timothy
Andrew Turnock Martin
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
DisplayLink UK Ltd
Original Assignee
DisplayLink UK Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by DisplayLink UK Ltd filed Critical DisplayLink UK Ltd
Priority to GB1801346.6A priority Critical patent/GB2556743B/en
Publication of GB201801346D0 publication Critical patent/GB201801346D0/en
Publication of GB2556743A publication Critical patent/GB2556743A/en
Application granted granted Critical
Publication of GB2556743B publication Critical patent/GB2556743B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/443—OS processes, e.g. booting an STB, implementing a Java virtual machine in an STB or power management in an STB
    • H04N21/4435—Memory management
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/37—Details of the operation on graphic patterns
    • G09G5/373—Details of the operation on graphic patterns for modifying the size of the graphic pattern
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1415—Digital output to display device ; Cooperation and interconnection of the display device with other functional units with means for detecting differences between the image stored in the host and the images displayed on the displays
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1423—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • G06F3/1462—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay with means for detecting differences between the image stored in the host and the images displayed on the remote displays
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/60—Memory management
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/14—Display of multiple viewports
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/363—Graphics controllers
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/39—Control of the bit-mapped memory
    • G09G5/393—Arrangements for updating the contents of the bit-mapped memory
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/39—Control of the bit-mapped memory
    • G09G5/395—Arrangements specially adapted for transferring the contents of the bit-mapped memory to the screen
    • G09G5/397—Arrangements specially adapted for transferring the contents of two or more bit-mapped memories to the screen simultaneously, e.g. for mixing or overlay
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/156—Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/174—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a slice, e.g. a line of blocks or a group of blocks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/4104—Peripherals receiving signals from specially adapted client devices
    • H04N21/4122—Peripherals receiving signals from specially adapted client devices additional display device, e.g. video projector
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44231—Monitoring of peripheral device or external card, e.g. to detect processing problems in a handheld device or the failure of an external recording device
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/4424—Monitoring of the internal components or processes of the client device, e.g. CPU or memory load, processing speed, timer, counter or percentage of the hard disk space used
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/02—Handling of images in compressed format, e.g. JPEG, MPEG
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2360/00—Aspects of the architecture of display systems
    • G09G2360/12—Frame memory handling
    • G09G2360/122—Tiling
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2360/00—Aspects of the architecture of display systems
    • G09G2360/12—Frame memory handling
    • G09G2360/128—Frame memory using a Synchronous Dynamic RAM [SDRAM]
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2360/00—Aspects of the architecture of display systems
    • G09G2360/18—Use of a frame buffer in a display terminal, inclusive of the display panel
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2370/00—Aspects of data communication
    • G09G2370/02—Networking aspects
    • G09G2370/022—Centralised management of display operation, e.g. in a server instead of locally
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2370/00—Aspects of data communication
    • G09G2370/02—Networking aspects
    • G09G2370/025—LAN communication management
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2370/00—Aspects of data communication
    • G09G2370/04—Exchange of auxiliary data, i.e. other than image data, between monitor and graphics controller
    • G09G2370/042—Exchange of auxiliary data, i.e. other than image data, between monitor and graphics controller for monitor identification

Abstract

Display data is received and stored at a display control device according to received ordering instructions. Also disclosed is managing display control device (16) memory (28) at a host (10), the memory used for storing display data sent from host to device for display, comprising maintaining a map (18) at the host corresponding to the device memory, the map indicating locations corresponding to addresses of the memory at which data is stored or not. The map is used to determine a size of a portion of the display data, and a location on the map where the portion of display data would fit. A memory address corresponding to the determined map location, together with the data, is then sent to the device, and the map location corresponding to the memory address is updated to indicate that the address is occupied. The data may be compressed using compression parameters, which can be selected so the compressed data will fit into a free space in the device memory. The decompression and combining order of portions can also be sent. An expectation of whether the memory will run out of space before all display data is sent can be calculated.

Description

(71) Applicant(s):
DisplayLink (UK) Limited (Incorporated in the United Kingdom)
140 Cambridge Science Park, Milton Road, CAMBRIDGE, CB4 0GF, United Kingdom (56) Documents Cited:
JP 080194642 A JP 080110943 A (58) Field of Search:
INT CL G06T, H04N
Other: Online: WPI, EPODOC (72) Inventor(s):
Colin Skinner Timothy Mark Edmonds Martin Andrew Turnock (74) Agent and/or Address for Service:
Mathys & Squire LLP
The Shard, 32 London Bridge Street, LONDON, SE1 9SG, United Kingdom (54) Title ofthe Invention: Management of memory for storing display data Abstract Title: Management of Memory for Storing Display Data (57) Display data is received and stored at a display control device according to received ordering instructions. Also disclosed is managing display control device (16) memory (28) at a host (10), the memory used for storing display data sent from host to device for display, comprising maintaining a map (18) at the host corresponding to the device memory, the map indicating locations corresponding to addresses of the memory at which data is stored or not. The map is used to determine a size of a portion of the display data, and a location on the map where the portion of display data would fit. A memory address corresponding to the determined map location, together with the data, is then sent to the device, and the map location corresponding to the memory address is updated to indicate that the address is occupied. The data may be compressed using compression parameters, which can be selected so the compressed data will fit into a free space in the device memory. The decompression and combining order of portions can also be sent. An expectation of whether the memory will run out of space before all display data is sent can be calculated.
Figure GB2556743A_D0001
1/7
Figure GB2556743A_D0002
Figure GB2556743A_D0003
bJO
2/7
ID
Figure GB2556743A_D0004
C\i
ΓΜ tub _Q
Ο
Γ\Ι
3/7
Figure GB2556743A_D0005
m ώ
4/7
Γ\Ι
Γ\Ι \ timestamps
Figure GB2556743A_D0006
m
5/7
LD ro
Figure GB2556743A_D0007
A
CD
Z5
CT CO
Π3
£ ω
o ro
-σ ω c
5? QJ (Λ C +j Π5 Π3 £_ qi -C Ο ΙΟ (_»
Figure GB2556743A_D0008
ω +j
CD
Ό
Q.
Z5
CD >. ω s_ +j O ro -t-J D Ξ2 Q. JZ Z5
Φ L_
+-» Φ o +-»
£ Φ
ω φ
Φ
TO Q. to
to
CD ω
t_
CD c
ω +j c
o m
c QJ +J C o o
Ό QJ to .
$ * Q_
E o o
-A
O to to
Φ
L_
Q.
Ξ o
u
LD tub
Figure GB2556743A_D0009
7/7
Figure GB2556743A_D0010
-1 Management of Memory for Storing Display Data
The present invention relates to methods and apparatus for managing memory for storing display data, and more particularly, though not exclusively, for managing a memory storing compressed display data at a display control device.
In desktop computing, it is now common to use more than one display device. Traditionally, a user would have a computer with a single display device attached, but now it is possible to have more than one display device attached to the computer, which increases the usable area for the worker. For example, International Patent Application Publication WO 20071020408 discloses a display system which comprises a plurality of display devices, each displaying respectively an image, a data processing device connected to each is display device and controlling the image displayed by each display device, and a user interface device connected to the data processing device. Connecting multiple display devices to a computer is a proven method for improving productivity.
The connection of an additional display device to a computer presents a number of problems. In general, a computer will be provided with only one video output such as a VGA-out connection. One method by which a display device can be added to a computer is by adding an additional graphics card to the internal components of the computer. The additional graphics card will provide an additional video output which will allow the display device to be connected to the computer and driven by that computer.
However, this solution is relatively expensive and is not suitable for many nontechnical users of computers.
An alternative method of connecting a display device is to connect the display device to a USB socket on the computer, as all modern computers are provided with multiple USB sockets. This provides a simple connection topology, but requires additional hardware and software to be present, as in general, USB has a bandwidth that makes the provision of a good quality video output a non-trivial task. It is also desirable that any additional hardware between the computer and the display device is kept as simple as possible. This means that when connecting a display device using a limited bandwidth technology such as USB, certain complex tasks can be difficult to achieve. In particular, a display control device needs to present that can receive the USB input and provide a suitable output to the display device.
As display technologies improve and the user's desire for display quality increases, the requirement placed upon the display control device that is receiving the incoming
-2display data will correspondingly increase, as the amount of display data will increase proportionally. It is obviously desirable that any new display control devices can is handle higher definition video inputs and higher resolution display devices, so the ability to produce a high performance display control device at as low a cost as possible is highly desirable.
GB 2485613 discloses a display control device that comprises a frame buffer store and a control component. The display control device is arranged to receive compressed display data, store the received compressed display data in the frame buffer store, and for each frame refresh of the display device access the stored compressed display data, decompress the accessed display data, and output the decompressed display data. Thus, the compressed data is stored in the frame buffer store without first being decompressed in the display control device.
Management of such a frame buffer store is needed, whether or not the display data is compressed. In order to keep the display control device costs as low as possible, it is therefore desirable that the computer, or host processing device, cooperate with the display control device to help in the management of the memory of the display control device. Therefore, aspects and examples of the invention are set out in the claims and address at least a part of the above described problem.
Examples of the invention may be implemented in software, middleware, firmware or hardware or any combination thereof. Embodiments of the invention comprise computer program products comprising program instructions to program a processor to perform one or more ofthe methods described herein, such products may be provided on computer readable storage media or in the form of a computer readable signal for transmission over a network. Embodiments of the invention provide computer readable storage media and computer readable signals carrying data structures, media data files or databases according to any of those described herein.
Apparatus aspects may be applied to method aspects and vice versa. The skilled reader will appreciate that apparatus embodiments may be adapted to implement features of method embodiments and that one or more features of any of the embodiments described herein, whether defined in the body of the description or in the claims, may be independently combined with any ofthe other embodiments described herein.
In a first aspect the invention provides a method, at a host device, of managing variable size memory of a display control device connectable to the host device, the memory being used for storing display data sent from the host device to the display control
-3device for displaying on a display, the method comprising:
maintaining a map at the host device corresponding to the memory of the display control device, the map indicating locations corresponding to addresses ofthe memory at which data is stored or not;
determining a size of a portion of the display data, the size being variable for different portions of data;
determining a location on the map where the portion of display data would fit into the memory;
sending, to the display control device, an address in the memory corresponding to the determined location on the map, together with the portion of display data; and updating the location on the map corresponding to the address in the memory to indicate that the address has data stored in it.
Preferably, the method further comprises determining free space available in the memory based on the map.
The memory preferably comprises a frame buffer memory for storing a frame of image data.
In one embodiment, the method further comprises compressing the portion of a frame of display data using compression parameters.
The method may further comprise selecting the compression parameters so that the compressed portion of display data will fit into a free space in the frame buffer memory of the display control device.
The compression parameters are preferably selected so that the compressed portion of display data has an optimum quality.
Selecting compression parameters so that that the compressed portion of display data has an optimum quality preferably comprises determining that the compression is too high and selecting compression parameters to reduce the compression,
The selected compression parameters preferably comprise quantisation parameters.
In one embodiment, the method further comprises determining, based on the map, whether the frame buffer memory of the display control device is expected to run out of space before all the display data of a current frame is compressed and sent, and if it is determined that the frame buffer memory of the display control device is expected to run out of space before all the display data of the current frame is compressed and sent, adjusting the compression parameters in order to increase the compression of the display
-4data so that all remaining compressed display data of the current frame will fit into the frame buffer memory.
The adjusted compression parameters preferably comprise quantisation parameters.
The map may corresponds to two or more frame buffers in the frame buffer memory at the display control device. The frame buffers may be of equal or unequal size.
The host device preferably sends to the display control device information regarding an order in which the portions of display data in the memory are to be accessed from the memory and combined to display an image on the display.
According to a second aspect, the invention provides a method, at a host device, of managing a random access memory of a display control device connectable to the host computing device, the memory being used for storing compressed display data forming parts of an image sent from the host device to the display control device, wherein the display control device decompresses the compressed display data from the memory for displaying on a display, the method comprising:
determining free space available in the random access memory;
selecting compression parameters so that a compressed portion of display data will fit into a free space in the random access memory of the display control device; compressing a predetermined portion of a frame of display data; sending, to the display control device, an address in the memory corresponding to the determined free space in the memory, together with the compressed portion of display data; and sending, to the display control device, information regarding an order in which the portions of compressed display data in the memory are to be decompressed and combined to display the image on the display.
In an embodiment, the method further comprises determining whether the memory of the display control device is expected to run out of space before all the display data of a current frame is compressed and sent, and if it is determined that the memory of the display control device is expected to run out of space before all the display data of a current frame is compressed and sent, selecting different compression parameters in order to increase the compression of the display data so that all remaining compressed display data of the current frame will fit into the memory.
The method may further comprise:
selecting compression parameters so that the portion of display data has an
-5optimum quality when it has been compressed;
Preferably, selecting compression parameters so that the compressed portion of display data has an optimum quality comprises determining that the compression is too high and selecting compression parameters to reduce the compression,
In a third aspect, the invention provides a method, at a host device, of managing a random access memory of a display control device connectable to the host device, the memory being used for storing compressed display data forming parts of an image sent from the host device to the display control device, wherein the display control device decompresses the compressed data from the memory for displaying on a display, the method comprising:
determining free space available in the memory;
determining whether the memory is expected to run out of space before all the display data of a current frame is compressed and sent;
selecting compression parameters so that all remaining compressed display data of the current frame will fit into the memory;
compressing a predetermined portion of a frame of display data;
sending, to the display control device, the compressed portion of display data; and sending, to the display control device, information regarding an order in which the portions of compressed display data in the frame buffer memory are to be decompressed and combined to display the image on the display.
The method preferably further comprises maintaining a map at the host device corresponding to the memory of the display control device, the map indicating locations corresponding to addresses of the memory at which data is stored or not.
Preferably, determining free space available in the memory comprises determining a location on the map where the compressed portion of display data would fit into the memory.
The method may further comprise updating the location on the map corresponding to the address in the memory to indicate that the address has data stored in it.
The map preferably corresponds to two or more frame buffers in the memory at the display control device. The frame buffers may be of equal or unequal size.
In an embodiment, the method further comprises selecting the compression parameters so that a frame of display data has an optimum quality when it has been compressed that can be sent from the host computing device in a target time.
The method preferably further comprises selecting the compression parameters so
-6that the portion of data can be sent from the host device in a target time. The target time may be chosen to be less than a predetermined period.
In an embodiment, the method further comprises selecting the compression parameters so that a frame of display data has an optimum quality when it has been compressed that can be sent from the host computing device based on resources available to transmit the compressed data to the display control device.
Preferably, if it is determined that the compressed portion of display data will not fit efficiently into the memory, then all previous portions of display data for the frame are compressed again using different compression parameters to produce smaller portions of compressed data for sending to the display control device for storing in the memory replacing the previously sent compressed portions of display data for the frame. The different compression parameters may comprise quantisation parameters.
In a further aspect, the invention provides a method, at a host device, of managing a memory of a display control device connectable to the host device, the memory being used for storing compressed display data sent from the host device to the display control device, wherein the display control device decompresses the compressed data from the memory for displaying on a display, the method comprising:
selecting compression parameters so that a frame of display data has an optimum quality when it has been compressed that can be sent from the host computing device in a target time;
compressing the predetermined portion of display data using the selected compression parameters;
sending the compressed portion of display data to the display control device.
According to a still further aspect, the invention provides a method, at a host device, of managing a memory of a display control device connectable to the host computing device, the memory being used for storing compressed display data sent from the host device to the display control device, wherein the display control device decompresses the compressed data from the memory for displaying on a display, the method comprising:
selecting compression parameters so that a frame of display data has an optimum quality when it has been compressed that can be sent from the host device based on resources available to transmit the compressed data to the display control device;
compressing the predetermined portion of display data using the selected compression parameters;
-7sending the compressed portion of display data to the display control device.
In another aspect, the invention provides a method, at a host device, of managing a memory of a display control device connectable to the host device, the memory being used for storing compressed display data sent from the host device to the display control device, wherein the display control device decompresses the compressed data from the memory for displaying on a display, the method comprising:
selecting compression parameters so that a portion of a frame of compressed data can be sent from the host device in a target time;
compressing the predetermined portion of display data using the selected compression parameters;
sending the compressed portion of display data to the display control device.
The target time preferably may be chosen to be less than a predetermined period.
Preferably, selecting the compression parameters is based on a time taken for a previous frame of compressed data sent from the host device.
Selecting the compression parameters may be based on content of a previous frame of compressed data sent from the host device.
According to yet another aspect, the invention provides a method, at a host device, of managing a memory of a display control device connectable to the host device, the memory being used for storing compressed display data sent from the host device to the display control device, wherein the display control device decompresses the compressed data from the memory for displaying on a display, the method comprising:
maintaining a map at the host device corresponding to the memory of the display control device, the map indicating quality of portions of the display data stored at particular addresses in the memory;
determining when the quality of a portion of the display data previously sent to the display control device was of low quality;
determining that resources are available to transmit the compressed portion of display data to the display control device in a higher quality;
selecting compression parameters so that the portion of the display data is of higher quality when it has been compressed;
compressing the portion of display data using the selected compression parameters;
sending the compressed portion of display data to the display control device; and updating the quality on the map corresponding to the address in the memory where
-8the portion of compressed display data having higher quality is stored.
The compressed portion of data is preferably packaged with a transport protocol before being sent to the display control device. The transport protocol may be an Internet Protocol, for example, TCP/IP protocol or USB protocol, or a wireless protocol, for example, WiFi Wireless protocol or WiGig protocol. The compressed portion of data is preferably packaged into a packet for sending to the display control device, the packet including the address, which address may be included in a header ofthe packet.
In one embodiment, the compressed portion of data comprises a partial update of the display data including only the display data that has changed from the previous frame. The compressed portion of data may comprise a tile of display data or a group of tiles of display data.
In a further aspect, the invention provides a host device configured to perform a method as described above.
In a further aspect, the invention provides a method, at a display control device connectable to a host device, of storing display data sent from the host device to the display control device in a memory for displaying on a display, the method comprising:
receiving a portion of display data together with an address;
storing the portion of display data at the address in the memory;
receiving information regarding an order in which the portions of display data in the memory are to be accessed from the memory and combined to display an image on the display;
accessing the portions of display data from the memory; and combining the accessed portions of display data to form an image for display.
Preferably, the memory comprises a frame buffer memory for storing a frame of image data.
The received portion of display data may be a compressed portion of display data. Preferably, the method further comprises decompressing the compressed portion of display data from the memory prior to combining the portions to form the image.
In a further aspect, the invention provides a display control device configured to perform a method as described above.
In a still further aspect, the invention provides a computer readable medium including executable instructions which, when executed in a processing system, cause the processing system to perform a method as described above.
Another aspect of the invention provides a display system comprising:
-9a host device configured to perform a method as described above; a display control device configured to perform a method as described above; and a display coupled to the display control device and configured to display the image. A still further aspect of the invention provides a display control device connectable to a host device and a display, the display control device comprising a random access memory for storing compressed display data received from the host device, a decompressor for decompressing the compressed display data from the random access memory for rasterising for output to the display, and a non-volatile programmable memory for storing compressed image data, wherein the decompressor decompresses the compressed image data for rasterising for output to the display when it is not required to display the compressed display data from the random access memory.
Embodiments of the invention will now be described in greater detail, by way of example only, with reference to the accompanying drawings, in which:
Figure 1 shows a schematic diagram of components of a display system;
Figure 2 shows a schematic diagram of a display control device used in the display system of Figure 1;
Figure 3 shows a schematic diagram of one embodiment of a buffer memory management system used in the display system of Figure 1;
Figure 4 shows a schematic diagram of part of a processing device used in the buffer memory management system of Figure 3;
Figure 5 shows other aspects of the processing device of Figure 4;
Figure 6 shows other aspects of the buffer memory management system of Figure
3; and
Figure 7 shows other aspects of the display control device of Figure 2.
A display system is shown in Figure 1. The system comprises a host processing device 10, a display device 12 and user interface devices 14. The user interface devices are a keyboard 14a and a mouse 14b. The system shown in Figure 1 is a standard desktop computer, with a display device 12, which is composed of discrete components that are locally located but could equally be a device such as a laptop computer or suitably enabled handheld device such as a mobile phone or pda (personal digital assistant) all using an additional display. Similarly, the system may comprise part of a networked or mainframe computing system, in which case the processing device 10 may be located remotely from the user input devices 14 and the display device 12, or indeed may have its function distributed amongst separate devices.
-10The display device 12 shows images, and the display of the images is controlled by the processing device 10. One or more applications are running on the processing device 10 and these are represented to the user by corresponding application windows, with which the user can interact in a conventional manner. The user can control the movement of a cursor about the images shown on the display device 12 using the computer mouse 14b, again in a totally conventional manner. The user can perform actions with respect to any running application via the user interface device 14 and these actions result in corresponding changes in the images displayed on the display device 12.
The operating system run by the processing device 10 uses virtual desktops to manage the or multiple display devices 12. The physical display device 12 is represented by a frame buffer that contains everything currently shown on that display device 12. In order to allow the display device to be connected to a USB port on the processing device 10, rather than the standard FGA port, as would be the case if the display device 12 is a secondary display device, the processing device 10 connects to the secondary display device 12 via a display control device 16. The display control device 16 is connected to the processing device 10 via a standard USB connection, and appears to the processing device 10 as a USB connected device. Any communications between the processing device 10 and the display control device 16 are carried out under the control of a USB driver specifically for the display control device 16. Such devices allow the connection of the secondary display device 12 to the processing device 10 without the need for any hardware changes to the processing device 10.
The display control device 16 connects to the display device 12 via a standard VGA connection, and the display device 12 is a conventional display device 12 which requires no adjustment to operate in the display system shown in Figure 1. As far as the display device 12 is concerned, it could be connected directly to the graphics card of a processing device; it is unaware that the graphical data displayed by the display device 12 has actually been first sent via a USB connection to an intermediate component, the display control device 16. Multiple additional display devices 12 can be is connected to the processing device 10 in this way, as long as suitable USB slots are available on the processing device 10.
The display control device 16 is external to the processing device 10 and is not a graphics card. It is a dedicated piece of hardware that receives graphical data via the
USB connection from the processing device 10 and transforms that graphics data into a
VGA format that will be understood by the display device 12. In topological terms USB
-11 and VGA are only examples of data standards that can be used to connect the additional display device 12 to the processing device 10. The general principle is that a generalpurpose data network (such as USB or Ethernet) connects the processing device 10 to the display control device 16 and a display-specific data standard (such as VGA or DVI) is used on the connection from the display control device 16 to the display device 12.
The display control device 16 is shown in more detail in Figure 2. The device 16 operates to receive encoded display data via a USB input port 24, which is processed with a processor 26 that is connected to a frame buffer 28 and provides an output via an output port 30. The device 16 is for generating the image that is displayed by the display device 12. The device 16 is effectively an image decoder, as it receives encoded display data which is then decoded to pixel data that is stored within the frame buffer 28.
The display control device 16 is a lightweight piece of hardware for adding an additional display 12 to a computer 10.
The incoming signal received by the USB input port 24 comprises encoded display data for one or more pixel tiles or groups of pixel tiles within the image to be displayed and also position data for each of the pixel tiles or groups of pixel tiles in the image. It will be appreciated that the image can be subdivided into any portion of display data that is less than the display data for an entire image, and reduces the amount of data that has to be transmitted between the computer 10 and the display control device 16. A pixel tile comprises an 8x8 array of pixels and the group size could be sixteen pixel tiles, for example, in a 4x4 array of tiles. However, any appropriate portion, or atom, of display data could be used. Often, only an update of part of the image is being sent.
The processor 26 takes the encoded tiles and decodes these tiles into pixel data and updates the frame buffer 28 with the new decoded pixel data is according to the position data in the received signal. In this way, the frame buffer 28 will always maintain pixel data for the entire image, and this pixel data is continuously being updated by the processor 26. The processor 26 will then output at least a portion of the frame buffer contents via the output port 30 to the display device 12 over the VGA connection. The display control device 16 is configured as a dedicated piece of hardware for the decoding process.
In a preferred embodiment, the video decoding device (embodied as a silicon chip or an end user product) receives an encoded video signal, decodes the video signal, and outputs the decoded signal to a video display device over a video link such as HDMI, DVI,
DP, such that the encoded signal arrives over some variant of the USB interface as
-12 standardised by the USB-IF, and the encoded signal makes use of the Haar transform for each 8x8 pixel square of the source images, in each colour channel, AC coefficients of each Haar transform are encoded as variable length bit fields, where the variable encoding is defined using a simple and compact UseVec form (consistent with the definition in the background section). The AC coefficients are ordered so that trailing coefficients represent highest-frequency data, and trailing zero coefficients may be omitted; the colour channels are all at the same resolution, with no subsampling of chroma channels being performed, except as dynamically determined by the number of coefficients provided to the Haar transforms. The encoded signal format is divided into groups of 8x8 tiles so that partial updates of video frame may be performed. Each tile group separately identifies its intended location in the displayed image so that local changes to not require a complete frame update. Each tile group contains fewer than twenty 8x8 tiles and the decoded signal is sent to a video raster display port such as HDMI, DVI, DisplayPort, or is displayed directly on a is visual display which forms part of the device.
Figure 2 shows one embodiment of the display control device 16, which includes a dedicated frame buffer 28 provided by, for example Random Access Memory (RAM). In other solutions, the frame buffer 28 may not be part of the device 16, but may be present on another separate chip, in which case the processor 26 writes to that frame buffer 28 via a suitable external output port. The device 16 may not be a separate device as shown in Figure 1, but may form part of the actual display device 12. In this case, the output device 30 is not an output port as such, but is the actual display device 12, which is written directly from the frame buffer 28.
As mentioned above, the frame buffer 28 stores portions of data that, together, form the complete image. Position data that is sent indicates to the processor 26 the position of the portions of data within the image, so that, when they are decoded, the complete image can be formed.
Nevertheless, managing the frame buffer 28 in order to make efficient use of its memory space, is required. As shown in Figure 3, the frame buffer 28 includes space for a number of different portions 20 of display data. The portions 20 may be of different sizes, as shown, particularly if they constitute updates of part of the image. It is therefore desirable that the different sized portions are efficiently packed into the frame buffer 28, so that the least amount of space is wasted between different portions 20. For example, the portion of data 20b would not fit into the space next to portion 20a, since it is portion 20b is
-13too big for that space. However, another portion of data may fit into that space next to portion 20a, and it would be more efficient to store it in that space, rather than on another row in the frame buffer 28.
As shown in Figure 3, in one embodiment, the processing device 10 (the host device), is arranged to include a map 18 corresponding to the frame buffer 28. The map is updated to indicate where, in the frame buffer 28, portions 20 of display data have been stored, so that the processing device 10 can best determine where to store further portions 20 of display data. The processing device 10, where the portions of data are encoded, and/or compressed, determines the size of the portion of data to be sent to the device 16 for storage in the frame buffer 28, the size being variable for different portions 20 of data. The processing device then determines a location on the map 18 where the portion 20 of data would fit in the frame buffer 28. For example, the processing device 10 may maintain a free space list 32, in which the free spaces in the map 18 are included, together with the sizes of the free spaces. The processing device 10 can then easily determine which is the smallest free space into which the portion of data can fit, thereby maximising the packing efficiency. The address in the frame buffer 28 corresponding to that location in the map 18 is then sent to the display control device 16, together with the portion of data, via a transport mechanism 22 and the location on the map 18 is updated to indicate that the corresponding address(es) in the frame buffer 28 are now full. The transport mechanism 22 can be any appropriate transport protocol for transferring the display data, and can include packaging the portions of display data into packets, together with the addresses (and other information), as required.
As mentioned above, the processing device 10 also maintains a list 34 of the portions of data with the order in which they are to be accessed to form the image to be displayed. The list 34 (or at least the information therein, i.e. the order in which the portions of display data stored at particular addresses in the frame buffer 28 are to be accessed and displayed) is also sent to the device 16. As shown in Figure 3, the portions of data, together with their assigned addresses, are inserted into a data stream 52, as is the display list 34, for transfer to the transport mechanism 22 for transport to the display control device 16. Therefore, the different portions of the image can then be sent in the correct order to a rasteriser 58 in the display control device 16, from where the image is sent to the display.
It will be appreciated that the display control device may have more than one frame buffer, for example two (or more) frame buffers, which may be provided in the same RAM
-14or in different RAMs. In such a case, of course, the map 18 would map all the frame buffers and there would be separate display lists 34 for each frame buffer to provide the display order for each display buffer. Furthermore, the different frame buffers may be the same size or may have different sizes. If desired, a separate map 18 could be provided for each frame buffer, whether or not they are provided in the same or different RAMs. Each frame buffer may store a frame of image data, so that one frame buffer is being read out and displayed while the second frame buffer is being filled with the next image. Furthermore, as mentioned above, the display data may be compressed by the processing device and sent to the display control device in compressed form. The received compressed data can then be stored in the frame buffer without decompression, and the may only be decompressed when needed for display.
Alternatively, the RAM in the display control device may not be specifically separated into individual frame buffers, but may have one or more memory spaces such that different portions of compressed data of different frames can be sent to addresses in the same memory space such that portions of one frame may be mixed up with portions of another frame. The list 34 sent to the display control device then provides to the display control device the information regarding an order in which the portions of display data in the memory space are to be accessed from the memory space for individual frames and combined to display an image on the display. The display control device can access from the memory the compressed portions of display data for one frame while receiving and storing in the memory compressed portions of display data for another frame.
If desired, the display control device can determine free space available in its memory, and send feedback indicative of the available free space to the host device. The host device then receives the feedback and updates the map based on the feedback.
Alternatively or additionally, if desired, the display control device can determine quality of a previous portion of display data received from the host device, and send feedback indicative of the quality of the previous portion of display data to the host device. The host device then receives the feedback from the display control device regarding quality of a previous portion of display data and updates the map based on the feedback.
In an embodiment, the host device can determine, based on the map, that the display control device memory requires defragmentation, and can send appropriate instructions to the display control device to perform a defragmentation procedure, and can update the map accordingly. The display control device receives the instructions from the host device, and performs a defragmentation of the memory in response to receiving the
-15instructions.
Figure 4 shows one embodiment of part of the processing device 10 of Figure 3 in more detail. As can be seen, the data is initially held in a data queue 36, from where individual portions of data are passed to a compressor 38 forming part of a compression unit 40. After compression, the compressed portion of data is passed to the transport mechanism 22 for transfer to the display control device 16. However, as mentioned above, once the portion of data is compressed, the size of the compressed portion of data is passed to an allocator 42, which determines (preferably, but not necessarily, from the map 18, using the free space list 32) which is the best free space into which the portion of data can fit. The address in the frame buffer 28 corresponding to that location in the map 18 is then sent to the display control device 16.
Furthermore, however, the allocator 42 can determines the available free space and can feed the available size of free space back to a parameter selector 44, where compression parameters, such as compression parameters, are selected for the compressor 38 to use in compressing the data portion. In this way, for example, if the compressed portion of data is too large to fit into an available free space, the allocator 42 can pass the size of the available free space back to the parameter selector 44, which can then select different compression parameters and the portion of data can be compressed again, so that the compressed portion of data will fit into the available free space. The parameter selector may also, of course, select the compression parameters so that the compresses portion of data has an optimum quality, and, it is found that that there is more available space than would be used for the size of the compressed portion of data, then the compression parameters can be (re)selected to reduce the compression to increase the quality of the compressed portion of data.
Also shown in Figure 4 is an estimator 46, which obtains an estimate of the available free space from the map 18 via the free space list 32 and also obtains a size of the data queue 36 still awaiting compression and transfer. The estimator 46 then determines whether the frame buffer 28 has enough space for all the portions of data still in the data queue 36 if they were to be compressed using the normal compression parameters. If the estimator 46 determines that the frame buffer 28 of display control device 16 is expected to run out of space before all the display data of the current frame is compressed and sent, the estimator 46 provides that information to the parameter selector 44 so that the compression parameters can be adjusted in order to increase the compression of the portions of display data so that all remaining compressed display data
-16of the current frame will fit into the frame buffer 28. Again, although the estimate of free space is described as being determined using the map 18, this is not essential, and any way of determining the available free space could be used.
Apart from determining whether the frame buffer 28 has enough space for all the portions of data still in the data queue 36 if they were to be compressed using the normal compression parameters, the estimator 46 can also control the parameter selector to select the compression parameters based on the time taken to send the compressed portion of data. For example, it is often the case that there is a maximum period of time available for a compressed portion of data to be transmitted from the processing device. This period may depend on the physical resources available for the transport mechanism, or may depend on the transport protocol being used. So, for example, a period of 16ms may be a maximum period available. It is desirable, therefore, to select compression parameters which would reduce the size of the compressed portion of data sufficiently that it could be sent out in a time less than the maximum period. A target time, for example, 14 ms, may therefore be aimed at. A comparator 48 may be used to compare the timestamps from the transport mechanism 22 for previously sent compressed portions of data with the target time, to see whether the target time was being achieved, or whether the actual times for previously sent compressed portions of data were longer or shorter than the target time. Based on that comparison, the estimator 46 can then factor in whether previous compressions were too high or too low to achieve the target time with the optimum quality and can control the parameter selector to select compression parameters accordingly.
It will be appreciated that the target time is the time taken for the processing device 10 to send out the compressed portions of data. Alternatively or additionally, the compression parameters can be selected to achieve the optimum quality based on the time taken for the compressed portions of data to be transmitted across the link from the processing device 10 to the display control device 16. This may depend on the resources available across that link.
The compression parameters may also be selected to make sure that frame buffer is filled in time before the frame buffer needs to be accessed for displaying the frame. For example, the time taken for a previous frame to be sent, transmitted and received may be used to select the compression parameters for the next frame. Furthermore, the compression parameters may also be selected based on the content of the previous frame of data. For example, a frame that comprises text may be compressed less than a
-17frame that comprises video data.
Of course, it is possible that the allocator 42 is not be able to find sufficient free space in the frame buffer 28 for a portion of compressed data, even if it was (re) compressed at a higher compression. If the allocator 42 therefore determines that it cannot allocate space in the frame buffer to the compressed portion of data, it notifies an overflow condition 50, which sends a command to the clear the map 18 and to the data queue 36 to rewind the queue back to the beginning, i.e. to start compressing portions of the frame again. In this case, the parameter selector 44 would select compression parameters from the beginning of the frame to provide a higher compression so that it was more likely that all the frame would fit in the frame buffer. Since the map 18 was cleared, the new compressed portions of data would be allocated any appropriate available space in the frame buffer 28 based on the map 18, and they would, of course, overwrite in the frame buffer 28 anything that was already there. Although this process could be carried out for the existing frame that did not fit, if it were determined that there was not enough time for the frame to be (re)compressed and transmitted to the display control device before it needed to be accessed and displayed, then the frame could be skipped and the next frame could be processed, so that the problem frame was skipped at the display.
Figure 5 shows another aspect of the host device 10 in which the map 18, or a separate map, is maintained showing the various areas of the image. A listing 54 is kept for each of the areas of their age, i.e. when they were last sent and stored in the buffer memory, and their quality, i.e. the compression parameters that were used to compress the portion of display data. These parameters (age and quality) are used by the parameter selector 44 to adjust the compression parameters to improve the quality of portions of display data that were previously compressed with high compression. If a portion of display data, for example, was previously sent with low quality, then, when the area corresponding to that portion next needs to be resent, it would be desirable, perhaps subject to the constraints described above, to select compression parameters that provide a higher quality, so that the area of the image may be considered to be “healed”. A multiplexer/comparator 56 receives information as to which areas of the image have changed for which portions of display data are to be compressed and sent to the display control device. The multiplexer/comparator 56 can then determine whether the corresponding portion of display data previously sent was of low quality, or is of more than a predetermined age, and can then instruct the parameter selector to select compression parameters to compress that portion with a higher quality.
-18Alternatively or additionally, the multiplexer/comparator 56 may be used to receive information regarding when resources may be available. When resources are available, the multiplexor/comparator 56 can select a particular portion of display data that was previously sent in low quality, or is of more than a predetermined age and insert it into the queue for (re)compression at a higher quality (lower compression) to be (re)sent to the display control device to “heal” the image. The resources that may be available may be resources at the host device for compressing the portions of display data, resources at the display control device for receiving the compressed portions of display data, and/or resources at the transport mechanism for transporting the compressed portions of display data from the host device to the display control device. It may be that when no or little new data is required to be sent, then more and more of the image can be healed in this way by sending through higher quality portions of data than were previously sent. Figure 6 shows an embodiment of the system, similar to Figure 3, in which portion 20c of compressed data stored in the frame buffer 28 is only a partial update of the portion of information for that area of the image. As will be apparent, if an area of the image is generally very similar to the same area in the previous frame, with just a small change (designated by “XX”), then a partial update providing only that small change and otherwise indicating that the rest of the area remains the same as before. This can reduce the amount of data, that needs to be sent from the processing device 10 to the display control device 16. The partial update is, of course indicated in the map 18 and the display list 34. The display list 34 is, as previously explained sent to the display control device 16, where it is used by the rasteriser 58 to appropriately access the data from the frame buffer and generate the raster image, with the changed portion XX to the display 12.
Figure 7 shows an alternative embodiment of the display control device 16. As shown, the data stream 52, such as that produced by the processing host device of Figure 3, is received via the transport mechanism 22 and the data portions, which in this embodiment are compressed, are stored in the frame buffer 28 according to the addresses sent with the data portions in the data stream 52 from the host device. The display list 34 indicates the order in which the display portions are to be displayed to a fetcher 60, which fetches the compressed portions 20a and 20b of display data from the appropriate locations in the frame buffer 28 in the order provided by the display list 34. The compressed portions 20a and 20b of display data are then passed to a decompressor 62, which decompresses them, and passes them to the rasteriser 58 to produce the rasterised image for display.
-19ln a preferred embodiment, the display control device 15 further includes a nonvolatile programmable memory 64 for storing compressed image data in the same format as the compressed display data in frame buffer 28. The compressed image data can be start-up splash image or some other image that is pre-programmed into the memory 64 so 5 that when the display control device is not operating to display data from the frame buffer 28, for example at start-up or when the host device has been disconnected from the display control device, the display control device can decompress the compressed image data for rasterising for output to the display in the same way as it would otherwise decompress the display data from the frame buffer 28.Although several embodiments 10 have been described in detail above, it will be appreciated that various changes, modifications and improvements can be made by a person skilled in the art without departing from the scope ofthe present invention as defined in the claims.

Claims (17)

Claims
1. A method, at a display control device connectable to a host device, of storing display data sent from the host device to the display control device in a memory for displaying on a display, the method comprising:
receiving a portion of display data together with an address;
storing the portion of display data at the address in the memory;
receiving information regarding an order in which the portions of display data in the memory are to be accessed from the memory and combined to display an image on the display;
accessing the portions of display data from the memory; and combining the accessed portions of display data to form an image for display.
2. A method according to claim 1, wherein the memory comprises a frame buffer memory for storing one or more frames of image data.
3. A method according to claim 2, wherein the received portion of display data is a compressed portion of display data.
4. A method according to claim 3, further comprising decompressing the compressed portion of display data from the frame buffer memory prior to combining the portions to form the image.
5. A method according to any preceding claim, wherein the portion of display data comprises a partial update of the display data including only the display data that has changed from any previous frame.
6. A method according to any preceding claim, wherein different portions of data may be portions of different frames and may be stored in the memory such that portions of one frame may be mixed up with portions of another frame.
7. A method according to claims 6, further comprising accessing from the memory the compressed portions of display data for one frame while receiving and storing in the memory compressed portions of display data for another frame.
-21
8. The method according to any preceding claim, further comprising determining free space available in the memory, and sending feedback indicative of the available free space to the host device.
9. The method according to any preceding claim, further comprising determining quality of a previous portion of display data received from the host device, and sending feedback indicative of the quality of the previous portion of display data to the host device.
10. The method according to any preceding claim, further comprising receiving instructions, from the host device, to perform a defragmentation of the memory, and performing a defragmentation ofthe memory in response to receiving the instructions.
11. A display control device configured to perform a method according to any preceding claim.
12. A display control device according to claim 11, wherein the display control device is integrated into the display.
13. A display control device according to either claim 11 or claim 12, comprising a random access memory for storing compressed display data received from the host device, a decompressor for decompressing the compressed display data from the random access memory for rastering for output to the display, and a non-volatile programmable memory for storing compressed image data, wherein the decompressor decompresses the compressed image data for rastering for output to the display when it is not required to display the compressed display data from the random access memory.
14. A display control device according to claim 13, wherein the compressed image data comprises a start-up splash image.
15. A display control device according to claim 13, wherein the compressed image data comprises an intermediate image indicating a disconnection from the host device.
16. A computer readable medium including executable instructions which, when executed in a processing system, cause the processing system to perform a method
-22 according to any one of claims 1 to 10.
17. A display system comprising: a host device configured to:
5 determine an address in a memory of a display control device where a portion of display data would fit;
send to the display control device the address, together with the portion of display data; and send to the display control device information regarding an order in which 10 the portions of display data in the memory are to be accessed from the memory and combined to display an image on a display;
a display control device according to any one of claims 11 to 15; and a display coupled to the display control device and configured to display the image.
Mr Iwan Thomas
14 March 2018
GB1801346.6
GB1801346.6A 2012-12-21 2012-12-21 Management of memory for storing display data Active GB2556743B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
GB1801346.6A GB2556743B (en) 2012-12-21 2012-12-21 Management of memory for storing display data

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
GB1801346.6A GB2556743B (en) 2012-12-21 2012-12-21 Management of memory for storing display data
GB1223329.2A GB2509169B (en) 2012-12-21 2012-12-21 Management of memory for storing display data

Publications (3)

Publication Number Publication Date
GB201801346D0 GB201801346D0 (en) 2018-03-14
GB2556743A true GB2556743A (en) 2018-06-06
GB2556743B GB2556743B (en) 2018-12-05

Family

ID=47682540

Family Applications (3)

Application Number Title Priority Date Filing Date
GB1223329.2A Active GB2509169B (en) 2012-12-21 2012-12-21 Management of memory for storing display data
GB1801346.6A Active GB2556743B (en) 2012-12-21 2012-12-21 Management of memory for storing display data
GB1801348.2A Active GB2556262B (en) 2012-12-21 2012-12-21 Management of memory for storing display data

Family Applications Before (1)

Application Number Title Priority Date Filing Date
GB1223329.2A Active GB2509169B (en) 2012-12-21 2012-12-21 Management of memory for storing display data

Family Applications After (1)

Application Number Title Priority Date Filing Date
GB1801348.2A Active GB2556262B (en) 2012-12-21 2012-12-21 Management of memory for storing display data

Country Status (4)

Country Link
US (1) US9947298B2 (en)
EP (3) EP3726838B1 (en)
GB (3) GB2509169B (en)
WO (1) WO2014096791A1 (en)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9892707B2 (en) * 2013-03-14 2018-02-13 Displaylink (Uk) Limited Decompressing stored display data every frame refresh
US9721528B2 (en) * 2014-11-10 2017-08-01 Xilinx, Inc. Processing system display controller interface to programmable logic
CN105549943B (en) * 2016-02-17 2018-10-23 京东方科技集团股份有限公司 Display drive method and display system
EP3472806A4 (en) 2016-06-17 2020-02-26 Immersive Robotics Pty Ltd Image compression method and apparatus
WO2018123801A1 (en) * 2016-12-28 2018-07-05 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ Three-dimensional model distribution method, three-dimensional model receiving method, three-dimensional model distribution device, and three-dimensional model receiving device
AU2018217434C1 (en) 2017-02-08 2023-04-27 Immersive Robotics Pty Ltd Displaying content to users in a multiplayer venue
AU2018372561B2 (en) 2017-11-21 2023-01-05 Immersive Robotics Pty Ltd Image compression for digital reality
AU2018373495B2 (en) 2017-11-21 2023-01-05 Immersive Robotics Pty Ltd Frequency component selection for image compression

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH08110943A (en) * 1994-10-11 1996-04-30 Canon Inc Address generator and image processor
JPH08194642A (en) * 1995-01-20 1996-07-30 Fuji Film Micro Device Kk Image processing system

Family Cites Families (37)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4394774A (en) * 1978-12-15 1983-07-19 Compression Labs, Inc. Digital video compression system and methods utilizing scene adaptive coding with rate buffer feedback
US4823108A (en) * 1984-05-02 1989-04-18 Quarterdeck Office Systems Display system and memory architecture and method for displaying images in windows on a video display
US5131080A (en) * 1987-08-18 1992-07-14 Hewlett-Packard Company Graphics frame buffer with RGB pixel cache
US5170468A (en) * 1987-08-18 1992-12-08 Hewlett-Packard Company Graphics system with shadow ram update to the color map
US5159678A (en) * 1990-06-11 1992-10-27 Supercomputer Systems Limited Partnership Method for efficient non-virtual main memory management
US6366289B1 (en) * 1998-07-17 2002-04-02 Microsoft Corporation Method and system for managing a display image in compressed and uncompressed blocks
KR20010057947A (en) * 1999-12-23 2001-07-05 신현준 Unitary processing apparatus for displaying information on a plurality of displaying unit
JP3600567B2 (en) * 2001-09-21 2004-12-15 株式会社東芝 Video distribution device
JP3977652B2 (en) * 2002-02-08 2007-09-19 日本電信電話株式会社 Quality control coding control method and apparatus for distribution service, and program thereof
TWI220246B (en) * 2003-05-23 2004-08-11 Mediatek Inc Method for generating an initial screen of an image display system
CN1319029C (en) * 2003-06-18 2007-05-30 联发科技股份有限公司 Method for generating starting picture of video playing device
JP4755642B2 (en) * 2004-04-26 2011-08-24 ストアウィズ インク Method and system for file compression and operation of compressed files for storage
US7386663B2 (en) * 2004-05-13 2008-06-10 Cousins Robert E Transaction-based storage system and method that uses variable sized objects to store data
US7302543B2 (en) * 2004-06-16 2007-11-27 Nec Laboratories America, Inc. Compressed memory architecture for embedded systems
US7616821B2 (en) * 2005-07-19 2009-11-10 International Business Machines Corporation Methods for transitioning compression levels in a streaming image system
JP4011591B2 (en) 2005-07-20 2007-11-21 シャープ株式会社 Manufacturing method of liquid crystal display panel with microlens
GB0516713D0 (en) 2005-08-13 2005-09-21 Newnham Res Ltd A display system and method of operating a display system
JP5061273B2 (en) * 2005-10-03 2012-10-31 新世代株式会社 Image generation device, texture mapping device, image processing device, and texture storage method
CN100499758C (en) * 2006-01-13 2009-06-10 深圳创维－Rgb电子有限公司 Television set capable of changing opening picture and method thereof
JP4493626B2 (en) * 2006-05-25 2010-06-30 株式会社ソニー・コンピュータエンタテインメント Multiprocessor system, library module, and drawing processing method
KR101120019B1 (en) * 2007-04-04 2012-03-26 삼성전자주식회사 Apparatus and method for receiving and reproducing broadcasting data through wireless communication
JP4755624B2 (en) * 2007-04-23 2011-08-24 ルネサスエレクトロニクス株式会社 Motion compensation device
JP2009055162A (en) * 2007-08-24 2009-03-12 Panasonic Corp Recording reproducing terminal
US7982752B2 (en) * 2008-07-28 2011-07-19 Mediatek Inc. Multi-format image display apparatus and method
EP2285111A1 (en) * 2009-08-07 2011-02-16 Canon Kabushiki Kaisha Method for sending compressed data representing a digital image and corresponding device
US8619866B2 (en) * 2009-10-02 2013-12-31 Texas Instruments Incorporated Reducing memory bandwidth for processing digital image data
JP5196666B2 (en) * 2009-10-03 2013-05-15 Ｋｄｄｉ株式会社 Data transmission device, program and method for completing data transmission by expiration time
US8351600B2 (en) * 2009-10-30 2013-01-08 Cleversafe, Inc. Distributed storage network and method for encrypting and decrypting data using hash functions
WO2011055490A1 (en) * 2009-11-05 2011-05-12 日本電気株式会社 Distribution system
US9075560B2 (en) * 2009-12-10 2015-07-07 Microsoft Technology Licensing, Llc Real-time compression with GPU/CPU
CN101827199A (en) * 2010-04-06 2010-09-08 福建新大陆通信科技有限公司 Method for realizing startup picture change of set top box through external memory medium
US20110299589A1 (en) * 2010-06-04 2011-12-08 Apple Inc. Rate control in video communication via virtual transmission buffer
GB2485576B (en) * 2010-11-19 2013-06-26 Displaylink Uk Ltd Video compression
GB2485613B (en) 2010-11-22 2014-08-13 Displaylink Uk Ltd Display control device
KR101158876B1 (en) * 2012-03-09 2012-06-25 엘지디스플레이 주식회사 Display device and method for controlling panel self refresh operation thereof
US9892707B2 (en) * 2013-03-14 2018-02-13 Displaylink (Uk) Limited Decompressing stored display data every frame refresh
US9396564B2 (en) * 2014-03-26 2016-07-19 Empire Technology Development Llc Atlas generation based on client video configuration

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH08110943A (en) * 1994-10-11 1996-04-30 Canon Inc Address generator and image processor
JPH08194642A (en) * 1995-01-20 1996-07-30 Fuji Film Micro Device Kk Image processing system

Also Published As

Publication number Publication date
EP2936298A1 (en) 2015-10-28
EP3726838A1 (en) 2020-10-21
US9947298B2 (en) 2018-04-17
GB201801346D0 (en) 2018-03-14
GB2556262B (en) 2018-11-21
EP3726838B1 (en) 2024-04-10
GB201801348D0 (en) 2018-03-14
GB2509169B (en) 2018-04-18
GB2556743B (en) 2018-12-05
GB201223329D0 (en) 2013-02-06
GB2509169A (en) 2014-06-25
US20160196804A1 (en) 2016-07-07
EP2936298B1 (en) 2020-06-17
WO2014096791A1 (en) 2014-06-26
GB2556262A (en) 2018-05-23
EP3734589A1 (en) 2020-11-04

Similar Documents

Publication Publication Date Title
US9947298B2 (en) Variable compression management of memory for storing display data
US9619866B2 (en) Image transformation
US20160125568A1 (en) Management of memory for storing display data
US9619916B2 (en) Method for transmitting digital scene description data and transmitter and receiver scene processing device
US9426476B2 (en) Video stream
GB2484736A (en) Connecting a display device via USB interface
KR20160080929A (en) Apparatus and method of adaptive ultra high definition multimedia streaming service based on cloud
US20160005379A1 (en) Image Generation
US20140355664A1 (en) Optimizing video transfer
US9226003B2 (en) Method for transmitting video signals from an application on a server over an IP network to a client device
US20140218379A1 (en) Device, system and method for transferring network data
US20120218292A1 (en) System and method for multistage optimized jpeg output
GB2528870A (en) Managing display data for display
US10249269B2 (en) System on chip devices and operating methods thereof
CN107318021B (en) Data processing method and system for remote display
CN107318020B (en) Data processing method and system for remote display
US7920747B2 (en) Pre-distribution image scaling for screen size
US20240087169A1 (en) Realtime conversion of macroblocks to signed distance fields to improve text clarity in video streaming
JP2022042214A (en) Image processing device and screen transfer program
CN116489132A (en) Virtual desktop data transmission method, server, client and storage medium
JP2016051945A (en) Image processing system, information processing device, projection apparatus, method and program