Classifications

• • G—PHYSICS
  • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
  • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
  • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
  • G09G5/42—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of patterns using a display memory without fixed position correspondence between the display memory contents and the display position on the screen
• • G—PHYSICS
  • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
  • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
  • G09G2340/00—Aspects of display data processing
  • G09G2340/02—Handling of images in compressed format, e.g. JPEG, MPEG

Definitions

• the present invention relates to a method for constructing an image to be displayed on a display system. , such as a screen from a plurality of objects.
• An object is a graphic element that can be displayed on the screen. It can be video, bitmap or vector, for example. If the "object” approach is a widespread concept at the application level, current display mechanisms are traditionally satisfied with going to look for the image already built somewhere in memory, in a predefined area which is generally the size of a video frame, hence its name “frame memory” or "frame buffer” in English.
• an object copying module is commonly used allowing an object to be retrieved in any area and to be written to the frame memory.
• the graphics and video systems according to the prior art use frame memories in which the images are produced from graphic and video primitives and from objects copied in these areas.
• these are generally recombined at the time of display using mechanisms of multiplexing, inlay ("chroma-keying" in English) or fusion by transparency ("alpha-blending "in English). This approach is very demanding on memory space, especially for systems wishing to offer very good visual quality, and which therefore require multiple memory plans.
• the present invention aims to remedy the aforementioned drawbacks by proposing an image construction method in which the useful memory for this construction and the useful power of the system are reduced compared to the systems of the prior art.
• the above object is achieved with a method for constructing an image capable of being displayed on a display system from a plurality of objects.
• the image is constructed line by line directly from the objects by carrying out the following steps: - for each line of the display system, this line is constructed on the fly by retrieving and storing, in real time, in at least one line memory, all pixels relating to the objects intended to be displayed on said line, and these pixels are sent to the display system according to a series of pixels, so that the the image is formed only on said display system; and in that the line construction mechanism is synchronized in line and in the frame with the plug system.
• the present invention makes it possible to display objects directly on a display system such as a screen.
• the object is at the heart of the image construction mechanism.
• the construction of the image is volatile since its mechanism does not use a frame memory and that the image does not exist anywhere else except on the display system.
• the image is first constructed From the objects, we store this image in a frame memory, then we display the already formed image.
• display system is meant a system comprising one or more screens synchronized online and in a frame, and controlled by a single controller generating screen timing signals, this controller being placed in the accelerator. For example, from the accelerator point of view, two screens side by side can be considered as a single screen of double definition.
• the display system can include a display memory capable of receiving the pixels forming the image. The image is then formed within this display memory rather than directly on a screen.
• the present invention has a definite advantage over standard systems since it requires little memory (absence of frame memory), therefore a lower cost price.
• the image is constructed directly, the operation of preparing the frame in a frame memory is avoided, hence a reduction in the necessary bandwidth at the memory level. Having fewer operations to display an image on the screen, the entire system can be less powerful, therefore less energy consuming, less disturbing, which is essential for an on-board system for example.
• the present invention therefore makes it possible to produce images of very high visual quality from low-power components.
• dynamic image is meant an image composed of 2D objects which can evolve over time, move, transform, change state, ...
• two are used. line memories performing, offset, each successively a construction step then a display step so that when a first line memory displays pixels on the current line, pixels are constructed and stored in the second line memory to be displayed on the next line.
• each object to be present on this line is independently identified according to a set variables specific to each object; a useful area corresponding to the line considered is detected in each object thus identified; and the raw data from said useful area is converted into pixels compatible with the display format.
• the memory where the objects are stored is accessed from electronic mechanisms and the raw data from the useful areas is temporarily stored in storage means.
• transformations and effects which are in fact digital processing, can for example be a partial display or "clipping" in English, resizing or “resizing” in English, transparency, thresholding, anamorphism, filters, ...
• each object is therefore independently configurable and we can change the parameters of an object without affecting the other objects.
• the realization of an HMI is therefore extremely simplified.
• the position of this pixel in the line considered it is first possible to determine the position of this pixel in the line considered, then further to determine the value of the pixel already present in this line at the same position. Furthermore, when writing a pixel in a line memory, this pixel being intended to be displayed at a given position on a line of the display system, it is possible to apply a mixer between this first pixel and a second pixel currently stored in the line memory at said same given position, the mixer being applied in a manner proportional to the level of transparency of said first pixel.
• the video frame comprising two distinct time zones which are a vertical dead zone, called “vertical blanking interval VBI" in English and a vertical active zone, called “vertical active interval VAI "in the English language, corresponding respectively to the interval between the two active frames and to the duration of display of an active frame
• the stages of construction and display are carried out on the fly during each vertical active zone, and we carry out, during the vertical dead zone, any preparation necessary for the construction process.
• Said “necessary preparation” can be any action for preparing the frame to come.
• said "necessary preparation” is preferably any action of preparation or anticipation of construction of the line to come.
• This preparation may include determining or updating (when the variables have already been determined) a set of variables specific to each object. These variables can be extracted from each hardware descriptor associated with each object, then stored within the accelerator.
• a hardware descriptor is the set of graphic and display parameters defining the corresponding object.
• the objects can be of different natures. We can have video, vector, bitmap, or other objects. Thanks to the descriptor, we can manage heterogeneous objects in the same way.
• Said preparation can also include sorting the objects in order of depth, for example decreasing or increasing so as to prioritize the order with which the objects will be superimposed on the screen.
• Sorting can be carried out by scanning all the objects for the first time to determine the most buried object, then by restarting the scanning of the objects without taking into account the objects already sorted, until all the objects are sorted.
• sorting can be done during construction.
• the volatile construction mechanism according to the invention makes it possible to take into account for each object a level of depth for the final display, making it possible to simplify the management of the recoveries of objects and windows from an application point of view, by delegating in "hardware" this complex and tedious work from a software point of view.
• the invention does not impose from a theoretical point of view a maximum number of managed levels.
• the present invention makes it possible to manage a very large number of graphic layers without being penalizing or sizing, both in terms of "hardware" resources, CPU power, and amount of memory. We can therefore easily produce complex transparencies: several transparent objects one above / below the other.
• the construction of the image is managed by a hardware accelerator by means of electronic mechanisms, that is to say a hardware management, an electrical management by state machines.
• This hardware accelerator is capable of generating the video frame of the display system. However, it can also synchronize and lock onto this video frame of the display system from synchronization information supplied by the display system itself.
• a system is proposed for constructing an image capable of being displayed on a display system from a plurality of objects notably stored in random access memory.
• the system comprises a processing unit such as a microprocessor associated with a hardware accelerator: to construct the image, line by line, directly from the objects, by retrieving and storing, in real time, in a line memory, all pixels relating to objects intended to be displayed on each active line of each video frame of the display system, to display these pixels by sending them to the display system according to a series of pixels at the pixel frequency, so that the image is formed as on said display system, and to synchronize the line construction mechanism, in line and in frame, with the display system.
• a processing unit such as a microprocessor associated with a hardware accelerator: to construct the image, line by line, directly from the objects, by retrieving and storing, in real time, in a line memory, all pixels relating to objects intended to be displayed on each active line of each video frame of the display system, to display these pixels by sending them to the display system according to a series of pixels at the pixel frequency, so that the image is formed as on said display system, and to synchronize the line construction mechanism, in line
• the hardware accelerator comprises the following elements: an object manager for activating and characterizing treatments linked to the objects, a memory for storing the parameters and variables of each object, - a hardware module of the DMA ("Direct Memory Access") type. "in English) controlled by the accelerator (and not by the processing unit) capable of recovering data from objects from a memory, a buffer memory for temporarily storing the data coming from the DMA hardware module, a decompression module and for converting raw data from the buffer into pixels, a multiplexer for selecting the pixel to be displayed between that from the decompression and conversion module and that from an external source, provided that the external source, the accelerator and screen are synchronized, a mixer to mix the pixel from the multiplexer and a pixel currently displayed checked as a function of the transparency of the pixel coming from the multiplexer, and two line memories ensuring successively in turn the display on the current line of the display system of pixels previously stored, and the storage of pixels coming from the mixer and which will be displayed on the next line.
• DMA Direct Memory Access
• FIG. 1 is a simplified schematic view of a process construction and display of an image according to the prior art
• FIG. 2 is a simplified schematic view of a process for constructing and displaying an image line by line in real time according to the present invention
• FIG. 3 is a diagram illustrating two time zones of a video frame according to the invention
• - Figure 4 is a diagram illustrating how two line memories are used for the construction and display of image line by line according to the invention
• Figure 5 is a simplified schematic view of a hardware accelerator implementing the method according to the present invention.
• a module 4 for copying objects constructs said image which is then stored in a frame memory 5
• the latter is generally the size of a video frame.
• a display module 6 is then content to fetch the image already constructed in the frame memory 5 and to display it on the screen 1 for the duration of a video frame.
• the present invention implements a completely different process.
• a hardware processor or "hardware" in English, otherwise called graphic hardware accelerator for constructing said image on the fly and in real time from objects 2.
• This accelerator in particular shown in FIG. 5, performs various operations during the video frame and repeats them with each new frame. So, during a frame, for each active line of the frame, the system according to the invention constructs in real time the pixels to be displayed on this line from objects
• the hardware accelerator includes a module
• VBI vertical blanking interval
• VAI vertical active interval
• the volatile construction For each active line of the screen, the volatile construction consists in filling a line memory with the line segments of the objects which are active in the line considered. The content of the line memory is then sent to the screen at the rate of the pixel frequency.
• the present invention allows that when a line (L1) is displayed, during this time, the next line (L) is under construction.
• the H synchro corresponds to the line synchronization (Horizontal)
• the V synchro corresponds to the frame synchronization (Vertical).
• the first step can be to decode a hardware descriptor in order to develop variables that will be used during the construction of the lines during the VAI zone.
• a hardware descriptor is associated with each object.
• "Material descriptor” means a coherent set of data, created and generally initialized by an application process. This descriptor contains all the information so that the hardware accelerator can display the object associated with it. This information, in particular stored in registers or memories, includes graphic parameters describing the nature of the object and display parameters. These can be separated into essential parameters (position, display attributes such as transparency level, etc.) and transformation parameters (partial display or "clipping", resizing or “resizing”, rotation, anamorphism, filters ).
• Each object can be of a different nature in that it belongs to a given class (vector, video, bitmap, ...) or includes a given color organization (palette, black and white mode, colors in 16, 24, 32 bits, with transparency or not, ).
• the descriptor can be local to the hardware accelerator or in an external memory. In the latter case, it is first necessary to recover the descriptor before starting decoding. Decoding the descriptor consists in extracting all the variables that will be used during the volatile construction. The decoding of the descriptor will be more or less long, more or less complex, depending on the capabilities of the hardware accelerator to perform advanced and complex functions (filters, "resizing”, anamorphism, "clipping", displacements, ...) .
• Decoding certain information from the descriptor may even be useless if the parameters supplied already correspond to the variables useful for volatile construction.
• the idea is to delegate the pre-processing of the raw data at the hardware level, so that it can guarantee real time and be synchronized in the frame with the volatile construction.
• This also makes it possible to store in the descriptor advanced parameters which will be directly (or almost) transmitted by an application called API ("Application Program Interface" in English) object oriented.
• API Application Program Interface
• the hardware accelerator is associated with a microprocessor which is programmed to so as to offer a set of predefined functions accessible via this API.
• Its descriptor then contains: the address where the image data in memory begins; the size of the image; the position of the object on the screen; the color organization of the bitmap image as it is stored in memory; the overall level of transparency of the object; ...
• the coordinates of the object are slightly negative compared to the origin of the screen. Only the visible part of the object will have to be processed to be displayed, the rest being clipped. This means that the actual position of the object will have to be determined, that the address of the object in memory will have to be updated to point directly to the first visible pixel.
• the size of the line segment useful for the volatile construction of a line must also be determined as a function of "clipping", but also as a function of the color organization of the pixels in memory.
• the decoding of the descriptor leads to a series of working variables which will be exploited by the hardware accelerator during the volatile construction.
• the useful variables will advantageously be stored locally at the hardware accelerator for reasons of accessibility. Some of these variables will undergo modifications as the line processes progress. For example, at the end of descriptor decoding, the address pointer points to the first active line segment of the object stored in memory. But the address pointer must point as the line processes progressively to the new active line segments.
• a second step can be to sort the objects in decreasing order of depth. This process allows you to prioritize the order in which the objects will be superimposed on the screen.
• Sorting can be done, for example, by scanning all the objects for the first time to determine the most buried object, then by restarting the scanning of the objects without taking into account the objects already sorted, until all the objects are sorted.
• each object could for example contain in one of its registers, the index of the next object in an order of decreasing depth, so that the process of line construction of the image can pass simply from from one object to another, and in the order required by complex transparency.
• Complex transparency is only obtained by the inventive principle of volatile construction according to the invention and described below
• the volatile image building mechanism is a line process, which means that it executes again for each active line of the screen, i.e. for each line of the VAI (see figure 3).
• Line process The line construction mechanism is synchronized with the display mechanism, which simply consists in sending a series of pixels at the pixel frequency to the screen. To do this, we can very well use only one line memory for these two mechanisms, knowing that the construction mechanism must then be timed so as not to update pixels which have not yet been sent to the screen. Two or more line memories can also be used. In this case, some so-called “off screen” used to build the following lines, while a so-called “on screen” is sent to the screen to fill the current line.
• retrieving the useful segment requires, among other things, to determine: the memory address where the first pixel of the object segment is located; - the number of pixels contained in the segment of the object; the law of horizontal and vertical variation between two pixels. The determination of these variables is not enough to recover the useful segment of the object.
• the following object parameters must also be considered: the width of the object stored in memory (in bytes); the color organization of the pixels; the compression law, if the object is stored in a compressed way.
• a process is carried out successively allowing for each object: a) to calculate the variables necessary for the recovery of the useful segment; b) retrieve the necessary object parameters; c) to update the variables in the registers in anticipation of the next lines, for example the memory address pointer.
• We also execute a process allowing access to the memory where the objects are stored, from hardware commands.
• DMA hardware Direct Memory Access
• the volatile construction "hardware" mechanism according to the invention can advantageously be implemented in an FPGA which makes it possible to integrate all the modules and processes necessary for implementing the method according to the invention.
• a hardware accelerator 21 can be architecture on such an FPGA.
• a memory 20 SDRAM, DDRAM, etc.
• This memory 20 is a random access memory external to the accelerator 21.
• An object manager 14 carrying out the activation and characterization of the processing operations specific to the object t (intra-object) as well as management between objects (inter-objects).
• a register 15 for storing the working parameters and variables for each object.
• a "DMA hardware” module 12 controlled by the object manager 14 and capable of retrieving the data (for example bitmap images) in the external memory 20.
• a buffer memory 13 for temporarily storing the raw data from DMA 12, when DMA processes and a decompression / conversion pipeline 16 are not synchronized.
• the object segment is retrieved by the DMA, it is a question of converting the raw data originating from the DMA into pixels in the output format selected so that it can be stored in the line memory.
• this requires knowing: the color organization of the pixels; the law of horizontal variation between two pixels; and - the compression law, if the image is stored in a compressed manner.
• This pipeline also includes means for transforming the pixels. Once the raw data has been converted into pixels, it is possible to apply digital processing, such as for example filters or effects on these pixels.
• chroma-keying the pixel becomes transparent if its value is equal to a reference value which one calls the value of "chroma-key”
• - thresholding by luminance the pixel becomes transparent if the value of its luminance level is less than a reference value which is called threshold value
• color filters modification of the chrominance of the pixel
• low-pass, high-pass filters influence of neighboring pixels to the current pixel by a suitable pipeline
• a pixel mixer 18 ("blending") according to the transparency of the pixel considered.
• Writing pixels to line memory is the last step in the line process of volatile construction. The writing of a pixel in the memory requires knowing: the position of the pixel in the line: coordinate "x" relative to the origin equal to the first pixel of the screen; and - the value of the pixel already present in the line at the same position, in particular when the object is semi-transparent.
• a multiplexer 17 selects either the pixel from the main video stream (video_in), or the pixel from the stream from the DMA 12. This mechanism requires that the timing of the accelerator be synchronized and locked on the video source.
• the operating frequency of the pipeline is several times the pixel frequency of the screen, and the multiplexer selects the video pixel at the rate of once per pixel period (at the pixel frequency). The rest of the time, the multiplexer manages the stream from DMA 12. When there is no main video stream, it is replaced by a background color.
• This architecture example also makes it possible to perform a video merge with the background color in proportion to a level of video transparency (alpha_video_in) without implementing a second merge module.
• the image construction method according to the invention makes it possible to manage the level of depth between graphic and video objects without limiting the maximum number of layers.
• This number of layers is not scalable in terms of the hardware resources of the accelerator.
• This method also makes it possible to manage the transparency between the graphic and video objects as a function of the positioning of the objects on the z axis regardless of the number of graphic layers which is not dimensioned to obtain the overall complex transparency.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Computer Hardware Design (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Controls And Circuits For Display Device (AREA)
• Image Generation (AREA)
• Processing Or Creating Images (AREA)
• Digital Computer Display Output (AREA)

Applications Claiming Priority (2)

Publications (2)

Family

ID=34944750

Family Applications (1)

Country Status (4)

Families Citing this family (1)

Family Cites Families (12)

• 2004
  • 2004-04-08 FR FR0403721A patent/FR2868865B1/fr not_active Expired - Fee Related
• 2005
  • 2005-04-08 EP EP05753728.4A patent/EP1738349B1/de not_active Not-in-force
  • 2005-04-08 WO PCT/FR2005/000857 patent/WO2005104086A1/fr active Application Filing
  • 2005-04-08 US US11/547,812 patent/US20070211082A1/en not_active Abandoned

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events