CA1257719A - Graphics display system - Google Patents

Abstract

GRAPHICS DISPLAY SYSTEM

ABSTRACT

A graphics display system in which a large number of object elements can be stored in a pattern memory and called up for display in real time.

Description

~S~7~9 BACKGROUND OF THE INVENTION

The existing architectures of personal home computers and video games provide graphics perfoxmance ~hich is severely limited due to technological restrictions imposed when the designs were developed. In the late 1970's, most graphics systems being designed used either 8 bit microprocessors or low performance 16 bit machines. By today's requirements, graphic processors of that time were mediocre in terms of resolution, color definition and animation support. The memory speed of these machines was insufficient for the high bandwidth encountered in the video domain. These and other restrictions caused the graphic display systems to be deliberately compromised.
The simplest approach to minimize bandwidth and memory re~uirement is to implement a "card" approach. This approach segments the screen into a matrix of small rectangles. Each rectangle may accept a simple outline pattern filled with a single color to represent a primitive object. This approach gives satisfactory representation of an image perceived in two dimensions. However, the realism of an image is related to perception in three dimensions.
Moreover, the basic single color card approach introduces severe handicaps when one attempts to portray overlapped or merged images.
A second approach taken to graphics systems has been to ~Z~S~77~9 employ a one to one correspondence between system memory and usable screen positions. This technigue is referred to as bit map or pixel (picture element) map. Unfortunately, the bit map approach has restrictions of its own. The image quality becomes unacceptable if the memory is minimized to remain with cost and speed constraints. Even if sufficient memory is provided, older processors cannot create the images fast enough to support animation.
In consideration of these problems, hybrid systems were created which provided unit screen bit map representation of a single "card" position. In this development bit map card approaches (still small rectangles) were joined with the notion of object independence, which allowed objects to be placed randomly around the screen and overlayed on top of each other. This concept aided in the generation of multiple planes containing objects to support three dimensional effects (which are rudimentary compared to the effects obtainable by the system here disclosed). While these innovative hybrids spawned an explosive business in programmable T.V. games, they are not easily enhanced, thus restricting their further use. To sustain personal computers and other graphics terminals throughout the late 1980's, more advanced and flexible architectures must be created.
The software environment available in most graphic ~2~

system architectures assembles characters and patterns in a simple sequential list organization. This format, al~hou~h easy to implement, is quite cumbersome and ineffective in environments which are constantly ~eing modified, updated and destroyed such as in image construction and animation.
Systems currently do not provide enough capability to support commonly encountered data structures that are used within a typical data base or programming environment. The reorganization of the sequential data pattern is essential for future generation graphic systems.
A ~raphics system which provided a very substantial improvement in the art is disclosed in applicant's U.S.
patent number 4,700,181 filed September 30, 1~83 (hereinafter referred to as the "earlier system"). In this system, which is described further hereinbelow, a pattern memory stores data representing a plurality of object elements. Another memory stores data identifying the object elements and data including instructions defining things such as the nature, location in pattern memory, display location and status of the object elements or parts thereof. In an embodiment of the system, this data is in the form of a linked list. For each line of a frame to be displayed, the linked list is traversed to determine which of the object elements impacts the particular line. A
processing means fetches the bit map of the appropriate line ~L25~ 9 of each such object element and enters it in one of two line buffers in overlay fashion. The object elements are overlayed in the line buffer in order of their visible priority (i.e., what is in front of what) so that when the line is completely formulated the higher priority object elements will be visible and object elements that are "blocked" by them will not be visible. The use of a pair of line buffers allows the next line to be constructed while the previously constructed line is being displayed. The display is achieved in real time.
Applicant's earlier system represented a si~nificant advance over the prior art, particularly in the ability to select from large amounts of data the information necessary to produce display images of desired precision and complexity, all within real time, but, as with any system, there was an upper limit on the amount of data which could be thus handled in real time.
There was therefore a strong incentive to produce a system which could further push forward the technology for producing quality display images, for expanding the amount of data which could be manipulated in real time to produce such images, and to further facilitate the ability to animate such images, all while still utiliæing commercially practical computer hardware. The display system of the present invention is the result of that incentive. It ~L~S7 7~

utilizes many of the novel method and apparatus approaches of the aforementioned earlier system, for example the features of storage of instructions in terms of linked lists, ways of choosing from a very extensivé color palette with only a minimal use of memory, painting of individual object elements in the display in terms of relative visible priority, and the use of a pair of buffers which alternately function to receive data to be displayed and to produce the desired display.
The enhanced capability of the system of the present invention significantly expands the potentialities of a graphics system, including the earlier system, particularly in terms of manipulation of data in order to create a scene and change it, all within the time constraint of a full motion video display system such as a CRT monitor or a TV
set. As was the case with the earlier system, the computer graphics system of the present invention allows the user to manipulate complex realistic images in real time, but to do so with greater flexibility and precision than had previously been thought possible with any but the most complex and expensive computer systems. Such speed and resolution is derived from the way information is stored, retrieved, and located.
As used herein, "real time" refers to the time required by a full motion video display system, such as one meeting standards of the National Television Standards Committee, to provide commercially acceptable representations of motion.
Typical of such display systems are CRT monitors, TV
receivers and the like. The system of the present invention produces instant interactive images within the time required to scan a frame of the display system, and can sustain those images indefinitely. Thus, designed with speed in mind, this system is capable of producing life-like animation for domestic television set or monitors. This animation can be made up of entirely computer generated shapes or pictures scanned into the host computer with a video camera. In either case, the resolution provided is far better than what current low cost computer video systems provide.
It is therefore an object of the present invention to devise a system to store and handle more detailed data about a scene than has previously been thought practical and which minimizes the time required to retrieve that data and produce a picture.
It is another object of the present invention to devise a process and equipment to represent an image in storage and to facilitate the retrieving of a maximum amount of data in order to form an image of maximum detail, all in a minimum amount of time.
It is another object of the present invention to devise a method and apparatus which arranges graphics and data, and ~5~

which retrieves that data, in a way to facilitate manipulation and animation of the produced images.
It is a further object of the present invention to enable stored data concerning an object element's appearance and/or instructions for the display thereof to be modified or changed in real time without interrupting the production of real time displays.
It is yet another object of the present invention to provide a system in which a very large amount of data is stored relating to the appearance of display objects and instructions as to the display thereof, only some of which objects are to be displayed at any given point in time, and enabling the display to be formed from selected object elements displayed in predetermined ways by a data processor which need not access all of the stored data in order to perform the task.
It is a still further object of the present invention to so design a display system that display object data and display instructions can be modified or augmented during, and without interrupting or delaying, the display process.
It is another object of the present invention to provide a display system in which display continuity is assured even though extensive stored data revision is taking place.
It is a further object of the present invention to so ~25~

organize the display system that a very substantial amount of stored data revision can take place without interfering with the continuity of changeable or animated displays, even to the extent of enabling the new stored data t~ be introduced from a "live" source such as a TV camera without interrupting th~ continuity of the dynamic display.

~2~

S~MMARY OF THE INVENTION

A key to the improved real time data hand]ing capacity of the system of the present invention i.s the use of three memory components, preferably acted upon by two different data processing units. In a first memory component ~sometimes referred to as a pattern memory) is stored the data corresponding to each of the object elements which might be displayed over a period of time. In the second memory component is stored, preferably in the form of a linked list in which the items are linked in order of desired visual priority, data comprising identification of particular object elements together with display instructions, e.g., the manner and location of representations of those object elements on the display to be produced. These first and second memory components are loaded with data from any suitable external source by means of a first data processing unit. The above describes a portion of the earlier system. In the present improved system there is a third memory component. The aforementioned data processing units, acting in accordance with appropriate program instructions, selects from the second memory component those identifications and display instructions which are appropriate for the display that is to be produced at any given instant, and thus produces in said third memory component a compiled list, preferably but ~2~

not necessarily sequential in character, of only those identifications and display instructions which are to be used at said particular instant to produce the display.
When the display/construction buffers are of the line type, each compiled list will relate to a given line to be displayed at that particular moment. Stated more generally, each compiled list preferably relates to the content, for an instantaneous display, of the display/construction buffers then in use. This third memory component may be constituted by two alternatively acting sections, so that one can be used to produce a display while the other is being loaded with the appropriate data, just as the two display/construction line bufers of the system of the earlier system (also preferably present in the system of this application) were alternately used. A second data processing unit, here often called a "painter", is instructed by the data in the third memory component to seek ~rom the first memory component the data corresponding to a particular object element selected to be displayed and to put the data into the alternately acting display/construction buffer memories at the proper location and in the proper fashion, all as instructed by the data read from the third memory component. The display/construction buffer memories function, as in the earlier system, to produce the desired video display, ~S~7~

including accurately producing the desired color at each point on the display.
The painter will access, in said third memory component, the data directly applicable to the particular display desired at a given instant in time, and need not access all of the identification and instruction data that is stored in the second memory in order to take care of all display eventualities. Hence highly sophisticated displays can be produced in real time. The time constraint of real time display production is in the amount of data that can be handled within that time. In the earlier system entire linked lists had to be traversed in real time, although only portions of those linked lists were relevant to the particular display that was to be produced at a given instant. In the system of the present invention, by way of contrast, the painter accesses only that data which the system has produced in the third memory component, and all or virtually all of that stored memory component data is relevant to the particular instantaneous display desired.
Hence considerably more data which is actually display-productive can be handled by the system of the present invention than could be handled by the earlier system.
While the construction/display buffers may if desired be formed on a line basis, the appropriate section or sections of the third memory component are loaded on the basis of a plurality of lines, and preferably on the basis of an entire field. Thus one has, for each of the sections of that third memory component, the complete field time (or plurality of lines time) in which to deposit the appropriate data from the first memory section, and the painter at any given instant need access only those parts of the appropriate section of the third memory componen~ which contains data appropriate to the particular line or lines then being constructed by the painter in the construction/display buffer.
From a geographical point of view, the several components of the memory may exist in the form of separate cards or units, or they may be located in different dedicated areas of a single memory structure. It is sometimes desirable to integrate different portions of the various memory components, and particularly those portions of the second and third memory components which relate to one another. Thus the memory unit may consist of one geographical area defining the identification and display (second memory component) instructions for a first object element, directly adjacent thereto is an area dedicated to receiving the data for the third memory component relating to that object element, directly adjacent thereto is the second data component data for second object element, directly adjacent thereto is the area dedicated to the third memory component data for that second object element, and so on.
It has been found desirable, when a given object element is made up of a plurality of sub-elements, to so structure the second memory component instructions as to enable the painter to select or "clip" from the data corresponding to a given object element only that data corresponding to one or more desired sub-elements. Thus even though the pattern memory for a given object element may comprise data representing a scene of appreciable width, a given instruction could cause the painter to take from that portion of the pattern memory only the data relating to a predetermined fraction of that scene, depending upon the particular view to be displayed.
The instructions in the second memory component may include animation instructions, identifying different views of a given object, all stored in the first memory component, which are to be displayed sequentially in point of time in order to produce an animation effect. Those instructions will preferably be in the form of linked lists in which the items are linked in terms of time sequence. When animation is desired the appropriate instructions can be deposited in the third memory component by the first data processor, and they then control the painter in constructing the data in the display/construction buffer memories.

In those linked lists, and in any other linked lists which may occur in the system, each item in the series desirably comprises linking instructions both forwards and backwards, so that each intermediate item of a given linked list is linked in both directions to adjacent items. This greatly facilitates the formation of identification and/or instructions in the third memory component where items are selected from only a portion of the items in a ~iven linked list. The double linking speeds the location and utili~ation of desired data in the list, and hence facilitates display~ and particularly animated display.
Further objects of the invention are accomplished through the use in particular ways of frame buffers, either in place of or in addition to the line buffers specifically disclosed in the earlier system or other embodiments hereof for construction and display purposes. (By "line buffers"
is meant a buffer with a capacity corresponding to one or more lines, but fewer lines than an entire frame.) In perhaps the simplest embodiment, a full frame buffer is employed, fed by the line buffers and retaining within itself the data for the last-constructed scene produced by those line buffers. Should those line buffers temporarily become inoperative because memory is being updated and therefore is not available for access by the buffers, the frame buffer will produce a scene on the display screen.

5L2~;~77~3 That scene will be static, not dynamic, but that is much more acceptable than if the screen were to go blank.
In further embodiments, the line buffers of the earlier system or of the embodiments hereof are supplanted by frame buffers. This has several advantages. One is that it eliminates the necessity for "painting" on a line-by-line basis, which is essentially uneconomic from a time viewpoint, and permits the painter to operate object by object, a much more efficient use of the "painter's" time, thus increasing the amount of data that can be handled in real time. Another is that particular objects can be displayed with considerably more detail than was previously possible. Yet another is that the frame buffers can also provide for a static display if access to the memories is cut off, as when the latter are being recharged with data.
Separate frame buffers can be provided for the odd and even lines respectively of the display, those separate frame buffers alternately functioning as construction buffers or display buffers analogously to the alternate construction/display functioning of the line buffers of the earlier system. An even more advantageous arrangement is to utilize separate full frame buffers for alternate construction and display modes. This arrangement has the advantage that since each of these buffers is constructed in one frame time and will then provide display data for two ~L2~

frame times, one for the odd lines and the other for the even lines, the nondisplaying memory unit can be made available to an external data source for updating or revision during one of those frame times. ~ence continuous variable and animated displays can be produced while at the same time allocating half of the total time to the extexnal data source, which in that case can be a "live" source, such as a TV camera.
To the accomplishment of the above, and to such other objects as may hereinafter appear, the present invention relates to a system (method and apparatus) for Eorming a graphics display, as defined in the appending claims, and as described in this specification, taken together with the accompanying drawings.

~ ;77~

BRIEF DESCRIPTION OF THE DRAWINGS

Fig. 1 is a simplified block diagram of the earlier system;
Fig 2 is a more detailed block diagram of the hardware portion of the earlier system;
Fig. 3 is a block diagram showing the use of a pair of line buffers alternately as construction and display buffers;
Fig. 4 is a representation of a particular arbitrary scene, chosen for purposes of explanation;
Fig. 5 is a view of a scene of Fig. 4 broken down or "fractured" into individual object elements;
Fig.s 6a-6g illustrate the various steps followed in the earlier system to produce the scene of Fig. 4 from the object elements of Fig. 5;
Fig. 7 is a diagramatic indication of two different ways in which color data is accessed and used in the earlier system;
Fig. 8 is a "tree" diagram illustrating the sequence of steps that are gone through, in accordance with the earlier system, in order to produce a display line from data with respect to "background" and "foreground" objects stored in memory;
Fig. 9a is a pictorial representation of a shaded three-dimensional object;

577~

Fig. 9b is a pictorial planar representation of that portion of the object of Fig. 9a included within the rectangle on that figure, with a limited number of different degrees of shading indicated, Fig. 9c is a l~-color palette that could be used with the representation of Fig. 9b in order to produce a display image of said object with a pronounced three-dimensional appearance;
Fig. 10 is a block diagram of an embodiment o~ the system of the present invention;
Fig. ll~is a "tree" diagram illustrating a typical linked list arrangement of data comprising display element identification and display instructions, together with indications of the details of typical data categories for each of the unit blocks;
Fig. 12 is a diagram similar to Fig. 11 but showing an alternative, more complex arrangement in which animation instructions are included in the linked lists;
Fig. 13 is a diagram showing a typical way in which individual data items may be included and arranged in the compiled list of the third memory component;
Fig. 14 is a combination of a block diagram and a representation of one particular geographical arrangement of portions of the second and third memory components generally designated A, B and C;

Fig. 15 is a block diagram of another embodiment of the invention;
Fig. 16 is a block diagram of a further embodiment of the present invention;
Fig. 17 is a block diagram of a still further embodiment of the present invention; and Fig. 18 is a time chart showing the performance of various functions by the embodiment of Fig. 17.

~25;~

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Fig. 1 is a block diagram showing the basic components of the earlier system that was previously referred to. A
graphics pre-processor 2 will convert the picture of the scene to be displayed into data stored at predetermined positions in the graphics memory 4. The image management processor 5, in conjunction with the instructions that it receives from software 6 via the system processor 7, will retrieve appropriate data from the graphics memory 4 and convert that data into a form which, after passing through the graphics post-processor 8, is fed to the graphics display 10, which may be a conventional TV picture tube, where a picture of the scene is formed and displayed. In many instances once the graphics pre-processor 2 has done its job, loading the graphics memory 4 with appropriate data, it is disconnected from the system, which thereafter functions on its own.
Fig. 2 is a more detailed block diagram of the image management processor 5. The input thereto from the software 6 and system processor 7 is at 12. Its connection to the graphics memory 4 is shown at 14. It includes a pair of buffers 16 and 18 (see also Fig. 3) each of which comprises a plurality of data blocks 20 arranged (at least conceptually) in a line, each buffer 16 and 18 containing the same number of data blocks 20 as there are pixels in a ~2S~

display line in the graphics display 10. Thus each display block 20 in each of the buffers 16 and 18 corresponds to a particular pixel in each of the display lines of the graphics display 10. The length of time available between the beginning of the scan of one line on the graphics display 10 and the beginning of the scan of the display of the next line is very short, on the order of 64 microseconds. In order to enable this system to construct and display successive display lines within those time constraintsr the two buffers 16 and 18 are used alternatively, with one being used to construct a line while the other is being used to actually display a line. The line buffer control 21 will connect one buffer, say the buffer 16, to the data input line 22, and will disconnect the buffer 18 therefrom, while at the same time the line buffer switching element 24 will connect the buffer 18 to the output line 26, while disconnecting the buffer 16 therefrom. That situation will continue while the graphics display 10 is scanning one line. When that scanning is completed the line buffer control 20 and the line buffer switching 24 will reverse their connections, so that during the next period of time the buffer 18 will be used ~or construction and the buffer 16 will be used for display. In this way sufficient time is provided to write data into all of the data blocks 20 alternatively in each of the buffers 16 and 18.
One of the fea~ures of the earlier system is the storing of data for the scene in terms of object elements, that is to say, the separate individual portions of the objects in the scene as they are viewed. The objects are "fractured" into individual visible portions, those portions are individually stored in memory, and are retrieved from memory whenever the object of which they are a part is to be constructed. In order to illustrate this we have shown in Figure 4 a scene comprising a cube 28, a parallelopiped 30 and a three-dimensional representation 32 of the letter B.
The width of the parallelopiped 30 is the same as the width of the cube 28. As indicated irl Figure 4, the scene there shown may be "fractured" into seven object elements A-G, of which object elements A and D are the same, so that only six object elements need be stored. Each of the individual objects in the scene is composed of separate surfaces. A
cube or a parallelopiped has six surfaces, but in the particular picture here shown there are only three surfaces visible, so data with respect only to those three surfaces need be stored in memory. Figure 5 illustrates the object elements in ~uestion, each set off separately, as they would be in memory.
Figures 6a through 6g illustrate how the object elements of Fig. 5 would be used to create the overall scene, the object element being written in each individual figure being shaded. It wil] be understood that the scene is produced on the graphics display 10 as a series of display lines, and the data corresponding to the appropriate portion of an object element on a given display line will be inserted into the appropriate data blocks 20 along the length of whichever one of the buffers 16 or 18 is functioning as a construction buffer for that particular display line. This is indicated by the broken lines 34 on Fig. 4, which represent a particular display line on the graphics display 10. This explanation, for purposes of clarity, will ignore the line-by-line construction process.
It will be noted that the object elements as shown in Fig. 5 represent the surfaces of the object in question which would be visible in the absence of all other objects.
Thus at the stage of Fig. 6c a cube 28, most remote from the viewer, is completely constructed, but when, in Fig.s 6d-6f, the parallelopiped 30 is constructed, it blocks out (erases) some of the representation of the cube 28. Similarly, as the letter B, designated by the reference numeral 32, is inserted into the scene, portions of the cube and parallelopiped behind the letter are likewise erased.
One of the most time-consuming aspects of creating a graphics display in color is selecting the particular color for each of the pixels on the display screen. In order for 77~

a color representation to be commercially acceptable, a truly large number of colors must be available. In the earlier system 4096 colors are available. This produces quite acceptable color quality, but if one had to retrieve from memory, for each pixel on a display line, the particular one of those 4096 colors that was desired, the data handling involved could not be carried out in real time by any data processor presently known which would be economically and spatially practical. To solve this problem, the earlier system employed a color selection system based on ~he premise that in individual sub-areas of the display screen the need for different colors is much more restricted than what is called for when the entire display screen is involved. There will be areas, sometimes small and sometimes large, where only a few different colors will be required, and it is wasteful of time to make a full color palette available where almost all of that palette will not be needed.
In the earlier system the color is defined by a 12 bit word, four bits defining the red component of the color, four bits defining the green component and four bits defining the blue component. It is these twelve bits which permit defining 4096 different colors. However, when one uses buffers 18 and 20 with 8 bits per pixel, to construct those buffers one can choose from only 256 of those colors ~lZS7~

(see Fig. 7, a color map comprising a sequential list of 256 colors, each defined by a 12 bit value). The system can be operated in such an 8 bit mode. However, to make the color list easier to work with, and more memory efficient, it is segmented into a series of 16 "color palettes". Each palette holds 16 colors, arbitrarily selected by the operator for the particular graphics task at hand. Each palette is numbered, and each of the colors in a given palette is numbered. In order to use one of the 256 colors stored in the palettes, one must first specify the palette that is desired, and then the actual number of the color in that palette to be used in painting an object is supplied by the object description stored in memory 4. That description is in the form of an 8 bit word defining the address of the color, and when that address is interrogated a 12 bit word is found which defines the particular one of the 4096 colors that is involved.
For further flexibility, the earlier system provides that, in displaying object, the user has a choice of yet another mode - 2 bits per pixel. The mode defines the color resolution of the object. Using 8 bits per pixel one can use any or all of the 256 colors in defining an object.
Using 4 bits per pixel lets one use one of 16 colors to define a pixel. Using 2 bits per pixel causes the pattern data to be read in groups of 2 bits and only gives one a choice of 4 colors to describe a pixel.
The earlier system, in order to conserve internal memory, makes some basic assumptions about the way people use color to paint pictures. On e of these assumptions says that 90 percent of the time one will be able to paint an entire object without needing a color your palette does not contain. As a result, only one color palette mav he used to create an object. More complex objects, having more colors, are able to be created by overlaying objects on top of each other. This allows one final new object to be composed of as many colors as one desires. With this in mind the two efficient color resolution modes take on new meaning. One must decide whether you will need to use more than 4 or less than 4 colors within an object, keeping in mind that having 16 colors readily available uses a lot of memory.
If, and only if, the 2 bits per pixel mode is in use, each of the 16 color palettes are further subdivided into 4 "mini-palettes". Each mini-palette contains 4 colors.
Since the list of 256 colors is still logically divided into the 16 color palettes, the correct way to reference a mini-palette is to first specify the color palette, then specify the mini-palette.
The actual process of constructing the 8 bit word defining the color for a given pixel begins with the selection of the color resolution mode. Color resolution is 7~

specified within an object when its character and/or bit map strings are defined.
Once a mode is chosen, the "palette select" argument addressing the palette you want to use must be filled in.
If the 4 bit mode is selected one need only specify one of the 16 color palettes. The 4 bits representing the color palette number are placed in the high order 4 bits of the 6 bit word (the remaining 2 low order bits are not used in the 4 bit mode). If the 2 bit mode is selected, one must specify one of the 4 mini-palettes in addition to the color palette. The 2 bits representing the mini-palette number are placed in the lowest 2 bits cf the 6 bit word.
If the 8 bit mode was selected the palette address argument is ignored.
A11 of the components needed to display the object are now present and actual construction of the 8 bit line buffer word can begin.
In 4 bit mode, the first 4 bits are read from the object pattern data and deposited in the lower order 4 bits of the 8 bit word. The upper 4 bits of the 6 bit "palette select" word are then placed in the high order 4 bits of the 8 bit word. With the 8 bit word now filled on the line buffer, one pixel is completely specified as to choice of colormap selection. The system uses this colormap address (the 8 bit word) and displays the color associated with the ~L~S~77~

12 bit value it finds there.
In 2 bit mode, the first 2 bits are read from the object pattern data and deposited in the low order 2 bits of the 8 bit word. The entire 6 bit "pa]ette select" word is then placed in the high order 6 bits of the 8 bit word.
With the 8 bit word filled, one pixel is ready to be colored in. The system looks at the colormap address (the 8 bit word) and displays the color associated with the 12 bit value it finds there. Note that in 2 bit mode, one colors twice as many pixels as 4 bit mode for each fetch of object pattern data.
Although this method of addressing 4096 colors is indirect, in fact it enab-les the system to provide the greatest amount of colors using the least amount of memory.
In 8 bit mode, the pattern data supplies all 8 bits required to completely define the color choice of one pixel.
With the 8 bit buffer word filled the system uses this word, as before, to address the colormap and display the actual pixel color specified via the 12 bit value found there.
Figures 9A, B and C illustrate how the earlier system supports the visual perception of a two-dimensional representation of a three-dimensional image.
Using a four bit per pixel mode by way of example, a close coupling is achieved between the luminance grading of the selected color palette and the binary coding of the object pattern as defined in the system memory. In this example, the visual perceived effect is that of a cone illuminated from the right, and the rectangle in Figure 9A
represents a selected portion of the image. Fig. 9B is an expansion of that selected image portion showing certain arbitrary luminance shadings (dark to light), it being assumed for purposes of this example that the cone is of a single basic color. Fig. 9C is an illustration of the contents of the color map for the given example relative to the luminance values to be employed. The address "0000"
represents transparency and the addresses from "0001" to "1111" represen~ gradations in shading from the darkest representation to the lightest representation. Fig. 9B may be considered as a unit screen bit map, with the individual bits having color definitions corresponding to the shading in Fig. 9B and the particular colors in Fig. 9C. It should be understood that while this particular example has been described in terms of gradations in a single color, the cone being postulated as a monochromatic object the polychromatic aspects of which arise from its illumination and the way in which it is seen, multi-colored objects could have two-dimensional representations producing comparable three-dimensional effects by using appropriate different colors in the palette of Fig. 9C. If more than 16 individual colors are thought to be re~uired to produce a given image, the ~S~7~

object may b represented in memory as a plurality of object elements, each with its own 16 color palette, the object elements being appropriately spatially related.
The system permits the display of character data as well as pictorial data, using known techniquesO The system is also capable of providing windows -- an area on the graphics display screen lO on which an independent image may be displayed. Each window is in essence a variable-size virtual screen.
The sequence of steps performed by the earlier system~
under the con~rol of appropriate software, producing a graphics display is schematically indicated in Figure 8, illustrating a linked list "tree" showing a procedure involved in connection with a given background ~row) and foreground (slice) characteristic or attribute. A given line of a given image may involve a large number of row and slice attributes, the number being limited only by the time available to handle them. The software will produce real time display by traversing the tree structure, filling in appropriate nodes with information about the objects that are to be displayed, their positions relative to the screen and each other, and the pattern data to be used for the current representation of the image.
Fig. 10 is a block diagram of a system in accordance with an embodiment of the improvement of the invention. A

first memory component A, hereinafter termed "pattern memory", receives and stores data, usually in the form of a bit map, defining the appearance of those object elements which, it is expected, will be displayed over a period of time, although in most instances not all of those elements will be displayed in any given moment. An object element may be considered as an independent pictorial entity, which may in turn be made up of a plurality of sub-elements. The system of this embodiment enables that object element, or !'clip"-selected sub-elements thereof, to be freely positioned over the display space and to be unrestrictively overlaid over previous patterns placed in that display space. Such overlaying involves user-definable visible priority in terms of whether a given object element will appear in front of or behind another object element, thus enabling three-dimensional animation effects to be produced.
Each individual object element is identified in some appropriate fashion, as by its location in that portion of the memory constituting the first memory component A. The pattern memory A forming a part of the display system may be augmented by memory structure external of the system proper, e.g., an attached disc storage instrumentality.
The second memory component B, hereinafter termed "system memory", may contain program instructions and will also contain data, preferably in the form of linked lists of ~2~S77~L9 the type generally described above in conjunction with the earlier system, identifying various components of a desired display and containing display instructions relative thereto, such as defining where on the screen the display of the object element is to be located, what its size is to be, what its color is to be and what, if any, manipulations (e.g. pan, zoom, warp, rotate) are to be performed on the relevant data stored in pattern memory A before that data is actually displayed. This data stored in the system memory B
relates to all portions of the display which are to be formed throughout the period of operation of the system, and is not limited to the data needed for a display at any particular moment.
A data processing unit, generally designated D, and hereinafter termed the "system processor", functions before a display run is commenced to deposit the appropriate data in the pattern memory A and the system memory B, obtaining that data from some external source, and the system processor D may also be used to update the data in pattern memory A and/or system memory B, in accordance with instructions and that either internally stored or received from an external host computer, while the system is Gperating to produce displays. It further loads color information into a color map memory G.
In accordance with the present embodiment of the invention, the system processor D performs an additional function. As display time passes it reads from the linked lists of system memory s that identification and display instruction data relevant to creating a display at a particular moment, and it deposits that data into the third memory component C. That which is deposited will hereafter be termed "the compiled list", which may well be in the form o~ a sequential list, and hence the memory component C will hereinafter be termed the compiled list memory C. The compiled list represents the object element identification and relevant display instructions of a particular instantaneous.display, this being usually only a small proportion of the corresponding data stored in the system memories A and B~
A separate data processing unit generally designated E, and hereinafter termed the "graphics painter", addresses the identification and instruction data stored at any given moment in the compiled list C and, for each object element identified in the compiled list C, reads from the pattern memory A the data defining the appearance of that object element and then, in accordance with the display instructions for that object element stored in the compiled list C, the graphics painter E produces display data which feeds to the two alternately acting display/construction buffers generally designated FI and FII which, as here -.

~ 7~

specifically disclosed, correspond to the alternately acting buffers 16 and 18 (Fig. 3) of the earlier system. The two display/construction buffers FI and FII are here disclosed as constructing lines of the display, one such buffer being constructed by having data put thereinto by the graphics painter E while the other such buffer is acting to produce a line display, the functions of the two buffers FI and FII
alternating in time. Hence the graphics painter E may access the system memory B and the pattern memory A on a line-by-line basis.
As in the earlier system, the output from the construction/display buffers F goes to the color map G, into which appropriate data had previously been stored by the system processor D, and from there the display data goes to digital to analog converter H, from which a composite video signal I system goes to the display instrumentality, in known manner.
It is desirable that the system processor D, in response to information received from outside the system or from the program in system memory B, modify the linked lists in the system memory B in real time without affecting the capability of the system to produce real time displays. To that end two compiled lists C-I and C-II are provided, each of which may contain the appropriate identification and display instruction data for a given frame. When one of the ~ 7 ~

compiled lists C-I or C-II is being accessed by the graphics painter E in order to produce display data, the other compiled list C-II or C-I is being constructed by the system processor D.
As has been explained, the data in the compiled list C
compresses the identification and the display instructions relating to the visible objects in the scene to be displayed at a given moment. In order to have those displayed objects be of adequate resolution, detail and color, and to simultaneously display a significant number of different objects, the amount of data required for the compiled list C
cannot, as a practical matter, be generated in line time, yet each of the construction/display buffers F are constructed in line time. But since the compiled list C
relates to an entire frame, that list can be generated in frame time, and since typically there are 525 lines to a frame in a conventional TV display, that gives 525 times more time for compiled list construction than for line buffer construction when the display is to be changed thirty times a second (the time to display a given frame), thus enabling the system to handle significantly more data than previous systems and thus produce considerably more sophisticated displays. If the display need not be changed so frequently, there is a corresponding increase in the time available to generate a given compiled list.

;77~

For all of these reasons the system processor D can handle much more data in real time than was possible in the earlier system.
Fig. 11 represents a typical linked list arrangernent of object element identification and display data stored in the system memory B. Data is there stored in a hierarchy of attributes, with some or all of those attributes being further arranged in the form of linked lists ordered in terms of visible priority. For example, and as shown in Fig. 11, the highes~ or most general attribute is the frame attribute 2, next in order are the window attributes 4, and~
for each window attribute 4, the various object attributes 6 associated therewith. In addition, and for purposes of enabling access to particular objects or lists, a series of symbol attributes 8 may be stored, each of which may also include a list of sub-identifications (called "children"), e.g., "dog" may be the main symbol and "dog walking", "dog sitting", "dog jumping", etc., may be "children".
The data stored for the frame attribute 2, which represents an overall scene to be displayed at a given moment, comprises its desired x and y origin points on the display screen, a pointer to the window list or lists that are to be used in that frame, and an identification of the highest priority window in tht window list which is to be used.

'77~

A "window", as here used, is a defined viewpoint, or rectangle through which selected object elements are to be viewed, the window itself defining he bounds of the viewing area and hence determining what portion of the selected object element is to be displayed. Each window attribute 4 contains data defining its desired location on the display, its size, linking pointers, preferably to both the preceding as well as the succeeding window in the linked list, a pointer to the object list or lists to be included in the window, a pointer to the highest priority object in that list which is to be displayed, and an identification of the window to match with the appropriate symbol at~cribute 8.
Each object attribute 6 includes a pointer 'co pattern memory A, identifying the pictorial data in that pattern memory A which relates to the particular object, the desired location of the object, its size, a definition of the number of bits per display pixel, identification of the color palette to be used, and identification of the symbol attribute 8 that is to correspond to that object, as well as data identifying the object itself. Each symbol attribute 8 may contain data defining an identifying name, so that it can be manually or automatically selected, together with data concerning its size, its location in the pattern memory A and, if desired, data concerning various manipulations which might be performed to controllably modify or distort ~a25~

the display image as well as links, preferably in both directions, to the allied "children" data.
It has been found to be advantageous to also include in the object attribute 6 data restricting the portion of the relevant display object which is to be displayed. This data can be in the form of words identifying the location of the top, left-hand side, bottom and right-hand side of the area to be clipped and, if a particular object within a composite object element is fractured by the clip, additional words defining the overall clip conditions with respect to that object. The clip in effect constitutes a restricted area of observation within that portion of the window of which the clip may be a part. Hence only that portion of the object element will be displayed which is both within the window attribute 4 definition and the clip instructions definition.
When a clipping is to be accomplished the relevant clip data is added to the object attribute data shown in Fig. ll.
What the system processor D does during the display process is to read the appropriate lists in system memory B, such as the one disclosed in Fig. 11, and produce, for each object element to be displayed at that point in time, data in the compiled list C. Fig. 13 is a representation of a particular body of data relating to a particular display object as it may be produced and temporarily stored in a compiled list C. The first line lO of that data is a 771~

pointer to pattern memory A identifying the particular line of that pattern memory to which the painter E should go.
That line typically comprises four bytes of eight bits each.
It usually takes more than one line of pattern n~emory to create one display line of the object, and therefore the compiled list for a given line of an object may require sequential reference to a plurality of pattern memory lines.
In such a case the data in line 10 initially points to the line in memory where the picture is to start. The "bottom line" and "top line" units 13 and 15 in line 12 indicate the position that the object should assume on the display screen. Each requires nine bits, but the memory is only sixteen bits wide, Therefore the "0 top" and "0 bottom"
units in lines 4 and 17 respectively represent the ninth needed bit in the top and bottom line items 13 and 15 respectively. In line 14 the "last" item 16 is a flag which appears only when the data block is a used for the last tlme in a se~uence. The "clip" item 20 is a flag indicating whether or not a clip is involved. The "pattern page" item 22 is used in conjunction with pattern pointer 10 in order to direct the painter E to the right spot in memory. The "full pattern width" item 24 in line 14 and the "relative width" item 40 in line 38 represent respectively an indication of the number of pixels which make up the entire object line and the number of pixels needed to make up the ;77~

object line taking into consideration the proportion of the entire object to be displayed. The "X position" item 28 in line 17 identifies the desired horizontal position where the display of the object element should start. The "left delta" and "right delta" units 30 and 32 are used when, because of clip or window constraints, not all of a given line in memory is to be used in painting the line of the picture. The second "pattern pointer" unit in line 36 identifies the first line of the relevant data in the pattern memory A. When, as has been explained, it is necessary to read more than one line of pattern memory in order to create a given display line, the first pattern pointer 10 and the second pattern pointer 36 are initially the same but, as the compiled list is followed and the painter E is directed to different lines in memory for a given object element, the first pattern pointer unit 10 points to those lines sequentially, being changed by the value of full pattern width 24 each time the data structure is run through and the object or portion thereof is to be displayed. The second "pattern pointer" unit 36, which remains constant, is used to return the first "pattern pointer" unit 10 to its initial value after the last sequence has been carried out. In line 38 the data unit 41 identifies the number of bits per pixel to be employed in making the display, and the "palette" unit 42 identifies the particular color that is to be used in displaying that particular portion of the object element.
It is to the data blocks of the type shown in Fig. 13 that the graphics painter E goes to determine where in pattern memory A it should look and what it should do with what it finds at the identified location in pattern memory A. It then deposits the relevant information, which we now call "display data" because it is the data actually to be used to produce a particular display image, into the construction/display buffers F in line basis real time, the system then functioning essentially as previously described in order to produce the display image.
All of the cornpiled list data such as is exemplified by Fig. 13 may be deposited in an area or unit of memory dedicated to that purpose, but this is not essential. The compiled list data may, from a physica] or geographical viewpoint, be integrated with the linked list data of the system memory B. This is schematically indicated in Fig.
14, where a given unit 44 from system memory B, such as a particular object attribute 6, is immediately followed geographically by that portion 46 of the compiled list C
which has been created by the system processor D in accordance with that particular object attribute 6. Next in line, at area 48, may be the next object attribute 6A in a given linked list of object attributes (see Fig. 1]), ~Z~

followed at 50 by the compiled list formed by the system processor D in accordance with that object attribute 6A, and so on, the links 52 of the object attribute linked list being located as disclosed, it being noted that those links operate in both directions so as to link a given object attribute 6 with both the preceding object attribute and the succeeding object attribute in a given linked listo The systems memory data in areas 44 and 48 will normally remain in the course of the display, unless changed by the system processor D in accordance with appropriate commands, either external or from the program portion of system memory B, but the compiled list data in areas 46 and 50 will be constantly changed during the display, as above described.
Fig. 12 is a block diagram of the same general character as Fig. 11 but showing a typical arrangement of linked lists and the data involved in those linked lists where animation instructions are inteqrated with the identification and other display instructions of the linked list system of Fig. 11. In Fig. 12 there is a first linked list 2A of frame attributes linked in terms of time because of the animation, each of the attributes thereof pointing to one or more linked lists 4A of window attributes. The system of Fig. 12 contains~ for each window attribute 4A, one or more lists 54 of animation attributes, linked in order of visual priority, each of which in turn points to 7~

one or more linked lists 56 of view attributes and one or more linked lists 58 of trail attributes. rhe view attributes 56 correspond generally to the object attributes 6 of the system of Fig. 11, except that the view attributes of a given linked list 56 represent views of the same object different from one another in a manner such as to produce an animation effect when sequentially displayed. Hence the view attributes in a given linked list 56 are ordered in time (visual priority is controlled by the linking in the animation attributes list 54). The trail attributes of linked list 58, also ordered in time, control the sequence of different physical locations where the individual view attribute objects are displayed, thus causing the objects to traverse a specified route on the display screen. The animation attributes give instructions as to how the view attribute linked list 56 and trail attribute linked list 58 are to be traversed (forward, backward or in circulatory fashion, sequentially or by skipping individual views).
When a particular animation attribute 54, at a given point in time, activates a particular view attribute 56 and trail attribute 58, the graphics painter E will be apprised, by the pattern pointer in the operative view attribute item 56 and by the bits per pixel and palette data also there included, what particular object should be read from the pattern memory A, how long it should remain on the screen, and how it should be displayed on the screen.
The double linking of the individual attributes in the linked lists of Fig.s 11 and 12, in which each intermediate item in the linked list has link instructions forward and backward to the item immediately after it and the item immediately before it, greatly facilitates modifying those linked lists in accordance with instructions received from the system processor D, as by adding or deleting items.
With such double linking it is not necessary, in order to make an insertion, to start from the beginning of the list to find the proper place where the insertion of the new item is to take place. The system processor D can go directly to the place where the item is to be inserted and insert it without having to modify the linking instructions of any of the objects in the list except for the two items immediately before and immediately after the item inserted. It further greatly facilitates the making of directional changes forward or backwards on the transversal of the list, something that is very important when animation of the type disclosed in Fig. 12 is involved. For example, if we want to show smooth motion we may use twenty sequential images~
but if we want to show rough motion or faster motion we may wish to delete every other one of those twenty images to produce a list of ten images. That can be done much more quickly with double linking than if the system has to search out the proper point for each deletion by counting again from the beginning in each instance.
In the above-described system a display can be produced only so long as the line buffers F have access, via the graphics painter E, to the pattern memory A, since it is only with such access that the display lines can be constructed in the buffers F. Whenever the pattern memory A
has to be available to other sources, formation of a display must cease. This greatly restricts the capability of the system to function while at the same time permitting updating of the pattern memory A. Yet it is often necessary to revise or update the contents of pattern memory A (and also system memory B) to an extent such as to be impossible to accomplish within real time constraints. When that occurs the display screen goes blank. That may occur for only a fraction of a second or for many seconds, depending upon the extent to which the contents of pattern memory A
are changed, but such blanking of the display is undesirable in any event.
The embodiment of Fig. 15 avoids this disadvantage by interposing between the construction/display line buffers F
and the color map G a frame buffer K. The lines constructed by the line buffers F are in turn constructed in the frame buffer K, where they remain until modified, and the line-by-line data is fed from frame buffer K to the composite video signal I in any appropriate known manner. Thus if for any reason construction of new lines in the line buffers F is interrupted -- for example, because the pattern memory A is being updated and hence access thereto by the graphics painter E has been interrupted -- the frame buffer K ensures that a scene is still displayed. That scene is static, because no changes are being made in any of the lines stored in the frame buffer K, but a static scene is infinitely preferable to no scene at all.
A second limitation in the functioning of the systems Gf previous embodiments is that construction is accomplished on a line-by-line basis. As a result the total number of objects that can be supported by the system is limited by the "highest band width" line. In addition, the entire object list in system memory B is traversed for each line to be constructed, although a good portion of that list will be irrelevant to the particular line under constructionO This results in a significant waste of operating time. Both of these factors significantly adversely affect the amount of data that can be handled by the system in real time and tend to limit the degree of detail with which particular objects can be displayed.
To overcome this disadvantage, the embodiment of Fig.
16 substitutes frame buffers F' for the line buffers F of the embodiment of Fig. 15. In that embodiment each of the frame buffers F' will accommodate half of the total frame, the buffer F'-I being adapted to contain the odd lines of the frame and the buffer F'-II being adapted to contain the even lines of the frame (it is conventional, in making up a display on a screen, to first sequentially display the odd lines of the frame and then to se~uentially display the even lines thereof).
This use of frame buffers F' rather than line buffers F
results in a significant overall improvement in the operation of the system. In the first place, since the display no longer need be constructed on a line~by-line basis, each object can be constructed in a given frame buffer F' in its entirety. Hence the "highest band width"
line no longer presents a limitation, and the construction time of the various-objects to be represented in a display can be averaged over a given frame time, thus enabling certain objects to be displayed in greater detail than had been previously possible, and permitting an increase of perhaps a full order of magnitude in the number of objects that can be displayed. In addition, because the display can now be constructed in the buffers F' on an object-by-object basis, considerable time can be saved in accessing relevant portions of the pattern memory A.
Moreover, because the two buffers F'-I and F'-II
contain data for a complete display, if access to the ~8 71~

pattern memory A by the graphics painter E is interrupted, as described above, a static display can be produced by the buffers F'-I and F'-II. Hence there need be no blanking of the display screen even when extensive updating of pattern memory A from the external source J takes place.
The broken lines in Fig. 16 show an optional modification of the system there disclosed. Instead of having the frame buffers F'-I and F'-II feed directly to the color map G, they can be caused to feed to an extra full frame buffer K', which in turn feeds the color map G. This extra full frame buffer K' is not needed to prevent blanking of the display when the pattern memory A is not available --the frame buffers F'-I and F~-II accomplish that, as has been described. The function of the extra frame buffer K' is to reduce the time that the display must be static by making the buffers F'-I and F'-II available to be used for construction while the buffer K' ensures continuity of static display, and also enables the buffers F' to be accessed by a third source, such as a TV camera, while continuity of the static display is ensured.
The embodiment of Fig. 17 is similar to the embodiment of Fig. 16 except that the construction/display buffers F"
are each full frame buffers having the capacity to store data for both the odd lines and the even lines of the display. With this arrangement considerably greater leeway ~s~

in updating or changing pattern memory A from an external data source J is possible, as is indicated by the timing chart of Fig. 18. That chart discloses four time slots, each typically representing the time required to display either the odd or even lines of a display, two such time slots of l/60th of a second each making up the conventional frame time of l/30th of a second. A first frame is displayed in time slots 1 and 2, a second frame is displayed in time slots 3 and 4, and so on.
In the first time slot the painter E constructs a frame in buffer F"-l, an earlier frame having been previously constructed in buffer F"-II. In that same time slot the even lines of the display are produced from the appropriate data in buffer F"-II.
In the second time slot the odd lines of the display are produced from buffer F"-II, but since the next frame has already been constructed in buffer F"-I, nothing more need be done with respect to that buffer. Hence the time of that second time slot is now available for the external data source J to update the data in pattern memory A via the system processor D. The graphics painter E is prevented from having access to the pattern memory A during the second time slot, but it does not need that access.
In the third time slot the graphics painter E is again given access to the pattern memory A and constructs the next , . - .

~2~7'73L~

frame in frame buffer F"-II, and in the same time slot the even lines of the display are produced from the appropriate data in buffer F"-l.
In the fourth time slot the odd lines of the display are produced from the appropriate data in buffer F"-I but, because buffer F"-II has already been completely constructed, the time of this time slot is available for updating of pattern memory A.
Alternatively, the system of Fig. 17 could be employed, in conjunction with a "live" data source such as a TV
camera, to construct one frame buffer F"-I from data provided by the graphics painter E and construct the other frame buffer F"-II from the data provided by the "live" data source.
The embodiment of Fig. 17 could, if desired, be provided with the extra full frame buffer K' of Fig. 16, to produce the results previously set forth above.

Claims (22)

CLAIMS:

1. A method for creating on a display screen having a given scanning time, a representation of a scene comprising selected ones of a plurality of object elements, said method comprising:
A. storing first memory data corresponding to said plurality of object elements;
B. storing second memory data identifying said plurality of object elements together with instructions as to the manner and location of representations of said object elements or parts thereof;
C. creating, from said second memory data, third memory data corresponding to identification and desired instructions with respect to selected ones of said-plurality of object elements;
D. creating, from said first memory data and in conformity with said third memory data, display data corresponding to the desired representation of said selected object elements in said scene;
E. causing the display data of step D to produce a display on said display screen corresponding to said desired representation; and F. carrying out at least steps D and E in real time relative to the scanning time of said display screen.

2. The method of claim 1, wherein said step of storing second memory data comprises storing said data in the form of linked lists in which the order of said data corresponds to the desired visible priority of said object elements.

3. The method of claim 1 in which step C and step D are carried out simultaneously by different processing means.

4. The method of any of claims 1-3, in which said monitor representation comprises a field made up of a series of lines, and in which steps D and E are carried out in real time relative to the scanning time of fewer than all of the lines of said field.

5. The method of any of claims 1-3, in which, when a preselected fraction of a given object element is to be displayed, step C identifies that preselected fraction and step D creates said display data for said object element which includes only the data corresponding to said preselected fraction.

6. The method of any of claims 1-3, in which said third memory data for each of said desired object elements are physically located so as to sequentially follow said second memory data for the corresponding object elements.

7. The method of claim 1, in which said second memory data comprises a series of data ordered in time to identify a series of object elements representing animation of a selected object element, and causing said display data at any point in time to include data from said series of data corresponding only to that point in time.

8. The method of claim 7, in which said data ordered in time is in the form of a linked list comprising a plurality of items in which each intermediate item is linked in both directions to adjacent items.

9. The method of claim 1 further comprising the step of utilizing at least one frame buffer memory capable of constructing from said display data a scene for said display screen and a depositing the appropriate display data into said frame buffer memory to construct its scene; and wherein said step of causing the display data to produce a display is performed on the display data in said at least one frame buffer memory and is carried out in real time relative to the scanning time of said display screen.

10. The method of claim 9, in which said step of utilizing at least one frame buffer memory comprises utilizing two frame buffer memories each capable of constructing from said display data a scene for different portions of said display screen, depositing the appropriate display data into a first of said frame buffer memories to construct its scene portion and then depositing the appropriate display data into the second of said frame buffer memories to construct its scene portion, and so on, and in which said display step comprises utilizing the display data in a given frame buffer memory while the other frame buffer memory is having display data deposited thereinto.

11. The method of claim 10, in which each of said frame buffer memories is capable of constructing a different half of said scene.

12. The method of claim 10 or 11, in which said display comprises a plurality of lines, and in which said first and second frame buffer memories construct the odd and even numbered liens of said display scene respectively.

13. In the method of claim 10, utilizing an additional buffer memory capable of constructing an entire display screen scene, and depositing in said additional buffer memory display data corresponding to a particular scene, and upon command, causing the display data in said additional buffer memory to produce the display on said screen.

14. The method of claim 13, in which the display data deposited in said additional buffer memory corresponds to the scene being displayed immediately prior to said command.

15. The method of claim 9, in which said step of utilizing at least one frame buffer memory comprises utilizing two frame buffer memories each capable of constructing from said display data a scene for said display screen and depositing the appropriate display data into a first of said frame buffer memories to construct its scene and then depositing the appropriate display data into the second of said frame buffer memories to construct its scene, and so on.

16. In the method of claim 13, in combination with an external data source, in a first period of time depositing the appropriate display data into a first of said frame buffer memories while the display data in said second frame buffer memory is caused to produce a display on said screen, in a second period of time causing said external source to modify said first and/or second memory data while said second frame buffer memory is caused to produce a display on said screen, in a third period of time depositing the appropriate display data into the second of said frame buffer memories while the display data in the first of said frame buffer memories is caused to produce a display on said screen, in a fourth period of time causing said external source to modify said first and/or second memory data while the display data in said first frame buffer memory is caused to produce a display on said screen, and so on.

17. The method of claim 14, in which said first, second, third and fourth periods of time are consecutive in that order.

18. Apparatus for creating on a display screen a representation of a scene comprising selected ones of a plurality of object elements, said apparatus comprising A. a pattern memory for storing data representing a plurality of object elements;
B. a system memory for storing data identifying said plurality of object elements and data comprising instructions defining the nature and location of representations of said object elements or parts thereof;
C. a third memory for storing instructions as to the identity, nature and location of display of desired ones of said preselected object elements:
D. a buffer memory for storing data corresponding to the desired representation of at least a portion of said scene;
E. first data processing means operatively connected between said system memory, said pattern memory and said third memory for transferring data therebetween;
F. second data processing means operatively connected between said third memory, said pattern memory and said buffer memory for depositing in said buffer memory data from said pattern memory in response to instructions from said third memory; and G. display means for causing said data in said buffer memory to produce a display on said screen corresponding to said desired representation.

19. In the apparatus of claim 18, means enabling said first and second data processing means to function simultaneously and independently.

20. The apparatus of claim 18, in which said data in said system memory is stored in the form of linked lists with data linked in the order of their desired visible priority.

21. The apparatus of any of claims 18-20 wherein said buffer memory comprises a pair of line buffers.

22. The apparatus of any of claims 18-20 wherein said buffer memory comprises a pair of frame buffer memories.