WO1991012586A1

WO1991012586A1 - Graphics system

Info

Publication number: WO1991012586A1
Application number: PCT/GB1991/000192
Authority: WO
Inventors: Stephen Freeman; Keith Dobbs
Original assignee: Crosfield Electronics Limited
Priority date: 1990-02-08
Filing date: 1991-02-07
Publication date: 1991-08-22
Also published as: JPH05505480A; WO1991012587A1; EP0514445A1; EP0514446A1; JPH05504013A; GB9002862D0

Abstract

A graphics system comprising an input device (31); a detector for detecting movement of the input device (31) relative to a surface (5); and a display (30) for displaying patterns which follow movement of the input device (31) to simulate the creation of patterns on a surface with a drawing instrument. The system also includes a display modifying apparatus (15-19) for modifying the display patterns over time in order to simulate the effect of wetness.

Description

GRAPHICS SYSTEM The invention relates to a graphics system comprising an input device; means for detecting movement of the input device relative to a surface; and display means for displaying patterns which follow movement of the input device to simulate the creation of patterns on a surface with a drawing instrument. Such a system is hereinafter referred to as of the kind described. Systems of the kind described have been developed during the last few years but one of the major drawbacks of current systems is that they are unable to simulate accurately the effect of painting on a canvas.

In accordance with the present invention, a graphics system of the kind described is characterised in that the system further comprises first and second stores for storing data representing the amount of dry paint and wet paint respectively in the finished image; combining means connected to the first and second stores for generating a resultant image on the display means by combining the data in the first and second stores; and processing means for modifying the data in the first and second stores to simulate the effect of increasing or decreasing the degree of wetness. We have realised that one reason why known systems do not accurately simulate painting on canvas is due to the inability of these systems to reproduce the effects of wetness in either the paint or the canvas. These effects are very important in painting and in many cases are used on purpose to create special effects in the finished painting. Various effects in painting have a time dependence, for example drying paint, fade, run, and smudge. These effects are used by artists as for example, smudging and fixing of chalk pastels, and the like. it should be understood that where multi-colour images are concerned there will be a pair of first and second stores for each colour component. Typically, the stores will be frame stores linked to a monitor.

In one application, the processing means can automatically cause the simulation of the drying of applied paint with time. Thus effectively, paint is transferred from the "wet" store to the "dry" store at regular intervals.

In more sophisticated examples, the processing means can perform a further function so as to simulate effects such as smudge, etc. This function would include modifying the positional and/or density functions of the paint while in the wet store, for example spreading the paint and the like. In addition, the "transfer" operation could be performed in real time or at intervals set by the operator. In addition to simulating drying by "transferring" from wet store to dry store, re-wetting can be simulated by "transferring" from dry store to wet store.

Examples of some effects which can be achieved using this system are set out below: l. Paint drvinσ

Paint is applied in the wet store and then the processing means copies a proportion in to the dry store each time period. The effect on the display is unchanging but the available "live" colour in the wet store is decreasing steadily. Smudge effects could then take place in the wet store, the effect of which could reduce over time. "Wetting" the dry store would copy part of the dry store into the wet store which would then "dry" back to the dry store. Of course, more brushing work could be applied to the wet store instead.

It should be understood that references to "copying" or "transferring" paint from one store to another include etching paint from the one store and repainting into the other. In this operation the merge and etch brushes used could have the same or different characteristics depending on the effects required. By brush "characteristics" we include: a) 2-dimensional overall size b) 3-dimensional shape (ie.2D size & height, the latter representing density) c) single colour or texture colours d) overall density limit (ie.a scaling factor applied to limit max.density) e) a combinational algorithm. 2. Drip, spread, run.

The drip effect is achieved by repainting over previously journalled brush tracks in such a manner as to simulate the look of dripping paint. Over time, these drips will build up by automatic repainting. This repainting will originate along the track of other paint strokes but will of course be done with a finer brush, in the (user definable) direction of gravity. The user may halt the process at any time and either manually edit the wet store or fix it into the image as dry paint by using an image grab-back facility. There is no reason why paint could not be "transferred" to the dry store simultaneously if required. To define the direction that gravity acts on the canvas (typically downwards) , and the magnitude of it, a vector could be placed on the screen, as part of the user interface. The user could define the degree of absorbency of the canvas which will determine the size and density of a brush to be applied when a spreading effect is required.

Spreading could be achieved by etching & repainting with a larger, lower density brush, into the wet store, as would simulating meniscus effects for very wet brushes. Other effects can be achieved by change ANY of the above brush characteristics described above.

3. Fade/mix

When paint is applied to a canvas, it may change colour as it dries, mix with wet colour already applied etc. These effects can be achieved by Suitably programming the processing means to "transfer" paint from the wet store to dry store. For fade, the wet store would be etched with an etch brush, and a merge or dye brush applied to the dry store at low density so that the overall effect is to decrease the overall paint density. As with all the these effects, the system would repeat the operations, thereby iteratively arriving at the effect the wanted at which point the image would be fixed by a grab-back operation.

An example of a- graphics system according to the invention will now be described with reference to the accompanying drawings, in which:-

Figure 1 is a block diagram of the apparatus; Figure 2 is a block diagram of a graphics image process of Figure 1; and

Figure 3 illustrates the appearance of images held in the stores shown in Figure 1;

The apparatus shown in Figure 1 can be divided into two primary parts. These comprise the host 1 and the graphics sub-system 2. The division is shown in Figure 1 by a dashed line. The host 1 is a 68020 microprocessor based system running UNIX which is a multi-tasking, multi-user operating system. The host comprises an I/O processor 3 coupled to a keyboard 4, a digitizing tablet 5 and associated pen 31, a system disk 6 and other data sources (not shown) . The I/O processor 3 is connected to a system inter-connect bus (SIB) 7 which is connected to ROM and RAM memory 8, a CPU 9, and an interface adapter 10. The interface adapter 10 is connected to a number of high speed image discs 11 which hold data defining the colour content of pixels of images at high resolution, the adapter also being connected via an interface 12 with the graphics sub-system 2. As mentioned above, the host has a conventional form and will not be described in detail. However, the SIB 7 is described in more detail in EP-A-0332417. The programme that runs on the host is a single "process" which reads and processes inputs from the digitizing tablet 5 under operator control and directs the graphics part 2 to display the host's responses to those inputs on the graphics monitor 30. Essentially, the system takes advantage of the host system in being able to perform a majority of the calculations so that only a small amount of control data is passed to the graphics sub-system. This graphics part 2 is much better than the host 1 at creating and manipulating graphical objects but the host is better at controlling input/output to peripherals, discs and tapes and is relatively easy to programme. The graphics sub-system 2 comprises an interface 13 which connects the graphics part to the host 1, the interface 13 being connected to a bus 14. The bus 14 is connected to five graphics image processors (GIPs) 15-19. In this embodiment, it is assumed that the images are defined by four colour components, namely cyan, magenta, yellow and black, there being a separate GIP for each colour. Thus, the GIP 15 processes the cyan colour component, the GIP 16 the magenta colour component, the GIP 17 the yellow colour component and the GIP 18 the black colour component. If the image was represented by a different number of colour components, for example red, green and blue then only three of the GIPs would be needed plus a mask GIP as mentioned below. The advantage of providing the GIPs 15-18 in parallel is that each component of each pixel in the image can be processed in parallel so that the overall processing time is reduced by up to four times over the processing time with a single processor. A further advantage of using the GIPs is that each has a bit-slice processor on which the programmer can define instructions useful for a particular application.

A fifth GIP 19 is provided for defining one or more masks and other features such as menus.

The construction of one of the GIPs of Figure 1 is shown in Figure 2. Each GIP comprises a bit-slice processor 20 coupled to bulk memory 21. This memory 21 will hold image data, brush profiles and text as required and is used as virtual image memory. The bit-slice processor 20 is also connected to a pair of framestoreε 22, 23 each of which has dimensions 1280x1024 and is 8 bits deep. In the GIPs 15-18, each framestore will hold 8 bit colour data while in the mask GIP 19 each framestore can be used to hold 8 bit masks or two separate 4 bit masks. Furthermore, one of the framestores in the GIP 19 can be used to display menus in one four bit plane and overlays in the other four bit plane. Overlays comprise construction lines and boxes and the like which are to be displayed on the monitor.

The eight bit data in each framestore 22, 23 is applied in four bit "nibbles" to respective scroll, amplify and zoom circuits 24-27 which operate in a conventional manner to perform one or more of the functions of scroll, zoom and amplify, the outputs from these circuits being fed to a mixer circuit 28. The circuit 28 mixes the data from each of the framestores 22 associated with the GIPs 15-18 with the data from each of the frame stores 23 associated with the GIPs 15-18 in accordance with the mask stored in the framestore 22 of the GIP 19. This mixer circuit which operates in two stages is described in more detail in EP-A-0344976.

The output from the mixer circuit 28 is fed to a two stage colour converter 29 which converts the four colour component data to three colour component data e.g. red, green and blue suitable for controlling the display on a monitor screen 30.

In use, images are stored on the high speed image disks 11 and these images may have been generated by scanning original transparencies or other representations or they may have been created electronically using an electronic paint brush. The host 1 causes relevant portions of these images to be "paged" in and out of the bulk memory 21 in the GIPs 15-18 and brush profiles to be loaded and unloaded from the bulk memory 21 in the GIP 19. The interface adaptor 10 has its own 68020 processor to allow it independently to control the disks 11. The GIPs 15-18 are directed by the host 1 to do various things to images in the bulk memory 21 so that when a GIP attempts to access an address in an image that is not currently in its bulk memory then part of that memory is written back to disc and a new portion read in. After the GIPs have finished processing, the data in the framestores is then scrolled, zoomed and/or amplified as necessary, mixed in the circuit 28, converted to monitor format and then displayed. If the host 1 wishes to display menus on the screen, these are drawn into the mask GIP framestore 23, known as the "overlay plane".

In normal paint mode, the operator selects a paint and a brush profile which, for example, may define a Gaussian distribution and this profile is stored in a special portion of each bulk memory 21 of the GIPs 15-19. The operator then "paints" by moving the pen 31 across the tablet 5 and the location of the pen 31 is monitored by the host l which then instructs the GIPs 15-18 to load the respective framestores 22 with paint data. Thus, at each position of the pen 31 the GIPs compute the data value to be located in each pixel covered by the brush raster within the appropriate framestore 22 in accordance with the well known formula New Pixel Value = (1-α) x Old Pixel Value + α x Paint value where α is a factor lying in the range 0-1 and is obtained by multiplying a current value defined by the brush profile with a percentage value selected by the operator which defines the maximum percentage of paint which can be used in this operation. The new pixel value is written into the framestore 22 and automatically copied into the appropriate position in bulk memory 21.

The contents of the framestores 22, 23 of each GIP 15- 18 are mixed and displayed as previously explained under the control of the data stored in the mask framestore 22 (of the GIP 19) . Initially, no use is made of the framestore 23 so that effectively no "mixing" takes place. In conventional electronic painting no account is taken of the effect of wetness of the paint being laid down. This can be dealt with however, by making use of framestores 23 of the GIPs 15-18. Essentially, the appearance of the image is varied with time under the control of the host 1 and the GIPs 15-19. This could be done destructively by modifying the contents of the framestores 22 and hence the bulk memory 21 but in the preferred example, use is made of the framestore 23. In this context, the framestores 22 will be referred to as the

"dry" stores and the framestores 23 as the "wet" stores.

In this example, it is assumed that a previously "painted" image is down loaded from the disk store 11 into the bulk memory 21 of the GIPs 15-18 and hence the corresponding framestores 22. This image is shown at 40 in Figure 3. A painting operation commences in a conventional manner with the operator selecting a brush profile and paint colour and movement of the pen 31 is monitored by the host 1 which stores in its memory 8 a journal of the specific locations of the brush path. In addition, the effect of moving the brush is generated by the GIPs 15-18 in a conventional way except that the resultant colour is laid down in the framestores 23. An example of the content of one of the framestores 23 is shown at 41 in Figure 3. Simultaneously, the brush path is recorded in the framestore 23 of the GIP 19 (the mask framestore) . In the case of the mask framestore, the data effectively defines a monochrome image.

The result of this operation is displayed on the monitor 30 as shown in Figure 3 via the mixer 28 which is controlled by the mask in the mask framestore 23. Due to this method, the image shown on the monitor screen 30 has been non-destructively obtained and the image in the framestores 22 has not been modified. If the operator is satisfied with the image as it now appears, he can actuate a so-called "grab back" facility in which the bulk memory 21 is modified in accordance with the displayed image under the control of the mixer circuit 28. The method by which the mixer circuit 28 operates is to apply an algorithm to each pixel read from the framestores 22,23 of the GIPs 15-18 under the control of a masking value read from the corresponding pixel in the mask framestore 22 in accordance with the following formula. Display Pixel = β x FS23 + (1-3) FS22 where β is obtained from the value in corresponding location in the mask framestore 22.

The invention enables the image stored in the framestores 22 to be modified automatically to take account of the wetness of the paint laid down and the effect of time. This is achieved by generating the effect of transferring paint from the wet framestores 23 to the dry framestores 22. In practice, this transfer is achieved by applying a conventional "merge" brush to the image in the framestore 22, that merge brush tracing the path of the original brush applied into the framestore 23 (as determined by the previously stored journal in the memory 8) and by applying an etch brush along the same path in the framestore 23. The two brushes need not necessarily have the same profile and different effects can be achieved by suitably selecting the profile of these brushes and the time intervals between performing the transfer operation. The merge brush will operate in accordance with the following algorithm: New Pixel Value = γ x FS23 + (1-γ ) FS22 where γ is obtained by multiplying the corresponding brush raster value by a previously selected percentage chosen by the operator corresponding to the maximum amount of paint which can be transferred from the framestore 23 in one operation.

The etch brush operates in accordance with the following formula:

SUBSTSTUTE SHEET p ^* = MAXIMUM_0F (s , (p- f (c - s) ) ) where : p^» = resultant image pixel, p = original image pixel, c = colour to remove, s = substrate colour (i.e. the limiting amount to which the colour can be etched set by user, and defaults to zero) and f = is a factor lying between 0 and 1 and is obtained by multiplying a current value defined by the brush profile with a percentage value selected by the- operator which defines the maximum amount of paint which can be removed.

The initial effect of these transfer operations will not be perceived on the monitor screen .30 unless it is arranged that the merge brush also applies a fade operation. However, the effect will become apparent during subsequent painting operations since the amount of wet paint still present will have decreased so that mixing new paint with the remaining wet paint in the framestore 23 will have a different result from mixing the paints during a single painting operation.

Other effects can be obtained as mentioned above. For example, smudge effects can be obtained by re-applying a brush (to the wet framestore) with a larger profile than the original brush, perhaps random in its distribution rather than Gaussian, and a different (lower) density which can be arranged to look like a smudging effect. The effect can be built up in the wet framestore by repeated automatic brushing with various types of brush and this could be stopped by the operator at any time when the desired effect had been achieved. The brush applied to the wet framestore may be a "data randomised" brush to get smudging of the original paint and this should not be confused with a random profile brush. A random profile means that the raster patch contains a random distribution of data that controls the amount of an effect (e.g. paint merging) but a randomized brush may have for example a Gaussian profile control data in a raster that controls the amount that the underlying image data is randomized in terms of individual pixel values.

For some effects it is only necessary to modify data in the wet framestore unless it is desired finally to modify the existing image.

Since a journal is kept of the brush tracks, the host could be adapted to deviate from those tracks for some effects. This could include random small "blobs" of paint appearing somewhere off the main track of the existing brush work in the dry store to obtain >a wet, splashy effect, or "runs" at tangents to the brush track.

Claims

1. A graphics system comprises an input device; means for detecting movement of the input device relative to a surface; and display means for displaying patterns which follow movement of the input device to simulate the creation of patterns on a surface with a drawing instrument, and is characterised in that the system further comprises first and second stores for storing data representing the amount of dry paint *and wet paint respectively in the finished image; combining means connected to the first and second stores for generating a resultant image on the display means by combining the data in the first and second stores; and processing means for modifying the data in the first and second stores to simulate the effect of increasing or decreasing the degree of wetness.

2. A system according to claim 1, wherein the processing means is adapted to modify the data in the wet and dry stores to simulate the transfer of paint from the wet store to the dry store at regular intervals.

3. A system according to claim 2, wherein the processing means applies a "merge" brush to the dry store and an "etch" brush to the wet store.

4. A system according to claim 3, wherein the processing means enables the "merge" brush and the "etch" brush to have different characteristics.

^SUBSTITUTESHE