JPH03164982A - Electronic graphic system - Google Patents

Abstract

PURPOSE: To display the work of an artist in real time during a preview by turning high resolution images to a low resolution, displaying them on a monitor, achieving desired change and then converting low resolution control image data to high resolution display. CONSTITUTION: This system is provided with a down-converter circuit 3 for reducing the amount of the information with the high resolution images so as to store the image with the reduced resolution as the observation images of an observation frame storage 4, a stylus/touch tablet device 6 capable of changing the images displayed on the monitor 5 and an up-converter 18 for performing the opposite operation of the down converter 3. Then, when the system is in a preview mode, the high resolution images are displayed on the color monitor with the lower resolution and the displayed images are changed as needed in real time by a user. When the system is in a synthesis mode, control image data are up-converted so as to control deformation. Thus, a high resolution image operation is made possible without recognizable delay between the operation of an operator and the effect.

Description

DETAILED DESCRIPTION OF THE INVENTION Field of the Invention This invention relates to electronic graphics systems.

BACKGROUND OF THE INVENTION Electronic graphics or image systems by electronic means are known in which the depiction of color images can be simulated or parts of one image can be merged into another. One such graphics system is described in our British Patent No. 2.089,625 and corresponding US Pat. No. 4,514,818, the teachings of which are incorporated herein by reference.

The system includes operator interface means that may be used by an operator to select from a range of colors and intensities and to select from a set of conceptual depiction tools for use in rendering.

Colors are selected by preset digital signals representing the components of the selected color, while tools are selected by selecting between different tool representations, each representing the image points formed by the tool to the image points it covers. represents the conceptual distribution of colors that could be produced. The system also includes a touch tablet and stylus combination, which is typically an interface means for generating a position signal specifying the position of the stylus with respect to the touch tablet. For some instruments, for example a paintbrush, it is acceptable for a position signal to be generated for each movement of the stylus depending on the distance between image points or similar distances, whereas for some instruments, for example an airbrush For other tools, position signals are generated at regular time intervals even if the stylus remains stationary on the touch tablet.

When a position signal is generated, new video signals (picture elements) are obtained for all image points of the patch covered by the selected tool. An image storage device is provided and each new picture element is written at the appropriate image point in the storage device. Such new picture elements are created in response to the selected color and the selected instrument distribution, and also to the pressure applied to the stylus and the respective image of the storage device, as described in the aforementioned patents. obtained by the processing circuitry in response to previously stored pixel values at the points. The user, who is expected to be an artist who typically lacks experience in the use of computer-based systems, selects the desired color and tool, and then manipulates the stylus to create a series of steps to define the path or positioning of the stylus on the touch tablet. by generating a position signal. Processing circuitry is responsive to the position signal to obtain picture elements for the patch of image points, and these picture elements are written back to the image storage device. In order for the artist to be able to observe this creation, the stored image is read repeatedly and the parentheses are applied to a TV-type color monitor, so that the assembly of the image can be observed and, of course, Such a system is not limited to TV-type formats, and any suitable video format may be employed. A system such as that described avoids the jaggy problem and unpleasant stepped appearance normally associated with lines that do not exist horizontally or vertically in a raster dene play.

It is important for the artist that the monitor displays the assembly of the image in virtually real time, and that virtually every patch of picture element that can be identified by the frame period position signal should be processed. , the parenthesis elements should be written to the image storage device (without interfering with reading from the storage device to the monitor) within a frame period when at least the time is averaged over several frame periods. It means that. The number of picture elements to be processed depends on the number of image points required per unit of image area, i.e. on the resolution of the system, and if the resolution is increased, for a given movement of the stylus In contrast, the number of position signals generated in a given time interval is also increased, as is the number of pixels in a given brush patch. Real-time processing typically requires 50
1250 lines per field at fields/second,
or even the high definition standard of 1125 lines/60 fields can be achieved in systems operating according to television standards. However, if for example 4
Difficulties are encountered if higher resolution of print quality images containing 000x5000 image points is required. Another difficulty is that TV-type monitors for displaying images of such resolution are not readily available as commercial items and, if custom made, would be very expensive. .

To reduce these problems, it has previously been suggested that rendering (hereinafter simply "rendering") should be done at the resolution of commercially available TV-type monitors. One system that takes this approach, and thus considers the creation or modification of images according to changes made to the form of the image displayed on the display screen, uses an initial Contains a converter for converting a resolution image to a lower resolution representation of the image. The lower resolution image is provided to a frame store which is read repeatedly for display of the image on a color monitor. The system is also compatible with our above-mentioned British Patent No. 2. No. 089,625 and corresponding U.S. Pat. allows the artist to select the colors and paint on the image. The processing circuit 3 reads the image data directly from the frame store, modifies that data according to the data generated by the artist's depiction on the touch tablet, and writes the modified image data back to the frame store for display. This depiction is done on a low resolution image so that the processing circuitry can keep pace with the artist's stylus movements and so that the results can be viewed on a monitor in real time. When the artist temporarily or finally finishes rendering, the system communicates the modified image data back to the high capacity storage device through an up-conversion circuit that converts the modified image back to high resolution.

While the depiction at low resolution allows the artist to observe the results of his actions in seemingly real time, down-conversion of the initial image followed by up-conversion makes it necessary at higher resolution. Despite the number of image points involved, it necessarily softens the high resolution image, effectively reducing the resolution of the image to a lower resolution.

4 Our European Patent Application No. EI'-A-0゜20
In issue 2.014, we store high-resolution image pixel data in a high-resolution image or high-resolution frame store, and draw new pixel data directly into that frame store through the use of a stylus and touch tablet. describes a system that can be modified by A converter is provided that reduces the number of pixels between the high-resolution frame store and the observed frame store, so that the artist can see when changes are being made to them, and it refreshes the monitor display. be read repeatedly to The converter is configured such that selected regions of the high-resolution image are transferred to the viewing storage and displayed on the monitor at full resolution, or a low-resolution representation of the entire image is created and for display. is arranged so that it is sent to the observation storage device. While this system provides real-time display of an image or portions thereof, it is assembled in a high-resolution frame store, so in most situations a large amount of silicon memory is required, and a high-resolution frame store is required. The assembly of the image may be delayed even after the movement of the stylus.

In a device according to our British Patent No. 2.089,625 as currently manufactured and sold,
Additional means are provided that allow the system to be configured to perform image composition in addition to rendering. Certain shapes that allow for synthesis are covered by our UK Patent No. 2.113.9.
No. 50 and corresponding U.S. Patent No. 4,602.28
No. 6, the teachings of which are also incorporated herein by reference. In such a configuration, storage means are provided having the capacity to store two independent images, herein referred to as foreground and background images, respectively, and a further image, referred to as a control image or stencil.

Control images, like foreground and background images, can be created by an artist using the system in rendering mode. The control image typically includes a region of picture elements with a maximum value representing opacity and a minimum value representing transparency, and may be used to overlay or mask selected portions of the foreground image while exposing the remainder. made to do.

The control image is defined by 8 bits and can be displayed as a single color superimposed on the foreground image during creation. a first for reading the picture elements of the background image and the control image in parallel from respective parts of the storage means and providing them to a linear interpolation circuit which is enabled during the image preview mode and during the image synthesis mode;
Addressing means are further provided. The second addressing means, incident to the reading of the foreground image pixels and the control image pixels, also provides background image pixels to the two interpolation means.

The first addressing means is connected to the manipulator, which addresses the foreground and stencils with respect to the background pixels so as to effect a spatial transformation of the foreground and stencil with respect to the background under control of the stylus or other user-operable means. The order in which control picture elements are read can be changed. While the system is in preview mode, the artist can manipulate the stylus or other control means to perform a series of desired transformations, such as zooming, panning, scrolling, rotation, and changing perspective. . During the frame period, the foreground image pixels in a modified order and the background image pixels are fed to the input of the first interpolator, while the control image is controlled to act as an interpolation coefficient for each pixel. given to input. The output of the interpolation means is provided to an image monitor and is displayed but does not affect any stored images. Interpolation creates an effect on the display such that the control image represents the foreground image that is opaque, and the control image represents the background image that is transparent, simulating portions of the foreground image above the background. have

Using a stylus or control means to vary the spatial transformation, the artist can try inserting various positions from the foreground into the background image until he perceives the desired transformation. Immediately, the artist switches the system to compositing mode by issuing a predetermined "stick" command.

The foreground image pixels and the background image pixels are then read into a twenty-eighth interpolation means and interpolated under the control of the control image pixels, the foreground image pixels and the control image pixels being determined on the last try during the preview mode. is read in the specified deformation order. Furthermore, in the compositing mode, the output of the interpolation means is read back into the storage means for the background image and produces a permanent change in that image, which simulates pasting the foreground portions over the background.

The specific synthetic steps described in the preceding paragraphs are merely illustrative, and many variations of the steps are possible. However, it should be pointed out that the difficulties in achieving real-time display of Archineto's works during composition and especially during previews may be more severe than during depiction. During the preview, the artist virtually attaches the portion of the foreground image that is to be pasted to the point of the stylus;
Turn and move it as seen on the monitor until he finds the desired transposition. The number of pixels to be processed for each deformation may be much larger than for any brush stamp, and the processing required to arrive at the deformation for each pixel to be deformed is particularly limited by the pixel address. can be complex when equations for evaluating changes in may be involved. Therefore, it can be difficult to achieve real-time motion during previews, and the hiccups will seriously impede the artist's creative process. It also reduces the number of pixels that need to be processed by limiting processing to pixels that lie within a computer-generated border slightly larger than the portion of the foreground image that is to be pasted. have been proposed, but even so, real-time operation is unlikely to be achievable in many cases at higher resolutions than foreseen.

OBJECTS AND DESCRIPTION OF THE INVENTION The present invention provides a system by which high resolution image manipulation is possible without appreciable delay between movements made by an operator and the resulting effects of those movements being viewed on a display. The purpose is to provide.

0 In accordance with one aspect of the invention, an electronic graphics system is provided for use in modifying data defining an initial high-resolution image, in which data related to a user-defined low-resolution control image is controlling the combination of image data of the initial image with data defining a low-resolution display of the initial image, generating data representative of the low-resolution combined image that is displayed sequentially on the display monitor; Once the desired changes to the image displayed above are achieved by the user, the low resolution control image data is converted to a high resolution representation thereof, which controls the combination of said other images with the initial high resolution image data. , used to generate data defining a modified high-resolution image.

Thus, the present invention in one aspect provides that initial high-resolution images may be of low resolution with only added paint or without features of the initial images touching that retain their initial high resolution or sharpness. To provide a high-resolution graphics system that allows a relatively low-resolution drawing system to be finished in a manner that softens the image quality. In this sense, the primary purpose of the control image is to delimit the area of the image that is touched by the paint.

As will become apparent from the detailed description of the preferred embodiments of the invention that follows, in the practice of the invention, as exemplified by the embodiments, a high capacity storage device, such as a disk storage device or a digital video tape. The first high resolution color image, for example 4000 x 5000 picture elements, stored on the recorder is down-converted to produce a lower resolution representation of the first high resolution image. The low-resolution image is applied to a viewing frame store of, for example, 1920×1035 pixels, and the contents of the viewing frame store are, for example, a conventional color T
It can be read repeatedly non-destructively for display on a V-monitor.

User-operable control means are preferably provided for use in making said changes, said control means comprising input means arranged to convert the user's descriptive behavior into data representative of said behavior. , the data is used to modify a control image as previously stored, and combining the low resolution representation of the initial image with the other image data according to the low resolution control image data;
Further comprising combining means for generating said low resolution combined image data for display.

Preferably, the combining means includes a linear interpolation circuit arranged to be responsive to control image data as a factor in the interpolation of the low resolution representation of said initial image data and other image data.

In the above aspects of the invention, the other image data may be data that defines another high resolution image, the low resolution representation of which is combined with the low resolution representation of the initial image during previewing, or that the user It may be specified color data. Thus, in the implementation of the invention according to the embodiments described below, means are provided to allow the artist, or even the rendering tool, to select the image information to be combined with the initial image by the user's rendering. It's okay to be hit. This selectable image information may be a single color selected from a palette of colors available in the system, or it may itself consist of further images or portions of images extracted from an image storage device. .

In embodiments of the invention, image data to be displayed is transformed from data defined in terms of subtractive cyan, magenta, and yellow (CMY) components to data defined in terms of additive red, green, and bicolor (RGB) components. Means may also be provided for converting the data into data. RG
The B component forms the basis of a visually displayed image, such as that seen on a color TV monitor, whereas the CMY component usually forms the basis of a printed image, and the CYM and RG components
By providing a means for converting to and from B, the image data to be displayed can be accurately colored by the user, regardless of whether the original image data is stored in RGB or CYM format. Can be easily controlled so that you can view the attached images.

A system embodying the invention allows a user to, if so desired, in no way affect the low-resolution initial images held in the viewing frame store.
And of course, it allows for experimentation by manipulating images displayed in real time without affecting the initial high-resolution images held in disk storage.

Therefore, there is no risk of the user inadvertently making destructive changes to the original high resolution image.

As will be explained in more detail below, in one of the preferred embodiments of the invention, the system:
For example, a color wash (Col
or wash). Select the color to be added to the displayed image,
The desired color wash effect can then be obtained in the first image by using a stylus on the tablet to draw five lower color intensities on the image. By increasing the intensity of the color imparted by increasing the pressure on the drawing tool, the user can exaggerate and replace portions of the original image with the drawing tool.

In another of the preferred embodiments of this invention, as will also be explained in more detail below:
The system can be used to combine portions of another image with the first image. By varying the transparency of the control image or stencil by increasing the pressure on the drawing tool, selected portions of other images may be transparently overlaid onto the first image, or portions of the first image may be transparently overlaid onto the first image. may be replaced.

The invention can also provide an initial high-resolution image modification according to the changes indicated on the displayed image once the user has finished making changes to the image. To this end, means are provided for upconverting the low resolution control image data held in the stencil frame store to a resolution equal to that of the original high resolution image held in disk storage. The up-converted stencil data is used as a high resolution weighting factor or interpolation coefficient in a one-time linear interpolation. In a first preferred embodiment of the invention, high-resolution image data is read from a storage device under the control of an up-converted high-resolution control image and by combining it into user-selectable color data. The modified and resulting combined image is then written back to storage.

In another preferred embodiment of the invention, the high resolution initial image data and the other image data are both read from the storage device, and the initial image is modified by combining it with the other image data, and as a result The resulting combined image is written back to storage. The high resolution combined image can then be read from storage and used in controlling any suitable high resolution printing device.

The user-operable control means may further include low-resolution transformation means for spatially transforming the low-resolution other image data and the low-resolution control image data with respect to the low-resolution first image data. good. Corresponding high-resolution deformation means for spatially deforming the high-resolution other image data and the high-resolution control image data with respect to the high-resolution first image data also provides a corresponding deformation on the high-resolution image. It should be established to enable this to be carried out.

In another of this aspect, the present invention includes a first storage means having a capacity for storing first picture element data defining a high-resolution image; providing an electronic graphics system comprising down conversion means for obtaining pixel data and defining said image at a lower resolution; and second storage means having a capacity for storing said second pixel data; It includes control input means for providing third picture element data defining said lower resolution control image; and at least one picture element data obtained from said second storage means in response to said third picture element data. user-operable means for obtaining a series of experimental image compositions comprising pixel data; means for displaying said trial compositions as they are obtained;
up-converting means for obtaining pixel data to define the control image at the high resolution; and at least pixel data obtained from the first storage means in response to the fourth pixel data. , and image composition means for obtaining an image composition corresponding to a selected one of said trial image compositions.

This aspect of the invention is based on the concept of restricting up-conversion to a control image and using it to control the processing of one or more high-resolution images (so that the processing The up-conversion of the control image may have the effect of softening transients in the control image, and this may also soften the resulting edges or transients in the final image.Furthermore,
9. The control image will often contain substantial regions of uniform values, and it will not suffer from up-conversion,
and will have no noticeable impact on the controlled image resolution.

The invention also provides a method of electronically processing an initial high resolution image to produce a modified high resolution image, wherein data related to a user defined low resolution control image is controlling the combination of image data with data defining a low-resolution display of the initial image, generating data representing the low-resolution combined image for display on the display monitor, and once displayed on the display monitor; Once the desired changes to the image are achieved by the user to display the image, the low resolution control image data is converted to a high resolution representation thereof and the control image data is combined with the initial high resolution image data. , thereby used to generate the modified high-resolution image.

Further features and advantages of the invention will also become apparent to those skilled in the art from the following detailed description of preferred embodiments of the invention.

DETAILED DESCRIPTION OF THE INVENTION Referring now to FIG. 1 of the accompanying drawings, a first embodiment of an electronic graphics system 1 is shown therein, which includes a high capacity storage device 2, which is e.g. F for storing data related to at least one initial high resolution image to be processed by the system 1.
It may be a Ujitsu disk pack or a 5ony digital video table recorder, or any other suitable high capacity storage device. The high capacity storage device 2 can be used to store pixel data for many different high resolution images, and can be quickly accessed by the system to pixel data for any one of these images. provide easy access. To simplify the explanation of the example,
Picture element data representing an image will hereinafter be referred to in relation to the image it represents, and therefore when reading the text below, reference to an image is effectively referred to in relation to the picture element data representing that image. It should be remembered that there is.

Typically, the images held in the high capacity storage device 2 are of a resolution suitable for high quality color prints or the like;
For example, for an image with a size of 25 cm x 25 cm, for example, 4000 x 5000 pixels or 8000 pixels
It would have a resolution requiring X10,000 picture elements. However, the invention is not limited to such resolutions or image sizes, and other high resolutions and sizes may be employed for use in providing images of desired quality. This is a place where being deaf is recognized.

Image data is stored in storage device 2 in a format defined by the manufacturer of the device. Thus, for example, image data may be stored in a high-capacity storage device in a byte-parallel, bit-serial format, or it may be
It may be stored on a digital tape recorder in CRCB format.

However, no matter how large the image data is held in the storage device 2, the system will typically be designed to operate in a byte-serial, bit-parallel format, and the extracted image data will be stored in the system. A buffer 2a is provided for converting it into the correct format for use.

Different stages of the system operate at different speeds and with different data formats. To ensure that data is passed satisfactorily between the different stages, buffers such as buffer 2a described above may also be used in the system under the control of a control subsystem (not shown in FIG. 1). is provided to facilitate the transfer of image data.

In use, the initial image is transferred from the storage device 2 to the buffer 2a
and a lower resolution image is obtained by a down conversion circuit 3 for storage in a viewing frame store 4 and display on a color display monitor 5. For example, the down conversion circuit 3 converts an image of 1920 pixels and 1035 pixels from the image in the storage device 2 by 15.
Reduce the resolution to one compatible with display monitors of 1125 lines at 60 frames per second or 1250 lines at 50 frames per second. The low-resolution image data of the viewing frame store 4 (hereinafter referred to as viewing images) is read serially from the frame store 4 in a raster sequence under the control of a serial address generator for display on a monitor 5.

The downconverter circuit 3 includes a two-dimensional spatial filter that filters the initial high resolution image and reduces the amount of information in the image so that the reduced resolution image can be stored as an observation image in the observation frame store 4. . Using a spatial filter in the down-converter circuit 3 will maintain the integrity of the first image of the observed image, and therefore the observed image will be a good representation of the first image.

The system 1 as shown in FIG. 1 also includes a stylus/touch tablet device 6 by which the artist can change the displayed image shown on the monitor 5. As the stylus is drawn by the artist through the tablet, many signals are output from the device. A signal XY representing the instantaneous position of the stylus on the tablet is output to a patch address generator 7. The patch address generator 7 converts the XY coordinate information from the tablet into a corresponding position, ie image point, address in the stencil frame store 8, the purpose of which will be explained in more detail below.

The stylus/touch tablet device typically provides coordinate information at a much higher resolution than that of the stencil frame store (same as the resolution of the viewing frame store 4, i.e. 1920 x 1035 pixels), and the patch address generator provides each XY coordinate. arranged to translate the pair from the touch tablet to the corresponding address in the stencil frame store. The conceptual drawing tool is used to pull the image into the stencil frame store 8, and after each update of the frame store, the system determines whether the incremental movements of the stylus on the touch tablet represent one image point or A similar interval may be exceeded and then arranged to be integrated until the frame store is again updated by stamping the modified patch of pixels in the stencil frame store 4. A signal representing the instantaneous pressure of the stylus on the touch tablet is also transmitted to the stylus pressure register 9.

As in the system described in our above-referenced patent, a set of artist's selectable conceptual tools are each stored in brush shape memory 10. The tool can be stored as a representation of a number of consecutive three-dimensional shapes covering a patch of image pixels, or as a mathematical expression representing the contour of the tool. The address signal output from patch address generator 7 is also used to synchronize the addressing of stylus pressure register 9 and plush shape memory 10. Selection means (not shown in FIG. 1) are provided for allowing the artist to select one of the drawing tools from the set. The selection means may be, for example, a thumbwheel arrangement that the user can turn to select the selected tool, or a menu-driven arrangement displayed on the monitor 5.

The manner in which position signals from touch tablet 6 are generated will depend on the particular conceptual depiction tool selected by the artist. For example, where the selected tool is, say, a paintbrush, the position signal is generated only when the integrated stylus movement is, for example, the distance between the image points as described above, whereas in, say, an airbrush, On the other hand, position signals are generated at regular time intervals even if the stylus is held stationary on the surface of the pen.

In use, the data output from the stylus pressure register 9 and the brush shape memory 10 as the artist moves the stylus via the touch tablet are multiplied together to provide an interpolation factor for use by the brush processor circuit 11. arise.

The brush processor circuit 11 performs a continuously circular read-modify write operation on the stencil or control image data in the stencil frame store 8 on a pixel-by-pixel basis. Control image data is extracted from the stencil frame store 8 and is negatively added to or subtracted from a preset image color or intensity value held in the intensity register 12 by a summing unit 13 (e.g. a 743381 device). The resulting sum outputted from the summing unit 13 is sent to the multiplication unit 14 (
The coefficient is then multiplied by the coefficient by a summing unit 15 (eg a 74S38 i device) to the data extracted from the stencil frame store 8. The data output from the summing unit is then written back to the stencil frame store 8, replacing the initial data in the stencil frame store 8.

The steps performed by the brush processor can be expressed in a mathematical algorithm for each image point of each brush patch as follows: S+=IK+S+-+ (IK)8 where S, = New stencil data stored in the stencil frame store 5l-1 - Previous stencil data stored in the stencil frame store ■ = Current intensity value - Interpolation coefficient ≦1. The read-modify-write steps are performed in successive cycles on the data held in the stencil frame store, but from the above algorithm the stencil data is K >
It should be clear that it will only be changed when Q, that is, when the artist is drawing on the touch tablet with a stylus. K is the product of brush shape and stylus pressure, and the stencil data in the stencil frame store 8 remains unchanged when the artist is not using the stylus/touch tablet device under pressure. read-modify-write cycles are not performed.

The degree of modification of the stencil data depends essentially on the brush shape, which it will be recalled is predefined in the brush shape memory 9 as a continuous three-dimensional shape, and on the instantaneous stylus pressure. It should be appreciated that each time an image point of a predefined brush shape moves conceptually over a new pixel in the stencil store 8, the value of the control image pixel is calculated again by the brush processor 11. . Thus, where only the edges of the brush shape move over a particular control image pixel, the value of K will be relatively low. In contrast, if the center of the brush shape moves over a particular control image pixel, the contribution from the brush to that pixel starts at a relatively low value at the leading edge of the brush shape, and It will increase to a high value of K at the center of the shape and then decrease to a low value of K at the trailing edge of the brush. Thus, in this latter case, the brush's total contribution to the control image pixels will be more substantial than in the former case. This method of processing data drawn to the stencil frame store avoids the problem of stepped edges by producing a stencil border contour that is not stepped.

The color is one byte (8 bits) that specifies the red component of the color in the system, one byte that specifies the green component of the color, and one byte that specifies the blue component, or alternatively, one byte for each CYM component. Defined using. Thus, at least 24 bits are used to define the color of each pixel.

However, the stencil need only be black and white, since it will be the same for each color component, thus controlling the combination of each of the three color components of the image with each color component of the artist-selected color data. This is because they can be used simultaneously or repeatedly to

Since the stencil is black and white, only 8 bits are sufficient for each picture element of the stencil. In this regard, although only a single line is shown connecting the various parts of the system to each other, there are often many parallel lines providing suitable connections, as will be obvious to those skilled in the art. It should be pointed out that.

1 Stencils or control images in the stencil frame store 8 are used to define areas of the image where the artist wishes to make the desired changes. In the illustrated case, the artist selected color is used to touch again the area of the original image defined by the stencil drawn by the user by manipulating the stylus on the touch tablet. During the creation of a stencil, the system arranges the array so that the artist-selected color data is used directly by the interpolator 16 under control of the stencil data to change the displayed image, i.e. to touch the displayed image again. may be done. In this way, changes in the images can be viewed by the user as they are being created by merging the artist-defined color data with the observation data of the interpolator 16. Alternatively,
The system 1 can be configured to display a representation of a stencil as it is drawn. This means that the frame store 4 observes the stencil values written to the stencil store.
This can be achieved by having only one of the color components, 2, be used as an interpolation factor, for example for red, and thus causing the stencil to appear as a red overlay on the displayed image. can.

Addressing the stencil frame store 8 is synchronized with the addressing of the viewing frame store 4 so that picture elements at corresponding positions in the two frame stores are read from the two frame stores at the same time. Image data from the viewing frame store is input to a linear interpolator or linear interpolator 16 .

Means (not shown in FIG. 1) are also provided to enable the artist to select color values from a palette of colors available in the system. The artist selected color is also input to linear interpolator 16.

Linear interpolator 16 interpolates the viewed image and artist selected colors so that they are selectively merged or combined. The stencil data in the stencil frame store 8 is used on a pixel-by-pixel basis as an interpolation factor α, which determines the contribution from the observed image, 3 and the artist-selected color to the resulting merged image displayed on the monitor 5. .

In its simplest form, the process performed by the linear interpolator 16 can be viewed as the addition of weighted artist-selected colors, the weights being applied to the observed image as defined by the pixel values of the stencil frame store. It is determined by the value of interpolation coefficient α. Thus, for low interpolation coefficient α values, the contribution from the artist selected color to the displayed image will be small, but for high coefficient values the contribution will be larger.

The contents of the stencil frame store 8 and the viewing frame store 4 are thus always kept separate from each other.

These frame store data are read and the observed image data is combined with artist defined color data by linear interpolation circuit 16 under control of the stencil data for display on display monitor 5. Therefore, the artist is free to adjust the shape of the stencil drawn on the stencil frame store 4a8 and the color selected for the displayed image combination.
Experiments can be performed without changing the observed images held in the .

An optional feature of the system is a color conversion circuit 17, which converts the image data output from the linear interpolator 16, typically used in graphics systems where images are to be printed using CYM or CYMB inks or the like. Subtractive color cyan, yellow, magenta (CYM)
or, in some applications, from a cyan, yellow, magenta, black (CYMB) signal to an additive red, green, blue (RGB) signal, such as that typically used in graphics systems where images are to be displayed using color monitors. Convert to signal. Providing this conversion circuit 17 allows for greater flexibility in the format of the initial image, i.e.
An RGB formatted or CYM(B) formatted image can be operated upon by the system 1 and displayed in the correct colors for the image on the display monitor 5. Of course, if the original image data is held in the storage device 2 in the form of RGB data, there will be no need to convert the data for display on the monitor 5. However, in this case, once the desired modified image is ready to be printed, a conversion circuit such as conversion circuit 17 is used to convert the image data from the RGB video format to the required CYM or CYMB print format. Can be used in high resolution. Our own color conversion circuit is particularly suited to this task, as described in our European patent application published as EP-A-0,245,943, the teachings of which are incorporated herein by reference. .

All of the parts of the system described so far, including interpolator 16, operate at the relatively low resolution of monitor 5, and this allows the artist to work in real time with the system being created in preview mode. That is, the artist can preview the work as it is being drawn or constructed using a stylus/touch tablet device. Control data is processed and pulled to the stencil frame store that affects the combination of artist selected colors and viewed images, and the resulting modified images are displayed on the monitor without appreciable delay. Thus, the portions of the system so far described are arranged so that the artist's depiction can be previewed in real time.

Once a modified image satisfactory to the user has been achieved, the initial high resolution image held in storage 2 can be modified according to the displayed image if the artist so desires. The system is again configured into a compositing mode in which the initial high-resolution image is maintained in the low-resolution stencil storage 8 by the implementation of commands predefined by the artist. changed under. The control image data held in stencil storage, ie, interpolation coefficient α, is passed to upconverter 18, which performs the opposite operation to that of downconverter 37. In other words, the control image data is 1920X10
A low resolution of 35 picture elements is converted to a high resolution of, for example, 4000x5000 picture elements.

This low-to-high resolution upconversion is accomplished in upconverter 18 by use of any suitable known technique, which converts a single picture element into a corresponding group of pictures while maintaining the integrity of the control image. or map a small number of low-resolution pixels to a corresponding larger number of high-resolution pixels.

Up-conversion of the control image may have the effect of softening the edges of control edges or other transients by making them spatially spread over a large number of picture elements. This may also result in softening edges or other transients in the final combined high resolution image.

With the system configured in its compositing mode, initial high resolution image data is read from storage 2 through buffer 2a and input to high resolution linear interpolation circuit 20 along with user selected color data.

8. The high-resolution linear interpolation circuit 20 performs only one interpolation on the high-resolution image pig and selected color data under the control of the up-converted control image data. The operation of linear interpolator 20 is similar in many respects to linear interpolator 16 described above.
The main difference is that the interpolation process is performed using high-resolution data.

The resulting modified high resolution image data output from the linear interpolator 20 is written back to storage via a buffer 21 where it is stored for subsequent use. Buffer 21 performs the opposite operation to that of buffer 2a. That is, buffer 21 is arranged to convert incoming high-resolution image data from a byte-serial, bit-parallel format to a byte-parallel, bi-soto-serial format for storage in storage device 2. Such subsequent use may be, for example, printing of the modified high-resolution image or further processing with different user-defined colors, for example by the system shown in FIG. Of course, 9 normally the system would be arranged to write over the first image with the modified image to avoid an inappropriate number of different formats of the first image stored in the storage device. It will be appreciated that storing a modified high resolution image on the high capacity storage device 5 need not result in the destruction of the original image.

Thus, the above-described embodiments of the system provide one arrangement whereby when the system is configured into preview mode, an initial high resolution image can be displayed on a lower resolution color monitor; and the displayed image can be modified as needed by the user in real time, and when the first image is formed in the compositing mode, the system up-converts the control image data to control the deformation. and can subsequently be modified according to the displayed image by using up-converted control image data.

Accordingly, the system 1 is used to modify the initial high-resolution image data held in the storage device 2 to remove imperfections, defects or other defects from the initial image. You can. By using a stencil with an unstepped border contour, the final combined or modified image will include contributions from both the original observed image and the artist-defined color data. , and this will result in a more visually pleasing final image. This merging process will now be explained in more detail with reference to FIG. 2 of the accompanying drawings.

Referring now to FIG. 2 of the accompanying drawings, a second embodiment of a graphics system 30 is shown.

In many respects, this second system 30 is similar to the system 1 described above, and the following description will be directed to those features of the second system 30 that differ from the first system 1. As will become clear from the description below, a first high-resolution image, which can be regarded as a background image, i.e. an initial image or an image to be modified, is extracted from the storage device 2 via a buffer 2a and initially or the background image by a down-conversion circuit 3 to form a low-resolution display of the image. The first image thus down-converted is stored in the viewing frame store 4 for display on the display monitor 5. A second high-resolution image, i.e. will be used to modify the first image,
And the image that can be considered as a foreground image is also stored in the storage device 2
and is down-converted by the down-conversion circuit 3. This down-converted or lower resolution foreground image is stored in image frame store 31.

The control image data drawn to the stencil frame store 8 in the same manner as described above in connection with the first system 1 is extracted pixel by pixel from the observation frame store 4 and the image frame store 31. It is used to control the combined linear interpolation circuit 16. A linear interpolator 16 interpolates the first or background image and the second or foreground image so that they are selectively merged or combined and the control image data in the stencil frame store is interpolated. Used as coefficient α. In other words,
The foreground image data is weighted by an interpolation coefficient α as defined for each pixel by the control image data, and is added to the background image data. Again, a low interpolation coefficient α value will result in a small contribution from the foreground image, and a high coefficient value will result in a larger contribution. The resulting combined image output from linear interpolator 16 is
If such conversion is required, it is displayed on the monitor 5 after conversion to RGB format.

In System 1 above, as in Framestore 4.8
．． The contents of 31 are always kept separate from each other and the user can view and view images in the form of control images drawn into the stencil frame store 8 without affecting the images held in the image frame store 4. You are free to experiment with the resulting combined images as you wish. By using a low interpolation coefficient, a ghost-like effect of a selected second image portion on the first image can be achieved in the resulting combined image. By using a high interpolation factor, the opposite effect, or alternatively a complete overlay or splicing of the selected second image portion in the first image, may be achieved.

It will be appreciated that in most situations the artist will want to make spatial changes to the foreground image before it is combined with the background image. Such spatial changes to the foreground image may include transformations such as zooming, panning, scrolling, rotation, and changing perspective. For this purpose, an image manipulation subsystem is provided as part of system 30. When the system 30 is configured in its preview mode, ie, when a low resolution image is being displayed on the monitor 5, the image manipulation subsystem, such as that shown in FIG. Predefined commands can be selected and enabled for operation by the artist.

4 As can be seen from FIG. 3, an image manipulation subsystem, indicated generally at 33, applies spatial transformations to image storage 31, such as zooming, panning, scrolling, rotation, and deforming perspective views as described above. operating under the control of the stylus/touch tablet 6 or any other user operable control means to effect both a retained foreground image and a control image retained in the stencil storage 8; A manipulator 34 is included. Address generator 35 is coupled to viewing memory 4 and controls the extraction of the background image from viewing memory 4 for display via linear interpolation circuit 16 on monitor 5 . The address generator 35 also sends the generated address data to a manipulator 34 which performs spatial transformations on the address data and to an image storage device 31 containing the foreground image.
and the corresponding transformed address to both the stencil storage device 8 containing the control image drawn by the artist. The manipulator 34 may be implemented as an arithmetic circuit that defines an artist-selectable transformation, so that the transformed address can be derived from the address from the address generator.

The calculations involved in many of the transformations are non-trivial and are repeated for each pixel of the transformed output. Previously, foreground image manipulation could only be accomplished in real time where a restriction was placed on the displayed picture elements that made up the foreground image as it was rotated by the artist on a display monitor. An improved manipulator that alleviates this problem is disclosed in our co-pending UK Patent Application No. 9013192.1 and corresponding US Application No. 534,713. This improved manipulator performs reduced transformations on only some of the incoming addresses, e.g., every other address, when the foreground image is being moved on the display screen, and the artist At other times, such as when finishing moving the image, it includes an arithmetic unit that is controlled to perform all transformations, ie for all incoming addresses. The manipulator 34 in the depiction can be based on the improved manipulator 6 described above.

In this manner, the manipulator 34 is responsible for addressing the viewing storage in order to effect the desired spatial transformation of the foreground image data and the control image data with respect to the background image.
The stencil storage and image storage are arranged to vary the order in which they are addressed. In general, the addresses obtained by manipulator 34 will not correspond exactly to the integer addresses of stencil store 8 and image store 31, and will rather consist of integer and fractional values. Integer and fractional values may be used to identify the closest group of each pixel address in the memory. These groups of picture element data are output from the stencil storage device 8 and the image storage device 31 to respective spatial interpolation circuits 36,37. Spatial interpolators 36 , 37 perform spatial interpolation on each pixel group sent from a respective storage device to obtain a representative single pixel value that is output to linear interpolator 16 .

7. A linear interpolator 6 also receives background image data from the observation store 4 and generates spatial data associated with the image store 31 according to transformed control image data from a spatial interpolator 36 associated with the stencil store 8. The transformed foreground image data from the interpolator 37 is combined with background image data pixel by pixel.

The contents of viewing memory 4, stencil memory 8 and image memory 31 are always kept separate and the contents of these memories are periodically read and combined for display on monitor 5.

In this way, the combined image displayed on monitor 5 provides a representation of the combined contents of the storage device in real time. Accordingly, manipulation of the control and foreground images by the artist will be visually immediately visible on the display with the system configured in preview mode.

In this way, the preview mode system appears to the artist to be operating as if the foreground image were attached to the end of a stylus. Foreground 8 The image moves over the background image on the display screen in real time as the stylus is moved across the touch tablet.

Once the resulting combined image as displayed on the monitor 5 is acceptable to the user, the high resolution images held in the storage device 2 can be extracted for combination. Referring now to FIG. 2 of the drawings, in the case where two images are combined by simply overlaying one image on the other according to the control image data, the two images are transferred from storage to their respective observation buffers 2a and 2a. The images can be extracted via an image buffer 32 and combined by a high resolution linear interpolator 20. Again, the high resolution form of the stencil or control image up-converted by up-conversion circuit 18 is used as an interpolation factor by linear interpolation circuit 20, which may result in softening the edges between the two images. The resulting high resolution combined image is passed to the storage device 2 via a buffer 921 for storage therein.

However, if the foreground image and the control image are transformed by the operator by the manipulator 34, an alternative arrangement, such as the arrangement illustrated in FIG. Can be used in combination.

Once the desired effect is achieved through the use of subsystem 33 of FIG. 3, the artist can instruct the system to combine the images at high resolution. This can be achieved, for example, by the artist selecting a compositing command from a menu on the monitor, which command causes the parts of the system to be re-formed into the configuration shown in FIG.

As can be seen from FIG. 4, the high-resolution image manipulation subsystem, indicated generally at 38, generates a high-resolution background image of the storage device represented by box 2' and a high-resolution background image of the storage device represented by box 2''. A high-resolution foreground image is arranged to be combined with a high-resolution foreground image according to a deformation parameter defined by a manipulator 34. An address generator 35 generates address data at high resolution, and the address data is used to extract a high-resolution background image 2'. The manipulator 34 is used to control the buffer 2a and the manipulator 34.
performs a spatial transformation on the high resolution address data according to transformation parameters previously defined and maintained by manipulator 34. (The deformation parameters are those that are maintained in the manipulator when the desired image is being displayed using subsystem 33 of FIG. 3 and the system is reshaped from its previous mode to its compositing mode. ) The high resolution transformed address data is output from the manipulator 34 to the buffer 32 for extracting the high resolution foreground image 2'' from storage. The high resolution address data is expressed in terms of lower resolution pixel positions before the low resolution stencil storage 8 can be addressed by the manipulator 34. It will be appreciated that the group of high resolution addresses provided by the manipulator 34 will have the same integer address, but different fractional addresses. The contribution made to each high-resolution picture element from the address of the storage device 8 depends on the respective fractional address, and they are added to the spatial interpolator 3
Combined by 9. Up-conversion is inherent in the process and is represented in FIG. 4 by block 18.

When control data from stencil storage 8 is sent to high resolution spatial interpolator 39, corresponding high resolution foreground image data is sent from buffer 32 to high resolution spatial interpolator 40. As far as subsystem 33 of FIG. 3 is concerned, the high resolution address obtained by manipulator 34 will generally not correspond exactly to an integer address, and therefore the group of picture elements surrounding the high resolution 2 address from manipulator 34 will be buffered. 32 and from the upconversion circuit 18 to respective spatial interpolation circuits 39,40. Spatial interpolation circuit 39.4
0 interpolates groups of pixels from upconversion circuit 18 and from buffer 32 to obtain each single pixel value that is sent to a one-time high resolution linear interpolator 20. Linear interpolator 20 also receives corresponding background image pixels from buffer 2a. The background image and the deformed and spatially interpolated foreground image are combined by a linear interpolator 20 according to the deformed and spatially interpolated up-converted control image to produce a combined high-resolution image. It occurs every element.

The combined image is sent to the storage device 2 via the buffer 21.

The second embodiment described above thus provides a system in which a first high resolution image can be modified by overlaying or splicing selected portions of a second high resolution image. Image changes are displayed in real time using a low-resolution display of images on a low-resolution monitor, and once the required combination of images has been reached on the display monitor, the first and second The high resolution images of the first and second images may be combined according to the displayed combination by upconverting the low resolution control image data to the high resolution of the first and second images.

Modifications to the above-described embodiments of the invention will be apparent to those skilled in the art. For example, while the two systems have been described with reference to the entire image being extracted from high-resolution storage, it is also possible to prepare portions of the high-resolution image to be extracted and modified by the user. It is possible. This arrangement may be desirable in situations where detailed changes are required to only a small portion of the image, and
and such features may be accommodated by appropriate modifications to the manipulator shown and described in connection with FIGS. 3 and 4 of the drawings. By preparing a display of portions of an image, the artist can view an enlarged version of the image details on the display monitor, and provide additional information for the displayed image that is subsequently stored in high-capacity storage. It would be possible to make changes to the original image while working on this expanded scale. Additionally, each of the systems described above can be arranged to divide an image into an array of sub-image portions or tiles, and to manipulate and communicate only the data regarding the tiles for which changes have been made. Of course, the tiles can be of other sizes as well.
Quantitatively, it would be 256 x 256 pixels. By using the tiling feature, the speed at which the initial image held in storage is modified according to changes to the displayed image can be increased, since changes are made to the image data of the affected tiles. This is because there is no need to do so. This also allows the system to occasionally make update changes to the initial high resolution image according to changes to the display interval. For example, an update change may be made each time the artist lifts a stylus from the touch tablet to take into account the most recent changes. Illustrating also facilitates panning or scrolling of the displayed image where the initial image is, for example, a panoramic image and is only the displayed image or a window portion of that image. Prepare the up and down converters by extracting all the tiles that contribute to the window being displayed on the monitor at a given time, and conveying these tiles to a buffer, and which should be transparent, i.e. 1 By arranging up and down converters to increase or decrease the resolution of the image data from the buffer by a factor of It can be communicated to the buffer and from there to the frame store. A surveillance region is defined in the initial image and can be used to indicate to the system when the displayed window is moving closer to the edge of the image. Image tiles can then be extracted from corresponding positions at opposite edges of the image to create a wraparound effect in the displayed image.

The invention is intended to include all such modifications, and is limited only by the scope defined by the following claims.

[Brief explanation of the drawing]

FIG. 1 is a schematic block diagram representing one preferred embodiment of an electronic graphics system. FIG. 2 is a schematic block diagram representing another preferred embodiment of an electronic graphics system. FIG. 3 is a schematic block diagram representing the image manipulation subsystem of the graphics system when formed in preview mode. FIG. 4 is a schematic block diagram representing the image manipulation subsystem of the graphics system when created in compositing mode. In the figure, 1 is an electronic graphics system, 2 is a high capacity storage device, 3 is a down conversion circuit, 4 is a viewing frame store, 5 is a display monitor, 6 is a stylus/touch tablet device, 7 is a patch address generator, 8 is a stencil frame store, 79 is a stylus pressure register, and 10 is a brush shape memory. 8 Procedural amendment (method) % formula % 1. Indication of the case 1990 Patent Application No. 232291 2. Name of the invention Electronic graphic system 3. Person making the amendment Relationship with the case

Claims (16)

[Claims] (1) An electronic graphics system for use in changing data defining an initial high-resolution image, the system comprising:
In that system, data associated with a user-defined low-resolution control image controls the combination of other image data with the data defining a low-resolution display of the initial image, and data associated with a user-defined low-resolution control image that is continuously displayed on a display monitor. generating data representing the combined resolution image, and converting the lower resolution control image data to its higher resolution representation once the desired changes to the image displayed on the display monitor have been accomplished by the user; , an electronic graphics system that is used to control the combination of said other image data with said initial high resolution image data and to generate data defining a modified high resolution image. 2. The electronic graphics system of claim 1, wherein the initial high resolution image data is stored on a high capacity storage device. (3) user-operable control means are provided for use in making said changes, said control means including input means arranged to convert a user's descriptive behavior into data representative of said behavior; The data is used to modify previously stored control image data, and the lower resolution initial image data is combined with the other image data in accordance with the lower resolution control image data for display. Claim 1 further comprising low resolution combining means for generating combined image data of
Or an electronic graphics system according to claim 2. 4. The electronic graphics system of claim 3, wherein the low resolution combining means includes a linear interpolation circuit arranged to be responsive to the control image data as a factor in the interpolation of the low resolution initial image data and other image data. (5) up-converting means for converting low-resolution control image data into high-resolution data representing the same, and combining said initial high-resolution image data and said other image data to produce said modified high-resolution image; An electronic graphics system according to any of the preceding claims, further comprising high resolution combining means for. (6) The high resolution combining means includes a linear interpolation circuit arranged to be responsive to the high resolution control image data in interpolating the first initial high resolution image data and the other image data. electronic graphics system. (7) The electronic graphics system of any of the preceding claims, wherein the other image data includes user-defined color data. (8) the other image data includes data defining a second initial high-resolution image stored in a high-capacity storage device;
7. An electronic graphics system according to any preceding claim, wherein a low resolution representation of the second image data is combined with the low resolution first image data for display on a display monitor. (9) The user-operable control means further includes a low-resolution deformation means for spatially deforming the other low-resolution image data and the low-resolution control image data with respect to the low-resolution first image data. Claims 3 and 4 comprising
9. An electronic graphics system as claimed in claim 8 depending on. (10) The method according to claim 5 or 6, further comprising a high-resolution deforming means for spatially deforming the other high-resolution image data and the high-resolution control image data with respect to the high-resolution first image data. An electronic graphics system according to dependent claim 9. (11) Low-resolution control image data and low-resolution first
An electronic graphics system according to any of the preceding claims, wherein image data of is maintained in a control image frame store area and a first image frame store area, respectively. 12. The electronic graphics system of claim 11 when dependent on claim 8, wherein the low resolution second image data is maintained in a second image frame store area. (13) a first storage means having a capacity for storing first picture element data defining a high resolution image; and second picture element data obtained from the first picture element data; An electronic graphics system comprising: down-conversion means for defining a lower resolution control image; and second storage means having a capacity for storing said second pixel data; control input means for providing third pixel data to be used, and in response to said third pixel data, obtaining a series of experimental image compositions comprising at least pixel data obtained from said second storage means; user operable means for displaying said trial composition when said trial composition is obtained; and means for obtaining fourth picture element data from said third picture element data to display said control image in said control image. up-conversion means for defining a high resolution; and at least pixel data obtained from the first storage means in response to the fourth pixel data; an image composition means for obtaining an image composition corresponding to the image. (14) A method of electronically processing an initial high-resolution image to produce a modified high-resolution image, wherein data associated with a user-defined low-resolution control image is combined with other image data. controls the combination of an initial image with data specifying a low-resolution display, resulting in data representing a low-resolution combined image for display on a display monitor; and Once the desired change to is achieved by the user, the low resolution control image data is converted to a high resolution representation thereof and controls the combination of said other image data with the initial high resolution image data thereby controlling said changed image data. A method used to produce high resolution images. (15) (a) storage means having a capacity to store at least one set of a large number of primary picture elements representing a high resolution initial image; and (b) a reduced number of said primary picture elements with respect to said primary picture elements; (c) means for obtaining a number of secondary picture elements from an element for representing either a part of said initial image or the whole of said initial image, but with a reduced resolution with respect thereto; (c) said secondary picture elements; means for displaying an image represented by picture elements; (d) varying some of the at least secondary picture elements for defining and representing a selected region of the displayed image; operator interface means for achieving desired changes in the displayed image; (e) said selected area according to the changed secondary picture element so that said desired change is transferred to said initial image; means for varying a primary picture element representing a region of said initial image corresponding to. (16) the initial image is stored as initial high-resolution data, which is modified to display a representation of at least a portion of the image as a monitor of a lower resolution than that of the initial data; and user-specified control data for display. used to effect a change in at least some of the modified data, the change being substantially such that the resulting change to the display of the image will be displayed in substantially real time. This is done in real time and once the desired changes have been made to the displayed image,
An electronic graphics system in which the control data is modified to effect a corresponding change to the initial high resolution data and thus the initial image.