GB2299881A - Image data manipulation in buffer of controller for bulk storage device - Google Patents

Abstract

A bulk storage controller having a memory for buffering data which has been requested from the bulk storage permits the modification of buffered data prior to onward transmission. The data in the buffer are used to index into a RAM containing a look up table and data from the corresponding addresses indicated in the look up table are written back into the buffer. The data may be image data and the look up table may be a colour cube for introducing colour or textural corrections, possibly with interpolation. Cubic and tetrahedral interpolation schemes are used. The colour cube look up table may be targeted at specific colour ranges eg. to prevent the modification of reserved colours used for "blue screen" techniques.

Description

Data Handling Systems This invention relates to data handling techniques and particularly but not exclusively to handling image data, for example in digital colour processing equipment of the type described in GB-A-2283384, the contents of which are incorporated herein by reference.

In a large number of cases it is required to read an image from disc and modify it in a "one dimensional" manner, that is for example, to create a new 'red' content purely as a function of the existing red image.

Thus data is read from a bulk strorage device, such as a magnetic hard disc or an optical disc, and under the control of a central processing unit (CPU) is routed through a data table stored in RAM (Random Access Memory) which is in the CPU's memory map. This results in modified data, which in itself may probably undergo further modifications. A conventional system uses a disc controller which includes a data buffer and coimmunicates with the remainder of the system via a SCSI or other interface.

According to an invention disclosed herein, there is provided a controller for a bulk storage device for digital data, comprising internal processing means and memory means as integral parts of the controller, the processing means receiving requests from external data processing menas for data to be read from the storage device and controlling retrieval of the data and transmission of data to the external data processing means, and the memory means being configured as a buffer for data read from the storage device prior to transmission to the external data processing means, wherein the memory means is further configured to modify the data read from the storage device in accordance with predetermined criteria and the modified data is transmitted to the external data processing means.

Generally the external data processing means will comprise a central processing unit including a microprocessor, random access memory and other components necessary for the operation of a microcomputer or a custom device such as a digital colour processor of the type disclosed in US Patent 5,450,500. Whilst the external data processing means is distinct form the controller, they may be incorporated in the sme unit as part of an integral system Preferably the memory means of the bulk storage devicehas a first area configured as the buffer and a second area configured as a lookup table containing information for modifying data in the buffer in accordance with the predetermined criteria. Preferably the predetermined criteria are user programmable.In a preferred arrangement the controller comprises part of a system for modifying image data, for example a digital colour processor, the controller providing primary colour correction of the data read from the storage device.

This system has many advantages, the principal one being that is allows two modifications to take place in parallel. The first of these modifications could take place through the lookup tables in the disc controller, under the control of the microprocessor in the disc controller that controls the disc activity, and the second being under control of the main CPU, and either involving the mathematical calculation power of the CPU, or further lookup table processing using Lookup tables on the CPU's RAM map. Preference is made to the accompanying Figure which illustrates one possible embodiment of the system. This shows diagramatically a disc, a controller having a buffer and a lookup table ("bUT"), and a main processing unit including a central processing unit and random access memory.

One important feature is that the incremental work to get this function incorporated into a disc controller is very small. Disc controllers already have a Microprocessor to control activity of the disc, and already have RAM, as it is usually necessary to 'buffer' data that is being written to or read from the controller. All that is lacking is the necessary control software in the microprocessor code for the disc controller.

Currently, disc controllers are complete microprocessor control systems. They incorporate the following elements:1/ A Microprocessor. Disc manufacturers use standard microprocessors, such as the ones used in desktop computers, as these give 'economy of scale'.

2/PROM (Programmable Read only memory). This contains the program information that makes the microprocessor control the disc.

3/ RAM (Random Access Memory) This is the memory that the program is executed in.

4/ External control input. This is an input to the Disc controller Microprocessor, usually along the minicomputer's 'bus', through which it receives data.

The typical flow of information is as follows: The main CPU will request that the Disc controller read, for example, logical blocks 3453 to 3457 from the disc.

This will activate the software running in the disc controller CPU. This will be typically running a 'wait' loop, until it is requested to do something. The Disc controller microprocessor will then check that the requested disc blocks are valid (i.e. that they exist) and probably then check that the disc is running. It will then probably decode the requested logical blocks into physical blocks (i.e. that the requested blocks 3453 to 3457 are actually on platter 3, track 25, records 26-30. It will then wait until the existing transfer is complete, then move the heads to the required track, and actually cause the 'read' to take place, into a buffer in the RAM of the Disc controller's microprocessor.

It will probably check the disc control register, to see if the transfer is complete, then check the 'checksum' to verify that the transfer has (most probably) read valid data. At this point, the Disc controller's microprocessor will signal to the system microprocessor that the transfer is complete and that the data is present & correct in the Disc controller's RAM.

In accordance with an embodiment of the invention, there would be a change to the control or program code in the PROM of the Disc controller. A further logic stage would take place after the checksum verification. The extra code would command the disc controller microprocessor to read the data from the buffer RAM, and use each point of this data to 'index' into another buffer of data in RAM, which contains the lookup table that we are using. The data found at this address is then written back into the original buffer. When this has been done for each point in the buffer, the disc controller then signals to the system controller that the transfer is complete.

A second important feature considered in the context of image data is the utilisation of three dimensional data structures, which can be thought of as 'colour cubes'.

A one dimensional lookup table is one that contains the only input parameter, and one (or one set) output parameter. An example of this may be a conversion from imperial measures to metric. The input parameter may be length in feet, and the output parameter length in meters. Correspondingly, a two dimensional look up table will take in two input parameters, and give one output parameter. An example of this may be the derivation of area from length and width. The two input parameters in this case are obviously length and width, and the derived parameter is the area.

Such lookup table principles can be extended into three or more dimensions. Colour measurement or reproduction requires three parameters. Conversion from one 'colour space' to another is a 'three-in' and 'three-out' transformation, and therefore well suited to a three dimensional lookup table.

One problem with such data tables is the amount of memory required. A sixteen input value one dimensional table takes sixteen memory locations. Correspondingly, a sixteen value two dimensional table takes two hundred and fifty six memory locations, and a sixteen value three dimensional table takes two hundred and fifty six memory locations, and a sixteen value three dimensional takes four thousand and ninety six memory locations.

Many techniques are known to avoid having to have too much memory. One such technique is disclosed in GB-A1369702, which uses a three dimensional interpolation technique. In summary, this technique is to subdivide a 256 x 256 x 256 colour cube into 16 x 16 x 16 'cubelets'. Real data points exist to define the 'cubelets', but interpolation takes place to get intermediate points within a 'cubelet'. This technique is a three dimensional interpolation technique. In the same way as in a one dimensional case we may measure or calibrate an output value for series (usually at regular values) of input values, this can be extended to a two or even three dimensional function. In the case of colour, it is usual to measure colour values as an 8 or 10 bit value. This means a range of 0-255 or 0-1023.

Let us consider the simpler 8-bit example. Because to measure a colour signal, we need to quantify its red, green, and blue values, all of which are independent from each other (meaning that just because we have measured the red and green, we cannot 'calculate' or guess the blue value).

Thus for each of the 256 red values, there are 256 values that the green value could have, with 256 blue values for each red and green values. Thus there are 256 x 256 x 256 possible different colours that we can describe with this system. Each of these input colours can have a different output colour, itself described by a red, green, and blue set of values.

We could envisage this transform table as a 'linear' system, with a table of each value or red from 0 to 255, combined with each value of green fro 0 to 255, combined with each value of blue 0 to 255.

An easier way of envisaging this data is as 'three dimensional cube'. We can imagine the Red, Green and Blue input axes as three mutually orthogonal (i.e. at Right Angles) axes. For a given combination of input red, green, and blue, we describe a unique point a 'pigeon hole' that contains an output colour, specified in output red, green, and blue.

If we can either measure or calibrate output colours for a regular grid of input colours, we can 'build' a grid of regular points within which we can calculate other points. Typically we can measure or calculate, say, input axes in multiples of 16. Thus we could 'load' the cube with values as below: R=0 G=0 B=0 R=0 G=0 B=15 R=0 G=O B=31 R=0 G=0 B=239 R=0 G=15 B=0 R=0 G=15 B=15 R=0 G=15 B=239 R=0 G=31 B=239 R=0 G=31 B=239 R=0 G=47 B=0 R=0 G=47 B=239 R=0 G=239 B=239 R=15 G=0 B=0 R=15 G=0 B=15 R=15 G=0 B=239 R=15 G=15 B=O R=15 G=15 B=15 R=15 G=239 B=239 R=31 0=0 B=0 R=31 G=239 B=239 R=47 G=0 B=0 R=239 G=239 B=239 Note:- In the above example, we have only 16 points, dividing the range up into 15 intervals, and covering the range from 0-239.We could alternatively have 17 points, dividing the range up into 16 intervals and covering the complete range. In practice, the very highest colours are not used in some colour 'scaling' systems, and thus we will continue the example with the range being specified by 15 intervals. It will be seen later that this move simplifies the necessary mathematics considerably.

Thus there is 'preloaded' the cardinal grid of points with 'output' values contained at these cardinal input points.

For any generalised input point, we can 'look' at the most significant four bits of the incoming R, G, and B values, and use could have the input point R=18, G=55, B=81 Thus the 8 bit 'Bit pattern' is as follows: R=00010010 G=00110111 B=01010001 Thus the 'Most significant four' bits for these values are: - R=0001 G=0011 B=0101 Or, in Decimal R=l G=3 B=5 Thus, remembering that Computer numbering starts at zero, we have derived the lower bounding 'cardinal point', as the first value of red, third value of Green, and Fifth value of Red (Which themselves correspond to the Values for R=16, G=48, and B=80) The upper bounding 'cardinal point' corresponds to R=32, G=64 and B=96. These two values, in three dimensional space, form a small cube, and are body centred diagonals of this cube.

All eight of these cube 'corners' or vertices, are as follows:- R=16 G=48 B=80 R=16 G=48 B=96 R=16 G=64 B=80 R=16 G=64 B=96 R=32 G=48 B=80 R=32 G=48 B=96 R=32 G=64 B=80 R=32 G=64 B=96 By the mathematical process of 'trilinear interpolation' we can use the lower four bits of the general input point as 'weighing factors' in the interpolation. In the above example, these are 0010, 1110, 0111, and 0001.

(or 2, 7, and 1) Thus on the red scale, we are 2/16ths of the way from the point at G=48 and the point at G=64, and 1/16ths of the way from the point at B=80 and the pint at B=96. The output values can thus be accordingly 'weighted' to find the output value that corresponds to this input point.

The already known techniques use a 'preloading' phase, where the cardinal points of the cube can be calculated using a mathematical algorithm implemented in software using a general purpose microcomputer.

GB-A-2053619 teaches that each 'cubelet' (defined by eight vertices) can be further broken down into six component tetrahedra sharing a common cube body diagonal axis.

The first step is to decide which cube is relevant, and then second is to decide which of six 'tetrahedra' within that cube one is bounded by. The improvement here is that say we have a point with the same upper four bits, but the lower four bits of the red, green, and blue signals are all very small, say 1, 2, and 2.

The 'upper' bounding corner is thus a 'long way'away fro the 'lower' bounding corner. Thus the bounding tetrahedron has the values of: R=16 G=48 B=80 R=32 G=48 B=80 R=16 G=64 B=80 R=16 G=48 B=96 3-D tetrahedronal interpolation can then be performed to find out an input value for he generalised input point.

It will be noticed that this tetrahedron is defined by four of the cube vertices. The six possible ways of choosing four of the eight points defines six possible tetrahedra.

There are mathematical advantages of interpolating in this technique, and a 'better' output control can be obtained from this technique. One reason for this is that all of the tetrahedra share a common 'body diagonal' since the 'body diagonal' represents the transition from Black through Grey to White. The human eye is particularly sensitive to small variations of colour in the near grey colours. Having such an axis common between the tetrahedra means no mathematical discontinuities in the 'gradient;' of colours near grey.

In the previous 'cubic' techniques, the undue influence of other neighbouring cube vertices could cause a discontinuity.

The cube systems described in the above referenced patents have a general purpose computer to pre-load the three dimensional data structure with alogirthmically calculated values.

Viewed from another inventive aspect there is provided a method of handling image data in which a colour cube of the type described above is preloaded by colour processing means, such as a digital colour processor (DCP) described in GB-A-2283384 with the necessary cardinal vertices for interpolation to take place.

This would enable a much slower (and cheaper) DCP to be built. The Present Standard TV Definition DCP handles data at a rate of 13.5 Million Bytes per second. A High Definition TV implementation would require a data rate of approximately five times this rate. By utilising the DCP engine to load the necessary number of cardinal points into a three dimensional data structure, and by using a 'double buffered' arrangement, where one 'cube' us being loaded whilst the other is being used for colour transform, there would be one twenty-fifth of a second (i.e. 40mS) to calculate only 4096 values. Thus the DCP compute engine only need run several orders of magnitude slower than at present.

In a significant number of real examples there is a requirement to subtly alter a small range of colours, leaving all other colours alone. Conventional 'colour cube' transforms are not at all good for this sort of work, as being interpolative they can 'smooth out' desired changes.

The solution to this is to load such a cubic data structure with a fine range of specific colours. Thus colours that are not within the selected range of colours will not get processed through the cube, whilst the finely selected range of red colours will get processed, but the interpolation will only be over a small range. It may even get to the stage where the data structure is loaded with adjacent colour (i.e.

incremental R,G,B values) in which no interpolation will be required.

Viewed from another inventive aspect there is provided a method of handling image data in which a colour cube is loaded with data corresponding to a relatively small range of colours, colours which are within the range are processed through the cube, and colours which are outside of the range are not processed through the cube.

The importance of this is several fold. Firstly, by the time we get to 1:1 mapping (i.e. no interpolation) then obviously we have no interpolation errors. Secondly, and less obviously, it allows the colour conversion of colours with 'key' signals in. This is a technique where one colour, or a narrow range of colours are used or 'reserved' to indicate 'switching' to another colour or picture. One such widely known technique using this is the so-called 'bluescreen'. In this technique, actors and objects are filmed or recorded against a background of a particular blue. In the video domain this blue is then 'recognised', and another picture source substituted for this blue. In this way, actors can appear to 'fly' across backgrounds. These techniques, although generically referred to as 'blue screen' quite often use colours other than blue.Greens and oranges are sometimes used. The important criteria is to ensure that the 'screen' colour does not occur in the subject. Thus it is important to keep the 'special' colours to the value that they start at, without changing them. A conventional 'cube' type colour correction may well interpolate these colour to a different value, causing the 'blue screen' process not to work! However, with the non-interpolative cube techniques, we can have a lookup table which manipulates a wide range of blues as the operator requires, without having the 'special' blue or limited range of 'special' blues change colour.

The use of interpolation 'cubes' in this case has many advantages over calculation techniques. We have to remember that the data that we wish to 'colour correct' is usually a motion picture sequence, usually arising from film or video sources.

US-A-4096523 teaches the concept of scene by scene programming. By this is meant that with combined with the use of a 'controller' (such as Pandora POGLE), a Colour Corrector (such as the Pandora DCP of GB-A2283384) and a 'picture source' (such as a Telecine) one can associate, or 'program' a colour correction sequence where a given set of optimal or user specified colour relating to a scene in the original.

An example of this is where there is a sequence of three consecutive scenes in a work. The first is shot outside, in bright sunshine. The second scene is shot indoors, and is too dark and 'yellow' due to the colour temperature of the lighting, and the third scene has the same requirements as of the first. We wish to 'program' a correction for each of these scenes so that in a mode where we want to transfer this material to videotape, we quickly reload the colour settings in the vertical blanking interval.

The system set out above has a variable range of interpolation within the cube. There are advantages gained by a 'cube' system even when loaded with a 1:1 interpolation, over a conventional DCP even though both of these may give identical numerical results. This main advantage is that the 'DCP' speed in the 1:1 cube does not need to be anywhere as fast as a conventional DCP. This is because with the cube, we only need to load 4096 points (which is like 'colour correcting' a tiny image that has only 4096 pixels, as opposed to the million to five million pixels in a SD or HD image.

Coupling the 'cube loading' (generally with a 'full range' cube) with the 'scene by scene' programming, a fairly simple computer system with one 'cube' could colour correct video picture streams in 'real time'.

This would be accomplished by the following steps: 1. Before the first frame of the first scene, the CPU runs the DCP 'Software' Implementation, that would load the first 'cube' with the colour correction lookups that correspond to the adaptor chosen and stored parameters for that scene. (Only 4096 'correction points' are needed.) 2. One then transfers the video data (typically 1 million) points per frame) through the lookup tables, to provide corrected data.

3. After the conversion of the last frame of the first scene, one reloads the cube, during the vertical blanking interval, with the new cube data, corresponding to the operator stored settings for scene two.

4. One then continues as in 2.

A further variation of the above, for slower computer architecture, is to have two 'cubes' (be they data structures within a single CPU computer, or a multiple processors on larger computers). This method allows for the creation of a second scenes 'cube' during the transferring of the data for a first scene through its cube. This removes the restriction to be able to recalculate the contents of a cube during the vertical blanking interval.

There is sometimes the requirement to perform several changes of colour within one picture. The already proposed architecture of the Pandora DCP in GB-A2283384 contains a multi-channel system, with a 'priority logic' that arbitrates for a colour that is within the selected colour range of more than one channel. This allows the rapid 'switching' of channels to try out different priority setups.

This effect would be difficult to emulate with the slower DCP 'compute engine' and cube technique, as changing the priority necessitate reloading the cube.

The solution to this is the use of multiple cubes. Each of the 'channels' could load a separate cube. Typical computers used within the Film & Video facilities companies certain multiple processors in order to 'render' graphics at a reasonable speed. One such computer is the Silicon Graphics 'Onyx' computer, which can have up to 20 processors. When such a computer is used for colour manipulation work, either a hardware DCP 'compute engine' could be added, or a software emulation of the algorithms could be used to load cube like data structures. Each of the processors in the Onyx could emulate one channel of the DCP. This would utilise the processing power of such a platform of the 'Onyx', and also enable the rapid switching of priorities of channels.

In GB-A-2283384 there are references to other picture attributes, such as the x and y addresses of the picture, and attributes such as "sharpness", or texture related attributes such as "grassiness" "graveliness", or "woolliness". These can be incorporated in the "cubic" data structures, by extending the dimensionality. Three dimensions are required for a three attribute picture, such as red, green and blue.

Four dimensions would be required for pictures with a "sharpness" attribute per pixel. Mathematically all of the above teaching can be extended into a 4-D space, and even to higher dimensions if required.

One practical example of the above is a country scene containing a number of constituents. For example consider a female model, in a green dress, sitting a field of grass, with a blue sky. It is required to artistically bring out detail in the green dress, and make it less blue. Assume that the attributes of the picture are derived and stored in this case as red, green, blue, sharpness and grassiness. We will need a 5-D low in blue and red, and low in "grassiness", we will make higher in "sharpness". This change will make only the green dress go sharper, and not over-enhance the detail in the grass.

Claims (8)

1. A data processing system comprising a bulk storage device for digital data, a controller for the bulk storage device and in data communication therewith, and main data processing means communicating with the controller by means of an interface, wherein the controller comprises internal processing means and internal memory means as integral parts of the controller, the internal processing means receiving requests from the main data processing means for data to be read from the bulk storage device, and controlling retrieval of the data from the bulk storage device and transmission of data to the main data processing means via the interface, and the internal memory means being configured as a buffer for data read from the bulk storage device prior to transmission to the main data processing means, and wherein the internal memory means is further configured to modify the data read from the bulk storage device in accordance with predetermined modification criteria prior to transmission to the main data processing means.

2. A data processing system as claimed in claim 1, wherein the internal memory means of the controller has a first area configured.as the buffer and a second area configured as a lookup table containing information for modifying data in the buffer in accordance with the predetermined modification criteria.

3. A data processing system as claimed in claim 2 for modifying colour characteristics of image data, wherein the second area of the internal memory means of the controller is configured as a lookup table containing information for providing primary colour correction of data in the buffer.

4. A data processing system as claimed in claim 2 or 3, wherein the contents of the lookup table are user programmable.

5. A data processing system as claimed in claim 3 wherein the main data processing means provides further colour correction of the image data supplied by the controller.

6. A controller for a bulk storage device for digital data, comprising internal processing means and memory means as integral parts of the controller, the processing means receiving requests from external data processing means for data to be read from the storage device and controlling retrieval of the data and transmission of data to the external data processing means, and the memory means being configured as a buffer for data read from the storage device prior to transmission to the external data processing means, wherein the memory means is further configured to modify the data read from the storage device in accordance with predetermined criteria and the modified data is transmitted to the external data processing means.

7. A method of handling image data in which a data structure in the form of a colour cube is preloaded by colour processing means with the cardinal vertices for interpolation to take place.

8. A method of handling image data in which a data structure in the form of a colour cube is loaded with data corresponding to a relatively small range of colours, colours which are within the range are processed through the cube, and colours which are outside of the range are not processed through the cube.