GB2239772A

GB2239772A - Generating images

Info

Publication number: GB2239772A
Application number: GB9022588A
Authority: GB
Inventors: Jaroslav Stark
Original assignee: General Electric Co PLC
Current assignee: General Electric Co PLC
Priority date: 1989-10-26
Filing date: 1990-10-17
Publication date: 1991-07-10
Anticipated expiration: 2010-10-17
Also published as: GB2239772B; GB9022588D0

Abstract

In a process for generating images using the mathematical framework of fractals implemented as a neural network architecture, processing for each pixel (i, j) of the image is carried out by a respective processing element (10, 11). All of the elements operate in parallel, thereby achieving very fast processing. The generated image is determined by the strengths of the connections between the processing elements. The processing elements may be individual elements or may be implemented in a computer. The image to be generated is preferably encoded as an iterated Function System. <IMAGE>

Description

Image Generation and Processing This invention relates to image processing, and particularly to image generation and image comparison using the mathematical framework of fractals.

It has previously been observed by a number of workers, for example B. Mandelbrot, 1982,"The Fractal Geometry of Nature", W. H.

Freeman, that a wide variety of images can be considered as fractals, i. e. a set such that two standard definitions of its dimension give different answers.

This offers the possibility of string, generating and processing images in entirely new ways. A particularly useful class of fractals for such purposes is given by so-called"Iterated Function Systems" (IFS), see, for example, M. F. Barnsley, 1988 "Fractals Everywhere", Academic Press and J. Hutchinson, 1981, "Fractals and Self-Similarity", Indiana U. J. Math., 30,713-747. As an example, Barnsley et al. have demonstrated image compression ratios of up to 104 : 1 using such techniques; see M. F. Barnsley and A. D. Sloan, 1988"A Better Way to Compress Images", Byte, 13, 215-223.

However, such schemes suffer from the disadvantage that prohibitively large amounts of computation are required, either to encode an image as an IFS or, conversely, to generate the image corresponding to a given IFS.

It is an object of the invention to provide an improved method and apparatus for image generation. The invention may also provide improved image comparison.

According to the invention there is provided an image generation process using the mathematical framework of fractals implemented as a neural network architecture, in which processing for each pixel of the image is carried out by a respective processing element, the processing elements operating in parallel; and in which the generated image is determined by the strengths of the connections between the processing elements.

Embodiments of the invention will now be described, by way of example, with reference to the accompanying drawings, in which Figure 1 is a schematic block diagram illustrating the interconnection of pixel processing elements for two pixels of an image, and Figure 2 is a schematic block diagram of a network architecture at one pixel for comparing images.

In image generation it is desired to plot an image corresponding to an IFS on to a computer screen. For simplicity it will be assumed that the screen is made up of square pixels and that the horizontal and vertical resolutions of the screen are identical.

If necessary, generalisation to include rectangular screens and pixels of other aspect ratios can be readily effected.

If N is the resolution of the screen, the pixels can be labelle by the pair (i, j) for 1 L i, j'N. Let pij denote the "intensity"of the (i, j) th pixel. If the display is a"grey-scale" screen, the"intensity"would be the grey level. In the case of a colour display,"intensity"could denote either the entry into a colour table or the intensity of one of the primary colours.

An IFS F is determined by maps {fl,----, fqW and probabilities fp,--, Pq. Here each fa (is a map of the plane to itself and pK is a number where 0d p C1 such that the sum of the US is 1. In Barnsley's original formulation of the theory, each f must contract distances, but this is not necessary in the present invention provided that the IFS corresponds to a unique image, i. e. in mathematical terms it has a unique invariant probability measure.

Given ,--, fq3 and iP1,---~Pq,} the corresponding image is generated iteratively as follows: The process is started with an arbitrary choice of initial intensity pij such that the sum of ij over all pixels is not zero.

By appropriate scaling it is asumed that this sum is equal to 1.

Then, given for an pixels at time t, its value ij (t+1) at time t+1 is calculated by :

The weights #iji'j' are computed from i1,..., fgjand {p1, ..., pq} as follows :

where y (o (, i,j, i',j') is 1 if the map fy sends the pixel ('j') close"to the pixel (i, j) and 0 otherwise. If the maps [f1,....., fql are scaled such that each f# maps the square 6 (x, y): 0#x, y#N into itself, then one possible definition of X(#,i,j,i',j'), is

fo = {1 if f B O otherwise where Bsj is the square {(x,y) : (i-1)#x < i, (j-1)#y < j This implementation therefore corresponds to the top right-hand corner of the pixel (i8, j') being sent anywhere into the square occupied by the pixel (i, j). Many other scalings and choice of rounding are possible. It should be noted that, at most, q of tt weights #iji'j' are non-zero for a given (i, j').

Figure 1 of the drawings illustrates, schematically, columns i and i' and rows j and j'of pixels of the image, and processing elements 10 and 11 used in carrying out the generation process. The functions may be performed by a dedicated simple processing element for each pixel (i, j), all operating in parallel to achieve fast processing, or by a more general computer, operatir in parallel mode. Dedicated processors may use elect, onic, optical or other suitable technology. In the case of a general computer, one processor in the computer would perform the calculations required for one or more pixel processing units.

Each pixel processor must be able to store the value and update it at each time step, in accordance with equation (1) above. The time steps are determined by a clock pulse generator 12 associated with all of the processor elements. The weights % are associated with the"strength"of the connection between the pixel processor (i', j') and the pixel processor (i, j) and need to be calculated only once for each image. This calculation may be effected by the pixel processing elements or by other suitable means. The weights -fjt can be stored by the pixel processor (1'j') or the pixel processor (i, j) or independently of each of those processors. The pixel processor (i'j') must be able to communicate the value j the pixel processor processor (i, j) at each time step, and the value may, if necessary, be multiplied by the weight q j.. As iteration proceeds, the values ij(t) will converge to a limiting value which is the desired image. Iteration is therefore continued until the value ij(t) ceases to change by an appreciable amount, as determined by the desired accuracy. The values are then passed to the display device. If the maps fNare all contracting, it is possible to give a realistic upper limit to the time required for the processing.

As an extension of the invention, computation of the Hutchinson metric (see"Fractals and Self-Similarity"mentioned above) can be effected. The Hutchinson metric gives a natural measure of the discrepancy between two fractal images obtained from Iterated Function Systems. Its main application is in encoding a given image * as an IFS.

This encoding process is basically a problem of optimization to find an IFS F* amongst some set of feasible IFSs F which minimizes some measure A tUF, y*) of the discrepancy between Up and p*, where F denotes the image corresponding to the IFS F.

Since there is no known explicit formula for F* in terms of the only feasible approach is an iterative one. A sequence Fm of IFSs is therefore generated such that rFm converges to a minimum of t. The choice of A is crucial in determining the efficiency of the search process.

In this embodiment of the invention, there are provided a choice ofd and a machine architecture which allows an extremely rapid evaluation ouf. 4. In particular it is assumed that #( F, *) = #H (MF * *) where is the Hutchinson metric and MF is the Markov operator corresponding to the IFS F. If u and Vare lmages on a computer screen, then #H is defined by

where f is a function which assigns a value qij to each pixel (i, j) which satisfies the conditions.

|#ij-#i'j'|#|i - i'| + |j - j'|, for all values where 0 di, j, i,'j' < N (5') and the minimum in equation (5) is taken over all possible functions which satisfy the conditions (5'). The Markov operator MF in equation (4) is defined by

for any image. It should be noted that this corresponds to one iteration of equation (1) which can be carried out rapidly, as described above.

A great advantage of the comparison given by equation (4) is that in order to compare images rif and r* it is not necessary to generate the image YFX but only to carry out one step of the iteration given by equation (1).

It is then necessary to effect efficient calculation of equation (5). This is equivalent to the linear programing problem : minimise

subject to the following constraints: - #i+1,j # 1 for 1#i < N and 1#j#N #ij - #i-1, j #1 for 1#i#N and 1#j < N 'cri, j+1 # 1 for 1#i#N and 1#j < N - #i,j-1 # 1 for 1#i#N and Kj N This can be solved using a neural net approach. One possible architecture is derived from a general algorithm of Tank and Hopfield (see D. W. Tank and J. J. Hopfield, 1986,"Simple Neural Optimization Networks; an A/D Converter, Signal Decision Circuit and a Linear Programming Circuit", IEEE Trans. Cir. Sys. 33,87-95.

A suitable analogue network for effecting the calculation is shown in Figure 2 of the drawings. This comprises, for each pixel, a linear amplifier 21 for the variable and a non-liner amplifier 22-25 for each of the above restraints, referred to as gjj (i+1, j), Bij(i-1,j), Bij(i,j+1),Bij(i,j,+1) and Bij)i,j-1), respectively.

The output of the amplifier 21 is,, and the outputs of the amplifiers 22-25 are denoted as Bij j'), The input to the amplifier 21 is a current zip of magnitude - ( ij - vij) - ( ij (i+1,j) + ij(i-1,j) + ij(i,j+1) + ij(i,j-1) + i+1,j(i,j) + i-1,j(i,j) + i,j+1(i,j) + i,j-1(i,j) for 1 < i, j < N, with appropriate modification for i, j s {1, N3 This is integrated by an RC circuit 26 to give C dX = + dt R The output is given byj = KIj for some constant K. The input to each of the amplifiers 22-25 is a measure of the degree to which the constraint is satisfied. Hence, for example, the input Jij(i+1,j~ to the amplifier 22 is Jij(i+1,j) = (#i+1,j - #ij + 1).

The amplifiers 22-25 are assumed to have a much faster response than the amplifier 21, and the outputs from the amplifiers 22-25 are therefore simply a function g (Jij (i',j')) of the input. Tank and Hopfield take g (z) =O, zO, g (z) =-K'z, z < O, where g is the output of the amplifiers 22-25 and z is the input to the amplifiers.

However, given the nature of semiconductor devices, it would be more appropriate to take g (z) =-K'expz, zZ O. This network will minimize the genergy"function.

where G is the integral of g, so G (z) =O, zozo, G (z) =iK'zZ, 2 < 0 With suitable choices of K, K'and R, this gives an arbitrarily accurate approximation to H (, ).

The term ( ij - vij) is fed, as a fixed input, to an inverter 27, together with the outputs of the amplifiers 22-25. The output of the amplifier 21 is fed to an inverter 28, which produces the output-% for feeding to the circuitry for the neighbouring pixel.

The linear programming problem could alternatively be solved using an implementation of any other neural network architecture for linear optimization. Furthermore, the architecture could be embodied directly as electronic or other circuitry or could be simulated on a digital or analog computer. As all of the connections in the network described above are"nearest neighbour", the architecture could be simulated efficiently on an SIMD processor, such as the DAP processor of ICL.

Claims

Cl ai ms 1. An image generation process using the mathematical framework of fractals implemented as a neural network architecture, in which processing for each pixel of the image is carried out by a respective processing element, the processing elements operating in parallel; and in which the generated image is determined by the strengths of the connections between the processing elements.
2. A process as claimed in Claim 1, wherein the processing elements are implemented in a computer.
3. A process as claimed in Claim 1 or Claim 2, wherein each processing element stores an intensity value for the respective pixel and updates said value at each of a number of successive time steps.
4. A process as claimed in Claim 3, wherein each intensity value corresponds to a grey level of an image to be generated.
5. A process as claimed in Claim 3, wherein each intensity value corresponds to the intensity of one of the primary colours of a colour image to be generated.
6. A process as claimed in any preceding claim, wherein the strengths of the connections for all processing elements are calculated once for each different image.
7. A process as claimed in Claim 6, wherein the calculation of the strengths is effected by the processing elements.
8. A process as claimed in Claim 6, wherein the calculation of the strengths is effected by means external to said processing elements.
9. A process as claimed in any preceding claim, wherein an image to be generated is encoded as an Iterated Function System (IFS).
10. A process as claimed in Claim 9, wherein the encoding is effected by iteratively minimising the difference between the intensities for two fractal images.
11. A process as claimed in Claim 9 or Claim 10, wherein a first image corresponding to an IFS is compared to a predetermined second image without generating said first image.
12. A process as claimed in Claim 10 or Claim 11, wherein the minimising is effected by a neural net.
13. A process as claimed in Claim 10 or Claim 11, wherein the minimising is effected by a computer.
14. A process for generating an image, substantially as hereinbefore described with reference to the accompanying drawings.