FR2531292A1 - Method and system for video image processing. - Google Patents

Abstract

Method and system for video image processing. Processing system comprising a processor 10 and an image memory 11. The image memory and the processor are controlled by the addressing mechanism 12. An image element which arrives is processed with information recorded beforehand and the proportion of processed and restored information is controlled in order to forestall errors in the density of image-information, owing to the fact that a memory location can be selected a number of times in the course of a frame period. The addressing device 12 can produce the desired item of information from address information which exists only at certain memory locations and which is updated only for a duration greater than a frame period. The device comprises spatial and temporal address interpolators in order to perform this operation.

Description

 A video image processing system for, inter alia, producing special effects on television is provided.

 In known systems used for special effects, video information, for example, is received by an image memory acting as a buffer and, in order to produce the desired special effect, an output processor receives, in a typical case, data coming from addresses selected inside the image memory, to reconstruct an image whose shape or dimensions are different from that of the image introduced into the memory.

 While systems of this kind operate in a limited range of special effects, as they are currently available, their mode of operation impedes their flexibility.

 One of the aims of the invention is to provide a system capable of giving greater flexibility in the manipulation of images - while maintaining their quality, so that the resulting image is not significantly degraded.

The subject of the invention is a video processing system for manipulating the shape of the images, characterized by
- frame recording means for receiving information on the image points at several locations equivalent to a video frame
- addressing means for accessing selected locations of the image memory several times within a frame period, depending on the manipulation required
processing means for processing the image information at any given location each time this location is selected,
- And control means used to vary the processing provided as a function of the density of the image information at a given image location, within a frame period.

According to another aspect, the subject of the invention is an addressing device for an image memory processing system, characterized by
means of determining only addresses selected from the desired addresses of the image memory to which access is requested
means for updating the addresses selected from the desired addresses, at a slower rate than the normal frame rate
- And address interpolation means to calculate all the desired addresses to which one must have access from the data available on the addresses in spatial and temporal modes, so that -all the addresses to which one must have access are available, updated, at normal frame rates.

The invention will now be described, with reference to the accompanying drawings, in which
Fig. 1 shows a basic embodiment of the system which relates to an aspect of the present invention
Fig. 2 shows different aspects of image manipulation
Fig. 3 shows in more detail the aspects of the system of FIG. 1 relating to processing
Fig. 4 shows further processing details
Fig. 5 shows another aspect of image manipulation
Fig. 6 shows a system capable of providing appropriate address manipulation
Fig. 7 shows address interpolation techniques for the system corresponding to a second aspect of the present invention
Fig. 8 shows a configuration for address interpolation, used to process spatial manipulation ;;
Fig. 9 shows an arrangement for spatial interpolation and temporal interpolation of addresses.

 Fig. 1 shows a basic embodiment corresponding to an aspect of the invention. A processor 10 receives the image information which arrives, at the same time as the information previously recorded in the image memory 11.

 The address x, y which has been accessed from the image memory, and designating a given point of the image, is determined by the output of the addressing device 12. In addition, a control parameter Z is supplied by the addressing device to control the processing inside the processor 10.

 The assembly shown departs radically from known systems in that, in reality, the addressing device defines "the shape" of the image to be produced by a process which establishes an image by accessing certain locations of the memory d image more than once within 11 frame period, so it is actually made so that all the points of the original image go somewhere even If their locations differ from their original locations of elements image.

 In addition1 the addressing mechanism produces the de-control parameter so that it is certain that correct fractions of the image points are added to the image memory.

 The processing and addressing device will now be described in more detail. The addressing and recording device is more complicated than it seems at first glance, because we have designed a system with which the points of the image can be considered as part of a grid. of cells and that these image points need not necessarily be assigned to a single cell, but may have parts assigned to a certain number of cells as shown in FIG. 2. Thus, a normal image (without processing), constituted by several image points, would be recorded in successive cells of the image memory, as represented by the image points 1 P2 P3 and P4 of lafig. 2 (a), and the addressing device 12 can be considered to produce a normal addressing sequence, the value of Z being set, typically, equal to 1. Although only the picture elements P1 to P4 are represented, it is clear that, with this operating mode, all the cells of the image memory would contain an image element.

 The representation of fig. 2 (b) shows the change of the points of the image, an image (always of normal dimensions) is moved horizontally and vertically by half a point of image. For simplicity, only P1 and P2 are shown. The picture elements P1 and P2 can now be considered to each have 4 parts allocated to neighboring memory cells.

In practice, the part of a given picture element which is added to a cell is known by the address x and y supplied by the addressing device 12, which is produced so as to produce a main address and a rest, the rest giving the part assigned to a given cell and including the parameter Z.

 Thus, for fig. 2 (b), the image memory 11 and the processor 10 controlled by the addressing device 2 actually receive, initially, the image element P1 and, after processing, partially attribute it to the cells C1 and C2 and to the appropriate cells on the next line, C and Cr. r The picture element P2 is then processed and placed at 02, C3, Cr and Cs. In practice, the process takes into account the information already allocated, originating from the picture element P1, in the cell C2, for example in such a way that this cell location is selected for a read operation followed by the processing dependent on the value of Z supplied, and then the processed output signal is written to memory.

 In practice, when the unsticking takes place without changing the dimensions of the image (zoom), the parts of the adjacent image elements allocated to a cell always add up to give 1 at the end of the processing.

In the case illustrated in fig. 2 (b), four operations of reading, processing and writing will be necessary to produce the desired image information, Z being equal, in this example, to 1/4.

 Fig. 3 shows an embodiment of the processor 10 suitable for providing the necessary manipulation of the basic image.

 A multiplier 15 receives the information that arrives and, after having multiplied it, it transmits it to an adder 16. The value of Z is always chosen between 0 and 1.

 In the example of fig. 2 (a) above, as already explained, the value of Z can be considered equal to 1 and, in the arrangement of FIG. 2 (b), it will be 1/4, since three-quarters of the adjacent picture elements are used to produce the information inside cell C2. Thus on condition that the content of the memory is reset to zero at the start of each image, the interpolation serving to produce the information for each cell is carried out automatically simply by the succession of the operations of reading-processing-recording.

 However, this mechanism is only suitable for shifting the image as it is. If there is a modification of the dimensions of the image, additional manipulation is necessary to avoid errors in the constitution of the image, as will now be explained with the aid of FIG. 2 (c). A halving of the dimensions of the image is represented there (without offset), so that the image element P1 is inside the cell C1 and that P2 is partly at l inside cells C1 and C2. Similarly, P3 is entirely inside cell C2 and P4 is partially inside C2 and C. If we did not now adapt the quantity of data, the information which would result inside cell C2, for example, would be double what we want (for example with regard to the intensity of the image). It is therefore necessary to provide the means to adapt to this situation, and FIG. 2 (c) shows that the adaptation coefficient (K) must be 1/2 to provide the desired adaptation to each cell.

 Thus, although this is a simple case, the general rule is really consistent with this explanation of the fact that K is the reduction ratio. Reduction ratios of 32 have been successfully achieved. Although K can be considered fixed for a given reduction (eg E 2 3 for reduction in a ratio of 3), the reduction need not be the same for the entire image and, in fact, the variable reduction gives rise to a large number of special effects which can be obtained with the present system.

Although the mechanism of the following operations
(reading - adding a new image point
manipulated - recording) in a single cycle may initially seem simple, it is, in fact, a very powerful tool which performs interpolation and filtering in a single operation or a series of operations without '' you need more sophisticated devices.

 Considering fig. 2 (d), it can be seen that the reduction and the offset can be carried out by shifting the addressing of a half picture element (for simplicity, only P1 and P2 are shown).

 Although fig. 3 shows the basic mechanism in question, it has been somewhat simplified, and FIG.

4 shows a more complete system intended to provide the desired treatment; we can see how the fractional part of the address and the density compensation mentioned above are used. There is now a multiplier 20 to give the density compensation.

 As shown in fig. 2 (b), a picture element can be manipulated in such a way that four portions are allocated to each of four neighboring cells.

In order to be able to carry out this manipulation at reasonable speeds, it is necessary, in a typical case, to add additional processing and memory to the mechanism of FIG. 3, where the four points should be calculated sequentially during an input picture element. The multiplier, the adder and the image memory separate from FIG. 3 have been replaced psr four image memories 34 to 37, each provided with its multipliers 30 to 33 and its adders 16 to 19 associated, respectively. The output signals of the image memories are received by the summing stage 38 to produce the combined output signal. Such a system allows the picture element which happens to be available for each of the four cells to record that it concerns and for their corresponding treatment. The addressing mechanism 12 is now represented as predicting the main address x, y for the four respective image memories (A, B, G and D) and, moreover, the fractional part (Z) of the address (a, b, c and d) and the density compensation coefficient (K). The main address for the image element P1, for any of the situations represented, will be A = CL, = C2, C = C and D = Cr. The fractions vary. So, for the.

fig. 2 (a), a = 1 and b, c and d = O, K = 1.

 For fig. 2 (b), when it comes to P1, the fractional addresses a, b, c and d are equal to 1/4, and K = 1.

Any combination of these portions is always equal to 1 due to the presence of the summation stage 38.

 For the situation in fig. 2 (c), for the picture element P1, we have aa = 1 and b, c and d = O and K = 1/2 for the picture element P2 B, G and D are as before but a and b are equal to 1/2 and c and b are zero with K = 1/2.

 It should be remembered that each of the memories 34 to 37 is a complete image memory. Therefore, when something is saved in the system, the addresses always express four different but adjacent cells, but, when reading what is in the system, the address of each of the image memories is the same. In other words, for processing, you have to access four cells, but, when reading, you need to access only one cell.

 The presence of the multiplier 20 effectively reduces the volume of incoming data, which is necessary when carrying out a reduction since, otherwise, an accumulation of contributions coming from numerous image points in a cell would give too great a density of information.

 For special effects, the compression is not the same for each cell inside the frame.

For clarity, the multiplier 20 has been shown separate from the multipliers 30-53 but, in practice, the multiplier 20 could, in a typical assembly, be incorporated into the dividers -30,31, 32 and 33.
Another special effect will now be described to illustrate the flexibility of the system. Fig. 5 shows an effect equivalent to turning page P.

The mere fact of producing the "forms" that we want allows us to carry out the correct succession of addresses - as well as the correction for establishing the image. Thus, at the edge E of page P, there will be greater accumulation (but it is compensated by the multiplier 20) than in the overlapping part F.

 In practice, it will appear that the flap F is transparent, so that. the image information below is also visible.

 If you want to make the flap opaque, so that the image below is not visible, you can do this thanks to the read-replace-record operation cycle, which can be obtained simply by removing the connection between the output of the image memory 11 and the adder 14, in the configuration of FIG. 3. In a typical case, it is convenient to generate an identification "index" through the addressing device, so that the system knows which area is above the other.

 Fig. 6 shows in more detail how the addressing mechanism 12 can operate for any desired shape.

 The examples above show that the addressing sequence chosen by the user effectively defines the shape and dimensions of the output image and that it also compensates for the information accumulated by providing parameters x, y, z, and E.

 your simple shapes in fig. 2 can easily be produced by the keyboard 21 for the introduction into the computer 20, so as to provide the sequence of locations of desired cell addresses to which one must access, as well as for entering the necessary value of Z and K .

 Thus, it can be made whether or not the grid of the image memory cells is within the limits of the shape of the desired image, when this is the case, these addresses are selected during the handling.

 As a variant, standardized mathematical formulas can be introduced to produce the desired shapes in the calculator 20. In the case of a circle, for example, the normal textbook equation for the circle is introduced, and simply by defining the address of the cell from its center and the radius of the circle, it is then possible to determine whether or not a certain cell address is inside the circle, and this defines the shape. of the resulting image.

The desired shape area is also an indication of the reduction ratio and this can therefore also be calculated to determine the value of K. The reduction value (K) for a given area of the image can be determined, for example, using usual surface calculation techniques (cf. pp. 129-131 of the Hewlett manual
Packard HP 25 from 1975), K being proportional to the area.

 The calculated values of x and y are then passed to disk memory 23, along with the appropriate value of E for this cell and for a given form.

In practice, the disc 23 contains a whole range of shapes, including suites of shapes to allow
the transformation of images from one form to another. A
once the forms have been introduced, the computer 20
is no longer required, and the system only uses the
disk memory 23 as a source of shape data.

Although access to discs is much faster than access to the computer, it is generally not fast enough to cope with video addressing rhythms. In order to overcome this problem, an additional device represented by the address interpolator 24 has been added, which operates in the manner illustrated in FIG. 7. In practice, the disk memory contains only approximate data on the addresses of the cells, as shown by the points K, L, M,
N, K and L typically representing the first and eighth successive addresses of image elements horizontally and K and N the equivalent address points, eight lines below. Points Q, R, S and T are, in a typical case, the equivalent address points eight frames away, so updating the address at this rate can be performed by the disk and the addresses located between the available points are interpolated from them, spatially and temporally, as shown in the figure.

It turned out that this technique does not give any noticeable degradation of the -1 image produced.
Although the computer 20 has been described as providing all the values of x, y, Z and K when only the approximate addresses are supplied to the disk 23, it may be indicated to supply only correspondingly approximated values, for the other parameters ; in this case, they are also interpolated to obtain all the desired information. Alternatively, the parameters can be calculated by following the address interpolation process in the interpolator 24, using the calculation discussed above.

 The address interpolation technique which has been described also works when the disc produces a series of effects and, while the change in the addressing is produced by the disc and updated every eight frames (in this example ), address interpolation produces a gradual change over the eight frames, giving more importance to the neighboring frame than to the distant frame.

 Fig. 8 shows an assembly used to produce the interpolation of spatial addresses. The approximate addresses reach the address delay circuit 41, which gives a delay equivalent to eight address lines. The delayed address is transmitted to the multiplier 42 had the approximate address of the moment is transmitted to the multiplier 43 before the addition in the adder 44. The output signal of the adder is transmitted to another delay circuit 45 whose delay is equivalent to eight addresses of picture points, and this delayed output signal is transmitted to the multiplier 46. The non-delayed output signal from the adder 44 is transmitted to the multiplier 47 before being received by the adder 48, which also receives the multiplier output signal 46.

 In practice, the assembly of FIG. 8 is doubled to give the interpolation- necessary for address x and address y.

 Thus, from fig. 7, when the approximate addresses K, X, M, N are available, any other interpolated address, for example the address W, can be determined from them. The values of k and l vary between 0 and 1, typically in del / 8 steps, when the addresses are calculated. These values, for multipliers, can be conveniently provided by consult tables attacked step by step by the addressing clocks.

 The spatial address interpolator is included in the assembly of fig. 9 used to produce the temporal interpolation. To simplify the explanation, the output signal from the disk memory 23 is shown as being supplied by a memory 23A of first form (containing for example the addresses K, L, M, N of FIG. 7) and a memory 23B of second form (containing the values Q, R, S, T). After the spatial interpolation, the values of the addresses are available for the temporal interpolation using the multipliers 52 and 53. The resulting output signals are available via the adder i The values of t, too , vary between 0 and 1 in steps of 1/8 for this it is convenient to use a table to consult 55. era allows the progressive introduction of any modification of the form of the address between the eight periods of frame.

 The system has been described as having approximate addresses operating approximately eight addresses and eight frames, but this value is not imperative.

In practice, although the system has been described generally with regard to the values of manipulation intensity for video information, when manipulating color data, this system could enter, in a typical case, tripled, so that one part processes the luminance data and the other the chrominance data (for example the color difference)
In the NTSC system, this is coded as Y, I and Q information respectively. This makes it possible to see that, although a relatively large number of picture memories is required, the results obtained justify such a configuration.

 Alternatively, the color can be treated on the basis of red-green-blue.

 Although the system has been described for use with special effects in television broadcasts, it can be used for other types of video systems requiring the manipulation of the image in a completely free form after their release. production.

Claims (19)

 1. Video processing system for manipulating the shape of the images, characterized in that it comprises  frame recording means (11) for receiving information on the image points, in several places equivalent to a video frame  addressing means (12) for retrieving locations for recording frames selected several times during a frame period, depending on the manipulation required  processing means (10) for processing the image information at any given location, each time that location is addressed, and  control means (12) for varying the processing provided as a function of the density of the image information at a given image location in a frame period.  2. System according to claim 1, characterized in that the addressing means-comprise a generator (12, 20) for defining the address locations with a precision exceeding the image element  3. System according to one of claims 1 or 2, characterized in that the processing means mentioned comprise an adder (16) used to add the information recorded previously and coming from the frame recording means mentioned in a proportion of the information on the image point which is determined by the aforementioned control means.  4. System according to claim 3, characterized in that the processing means mentioned comprise a multiplier (15, 20) used to vary the proportion of information transmitted to the adder mentioned by varying a control parameter supplied by the aforementioned control means.  5. System according to any one of claims 1 to 4, characterized in that the mentioned addressing means comprise a memory (23) for storing the address information corresponding to at least one desired form of image, and in that the control means mentioned comprise a memory (24) for storing the control information associated with at least one desired image form.  6. System according to claim 5, characterized in that the aforementioned memories are in a common information medium.  7. System according to any one of claims 1 to 6, characterized in that the aforementioned frame memory means comprise several image memories (34 to 37), the mentioned addressing having simultaneous access to an address location in each of the selected image memories and in that the processing means mentioned comprise several processors (16 to 19) designed to operate simultaneously, the information which arrives at the subject of an image point being processed and recorded in each image memory.  8. System according to claim 7, characterized in that there are provided addition means (38) for receiving each output signal from the mentioned image memories, so as to produce, from these signals, a totalized information on the picture points.  9. System according to any one of claims 1 to 8, characterized in that the mentioned addressing means comprise - means (23) serving to determine only some of the selected addresses mentioned among the addresses of desired image memories that it is necessary to select, - means (23) for updating the desired addresses at a slower rate than the normal frame rate, and - means (24) for address interpolation to calculate all the desired addresses to be selected, from the available address data, in the spatial and temporal modes, so that all of the addresses to be selected are available updated at normal frame rates.  10. System according to claim 9, characterized in that the aforementioned determination means comprise a memory (23) for keeping at least one addressing sequence.  11. System according to claim 10, characterized in that the mentioned updating means are adapted to select different parts of the mentioned memory.  12. System according to claim 11, characterized in that the mentioned memory and the mentioned updating means are provided by a mobile recording medium (23) containing only the aforementioned image memory addresses of the selected frames.  13. System according to any one of claims 10, 11 or 12, characterized in that there are provided production means (20 to 22) for determining desired addressing sequences before reception by the memory mentioned.  14. System according to any one of claims 9 to 13, characterized in that the mentioned means of address interpolation comprise - a first spatial address interpolator (40) to determine all the useful addresses from the information available associated with a first frame, - a second interpolator (40) of spatial addresses for delivering all the useful addresses from the available information associated with the next available frame, and - an interpolator (52 to 54) of temporal addresses for deliver the useful addresses associated with intermediate frames1 from information provided by the first and second abovementioned spatial address interpolators.  15. The system as claimed in claim 14, characterized in that each of the mentioned spatial address interpolators, comprises an arithmetic processor (42 to 44 46 to 48) for variably providing synthesized addresses as a function of their spatial relationship with the information mentioned, then available, 9 additional addresses.  16. The system as claimed in claim 15, characterized in that the mentioned interpolator of temporal addresses comprises an arithmetic processor (52 to 54) for variably providing synthesized information on the addresses, according to their temporal relation with the address information, available at this time, the addresses in a frame being changed during intermediate frame periods, to effect a gradual change from frame to frame.  17. System according to one of claims 15 or 16, characterized in that a generator (55) of arithmetic information variably controls the desired synthesis, on a time basis.  18. Method for processing information on images, characterized in that it comprises the following steps  - reception of information on the image points in several locations equivalent to a video frame,  - addressing selected locations of image memories several times during a given frame period, depending on the manipulation required,  - processing of image information at any given location each time that location is selected, and  variation in the processing performed as a function of the density of the information on the image at a given location of the image, during a frame period.  19. The method of claim 18, characterized in that the addressing operation comprises  the supply of addresses selected only from the desired addresses of image memories which it is necessary to select,  - updating the addresses selected from the desired addresses, at a slower rate than the normal frame rate,  - the calculation of all the desired addresses to be selected, by interpolation of the available addressing information, in spatial and temporal modes, so that all the addresses to be selected are available, updated, at rates of normal frame.