WO1994010796A1

WO1994010796A1 - Process and device for correcting the image orientation of film image sequences

Info

Publication number: WO1994010796A1
Application number: PCT/EP1993/002996
Authority: WO
Inventors: Ulrich Solzbach; Meinrad Rombach
Original assignee: KLINIKUM DER ALBERT-LUDWIGS-UNIVERSITäT FREIBURG
Priority date: 1992-11-02
Filing date: 1993-10-28
Publication date: 1994-05-11
Also published as: DE4236950C1

Abstract

A device (1) is used for processing and reproducing image sequences. A film projected by a film projector (2) is recorded by a positionable video camera (3). This device should allow details recognisable in a film, for example coronary vessel segments, which move for example because of heart muscle activity, to be reproduced in such a way that they seem to stand still to an observer and may thus be better recognised thanks to the integration faculty of the eye. A first individual image is stored by the video camera as reference image and the position of the relevant image details is compared to that of a subsequent image. When position deviations are present, a correction signal is generated which is used for driving motor-driven positioning drives (12, 12a) which correspondingly move the video camera (3) sideways. It is also possible to modify the distance from the film projector (2) in order to amplify sections of the image.

Description

Method and device for correcting the position of film image sequences

The invention relates to a method for processing and displaying image sequences for the medical evaluation of, in particular, angiographic details of a coronary angiography film, a projected film being recorded by a positionable image sensor, for example a video camera, and image position correction being carried out from image to image.

For coronary angiography films, for example, it is known to pick out certain image details from individual images of the projected film by positioning, for example, a video camera arranged in the projection path of a (cinema) film projector and then being able to view and examine them correspondingly larger.

For example, in the case of coronary angiography films with complicated back and forth movements of a coronary vessel segment to be examined, jerky movements of the eye (saccades) are necessary to adequately detect the object in the region of the keenest vision of the eye (fovea centralis). The faster and more uneven the movement of the coronary segment, the less time is spent during which the object gion of the sharpest vision and thus perceived.

As a result of the subsequent movement of the eye, the ability to integrate the eye is severely hampered by the displacement of the image of an object on the retina, for example by the movement of the coronary segment from image to image in the coronary angiography file. Thus, despite an enlarged representation of a certain image area, there is no significant improvement in visual perceptibility.

For image position correction, that is to say for positioning individual images of a film, methods are already known from DE 41 15 385 AI and from GB 2 187 913 A for measuring an image position error and thus the positional displacement of the individual images of a film relative to one another or to a reference position to capture. From DE 37 36 790 AI a method is also already known with which image position errors can be corrected. The aim of all these methods is to bring the individual images into an exactly same position after a position correction has been carried out, and thus to avoid image error.

However, an improvement in the viewing of image details moving within the image is not possible.

The object of the present invention is to provide a method and a device with which the creation of an image sequence can be carried out, in which image details of interest can be recognized much better or perceived visually. There should be the possibility of practically automatically postprocessing an existing film.

The achievement of this object consists in particular in the features of claim 1. In contrast to the correction of image position errors, where the entire individual image is positioned in relation to its outer position, the image detail to be examined in detail is used according to the invention as a reference. The position is corrected in a dimension and in a direction that corresponds to the movement of the image details. This results in a standstill of the image details to be viewed, so that viewed in the sequence, a significantly better evaluation is made possible.

Investigations have shown that when the individual images are in rapid succession, for example with an image display frequency of 25 images per second, image details are perceived much better than in still individual images. The ability to integrate the eye is used here. The human eye is able to perceive structures in a moving image sequence due to the temporal integration of successive images, which are covered by the noise in the individual images. This visual "post-processing" of an image sequence therefore ultimately provides a higher resolution of structures in a moving scene by reducing the quantum noise. However, there are clear limits to the human eye for this visual postprocessing during movements if the object of interest is imaged at different points on the retina during the movement.

This physiological phenomenon can be explained by temporal properties of the eye movements and the signal processing of the eye. When looking around, the eyes move rapidly from one fixation point to the next, known as saccades. The duration of these saccades fluctuates between 10 milliseconds and approximately 80 milliseconds. Longer fixation periods of about 0.15 to about 2 seconds occur between the saccades. The length of these fixation periods depends on the size of the fixed object, the lighting conditions, the structure in the vicinity of the object and the current position of attention. This is during these fixation periods Eye, especially in the region of sharpest vision (fovea central), is able to integrate the image information projected onto the retina in time. The integration time can be up to one second for stationary objects. Therefore, morphological, for example angiographic details, which are mapped onto the fovea central during the entire fixation period, show a smaller relative noise level in the chronological sequence of the cinema film and are therefore better perceived by the viewer when the images are in rapid succession than in the still frame.

For example, with an image display frequency of 25 images per second and a fixation period of 0.15 seconds, the eye integrates over 0.15 x 25 = 3 to 4 images. As a result, the relative noise level is reduced by 40% to 50%.

In the method according to the invention, the individual images can be captured electronically by an image recording device, for example by a video camera, and digitized on a downstream image computer. The image-to-image displacements of a selected image area are then calculated with the aid of digital image processing. On the basis of this calculated displacement, for example of a vessel segment, this detail of the detail can be centered on one location, so that the detail of the detail can then be reproduced while the background is moving. This means that even small image details can be easily recognized and assessed by utilizing the eye's ability to integrate.

According to one embodiment of the invention, the respective correction signal obtained by the calculated shift of the individual image details of interest can be used to shift the, for example digitized or digital, image signals which differ from one another in the selected image area. The result is then an image sequence with individual images, in which the image detail to be viewed stands still for the viewer because of the position correction from image to image.

According to another embodiment it is provided that the correction signal is used for the mechanical tracking of the image sensor.

In this case, the position correction signals can be used for automatic tracking, for example a video camera. The time required to create an image sequence can be significantly reduced and simplified compared to manual adjustment.

It is particularly advantageous if the selected image area is reproduced in a position-corrected form in an image format enlarged up to 10 times.

This means that even very small details can be easily recognized and assessed. Such details can be, for example, stenosis surfaces, dissections or thrombi in the vessel. In particular, the high resolution of the cinema film can be optimally used by the enlargement.

For a longer and precise observation of image details, it is provided that the position-corrected image areas are preferably reproduced in a loop pass. The image sequence can thus be viewed continuously for practically any length of time. This is particularly advantageous if only a few individual images are available. The position of the selectable image areas can be detected, for example, by image analysis methods, for example pattern recognition algorithms, within the overall image and compared with the position of this image area or the image detail of another single image contained therein. With such image analysis methods, the identification of previously marked image details can be carried out easily. In the device according to the invention for carrying out the method, it is provided that the image recording device is followed by a memory for storing the image signals of at least two individual images and a comparison device for pattern or marking recognition of selectable image areas or markings located within the individual images, that the comparison device either has a control connection to the memory for the transmission of correction signals formed in the comparison device or to motorized positioning drives of the image recording device.

With comparatively little effort, this enables the automatic processing of film sequences with position correction of image details.

It is expediently provided that the image recording device is a video camera and that it has an interchangeable or zoom lens. It is therefore possible to fall back on commercially available assemblies through which the total outlay can be kept to a minimum. By using interchangeable or zoom lenses, an exact adaptation to the respective circumstances with regard to the available film is possible.

The invention is explained in more detail below with its essential details using the drawing.

It shows in a more schematic form:

Fig. 1 is a block function diagram of an inventive

Device with mechanically positionable video camera and

2 shows a block function diagram with electronic image position correction. The device 1 according to FIG. 1 is used for processing and displaying image sequences, the images supplied by a film projector 2 being processed in such a way that image details contained in certain selectable image areas, which move when the film is played back, are reproduced after processing can be. As a result, the ability of the human eye to integrate can be exploited and even small image details can still be easily recognized and evaluated. In angiography in particular, small angiographic details such as stenosis surfaces, dissections or thrombi in the vessel can still be easily recognized and assessed. The film projector 2 delivers image data from a first single image to a video camera 3. This is done by arranging the video camera 3 in the projection path of the film projector 2.

The video camera 3 is connected to an analog / digital converter 5 belonging to a computer 4 via a line 22, which feeds the digitized image data of a first individual image to a marking device 6. The region of interest or the corresponding image detail is marked in this marking device 6 and this marked image is stored in a reference buffer 7. The marking of the image details can be checked on a screen 8. If the image data of the first frame is stored in the buffer 7, a control signal is sent to the controller 9 and the film projector 2 is switched to a next frame via a control data connection 17. The image data of this next individual image then arrive again via the video camera 3 and the analog / digital converter 5 either to the marking device 6 or else directly to a comparison device 10. If the image data of the second individual image is fed to the marking device 6, then the Marking of the image details of interest stored from the previous image are adopted. The image data contained in the reference image buffer 7 are then compared with those of the next one in the comparison device 10 Frame compared. If there are deviations with regard to the position of the marked image detail, a correction signal is output at the output 11 of the comparison device 10. It is also possible for the image data of the second individual image to be fed directly from the analog / digital converter 5 to the comparison device 10 and to be compared with the reference image data. In this case, the computing effort is greater.

To analyze the position of the selectable image areas and to compare images for positional deviations of certain image details, image analysis methods, for example pattern recognition algorithms, are used.

The output 11 of the comparison device 10 is connected to motor-driven positioning drives 12, which in turn are mechanically connected to the video camera 3. The video camera can be positioned in a plane lying transversely to the projection direction of the film projector 2. If the position correction signal is now supplied by the comparison device 10, the position of the video camera is adjusted accordingly until the marked image detail of the second individual image is positioned congruently with the image detail of the first reference image. A position feedback 13 is also connected to the video camera, which in turn is connected to the comparison device 10, so that a position control loop is formed here. If necessary, the selected image area, which contains the image detail of interest, can be enlarged in a position-corrected form, which in the simplest case can be done by changing the distance between the video camera 3 and the film projector 2 or by adjusting a zoom object. For this purpose, a corresponding positioning drive 12a is connected to an enlarging device 14 for controlling this drive. The video camera 3 is also connected via the line 23 to a video recorder 15, on which the image data of the first image and then the image data of the position-corrected second individual image have been stored. After the second image has been stored, the control 9 is released via the control data connection 16 in order to switch the film projector 2 to a next individual image via the connection 17. The process as described for the second individual image is then repeated, in which case either the first individual image remains in the reference image buffer 7 and also serves as a reference image for the further individual images, or it is preferably provided that the reference image buffer 7 is overwritten by the next picture. This means that for a third frame, the second frame serves as a reference frame. For example, a hundred individual images can be processed for an image sequence and the position can be corrected with respect to the marked image details. Apart from the marking of the image detail of interest required at the beginning of the processing, no manual intervention is necessary, so that the processing of the film section proceeds automatically frame by frame.

The result is an image sequence consisting, for example, of a hundred individual images, in which the movement structures were practically reversed. This means that the image detail of interest, for example a coronary vessel segment, stands still for the viewer and the otherwise essentially stationary background moves. The eye can thus concentrate on the still image detail, making use of the eye's ability to integrate.

FIG. 2 shows a device 1 a, which works with an electronic image position correction without mechanical positioning. In this embodiment, the film projector 2 is followed by an image sensor 18, which supplies image data in digitized form to an image memory 19 belonging to a computer 4a. finished. When the image data of a single image is stored, there is the possibility of recording each individual image in digitized partial images and, if necessary, composing these partial images internally to form an overall image with a correspondingly high resolution. This also makes it possible to select the drawing file as a section of the overall picture which contains the picture detail of interest.

In principle, the further processing of the image data contained in the image memory 19 is carried out as has already been described with reference to FIG. 1. Accordingly, a marking device 6, a reference image buffer 7 and a comparison device 10 follow the image memory 19. The correction data formed at the output 11 are supplied as x, y control data to a function block 20 in which the single image to be processed, which is taken over from the image memory 19, is correspondingly shifted in its position until the image detail of interest is congruent with it the same image detail of a reference image stored in the reference image buffer 7. The image data of the image area with the position-corrected image detail are then stored in a section memory 21. This is connected via the control data connection 16 to the controller 9 which, after a position-corrected image or image section has been saved, switches the film projector 2 to a next individual image.

The device 1 a works with electronic image processing and thus enables very high working speeds.

Claims

1. A method for processing and displaying image sequences for the medical evaluation of, in particular, angiographic details of a coronary angiography film, a projected film being recorded by a positionable image sensor, for example a video camera, and image position correction being carried out from image to image, characterized in that that a selectable image area with a moving object from individual images within an image sequence is detected in its position within the respective individual image and compared with the position of this image area within another individual image and by shifting the individual images in accordance with the object movement occurring from image to image within the selected image area is brought to a standstill and that the viewing and evaluation of the selected image area takes place on the basis of a plurality of successively reproduced, position-corrected images or their image areas.

2. The method according to claim 1, characterized in that the selectable image areas are detected by image analysis methods, for example pattern recognition algorithms in their position within the overall image and compared with the position of this image area within another single image.

3. The method according to claim 1 or 2, characterized in that the correction signal for shifting the deviating from each other in the selected image area, for example digitized or digital image signals is used.

4. The method according to any one of claims 1 to 3, characterized in that the correction signal for mechanical

Tracking the image sensor is used.

5. The method according to any one of claims 1 to 4, characterized ge indicates that the selected image area in position-corrected form in up to 10 times enlarged

Image format is reproduced.

6. The method according to any one of claims 1 to 5, characterized ge indicates that the position-corrected image areas are reproduced in an endless repetition of the individual, position-corrected images in a loop pass and that the playback speed is more than 8 images / second, preferably is more than 12.5 frames / second and in particular 25 frames / second.

7. Device for performing the method according to one of claims 1 to 6, for processing and displaying

Image sequences of a film projected by an image projector, an image recording device being positioned in the projection path of the film projector, in particular for medical evaluation of angiographic details of a coronary angiography film, characterized in that the image recording device (3, 18) has a memory for storing the image signals At least two individual images and a comparison device (10) for pattern or mark recognition of selectable image areas or recordable markings located within the individual images are connected downstream, that the comparison device either has a control connection to the memory for transmitting data in the memory Comparison device formed correction signals or motor positioning drives (12, 12a) of the image recording device (3).

8. The device according to claim 7, characterized in that the image recording device is a video camera (3) and that it has an interchangeable or zoom lens.