DE102011079067A1 - Method for equalizing image of high-speed camera of endoscopic inspection device for e.g. inspecting hole-like openings in intestine, involves determining equalization of image in reference to scan-axis of testing body - Google Patents

Abstract

The method involves detecting a complete image of a testing surface (21) that lies inside a testing body (20), where the complete image of the testing surface consists of individual image points of an image stack in an image post-processing process. The image points are provided in an annular focal point region (40) of the complete image. Equalization of the image is determined in reference to a scan-axis (30) of the testing body. An actual position at the testing body is determined, and the axis of the testing body is derived from a CAD-model of the testing body. An independent claim is also included for a device for equalization of an image of an image detection system of an endoscopic inspection device for circumferential visibility.

Description

The invention relates to a method for equalizing an image of an image acquisition system of an endoscopic inspection device for all-round visibility, wherein the image acquisition system captures an overall image of a test surface inside a test specimen and in an image postprocessing process an overall image of the test surface is assembled line by line from individual image points of an image stack are located in an annular focus area of the overall picture.

The invention further relates to a corresponding device for carrying out the method.

State of the art

Endoscopically based inspection systems, with which cavities can be imaged, are u. a. from medical technology for the exploration of intestine, stomach, blood vessels, etc. and from the workpiece quality control known.

For example, in the patent DE 10 2004 026 004 B4 an endoscope with a central working channel having endoscope shaft described at the distal end portion of an endoscope head is formed, which is traversed by the working channel, around which an optical composite, consisting of a number of Einzeleloptiken, arranged on a circular path. These each have an opening angle of greater than 90 ° and are directed with respect to the longitudinal axis of the endoscope at an angle to the outside, that their respective fields of view extend beyond the perpendicular to the endoscope longitudinal axis to the rear and beyond the longitudinal axis forward, resulting in advance of the endoscope head results in a common overlap region of all fields of view.

The publication DE 10 2006 019 124 A1 discloses an omnidirectional imaging system comprising at least one camera unit of line-shaped optical channels having a microlens and a detector located in its focal plane extracting a pixel from the microimage behind the microlens, the optical axes of the individual optical channels having different inclinations, so that they represent a function of the distance of the optical channel from the center of the side facing the image of the camera unit, whereby the ratio of the size of the field of view to the image field size can be determined specifically. It is provided that the camera unit is mounted on a rotating or rotation-oscillating axis of rotation, whereby an all-round view of the image acquisition system is made possible.

In existing on the market systems that, like the o. G. Examples show that different principles work to scan a component with a 360 ° interior view. In the methods under consideration, an all-round endoscope is inserted into the component. The endoscope images the inner wall of the component on a sensor.

The result is an image on a 2D sensor surface, which maps the 90 ° normal of the component within an annular circle. A narrow line in the detection plane is in focus. To achieve a complete image of the component, the endoscope is moved further in the axial direction. In this case, the next recording plane of the component is now in focus at the same location of the image sensor. Merging the image lines now allows creation of an unrolled image of the interior surface of the component which can be inspected.

In principle, the technical methods for such all-round endoscopes can be divided into two groups. For the most part, a special ring CCD sensor is used in order to be able to read out exactly this ring surface at maximum speed. Alternatively, the use of a high-speed camera is common, with the entire sensor surface is exposed. In postprocessing, it is possible to read out the pixels in the focused annular region. These pixels are unrolled and form a line in the receptacle of the inner surface of the specimen. Due to the fast image recording of the high-speed camera, a comparable fast scanning of the surface is possible. This method focuses on the following invention.

A disadvantage, however, is that the orientation of the scan axis and the component axis must be extremely precisely aligned in order to prevent artifacts in the unrolled image.

It is therefore an object of the invention to provide a method and a device with which in particular errors in the alignment of the two axes can be compensated for each other.

Disclosure of the invention

The object of the method is solved by the features of claims 1 to 6.

The object of the invention relating to the device provides for the implementation of the method an image acquisition system in which an image equalization is implemented by software with which during or at the end of a scan process, the current Position in the test specimen along a scan axis determinable and more distant distinctive component geometries are evaluated, and from the position of the scan axis to the component axis can be determined and equalization parameters for correcting position and angle errors can be derived.

The method according to the invention provides that the equalization of the image is determined from the reference of a scan axis to the axis of the test body, wherein in a preferred process variant it may be provided that the equalization of the image during the recording during immersion of the image acquisition system along a scan Axis in the specimen or after recording the image stack is carried out, on the one hand determines the current position in the specimen and on the image further removed distinctive component geometries, the position of the scan axis to the component axis is determined and from this equalization parameters are derived.

With the method and the corresponding device according to the invention artifacts due to position and / or angle errors in the all-round inspection can be minimized and the image quality of the image can be improved, since the effective information content of the resulting image is higher. The image is rectified on-the-fly, d. H. still during the recording, wherein the equalization software-based is preferably implemented in a high-speed camera as an image capture system.

In another variant of the method, the axis of the test specimen can be derived from a CAD model of the test specimen. This offers the advantage that the component axis is already predetermined and does not have to be recalculated.

In conventional solutions, the equalization in the result image is required. The not yet rectified image has already been created by interpolating the pixels of the 2D CCD sensor. The required subsequent equalization again interpolates the pixels, which results in a further loss of information, i. H. Sharpness is connected. This is improved by the method according to the invention.

Correcting angular errors and / or positional errors during the image post-processing operation on the basis of the equalization parameters results in a further advantage over the prior art since an automatic correction of the axial position of the recording system relative to the component can be carried out. As a result, the optimum focus position of the sensor system can be adapted, whereby readout of the image information can always take place in the correct focus range.

In a further variant of the method, the equalization parameters are redetermined for each image step or at specific intervals along the scan axis. By this tracking the image quality can be further improved. In addition, the information content of the result can be further increased.

In many cases it may be advantageous if, at the beginning of the scanning process, component edges of the test specimen, which are arranged along the scan axis at the end of the scanning process, are detected and evaluated as distinctive component geometries. For example, the radii of the bottoms of blind holes or the Bohrradien can be determined in holes with high precision, as they emerge as a sharp edge. When the scan is performed from the opposite side, pixels in the upper and lower levels of the image are determinable. This makes it possible to calculate the coefficients for the correction parameters for the position and / or angle correction before the scan and to apply them directly during the scan.

In a further advantageous variant of the method it is provided that a shading correction is implemented during the image equalization, which can be realized without additional effort. The Shading Correction corrects for unevenly lit images and allows for better stitching, d. H. Tying together the single lines, as lines that are closer to the light source are displayed darker and brighter lines farther away. In a scan that reads more than one row of pixels for speed reasons, the result image will have the rows too light next to the rows that are too dark, creating a raster effect that would make it difficult to detect defects in the surface being examined ,

In addition, it may be provided that a pixel image is calculated with virtual pixels to improve the image quality from the captured with the image acquisition system pixels, the virtual pixels are calculated by means of a weighting matrix of the pixels, and on the basis of this pixel image with the virtual pixels the Entzerrungsparameter be determined , In particular, a higher image sharpness can be realized by this interpolation.

The method described above with its variants can be particularly advantageously used within endoscopy in medical technology, e.g. for examining body cavities, for inspecting hole-like recesses or ducts in the field of mechanical engineering or vehicle engineering, where high precision and fault-free execution are required, or for 360 ° image acquisition in rooms.

Brief description of the drawings

The invention will be explained in more detail below with reference to an embodiment shown in FIGS. Show it:

1 in a schematic representation of an inspection device,

2 an object shot,

3a and 3b Aberrations due to position and angle errors,

4a another object shot,

4b and 4c one unrolled image of the object shot in each case 4a and

5a and 5b each a schematic representation of a pixel image in the original and an optimized pixel image with calculated virtual pixels.

1 schematically shows an inspection device 1 , at the tip of an endoscope 10 an image capture system is arranged in the form of a high-speed camera, which runs along a scan axis 30 into a test piece 20 for detecting an internal test surface 21 is introduced. The scan axis 30 should ideally be with the component axis 22 of the test piece 20 to match. The image acquisition system can be designed as a high-speed CCD camera and thereby captures the test surface 21 in a 360 ° all-round view, taking a sharp picture of the test surface 21 in an annular focus area 40 within the detection level 50 he follows. By moving the image capture system along the scan axis 30 can the test surface 21 be scanned line by line.

2 shows an example of the image of a panoramic view endoscope, in which the inside of a hole in a test specimen 20 is shown. The bright ring marks the focus area in this shot 40 , which in this case is less than 90 ° to the scan axis 30 sharply displayed and evaluated for inspection. In this case, the image pixels in the focused annular region are read out in a post-processing process and joined together line by line to form an unrolled image. This process is called stitching. It should be noted that the method is not necessarily fixed in a 90 ° viewing direction. However, this is the most common use case.

As a lack of previous systems, it turned out that the orientation of the scan axis 30 and the component axis 22 had to be extremely precisely aligned to avoid disturbing artifacts in the image analysis. Typical aberrations 70 result from position errors 71 and / or angle error 72 where the scan axis 30 to the component axis 22 moved laterally and / or tilted to each other, as in 3a and 3b is shown schematically.

The method according to the invention provides for the correction of these errors, in particular, to carry out an equalization of the image on-the-fly, that is still during the recording. This is done in a preferred embodiment by means of special software within the image acquisition system, ie within the high-speed camera. It is provided that the equalization of the image during recording when immersing the image acquisition system along the scan axis 30 in the test specimen 20 takes place, on the one hand, the current position in the specimen 20 determines the position of the scan axis by means of the prominent component geometries further away 30 to the component axis 22 is determined and from this equalization parameters are derived. As a result, the optimum focus position of the sensor system can be adapted, whereby a readout of the image information always in the correct focus area 40 can be done. In this case, it can be provided that at the beginning of the scanning process component edges of the test specimen, which are along the scan axis, are detected and evaluated as distinctive component geometries 30 are arranged at the end of the scanning process. This makes it possible to calculate the coefficients for the correction parameters for the position and / or angle correction before the scan and to apply them directly during the scan.

In addition, a shading correction can be implemented in the equalization, in which the object image with a reference image is set in relation to each other pixel by pixel to a ratio image. The pixel-by-pixel ratio formation is determined on the basis of an experimentally or computationally determined table that corresponds to a photosensitivity characteristic of the image sensors. The ratio image enhances contrast and is much more independent of surface properties such as viewing angle, color, brightness, surface roughness and slope or texture, allowing for unambiguous detection of surface defects.

4a shows an object shot 60 with a 360 ° all-round optics. In 4b is the object shot 60 in a post-processing process to an unrolled image 61 has been processed, resulting in an image error due to an incorrect axis position. 4c shows as a result of the method according to the invention, the unrolled image 61 in which a correction of the axis position took place during the recording.

To improve the image quality from the captured with the image acquisition system pixels, as a schematic view 80 in 5a and 5b shows a pixel image 81 with virtual pixels 83 calculated, the virtual pixels 83 by means of a weighting matrix 84 from the pixels (original pixels 82 ) and based on this pixel image 81 with the virtual pixels 83 ( 5b ) the equalization parameters are determined. Different image radii and positions can be sharper, with the resulting resulting image being the same number of virtual pixels 83 has like the pixel image 81 with original pixels 82 ( 5a ). It should be noted that the scanning with the virtual pixels 83 not by the number of available original pixels 82 must orient. Under- or oversampling is directly possible with the method. If a read is made of more than one pixel ring width, such interpolation is mandatory because the number of smaller radius peripheral pixels becomes less and less as a result.

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• DE 102004026004 B4 [0004]
• DE 102006019124 A1 [0005]

Claims (11)

Method for equalizing an image of an image acquisition system of an endoscopic inspection device ( 1 ) to the all-round view, wherein with the image acquisition system an overall picture of a in a test specimen ( 20 ) inside test surface ( 21 ) and in an image post-processing operation, an overall image of the test area ( 21 ) is composed line by line from individual pixels of an image stack, which are in an annular focus area ( 40 ) of the overall image, characterized in that the equalization of the image from the reference of a scan axis ( 30 ) to the axis of the specimen ( 20 ) is determined. A method according to claim 1, characterized in that the equalization of the image during the recording during immersion of the image acquisition system along the scan axis ( 30 ) in the test specimen ( 20 ) or after recording the image stack, on the one hand the current position in the test body ( 20 ) determines the location of the scan axis ( 30 ) is determined to the component axis and from this equalization parameters are derived. Method according to claim 1, characterized in that the axis of the test body ( 20 ) from a CAD model of the test specimen ( 20 ) is derived. Method according to one of claims 1 to 3, characterized in that the equalization parameters for each image step or at specific intervals along the scan axis ( 30 ) be redetermined. Method according to one of claims 1 to 4, characterized in that based on the equalization parameters angle error ( 70 ) and / or positional errors ( 71 ) during the post-processing operation. Method according to one of claims 1 to 5, characterized in that at the beginning of the scanning process as a distinctive component geometries component edges of the test specimen ( 20 ) are detected and evaluated along the scan axis ( 30 ) are arranged at the end of the scanning process. Method according to one of Claims 1 to 6, characterized in that a shading correction is implemented during the image equalization. Method according to one of claims 1 to 7, characterized in that to improve the image quality from the captured with the image acquisition system pixels a pixel image ( 81 ) with virtual pixels ( 83 ), where the virtual pixels ( 83 ) by means of a weighting matrix ( 84 ) are calculated from the pixels, and from this pixel image ( 81 ) with the virtual pixels ( 83 ) the equalization parameters are determined. Use of the method according to one of claims 1 to 8 within endoscopy in medical technology, for inspection of hole-like recesses or channels in the field of mechanical engineering or vehicle technology or for 360 ° image acquisition in rooms. Device for equalizing an image of an image acquisition system of an endoscopic inspection device ( 1 ) to the all-round view, wherein with the image acquisition system an overall picture of a in a test specimen ( 20 ) inside test surface ( 21 ) is detectable and by means of an image post-processing unit an overall image of the test area ( 21 ) can be assembled line by line from individual pixels that are located in an annular focus area ( 40 ) of the overall image, characterized in that an image equalization is implemented by software within the image acquisition system, with which during or at the end of a scan process, the current position in the test specimen ( 20 ) along a scan axis ( 30 ) can be determined as well as more distant distinctive component geometries can be evaluated, and from this the position of the scan axis ( 30 ) can be determined to the component axis and equalization parameters for correction of position and angle errors ( 71 . 72 ) are derivable. Apparatus according to claim 10, characterized in that the image acquisition system is designed as a high-speed camera.

Images