ES2367746A1 - System and methodology for the transmission of images using incoherent fiber bundles. (Machine-translation by Google Translate, not legally binding) - Google Patents

Abstract

System and methodology for the transmission of images using incoherent fiber bundles. An opto-electronic system is presented that allows both the calibration of incoherent fiber bundles (iofb) to be able to transmit images with them, as well as the transmission of images. The system consists of an optical measuring bench, an input lens, an iofb, a coupling system between the deck and a high resolution sensor, a calibration screen and a control and processing unit. The methodology that allows reorganizing the information captured by the iofb is described, so that the image present in the entry can be reconstructed later. The methodology includes several processes such as: iofb approach, location of fiber positions, calculation of an image reconstruction table (lut), reconstruction of the image and readjustment of the calibration. Once calibrated the mallet can be used in the exploration of enclosures with risks of explosion, with high temperatures or radiation. (Machine-translation by Google Translate, not legally binding)

Description

System and methodology for the transmission of images using incoherent fiber mallets.

Technical sector

This invention relates to the transmission of "encrypted" visual images using fiber mallets non-coherent low cost optics.

State of the art

Visual inspection systems through Electronic cameras are widely used today in quality control systems of industrial processes, systems of surveillance, positioning of mobile robots, among others Applications. However, in hard to reach places, with extreme conditions or over long distances, it may not be possible or convenient the use of electrical signals and electronic devices. Examples of its use in hard-to-reach sites may be in medical applications linked to endoscopy, periscopes and in the inspection of hostile environments exposed to high temperatures and / or pressure, nuclear radiation, or simply enclosures with a risk of explosions and / or corrosion as well as in surveillance systems of transport infrastructure (rail or road). For the transmission of images under these conditions, it is possible to use of optical fibers grouped in mallets that although it is coupled in their output terminal to an array sensor (camera), this is isolated from the medium of interest. The mallet only behaves like a Image transport element, and if encryption is desired, But not as a focusing device.

A fiber optic mallet for the transmission of images is made up of many optical fibers (tens of thousands), arranged in a compact manner, so that the ends of the mallet can be modeled as finite planes composed of several fiber points. With this arrangement, any image projected into the mallet's entry plane decomposes into finite fiber-points, and would appear in the output plane as a series of luminous points. The core of each fiber captures a portion of the image and delivery to the other end of the deck. The decks in general can be classified as to the way in which distribute the fibers longitudinally, in coherent and incoherent.

Consistent mallets are the most used in image transport using fibers and have found great application in medicine, in enclosure inspection systems, videoscopes, etc. These decks have some drawbacks such as which are generally designed to work with distances relatively small (few meters) and are more expensive. Without however they guarantee a good image quality.

IOFBs are generally designed for the distribution of light so they do not maintain the spatial relationship of the fibers between the input and the output, producing a kind of natural coding of the image, which is specific for each mallet. They are not usually subjected to the process known as fusion , which, in the coherent mallets, is responsible for the fact that at distances close to 4 meters the interferences between the fibers produce a blurred effect on the transmitted image. These characteristics mean that if they want to be used in the transmission of images, IOFBs are devices capable of transmitting information at greater distances than coherent ones. In addition, more flexible elements result that can be used in enclosed spaces and under hostile environments being cheaper. However, they require greater complexity in the treatment of information.

The Dujon et al . Patent, "Visual image Transmission by fiber optic cable" (No. US Patent 5327514, Jul. 5, 1994), describes an IOFB calibration method for image transmission, using several standard images to determine the input-output relations of the deck. The pattern images are made up of strips (vertical and horizontal) of different width each time, according to a coding of spaces. Although few calibration images are used, the calibration of the mallet requires a slightly more demanding optical system in terms of optical resolution.

In the patent of Roberts et al ., "Robust incoherent bast optic bundle decoder" (US Patent No. 6587189 B1, Jul. 1, 2003) an apparatus and the calibration methodology of an IOFB is described. The described system manages to calibrate the fiber mallet by scanning images consisting of lines (vertical or horizontal) but unlike Dujon's work they are fixed width. Decoding is easier although it is necessary to process a larger number of images. Searching for each fiber in which position of the strip the maximum excitation is obtained (both in the horizontal and the vertical scan), a LUT table is constructed that allows the image to be reconstructed. For this, the authors suggest the location of the fiber positions through morphological transformations to facilitate the construction of the LUT.

Explanation of the invention.

This patent describes a calibration system and image transmission using IOFBs. Also, the different procedures involved in the optics approach of IOFB input, calibration, calibration reset as well as, the reconstruction of the captured images.

The system uses an IOFB as a transport element. Although these mallets have notable disadvantages with respect to the coherent ones, they allow, through a calibration process, the transmission of images over greater distances, as well as transmitting the useful signal fully encoded. These characteristics make it potentially interesting in industrial video surveillance applications in areas of high corrosive risk, high humidity, with danger of radioactivity or explosion, and in those cases where it is necessary to transmit images optically over large distances.
cias.

The system described serves both for transmission system calibration as for reconstruction of the transmitted images. Figure 1 shows a General diagram of the calibration system. The different elements have been represented separated from each other to give a clearer idea. The system is basically constituted by (Figure 1):

?

IOFB (1) : Image transport element.

?

Dark enclosure (2) : It allows to isolate the system of entrance of the mallet of any reflex or external effect, during the calibration. It is not necessary to be able to reconstruct real images once the system is calibrated.

?

Control and Process Unit (3) : This block is responsible for controlling the calibration of the mallet and the entire camera control. Once the system is calibrated, it is responsible for the capture and storage of images, for the reconstruction of these and for the readjustment of the calibration of the system when it is required to readjust the system to new conditions.

?

Fiber adapters (4) : Its shape depends on each specific mallet and allows the connection of different models to the system.

?

Deck clamping system (5) : Allows both ends of the mallet to be fixed to the corresponding structure.

?

Coupling system of the mallet to the camera (6 and 9) : Attaches the mallet to the camera by means of a lens system (6) and allows to control the depth of field by means of a diaphragm or iris (9).

?

High brightness and resolution LCD screen (7) : Projects standard images during the calibration process. It must be able to emit enough energy to excite the fibers.

?

Screen support (8) : Ensures that the optics are perpendicular to the screen and avoids perspective errors in calibration.

?

Camera or sensor (10) : High resolution device governed by the Control and Processing Unit . Capture an encoded image from the mallet through the mallet-chamber coupling system .

?

Optical coupling (11) : Allows adapting optics with different frames to the harness clamping system .

?

Entry optics to the deck (12) : Projects the images on the entry side of the deck. It can be a standard mount optics and may require a coupling system to the harness clamping system .

?

Mobile support and rail (13 and 14) : Adjusts the system, in order to maximize the scanning area of the screen with influence on the input of the IOFB. This allows you to optimize the scanning resolution based on the field of view of the input optics.

The calibration principle consists of project by means of a suitable optic (12) on the entrance face of the IOFB (1), images from a flat screen (7). Be progressively sweep the entrance with a line (width, position and known orientation) and are checked at the exit of the deck fibers on which greater influence is achieved (17-18). For this, a high camera is used resolution (10). The scan is performed first in the direction horizontally and then vertically (15-16). Verifying these results and estimating the actual locations of the fibers in the image that receives the camera, a LUT is built that once purified is the basis of the reconstruction of any image captured by the sensor. In this procedure image processing techniques are used that will be described later.

The mallet is coupled to the entrance with a certain mount (12) by means of an adapter (4), which also allows to fix it to a mobile support (13), which can be moved using a rail (14) and thus optimize the field of vision. For ensure correct coupling to different optical standards may require an adapter (11). The exit end of the mallet is introduced in a mallet coupling system to the chamber (6) by means of an adapter (4) and a clamping system (5). The adapter (4) is a ring-shaped element, with a diameter such that it fit in (5) but with an inner hole that depends on the mallet. This guarantees the adaptation of the system to different IOFB with just replace (5) with another related to the IOFB of interest.

The coupling system of the mallet to the chamber (6), It has an iris that allows you to control the depth of field and the energy received in the chamber, coming from the mallet (1). The entire procedure for image calibration and reconstruction is carried out through a Control and Processing Unit (3).

Regarding the patents of Dujon et al . and Roberts et. al . (US 5327514-1994 and US 6587189 B1, -2003), although the same concept of decoding the mallet by known standard images is followed, a sequence algorithm and image projection totally different from those described in other patents are used, using a different installation and being the totally novel processing methodology. These previous works focus more on the calibration of the mallet, however the methodology presented here includes all the necessary methodology to define a real system of this type fully functional. Some new procedures are added, such as the focus, correction or equalization of the fiber responses and the readjustment of the system calibration.

For each IOFB, there is a single standard LUT and This is unchanged. This feature is used together with characteristics of each IOFB, so that any process of Calibration readjustment can be faster. As soon as an IOFB It is calibrated, it can be reused in another similar system independent or can be readjusted when moved from position original during calibration, as any change in relative positions of the fibers relative to the chamber can be easily corrected without having to do a long calibration.

Description of the drawings

The following figures allow us to understand the general operation of the system.

Figure 1 shows a general scheme of the different elements used in the calibration. They have represented all elements separately for greater clarity. Once the calibration is done, the system is ready for the capture, transmission and reconstruction of images and Elements (2), (7), (13) and (14), are not required.

Figure 2 shows a block diagram of the methodology to follow for the location of the fibers in the image of the IOFB output face, captured by the sensor (10)

Figure 3 shows us the way in which the calibration is performed ( decoding the mallet ) starting from standard images formed by lines that move horizontally and then vertically.

Figure 4 illustrates a block diagram of the methodology for gavel calibration.

Figure 5 shows a diagram of the Image reconstruction procedure.

Figure 6 represents a diagram of the method of calibration readjustment using the recording technique images.

Embodiment Fiber focus and location procedures

It is important to ensure the optics approach input before proceeding to calibrate the system. Since the image taken by the camera appears messy some method is required and algorithm that allows to determine the correct approach. Notice that You can't tell if the optics are focused, since you can't Rebuild the image before calibrating. For this you can represent a small square region or line on the screen of sufficient size to be able to excite a certain fiber group From the video frame it is determined in which fibers more energy is being received. Next, the optics are adjusted until this average energy value of each of the fibers detected maximum. This correspondence between the level of focus and energy level (average gray level) is given in that the better the focus the energy is better concentrated on the fibers

The field of vision is inescapably linked to the input optics. To ensure that the optics capture the largest possible area within the screen you can illuminate the blank screen and check in which position of the rail it is guaranteed that no fiber darkens. This process is linked to the previous one and you may have to repeat one or both another to cover the entire active area of the LCD monitor with the Best possible approach.

As shown in Figure 3, an IOFB does not maintains the spatial relationship between fiber positions in the input terminal with respect to those of the output terminal. Whether want to use such a system in the transmission of images, it requires locating beforehand the regions of the image, captured by the camera, where the useful information is found. To do this you must perform a search or detection of the elements of transport: the fibers. This procedure also allows to know about Beforehand the fiber light transfer function. This is important since the fibers in general suffer from deficiencies in polishing and finishing bringing with it that not all transmit equally, even when submitted equally to the same stimulus, for example, an extensive source of homogeneous light. Each of the fiber positions is stored in the LUT.

The location of the fibers is also a way to effectively reduce the processing time. This decrease in calibration time is due to the fact that the real positions of the fiber cores in the sensor are known a priori , which are responsible for carrying the useful information. Therefore, during calibration only a smaller number of pixels is verified in each image focused on the input, thus reducing the number of operations per second, the amount of memory needed and the necessary calculation times.

Another advantage associated with the detection of the fibers is verified when there is some slight rotation or movement of the mallet in front of the chamber or a replacement is made of it, then, the positions of the fibers need to be re-located because they have been physically modified . This movement can be modeled as a combination of translational and scaling rotation changes (of certain pre-established points of each fiber mallet - footprint -) of the fiber positions in the original LUT. Detecting this change can estimate the new positions of the fibers and allows the recalibration of the transmission system or can even be used to encrypt the
system.

To make a proper detection of fiber positions in the image projected on the sensor, the mallet is illuminated by the entrance with a diffuse light source of so that all fibers light up homogeneously. This source of Light is provided by the calibration screen itself. This shape seeks to clearly differentiate the fibers of the spaces intermediates and through a pattern localization algorithm circular, estimate the geometric centers of the fibers. For the location a method based on transformations was developed morphological binary images and the transformed of the distance.

The fiber localization methodology is You can define in six general steps (Figure 2):

?

Conditioning the image captured on the sensor : Filters are applied to reduce noise and increase the signal to noise ratio of the system.

?

Binarize the image : Obtain a black and white image where the illuminated fibers are partially separated from the background of the scene.

?

Eroding and dilating : The black and white image is eroded and dilated to improve segmentation since after binarization there may be circles that remain interconnected.

?

Distance transformation: Apply the distance transform to that image but complemented: Since the fibers are small and round, the distance transformation gives a peak at the centroid of each fiber.

?

Binarization of Distance Transformation : Depending on the fiber radius in pixels, a threshold is chosen to binarize the resulting image.

?

Calculate the geometric centers of the different regions : It is done using an intensity weighting method that allows us to choose the best emission point near the center. This point is chosen as centroid.

The distance transform is an operation which converts a binary image into an image in shades of gray, where all pixels have a value corresponding to the distance to the characteristic pixel of opposite value plus near. Therefore, for circular figures the geometric center is the one that presents more distance with respect to the edge.

Calibration procedure

The calibration procedure consists of decode the image by exposing different images patterns on the LCD screen above the system input. The pattern images are made up of strips of a certain width and that once projected at the entrance, they are able to excite each Once an undetermined number of fibers. How interest is given in 2D images, you need to excite the input of the two-dimensional IOFB: horizontally and vertically (Figure 2).

The stripes are first displaced in one dimension and then in another. In this way the entrance face of the mallet is completely swept with a resolution defined by the width of the strip and the displacement step. Each time the image is changed from the Control and Processing Unit, the image is captured and stored with a serial name. This allows a posteriori to easily recognize in which positions of the strips each fiber was significantly excited.

The impact of the strip on the entrance face of the mallet, it should be such that its width is slightly smaller or as much equal to the average diameter of the fibers (usually tens microns) This ensures that the maximum fiber response is Achieve when there is greater exposure of the strip on the fiber. TO The greater the area of exposure, the greater the response. The number of steps necessary to completely sweep the mallet inlet face defines the size of the reconstructed final image and is related with the maximum number of fibers in each dimension. Not however It necessarily means higher image resolution. By example, if the step is too small, it causes the image rebuilt increase the number of empty spaces that are not You can assign some value of gray.

For the analysis it is important to take into account the response of each fiber to give each and every one of them the same importance during the analysis. That is why during the calibration process a multiplicative factor α_ {i} must be calculated for each fiber i such that if they are all exposed to the same level of gray in the screen corresponding to the target, all average gray levels of centroids (NG) multiplied by this factor, result in the maximum possible gray value (Ex. 255 for 8 bits of resolution).

255 = α_ {i} NG

The parameter α of each fiber must be recalculated once this phase is over. This is done using an unsaturated image formed from a blank standard image. Taking into account the centroids and average gray levels in an area surrounding these coordinates, the values of α_ {i} are recalculated. This proportional method is ideal if there are no alignments in the system responses. If not, the methodology changes. This means that instead of storing a single proportional value [alpha], the coefficients of a polynomial that approximates the fiber response for different levels of gray should be stored. This means that the value α_ {i} that is used to reconstruct an image is obtained from said response by verifying the average gray level in the fiber and the value to which it should approximate. In case the system manipulates RGB color images, the table must store the analogous values of \ alpha_ {Ri}, \ alpha_ {Gi}, \ alpha_ {Bi} or the coefficients of the aforementioned polynomials but for each channel of
color.

How pattern images are intentionally in black and white, each change of the pattern image is verified if each fiber changed its state (lit up) and if this change was the most important of those received previously.

Through the above procedures a reorganization table or LUT is constructed. This table is stored in the Control and Processing Unit . It stores:

?

Fiber positions (coordinates)

?

For each scan image, it check each fiber position if any result bright enough If so, it is stored in the LUT in which row and column said condition was reached. If that fiber has already been remarkably excited, it is verified if the new position allows a more excitement If so, the Table is updated.

?

Compensation factor of the fiber transfer function i (? {I}).

For all this the structure of the LUT result:

one

Once the LUT is built, it should be verified if there are registered positions that are repeated or simply of the that there is a value for one dimension and none for the other. In such cases are eliminated or simply redistributed to positions next and not included in the LUT.

Calibration reset process

Each calibration process is only valid for A specific deck. If this is replaced by a new one, it is it is necessary to indicate to the system the new correspondence law (LUT). It is significant that the calibration process is only it is necessary to do it once and therefore, each fiber mallet is Associates an own LUT. However, any minimal movement of the deck in front of the camera due, for example, to a sensor change or a bad adjustment in the support that receives the mallet, causes the Image to rebuild does not form correctly.

This is because the reconstruction process try to relocate information from positions that are no longer They correspond to the originals. However, the set of fiber locations, have moved from their original positions under a law of geometric transformations that include the rotation, translation and to a lesser extent scaling.

To achieve a recovery of the LUT, it must be deduced to what extent these movements of the fibers have occurred in order to correct them. We call footprint the set of most distinguishable characteristic regions of each deck. For example, the set of closed regions inside the mallet that are empty of fibers. These regions are due to defects in construction and never change, being specific to each deck. Starting from the original LUT and the common information between the original mallet footprint and the new footprint present in the mallet, the LUT already adapted to the new conditions is recalculated.

First the regions of greater area are located and verified and a correspondence between the most characteristic regions of the original image and the current one is established. With this information a transformation matrix is obtained that allows to make a correspondence between transformed images ( matching ). This transformation matrix allows to relocate each and every one of the mallet fibers and therefore the LUT is obtained directly, which will allow the reconstruction of images.

The process of locating the tracks process the image captured by the sensor with all the fibers illuminated From the first calibration of a given IOFB, they extract the centroids of the largest regions of the mallet with absence of fibers which are used as control points. From such regions are also stored some descriptor of each contour of the region (eg Fourier descriptors). This last feature is what allows us to search any other image, the corresponding regions, regardless of having suffered globally in the image, some rotation, translation or scaled up

Regions are obtained through a process of binarization through a certain threshold and a subsequent Image tagging. To locate in another footprint of the deck the same regions, you should look for the optimal binarization threshold that allow correspondence between regions. This is implemented through an iterative process that verifies within the regions of greater area when there is a correspondence of regions according to the contour descriptor. Once the regions are determined that correspond, the transformation matrix that allows recalculate the new positions of all fiber centers and with this update the LUT (Figure 6).

Image reconstruction process

Once the system is calibrated, the images captured from the sensor can be transmitted and subsequently reconstructed or decoded. The first thing is to build a primitive image that is obtained by reorganizing the information obtained from the captured image, according to the LUT. The process is carried out as follows:

one.

Image capture

2.

We create a two-dimensional array of size p x p , where p is the number of positions used in the sweep or step.

3.

For each fiber position, the average gray level of the proximity of said centroid. This level of gray, multiplied by the compensation value \ alpha_ {i} proper to the fiber, is stored on the 2D array (primitive image) in those row and column positions declared in the LUT.

The primitive image, although it is an image that is related to the input image, has a series of points to which no gray value has been assigned. This is due to the non-uniformity of the mallet in terms of fiber distribution. That is why an algorithm is needed that completes or fills these undetermined regions of the image space. To solve this, a filter is used, which is iteratively convolved n times with the image only in those empty positions of the primitive image. With each step the gray values known in the primitive image are restored, however the "gaps" are gradually filled with gray levels that are related to the neighborhood. You can use a filter like the following or similar:

2

If the number of empty regions in the primitive is smaller, the reconstruction is better and the time spent in the reconstruction ( inpainting ) of the final image is less.

Claims (5)

1. System for the transmission of remote images using incoherent optical fiber mallets (IOFBs) and for the calibration of the mallet, characterized by being: a camera (10) that captures the information transmitted by an IOFB (1) through the elements optical (6, 9 and 12); a Control and Processing Unit (3) that governs a monitor (7) and processes the images captured by the camera (10); two accessories (4 and 5) that allow the fastening and coupling of different IOFBs to the system; an accessory (11) for coupling different optical frames at the entrance and a calibration bench necessary for the calibration of the system and consisting of a dark enclosure (2) and supports (8, 13 and 14). 2. Method for optimal configuration of a system for transmitting remote images using IOFBs, according to claim 1, comprising the following procedures:

to)

System calibration to calculate the transfer function, between input and output, necessary to recover and correct any transmitted image.

b)

readjustment of the system calibration when the mallet (1) is placed in a different position from the one it had during the original calibration, it is replaced by a different one or the camera (10) is changed.

C)

reconstruction of the final image captured by the camera (10) using the results of the previous calibration.

         \ vskip1.000000 \ baselineskip
      

3. Method for the optimal configuration of a system for the transmission of remote images using IOFBs, according to claim 2, characterized in that the system calibration procedure comprises the following steps:

to)

A System preparation stage for calibration based on the following non-separable set of actions:

i.

Focus on the input optics (12) of the IOFB (1) measuring the level of energy that a group is capable of transporting located of fibers and that is necessary to obtain a good calibration quality

ii.

Location of fiber positions in the image captured by the camera (10).

iii.

Calculation of correction factors for intensities or equalization of the responses of each fiber (\ alpha_ {i}) to match the transfer functions of the fibers

b)

A Calculation stage of the reconstruction table or LUT based on the following non-separable set of actions:

i.

IOFB input sweep (1) using a series of images formed by light strips projected from a monitor (7) on the entrance of the deck. The stripes should illuminate uniquely and in two dimensions to all the fibers of the IOFB

ii.

Analysis of each image resulting in the output of deck to determine in which positions of the entry, each of The fibers reached their best degree of excitation. These results are stored in a LUT or rebuild table.

iii.

Debugging the results obtained in the table reconstruction with the intention of eliminating possible redundancies and mistakes

         \ vskip1.000000 \ baselineskip
      

4. Method for the optimal configuration of a system for the transmission of remote images using IOFBs, according to claim 2, characterized in that the system calibration readjustment process comprises the following steps:

to)

Identify the regions that correspond in both images (at least 4 regions) by shape descriptors based on an original footprint of the mallet and the new fingerprint captured by the sensor.

b)

Calculate the transformation matrix geometric that would allow correspondence between the positions of the previous and current fibers.

C)

Through the transformation matrix you recalculate and update the fiber positions in the LUT

         \ newpage
      

5. Method for the optimal configuration of a system for the transmission of remote images using IOFBs, according to claim 2, characterized in that the following non-separable set of actions is performed in the reconstruction procedure of the final image :

to)

rearrange image information messy shot in the camera (10) through a table of reconstruction (LUT) obtained from the calibration procedure, according to claim 2, and which consists in relocating and correcting the gray levels contributed by each fiber in the sensor towards the positions indicated in each record of the LUT forming a primitive image.

b)

reconstruction of the final image that consists in convolving the primitive image, iteratively, with a known digital filter on empty positions and subsequently restoring the pixels to their original state initially known. This action progressively fills the empty spaces present in the primitive image with gray levels that are related to your neighborhood.