DE4447434A1 - CCD sensor for reading out image zones or lines - Google Patents

Abstract

The system uses the anti-blooming electrode of a CCD image sensor to delete the CCD contents before they are read out. Thus, the system only reads out the required parts of the image, which can even be individual lines or parts of lines. The rest of the image is erased with the anti-blooming electrodes of the sensor.The anti-blooming electrode is thus used to prevent overcharging between the CCD chip potentials. The partial image is shifted into the storage zone of the CCD chip and is further shifted to the read out register. The anti-blooming drain of the chip is used to erase the remaining image still in the image zone. During shifting or reading out, a new image is formed in the image zone of the chip.

Description

Today's CCD image acquisition for robot systems is becoming more and more common higher requirements regarding their accuracy and Speed set. Tasks for which the robot master all six spatial degrees of freedom, occur more and more often. A path control that the contour to be traced solely from the data of a CAD System or calculated from subsequently learned base values, can meet the increased accuracy and accuracy requirements No longer do justice to speed. Often the problem occurs that the parts to be machined are process-related be positioned differently so that the path contour Is subject to scattering, or is the contour to be traversed not known at all in advance. Then at the latest sensor-assisted path correction or sensor-guided Track tracking required. For this purpose, e.g. B. Laser distance sensors or CCD line cameras are used. These Sensors have the disadvantages that they are either few Can measure degrees of freedom at the same time or one for Real-time requirements achieve too low a measurement rate. Around Compensating for deficits are sometimes systems with several the same or different sensors. Such "Multi-sensor systems" hardly ever reach one dominant complexity.

It is therefore an object of the invention to control speeds (or: sampling rates) from known CCD image sensors improve without increasing the measurement or evaluation effort and as often as possible to certain areas of the image (lines or Drawing files) to get to the robot controller quickly to provide accurate measurements.

To this end, the invention proposes the elimination of the storage zone known CCD chips before (claim 6). To avoid that those image contents from the CCD charge image area (Image zone) time-consuming must first be read out only the required partial area can be read out, this can also be done  be single lines or parts of lines; the rest of the picture is still in with the anti-blooming electrode of the CCD image sensor the image zone deleted (claim 1). The anti-blooming electrode is used contrary to its actual purpose - the Preventing charge overflow ("blooming") between Potential wells of the CCD chip - used.

If the image section essential for the measurement changes, so it can be adjusted in size and position become. The "dynamically changeable." Image windows "are read out so that a time-consuming reading and moving from the image zone into the Storage zone is kept as short as possible (Claim 4). To accelerate the change of Readout direction of the readout register (claim 5) serve.

The invention proposes the fast image sensor especially for the autonomous acquisition of trajectories by robots; with this Measuring system can have five degrees of freedom in space in real time are measured with a sampling time of approx. 1 msec.

Faster pattern recognition, quality control and others Tasks for CCD image sensors are areas of application of the Invention. The high speed and flexibility opens up additional new areas of application for the use of CCD Image sensors, partly in areas that were previously only used by CCD Line sensors were reserved. The great gap between picture and line sensors can be almost completely eliminated.

The use of a transputer as a microcontroller offers simple and fast by means of a link connection Interface for coupling the sensor to a transputer network or to any other device that has a link interface (e.g. a PC with link adapter).  

A camera can be used as a sensor which corresponds to the Proposals of the invention are modified and adapted as well can also be a mere optics with a CCD chip behind it be provided.

The control electronics can also be made with few changes other CCD sensors use what in practical trials with the TH 7864A (from Thomson). A "Universal sequencer" will be the most realizable common CCD sensors flexibly supported. Hardware changes would then only result in the implementation different sensor interface components.

The core of the camera mentioned is a CCD image sensor with z. B. 768 _* 576 pixels, control electronics specially developed for the CCD chip, and digital evaluation electronics matched thereto. These components enable targeted and time-optimal reading of the required image sections from the image matrix. So z. B. the first image line can be read up to 2500 times per second. All other image lines are already deleted in the sensor with the anti-blooming electrode in order to save the otherwise required readout time (claim 1). One, two or three selectable image lines arranged in any way in the image zone can be read out and processed every millisecond.

The projector is projecting e.g. B. six to the image lines substantially perpendicular (thin) measuring lines on the surface of the object to be measured. A line detection algorithm determines the high-resolution positions of the measuring line centers. The coordinates of points on the object surface can be calculated from the intersections between the measurement lines depicted on the sensor and the evaluated image lines using a triangulation method, similar to a laser distance sensor. The measuring system thus simulates a matrix of n _* m laser distance sensors, where n is the number of projected measuring lines and m is the number of evaluated image lines. The position of a significantly stronger line located on the object (path marking or object contour) can also be recognized. If the coordinates of several object points in the vicinity of this line have been measured, the coordinates of a line point on the object can be determined with high accuracy by interpolation between the corresponding image points and the intersection of the line with an image line on the sensor.

The measurable degrees of freedom are:

• - the lateral deviation of the robot hand from the Lane marking (y component)
• - the distance from the robot hand to the object surface (z- Component)
• - The complete orientation of the robot hand with respect to a path line on the object (angle α, β and || corresponding to the rotations around the coordinate axes x _h , y _h and z _{h of} the hand-specific coordinate system).

The remaining sixth degree of freedom - the feed direction v along the path marking B - is not regulated. A feed rate v is explicitly specified by the path control. The orientation of the robot hand 1 can e.g. B. be regulated so that a tool is always aligned perpendicular to the object surface and the x-axis of the hand-specific coordinate system shows exactly in the direction of movement along the path B.

To measure the five degrees of freedom, the evaluation of only two image lines is sufficient. The actual position of the robot hand 1 relative to the path marking B is detected with the first image line. The position of a point of path B lying further ahead can be determined from the second, leading image line (the point is formed by the intersection of the scanning line and the path curve). This data can be used for path planning and pilot control concepts. The quality of the feedforward control can be increased by evaluating a third, trailing line.

The measuring frequency of 1 kHz achieved with the measuring system corresponds the interpolation cycle (F1 ms) of an experimental robot controller on a transputer basis.  

The following examples are intended to help understand the invention deepen:

Fig. 1.1 shows schematically the viewing angles of the projector for the measuring lines and the camera with the CCD sensor. The viewing angles of the camera and projector are at a fixed angle and look at the object along the path tracking line, which is to be recognized and tracked, with an essentially identical section. The dark area in Fig. 1.1 projects measurement lines or other symmetrical structures on the surface of the measurement object. The camera detects these measurement lines together with the path curve running between two of the measurement lines (cf. also FIG. 2.2). The scanning direction a of the line-oriented CCD chip runs perpendicular to the measuring line direction and approximately perpendicular to the path line direction B. The viewing angle of the optics of the camera is shown in light gray in Fig. 1.1.

Fig. 1.2 illustrates the movement of the object with its surface and the resulting overlap of the viewing angles of the projector and camera.

Fig. 2.1 is an enlargement in schematic representation of the front end of a robot hand. 1 The camera 4 with the CCD image sensor 5 and the lens 6 sketched in the aforementioned figures is arranged on it, while the measuring line projector housing 8 with its condenser lens 9 , the measuring slide 10 defining the line structure, is arranged at an angle to the central axis of the camera 4 and the imaging lens 11 . Arranged behind the condenser lens 8 is a field with a large number of luminous LED diodes 7 , which stroboscopic project the measurement lines of the slide 10 onto the object surface. The reading of the image zone of the CCD chip 5 is synchronized with the stroboscopic illumination of the surface, so that the influence of extraneous light can be suppressed.

If the robot hand 1 moves, the measuring device 4 and 8 moves together with it, since the projector and camera are arranged in a common housing 3 or on a common mounting plate of this housing 3 . The mounting plate 3 is immovably fixed to the robot hand via a flange 2 .

The view of the measuring surface is explained in Fig. 2.2, the measuring lines l₁ to l₆ can be dark lines, while the rest of the field is brightly illuminated. The contour of the trajectory B runs within two measuring lines, the course between the measuring lines l₃ and l₄ is shown. The length of the measuring lines in the direction of movement v of the robot hand 1 can be made dependent on how strongly curved the trajectory B is.

When using a CCD line sensor for an autonomous Line tracking raises the question of how to attach the Sensors - with or without lead, optimal lead distance. The The type of attachment affects the properties of the system, being accurate from the point of view, maximum travel speed and options for one Path planning and feedforward control should be optimized. A dynamic variation of the line feed is desirable technically difficult to achieve.

By evaluating two lines of the CCD image sensor, all necessary information can be obtained in order to completely determine the position and orientation of the robot hand 1 with respect to the path B. The disadvantages of the higher readout time of a CCD image sensor and its predetermined rigid (usual) readout procedure to comply with the CCIR-TV standard of the output signal curve would reduce the maximum achievable image sequence to 25 frames or 50 fields per second if not the speed with the invention would be increased considerably. The invention also "handles" the large amount of data which is produced by image sensors and which has to be processed or at least recorded.

The accelerating suggestions are given below as examples presented to ensure optimal read-out for just a few to enable required image lines from a CCD image sensor. The procedures are e.g. B. with the THX 7868A or TH 7864A from Thomson particularly efficient.

Image sensors are generally related to the duration of a display Adapted television picture. The picture is in two successive fields (interline method) read out, the readout time for each field 20 ms is. 40 ms are therefore required for a complete image. This factor alone limits the measuring rate to a maximum 25 measurements per second - theoretically. Add to that the Processing time for the data (in) quantities. The Processing the data usually takes a multiple of that Selection time. For high quality robot control with an interpolation cycle of a few milliseconds these measuring rates have so far not been suitable. Measurement rates would be necessary ideally correspond to the interpolation cycle or at least are less than an order of magnitude lower.

Through flexible control of a suitable CCD Image sensor, d. H. by abandoning the rigid control according to the CCIR-TV standard, such a high measuring rate can be used achieve a reasonable effort. All of them image information that is not required is already deleted in the sensor save the reading and processing time. This allows the Access (only) to the required data is time-optimized.

The suggestions were made using a six-axis articulated arm robot of inventions tested. For the detection of five spatial Degrees of freedom are the image information from two to three (preferably arbitrarily selectable) picture lines needed. At The feed speed is up to 1 m / sec (36 km / h) Path contour B or measuring line still resolved with sufficient accuracy (1 measuring point / mm). This results in a measuring rate of 1000 Measurements per second. This measuring rate corresponds to that Interpolation cycle of the mentioned control on a transputer basis.  

The background of the CCD image sensors, which are divided into three groups to be grouped:

• - Interline transfer sensors
• - Frame transfer sensors
• - Frame interline transfer sensors

A typical frame transfer image sensor is shown in simplified form in FIG. 3.1. It consists of the following functional units:

• - an image zone consisting of a two-dimensional matrix individual picture elements (pixels) is formed;
• - an equally large storage zone;
• - The one immediately adjacent to the storage zone Readout register, the length of which is the number of pixels in a pixel line corresponds to the image and storage zone;
• - The to the first pixel element of the readout register connected output stage.

The image and storage zones are largely identical built up. However, the storage zone is marked by a opaque layer covered so that incident light has no effect there. Also missing in the storage zone all devices for exposure control, since they not needed there.

The charges generated in the image zone by exposure to light (Charge image) can be within the two zones with the help of (usually four clock signals out of phase by 90 °) move the bucket chain principle vertically. For picture and Memory zone there are separate clock signals (P clocks and M clocks) are available so that the charges in the Have the storage zone moved separately without the loads in the image zone.

The charges from the row that are immediately attached to the Readout registers can be connected in a single step the readout register can be inserted. In the readout register you can then use additional clock signals (L clocks)  be transported horizontally to an output stage. The The output stage converts the charge packets into one of the charge quantities proportional voltage around that at the video output of the sensor can be tapped.

Fig. 3.2 shows the timing of image acquisition and of the subsequent read-out operation.

Starting from the initial situation (a), in which both the picture and the storage zone are free of charge carriers Sensor exposed for the first time. During the Exposure time generated by exposure to light Charge carriers integrated in so-called potential pots (b to d). The integration time is almost 20 ms according to the CCIR TV standard. However, in many modern sensors it can be Reduce suitable clock control as desired (electronic shutter).

A vertical transfer takes place after the exposure time has expired Charges from the image zone to the storage zone (e and f). Of the Transfer typically takes about 350 µs. During vertical transfer charges continue to be generated by exposure to light also be integrated and slightly smear the image (smearing effect). The pollution starts in the first Line of course not noticeable and is in the last image line the strongest. It is usually. low because of the ratio Transfer time / integration time is very short (350/20 000 = 0.0175). However, the influence of the smear effect is increasing Meaning, the further the exposure time using the Electronic shutter is reduced. Extremely bright pixels can also have a clearly visible tail (pulling effect) cause.

After the vertical transfer is complete, the image is located completely in the memory zone, being the first image line comes to rest directly on the readout register. It follows Read the image line by line from the memory zone using the Read register for the output stage. In the time required for this  - in other words, the next image in the image zone on integrated (g to i). After the selection process has ended the state shown in (d) is reached again and the cyclical process begins again.

So are only in the image zone of the sensor due to light once charges have been generated, they can only be time-consuming, sequential pushing out of all image lines be eliminated via the readout register in the output stage.

Fig. 3.3 illustrates the results when the entire middle third of the image is pushed together in the readout register in order to access the image contents in the last third of the image more quickly. The area shown in black in the picture represents the collapsed area. Although the picture as a whole is quite dark, a large part of the last section of the picture is "flooded" with charges.

To get the information from just any two "Useful lines" - the image section - as quickly as possible to get there without destroying the contents of the useful lines or are affected:

• 1. Vertical transfer of the image from the image zone to the Storage zone.
• 2. Push the lines that are not required down to approx. eight lines before the first useful line. Meanwhile running a quick horizontal transfer.
• 3. Reading out seven lines with fast horizontal transfer.
• 4. Reading out the last line before the actual useful line with normal, quality-preserving shift frequency.
• 5. Reading out the first useful line with normal shift frequency.
• 6. Repeat steps 2 to 5 for the second Usable line.
• 7. Delete all of the following image lines any way.

The last step can also be omitted.  

In the image and in the storage zone become horizontal neighboring pixels by vertically running fixed Separate potential barriers generated by means of a gate become. The potential at the gate can be used Set the height of the potential wall. Even if the amount of Potentialwalls is set large, it can happen that overexpose the potential wells by overexposing the sensor and charges into neighboring, less exposed Drain off picture elements. The related disruptive Impairment of the image is called "blooming". In modern CCD sensors, a so-called anti- Blooming unit integrated to prevent this effect. The Gate between two neighboring pixel columns is shown in divided two individual electrodes, with a so-called. Anti-blooming drain is inserted. The anti-blooming drain takes off the excess electrons from the overfilled potential wells, that otherwise flow into the neighboring potential wells would.

By means of the anti-blooming unit controlling the drain an electronic shutter for setting the exposure time realize. By raising the potential well bottoms to the Level of the potential wall - the potential pots then disappear completely - flow all generated by exposure to light Loads immediately into the anti-blooming drain. Raising the Potential well bottoms are achieved by opening all four P-cycles the same, appropriately chosen potential. Only when the bottoms of the potential are lowered again, the begins Integration of the optically generated charges. With the electronic Shutter can be extremely short exposure times of some a few microseconds.

Activation of the electronic shutter function immediately before the vertical transfer of the image from the image into the Storage zone provides an absolutely black picture. This could based on tests on the sensors TH 7864A and THX 7868A be detected. This opens up the possibility during the vertical transfer, immediately after the takeover  a specific, freely selectable line from the image zone into the Storage zone to activate the shutter function, which is the deletion of the residual image still in the image zone Has. The part of the Image is not affected in any way. The shutter Function can be used to set the number m Use the lines to be read (0m292 for TH 786A).

Since the charge of the (292-m) remaining image lines completely can be eliminated, only the charges of the first m Lines from the memory zone via the readout register be pushed out. The time spent processing the last (292-m) irrelevant lines are therefore completely eliminated. If in particular only the first image line is required, i. H. of the Image sensor is used as a line sensor, are in this way Image sequences of up to 2500 images per second can be achieved. Of the "Abuse" of the anti-blooming electrode to erase the image zone is not intended by the manufacturer.

Fig. 3.4 illustrates the read cycle described in the example, only the first half of the image is read out. Starting from a completely exposed image in the image zone (a), a vertical transfer of the first 146 lines into the storage zone is first carried out (b). As soon as the last line to be read is in the memory zone, the P clock pulses are stopped and the remaining partial image in the image zone is deleted with the anti-blooming unit (c). During the deletion process, which lasts only a few microseconds, the partial image to be read out is shifted further in the direction of the readout register (d). Deleting effectively does not contribute to the total readout time.

After the deletion process, the image begins in the image zone Integration of the next picture. This process is running again at the same time as reading out the first half of the image from the Storage zone (e, f).  

With the measures discussed so far, two would be designed any picture lines with the TH 7864A in only about three to six milliseconds possible.

The THX 7868A CCD image sensor offers a special feature So-called. Windowing device (windowing device), with which any Image lines individually and in only typically 2 µs completely deleted can be. In contrast, reading a line takes time approx. 64 µs.

The windowing unit is parallel to the readout register arranged extinguishing device. It essentially consists of the windowing drain through the windowing gate Readout register is separated. During normal operation a potential is applied to the windowing gate, one for the Insurmountable charges contained in the readout register Potential barrier created. When the potential barrier is lowered through a suitable voltage control of the windowing gate the charges flow from the readout register into the windowing Drain off, which equates to deleting the line information is. A targeted control of the windowing gate allows consequently the quick deletion of individual image lines and areas, whereby the high expenditure of time for the serial readout of the relevant picture lines can be saved.

Experiments have shown that windowing Unit managed to delete parts of a line cleanly can be. So there is the possibility, only that the first n pixels (0n845) of a line, the relevant one Contain information. The unneeded part of the line (the 845-n last pixel) can be deleted to add another Realize time savings.

As a further special feature, the THX 7868A has a second one Output stage, which is constructed analogously to the first stage, however with the difference that it is with the last instead of the first picture element of the readout register is connected. By suitable control of the shift clocks on the readout register  influence the horizontal transfer direction. By repentance In the normal shift direction, the line information gets into reverse order to the second output stage. An over Image read out this output appears horizontally mirrored. Are the desired image contents more in the left half of the picture, the lines become normal read out. However, if the information is predominantly on the right in Image, the lines are read out mirrored.

In combination with the option to delete parts of a line the mirroring function can be used even faster to get the desired information of an image.

Since only a few for fast contour tracking, but far apart lines can be read in a very short time is the windowing function of the sensor for that intended application advantageous.

The so-called tracking function may be mentioned here as an example. With the tracking function a z. B. particularly bright point can be tracked due to the fast achievable image sequence with a high temporal and spatial resolution. The image section can be tracked in real time - following the position of the point - the required window size and position can be selected depending on the current speed and direction of movement ( Fig. 3.6). Interesting areas of application are conceivable for this, for example motography.

Since in the planned application of the camera due to the selective reading of partial images no longer the entire normally available exposure time for the each next picture is available, a stroboscopic Illumination with intense flashes of light may be provided to to increase sensitivity.

The storage zone can only be illuminated in the manner of a stroboscope omitted.  

An overview of the circuit components of all camera electronics combining all features is given in the block diagram in FIG. 4.1. Regarding the technical implementation of the camera sequencer, it should be noted that the sequence control units were designed as simple Mealy automatons, which are accommodated in a total of five PLD modules of the type MACH210-15 from AMD.

The camera electronics contain their own microcomputer Transputer base. The IMS T222 is integrated into the system logic hierarchical structure of a superordinate transputer node (Up system) as subordinate node (down system) embedded. The transputer receives control commands via link 0 from a controlling transputer. That is in general the transputer of the frame grabber assembly. But the camera can also from any other transputer assemblies or from one PC with link adapter can be checked. The only requirement to this is that the link connection be made differentially must, since the link drivers are specified in accordance with the RS422 standard. The differential transmission was chosen because it is special is robust against electromagnetic interference. she ensures trouble-free operation in the strong disturbed environment.

Via the 2048 x 16 bit (4 kByte) large two-port memory and the The camera sequencer has four 16-bit registers from the transputer be programmed and controlled. For the practical Using the camera means that reading everyone Image from the CCD sensor can be programmed individually. The mentioned tracking function is thus from the sequence directly supported.

The digital clock signals generated by the sequence for the CCD Sensors are transmitted to the camera via coaxial cables. There they are from the sensor interface to the voltage level implemented as for the sensor control in the data sheet are specified. The sensor interface also ensures in particular for compliance with the required steepness. For Generation of all clock sequences was a camera sequencer  built up, each in three time levels several simple Can perform basic functions, the flow of which by Programming can be modified. The three time levels are justified as follows:

• - Cycles for processing individual pixels (approx. 67 ns)
• - Cycles for processing individual image lines (up to approx. 61 µs)
• - Cycles for processing fields (up to approx. 20 ms)

Fig. 4.4 gives an overview of the function blocks of the camera sequencer. The sequencer is made up of two control units, which are arranged hierarchically, but otherwise work largely independently of one another. It generates 16 different digital control clocks for time-optimized reading of the relevant image sections and for deleting all unnecessary image areas. Different sub-functions of the control units are generated depending on programmable input parameters and input flip-flops.

The shutter register

The two shutter registers are used for electronic exposure time control. At the beginning of an image cycle, the exposure counter is loaded with the content of a shutter register. The exposure counter is then counted down to zero with a clock cycle of approximately 533 ns. As long as the counter content is not zero, the formation of potential pots is prevented by suitable control of the P clocks, so that all generated charges flow into the anti-blooming drain and the integration of an image is prevented (electronic shutter). As soon as the value zero is reached, the integration of the image is released. The exposure release is also used as an external signal to control a stroboscopic light source 7 , so that the flash light can be fired at exactly the right time - at the beginning of the exposure.

The two shutter registers can be used independently delay times of different lengths can be programmed. So  it is conceivable, for example, to have one scene in two successive fields among others To record lighting conditions. Is z. B. a shutter register initialized with the value zero and the other with a value, which leads to an exposure time of only 100 µs, so the Image scene in one image in daylight or ambient light recorded while using the scene in the following image - possibly structured - flash light is recorded. Of the The influence of daylight and ambient light is on the Flash photography is practical due to the short exposure time completely suppressed.

The image parameter register

Apart from the exposure delay, all image parameters can be programmed via the two 16-bit image parameter registers. Fig. 4.5 shows the assignment of the individual bits of an image parameter register.

begin

The sequencer is initialized by setting this bit after guaranteed after switching on the camera or after a reset a defined start of the control units and is used for Synchronization with the transputer.

Reset

Reset the sequencer. The sequencer that Clock control unit and all counters are in the basic state transferred. All clocks and other output signals are on a defined, uncritical rest potential. After this Reset must start for a defined start of the sequencer Bit can be set.

Fig. 4.5 shows the assignment of the bits of a single line parameter register.

Number of pixels from the line (10 bits)

The pixel counter is loaded with the least significant 10 bits, the number of pixel output cycles for the current  Line decides. After output of the programmed number of pixels the windowing unit is activated and the rest of the line deleted.

Left shift

Each line can either go to the left or right sensor output be issued. The bit gives the control unit the desired one Sliding direction known. If the bit is set, the line is in read out in normal direction via VOS1. If the bit is cleared the line is read out in the opposite direction via VOS2. The Control unit automatically takes into account the different Number of irrelevant pixels at the beginning and end of the line. A Image line is made up of 845 pixels, although only 768 pixels contain image information. The remaining pixels are called pre-scan, post-scan and dark reference pixels used. Depending on the sliding direction appear in a different order at the sensor output:

Output VOS1

• - 14 post-scan pixels
• - 50 dark reference pixels
• - 768 image pixels
• - 13 pre-scan pixels

VOS2 output

• - 13 pre-scan pixels
• - 768 image pixels
• - 50 dark reference pixels
• - 14 post-scan pixels

Via the interface to the "frame grabber" (capture device) the sequence only signals the output of the relevant ones Image pixels. The non-relevant pixels are from the frame grabber therefore not processed at all.

Last line

The bit must be set in the last line to be processed, because it tells the sequencer to end the frame cycle and use  to start the next image cycle.

Read out

The bit signals the sequencer whether the line has been read out or to be deleted. If the bit is set, the picture line becomes read out up to the programmed number of pixels. Otherwise it will the line immediately instead of the horizontal transfer Activation of the windowing unit deleted. Then the next line processed.

The picture is optimized by the control unit in the following way read from the CCD sensor:

• - Extend lines 1 to 50 from the image zone into the Storage zone.
• - Delete the remaining lines in the image zone. From now can already run in parallel to all subsequent read operations the exposure of the next image can be done.
• - Move the 50 lines in the memory zone to Output register.
• - Read out line 1.
• - Delete lines 2 to 49 using the windowing unit.
• - Read out line 50.

After the last step, the storage zone is complete cleared. In the meantime, that was already happening in the image zone next picture is exposed and can be read out.

After programming the sequencer, the readout process described runs periodically. The external synchronization mechanism for triggering the readout process (signal start image cycle in Fig. 4.1) and the signal for synchronously controlling a stroboscopic lighting can be used to implement an operation synchronized with the scanning and control cycle of the robot controller.

The functions of the flexible camera described are particularly effective with a suitably tailored "universal frame grabber" for digitizing the image data. Such a frame grabber must be able, in deviation from the "grabbing" of a full image, also to correctly process the image segments supplied by the camera and in some cases not connected. In order to achieve this goal, a suitable interface between camera and grabber is being worked out. The interface between the camera and grabber consists of seven signal lines (Fig. 5.2). In addition to the actual video signal, five other control signals are transmitted from the camera to the grabber so that the sequence control unit of the grabber can understand the image output from the camera in all possible operating modes. The five control lines have the following functions:

(a) New image (image synchronizing signal)

At the beginning of each frame cycle (for one field) a Image synchronous signal transmitted. The signal does that Control unit to start "grabbing" a new picture. The line counter of the control unit is reset.

(b) Field 1

The signal is required to be able to distinguish between the two successive fields (with even and odd picture lines in CCIR TV mode). If field 1 is active, the digitized image data are written in odd line positions in the image memory, otherwise the even line positions are written.

(c) New line (line synchronizing signal)

At the beginning of each line cycle, a line Transmit synchronous signal. The signal causes the grabber to To increment the line counter and the pixel counter reset. The following digitized pixels are thus in the next "line" in the image memory of the grabber registered.  

(d) PHI (pixel shift clock)

At a defined sampling time with one if possible To obtain low sample jitter, the pixel shift clock became PHI from the signal F2L by the camera control electronics is generated, derived. This synchronization signal will placed directly on the clock input of the A / D converter of the grabber. The signal can be varied in phase Runtime effects in the transmission line from the camera to the Compensate for grabber. This is always an exact Pixel-synchronous sampling of the video signal guaranteed.

(e) Pixel valid

The signal is used to distinguish between valid pixels and pre, post and dark reference pixels, respectively. Only if this signal is active, it concerns the incoming Pixels around relevant pixels. While issuing the pre, Post and dark reference pixels of a line become the signal deactivated so that these pixels are not digitized by the grabber or transferred to the image memory.

Another control signal is sent from the grabber to the camera broadcast: camera start

If the signal is active, the camera sequencer can Read out image sensor cyclically. The signal becomes inactive switched on, the camera sequencer remains after processing the currently running picture cycle. With a short impulse a single image cycle of the Trigger the camera from the grabber. So that the frame rate the camera can be set as long as the period lasts is not chosen smaller than that of the camera sequence required time for the output of all desired parts of the picture.

The image sequence can be z. B. with the Synchronize the sampling and control cycle of the robot controller, or an extremely short sequence of images can be realized, e.g. B. for Time-lapse recordings.  

The frame grabber includes a programmable, autonomous Working sequential control unit for the cyclical recording of the Sensor supplied image information. The video signal will digitized in 256 shades of gray. The frame grabber forms the physical pixel matrix of the CCD sensor as virtual pixel matrix in a 1 Mbyte two-port memory from. For every pixel of the pixel matrix in the CCD sensor thus corresponds to one byte in the memory of the frame grabber. The grabber is also equipped with a transputer (IMS T805) and 4 MB of RAM, so that it can be linked can program and monitor the camera functions. Of the Transputer has direct access via the two-port memory on the digitized image data. He can do it over several Link connections with a considerable transmission bandwidth into another transputer network for real-time data processing transfer (transputer 1, 2, n).

The complete reading of an image is the transputer by means of a picture interrupt generated by the sequencer announced to short terms and thus short Achieve response times of the scheme.

Reading the image data into the memory of the frame grabber is with writing a page of paper with a printer (e.g. type wheel printer) comparable:

• - The grabber positions after receiving the image Synchronous signal New image the write pointer to the first Pixel position in the first line of the virtual pixel Matrix - depending on the signal field on the first odd or on the first even line. This Step is with the paper feeding into the printer as well as the Align the printhead to the first writing position comparable.
• - Digitized pixels are sequentially in as long consecutive byte positions written until a Line sync signal New line (corresponding to a Line return at the printer) is received. So it has to  under no circumstances always transfer a complete image line become. Rather, the number of pixels read out in each Vary image line arbitrarily.
• - Hit multiple line sync signals immediately one after the other, the corresponding picture lines in the Grabber's two-port memory skipped. Doing so only the line counter increments (according to the Insert blank lines when printing).

In contrast to conventional frame grabbers, the "Universal Frame Grabber" presented here is able to correctly read any image format from a 1 _* 1 image matrix to a 10241024 image matrix without reprogramming the control unit. It directly supports different CCD sensors, especially the THX 7868A with 768 _* 576 pixels. Successive lines may vary in length as desired.

The control unit automatically updates the image information the positions written in the two-port memory that the physical pixel positions in the CCD image sensor correspond.

Grabber and camera form in connection with the interface a particularly powerful unit for an advanced Real-time image data acquisition and image processing.

From the test image in FIG. 6.1 shows that each row can be individually programmed to unmirrored readout via VOS1 or mirrored read via VOS2. In the top section of the picture all lines were read out normally, in the second fifth of the picture each straight line was read out mirrored. In the middle of the picture all lines were read, in the following part of the picture only the odd lines were read in mirror image. The bottom section contains only non-mirrored picture lines. The mirroring function is e.g. B. for the optimal implementation of the tracking function of interest (see. Fig. 3.6).

The function of the windowing unit is illustrated in FIG. 6.2. The enlargement of an image shows the transition area between the completely deleted half of a camera image and the adjoining second half of the image that was read out. The first line read is not lighter than the following lines, which suggests that all charges from the deleted lines have actually been completely deleted (see Fig. 3.3; problems with the TH 7864 without a windowing unit). However, the first line read is also not darker than the following lines, which proves that the windowing function switches between deleting and reading out in good time, so that the information in the first useful line is not or is not affected by the deletion of the previous line partially deleted with.

Fig. 6.4 illustrates that the number of pixels to be read out can be set individually for each line. The black areas indicate parts of the image that have not been read from the CCD sensor but have already been deleted in the sensor. The readout times saved for each incompletely read line add up, so that the readout time is shorter, the larger the black areas. The areas of interest in an image can thus always be read out at optimal times. In particular, the effective pixel matrix of the sensor can be programmed between 1 _* 1 and 768 _* 576 pixels depending on the task and real-time requirement.

In extreme cases, the number of rows read out can be zero, which of course makes no sense. "Scanning" with any line - or even with only part of a line - is a very interesting application. Fig. 6.5 was created by scanning with (only) line 147. The image sequence was set by synchronization with a robot controller to 1000 image cycles per second - this corresponds to the interpolation cycle of the robot controller. In front of the camera, the playing card was swiftly swiped from top to bottom. When the edge of the card reached the "sensitive" line 147, the brightness reflex was registered by the grabber and the evaluation of 1024 successive image lines was triggered every millisecond. The frame grabber digitized the incoming image lines and copied them from the image memory into successive line positions in the main memory. The sequence of 1024 image lines resulted in the displayed scan image with 1024768 pixels.

It is due to the fact that it is actually a scan image recognize that due to the non-constant speed, with which moves the card in front of the sensitive image line when scanning the upper half of the image was squashed and the lower half Half stretched appears. When scanning with the first one Image line, a scan frequency of 2500 Hz can be achieved which is in the order of magnitude of common CCD line sensors.

Fig. 6.6 shows the time required to read 1 to 3 image lines, depending on the number of the last image line read out. In Fig. 6.7, the time required for time optimal reading of any n and the n first lines finally is shown.

The measuring system based on the camera only uses in the example three freely selectable image lines of the CCD image sensor for the Measurement of five spatial degrees of freedom with one measuring rate of 1000 measurements per second (sampling frequency of 1 kHz).

• (A) From a picture line running before, data for a efficient path planning and Cartesian pre-control be won.
• (B) The middle line records the actual position of the Tool reference point of the robot, its position relative to the railway line.
• (C) A trailing line finally supports the measurement some parameters, which increases the measuring accuracy can be.

The choice of the leading and trailing line can be made dynamically to the Path course and adapted to the current path speed become.

The principle of how a line detection algorithm with a high sampling frequency can determine the position of the trajectory B between two measuring lines 13 and 14 has been explained with reference to FIG. 2.2, although only a single transverse scan line is read from the CCD and processed.

This performance result is potentiated by a
Interpolation. It is used to measure and process the distance between the point Z of the trajectory B lying in the scanning line a between various of the other measuring lines l _i → l _j . In the example of Fig. 2.2 there are even 18 ways to calculate the position of Z (from l₁ to l₄, from l₂ to l₄, from l₃ to l₄; from l₁ to l₅, from l₂ to l₅, from l₃ to l₅; etc .). If an average is formed from this multitude of measured values - one interpolates - the position of Z can even be given on the basis of the center point of the width of the trajectory B. Even wide, even irregular train lines are compensated. One (among many) algorithms for interpolation is given in Fig. 1.1.

Because of its enormous flexibility, the frame grabber can also used for various other tasks especially for tasks where it is quick Provision of data from selected image areas or from arrives a few freely selectable lines. Especially for the construction The camera has faster and more precise optical measuring systems decisive advantages due to its high flexibility compared to conventional CCD full-frame and line cameras.

Claims (7)

1. Using the anti-blooming electrode of a CCD image sensor to delete the CCD content (the ones on it) Charge) before the CCD content has been (completely) read out. 2. Use according to claim 1, wherein the charge image with the Anti-blooming electrode after reading out a few lines is deleted. 3. Use according to claim 1 or 2, wherein the charge pattern of the Sensor is deleted after (only) one line has been read out has been. 4. Use according to one of the mentioned claims, in which the readout and the integration of the charge stored in the CCD chip is nested, wherein

• (a) the sub-image shifted into the memory zone of the CCD chip is (further) shifted to the read-out register;
• (b) the anti-blooming drain of the CCD chip is used to erase the residual charge image (still) in the image zone of the CCD chip;
• (c) according to (already) during the moving or reading out
• (a) a new charge image is exposed or built up in the image zone of the CCD chip.

5. Use according to one of the claims mentioned, in which the / a read-out register is read out on one side or the other depending on the position of the partial image ( Figure 3.6). 6. CCD image sensor with image zone but without memory zone, whereby the serial readout register directly at the (charge) Image zone is arranged. 7. CCD image sensor according to claim 6, wherein a window Masking unit is provided.