WO2001099414A1

WO2001099414A1 - Image processing system

Info

Publication number: WO2001099414A1
Application number: PCT/CH2001/000389
Authority: WO
Inventors: Nikolaus Schibli
Original assignee: Fastcom Technology S.A.
Priority date: 2000-06-23
Filing date: 2001-06-21
Publication date: 2001-12-27
Also published as: AU2001265730A1

Abstract

The invention relates to an image processing system comprising an image reading unit which is made up of an electronic image sensor and at least one AD converter for converting the analog signals read out of the image sensor into digital image data; a processing unit; and at least two image memories, whereby each image memory can be controlled by the image reading unit and the processing unit and can be used to read or write image data. The inventive system is characterised in that it comprises a configuration which enables the image reading unit to write a read image to a first image memory and to write the next image in a next image memory. As soon as all image memories contain an image, the newly detected image is written again in the first image memory. At the same time as the recording of the image, the processing unit has access to the last image memory written by the image reading unit and which is no longer being written by the image reading unit.

Description

Image processing system

Field of the Invention

The invention relates to a processing system and its use, which allows the image information to be made available to a processor as quickly as possible.

Nowadays, compactly integrated image reading and processing systems, so-called "smart cameras" or "intelligent cameras", are used for the digital processing of images. These cameras can not only record and digitize the images via optics, but also process the images with a processor and finally deliver the results of this image processing to another computer via a communication interface (for example a control unit, a robot arm, etc.). Such systems are used in many places in industry. In this way, an "intelligent camera" works like a sensor that forwards a measurement result from the image information to a central processing unit.

However, industrial processes need the results as quickly as possible in order to keep the sampling times of the control systems as short as possible. As a result, the control systems are more efficient and most processes can run faster. For the "intelligent camera", this means on the one hand that the processor has to process the images very quickly, but also that the image reading unit only provides the processor with the necessary image information as quickly as possible.

State of the art:

US Pat. No. 5,396,288 of March 7, 1995 to Fuji Foto Film Co. describes an image reading system for a video camera or digital camera. In this invention two image memories are used, in which simultaneously the same image is read in, but for each image with a different gain value of the analog signal. This will read a lighter image and a darker image into memory. The goal is to achieve a new image that is more dynamic than the two original images. To do this, the two images are combined: If one part of the picture is oversaturated, the new part of the picture is taken from the darker picture. If another part of the picture is too dark, the new part of the picture is reconstructed from the lighter picture.

The US Pat. No. 5,146,340 of September 8, 1992 from Georgia Tech Research Co. describes an image reading and processing system which allows an electronic image sensor to be read at two different speeds. This is done in order to read out a desired image zone at the correct speed, while the rest of the image is read out at an increased speed, only to get to the interesting part of the image more quickly. The unnecessary parts of the picture are not digitized. With this method the clock frequency of the image reading process can be increased. The read image is placed in an image memory. This image memory can be read out and processed by a processor after the buses have been switched over.

Canon Kabushiki Kaisha's US Pat. No. 5,075,788 describes an image readout method with a CCD image sensor in order to classify an image into different zones. A distinction should be made here between where in the picture characters are shown and where there is purely photographic information. In order to implement this invention, a camera system is proposed where the image is stored in a memory and then processed with a computer system.

The US Patent No. 5,060,074 from Hitachi Ltd proposes an image reading unit for a video camera which, like in US Pat. No. 5,146,340, allows the image field to be read out at two different speeds. This is done in order to read an interesting image zone faster than the entire image. Afterwards, vibrations of the image caused by the camera user can be balanced. The interesting image zone can be displayed without vertical or horizontal shift.

Description of the invention

The invention relates to an image processing system consisting of:

An image reading unit, consisting of an electronic image sensor, one or more AD converters, in order to convert the analog signals read out by the image sensor into digital image data,

- a processor unit,

At least two image memories, each image memory being controlled by the image reading unit and by the processor unit and being able to be used for reading and / or writing image data,

characterized in that it has a configuration that allows the image reading unit to write a read image to a first image memory and to write a next image to a next image memory; as soon as all image memories are written with an image, the newly captured image is written back into the first image memory;

At the same time as the image recording, the processor unit has access to the image memory which was last written by the image reading unit and is no longer being written to by the image reading unit.

In contrast to the conventional solutions, the proposed invention can simultaneously capture an image and read it into an image memory, while the previous image is available to the computing unit for image signal processing. The image reading and processing system consists of an image reading unit, an image signal processing unit and several image memories. With the application of this invention, for example, the clock rates of an image processing system can be increased. In addition, the invention relates to the use of an image processing system, characterized in that that the image reading unit writes a read image in a first image memory and writes a next image in a next image memory; as soon as all image memories are written with an image, the newly captured image is written back into the first image memory; At the same time as the image recording, the processor unit has access to the image memory which was last written by the image reading unit and is no longer being written to by the image reading unit.

The image processing system is preferably characterized in that the image reading unit binarizes the read-in image in comparison to a predefined threshold value before the image is stored in one of the image memories.

List of figures

Figure 1: Schematic representation of the image reading and processing unit, also called an intelligent camera.

Figure 2: Overview of a possible implementation of the electronics for the proposed image reading and processing unit.

Figure 3: Detailed overview of an image reading unit with several readout channels.

Figure 4: Representation of the possibility of reading out a partial image with the proposed system.

Implementation of the invention

Regarding Figure 1:

Compact image reading and processing systems, also called "intelligent cameras", are used for industrial purposes. The "intelligent camera" can, for example, read an object as an image, measure this object (distances, diameters, etc.) and these measurement results can be combined with one additional Processor system are transmitted. FIG. 1 shows a simplified illustration of an “intelligent camera” 51. It consists of an optical system 52, which is responsible for projecting a desired image zone onto a surface inside the camera.

A matrix of light-sensitive elements (also called an image sensor) is located on this surface and creates an image. This image information is digitized and made available to a processor system as an image. This is the task of the image reading unit 53.

A processor system is equipped with the necessary peripherals (memory, timer, etc.) to process the image in order to achieve results from it. This part of the “intelligent camera” is the image processing unit 54.

After that, the results often have to be transmitted to a process computer (robot controller, PC, etc.). The communication unit 55 is responsible for this. The external devices that communicate with the intelligent camera are referred to as “external peripherals” 56.

Regarding Figure 2:

The invention described here is based on the principle of paralleling the data flow from the image reading unit to the image processing unit as far as possible in order to achieve the shortest possible pipeline time. An image is read into a memory while the image is used by the processor in another memory. The functions of these memories are subsequently exchanged. In order to explain this, FIG. 2 shows an implementation example.

The example given consists of three elements: an image reading unit, the image storage elements and the processor unit. An optics 1.3 is located on the image reading unit, which is responsible for focusing an image on an image sensor 1. An image sensor is a matrix of light-sensitive semiconductor Elements defined. The image sensor 1 can consist of a CCD image sensor (charge coupled device), but also of a CMOS image sensor. The image sensor is confessed with the image with a fixed integration time. Then the image is read out. The horizontal driver 3 is used to push the entire image matrix into the readout register 1.1. The shift register is then read out with the horizontal driver 3, in which one pixel after the other is led from the image matrix to an analog signal processing 4 and is subsequently converted digitally. It is possible that several reading registers are integrated in the CCD image sensor. Analog signal processing 4 is understood to mean, for example, the “correlated double sampling” CDS and also the “automatic gain control” AGC. The conversion of the preprocessed signal is done with the AD converter 5.

The control element 6 of the image sensor 1 is now responsible for storing the image information in the one memory. The control element thus generates the entire control signals for the image sensor drivers 2, 3 for the readout register 1.1 for the analog signal processing 4 and for the AD converter 5. The control element can be a processor, but also a programmable gate array (FPGA). The addresses for the respectively selected memory (9), (10) are generated synchronously with the output of the digital pixels on the AD converter 5. The control element 6 looks in the control register 21 into which image memory is to be written. The control register 21 retains the information as to which image memory 9, 10 is not currently being addressed by the processor 13, ie not being used, but also which image memory is to be written by the control element 6. The image memories 9, 10 can be activated cyclically by the processor 13, for example. If there are only two image memories, they alternate between the write and read cycles. This communication takes place via the control signals 11. The image memories can now either be assigned to the processor or to the image reading unit. The assignment is made via the bus switch 12. The addresses and data of an image memory are either routed to the image reading unit (for writing new images to the memory) or to the processor unit (to process the images already read). This Control is made via the control register, which generates these control signals 22. The image data 7 are now written in an image memory 9, 10, the addresses 8 are generated by the control element of the image reading unit 6. If, for example, the image memory 9 is now written by the image reading unit, the image memory 10 can thus be connected to the processor unit. The processor unit can read this image, also copy it to another local memory 20, etc

The processor unit can be implemented in different ways. The central element is the processor 13. This can be a microcontroller, but a digital signal processor (DSP) is also possible. This processor generates the addresses in order to read the image memories 9, 10 and the local memory 20. The data bus 16 can also contain image data but also other data (for example for telecommunications). The processor is driven by a clock signal 14. If possible, the clock (14) of the processor and the clock of the image reading unit 23 are the same. The clock of the image reading unit is introduced via the control element of the image reading unit. The control register 21 is written via the processor 13. Information is written therein, such as the desired integration time of the CCD image sensor, the amplification of the analog image signal, the image memory in which the image is to be written, etc. If, for example, an image has to be recorded, the processor 13 issues an "image request" with all the necessary information Parameters are written into the control register 21. The image reading unit then immediately starts reading in the image with the desired parameters. During this reading time, the processor can calculate with the previously read in images in the image memories and does not have to wait for the new image. Results from The image processing that has been achieved with the processor can be sent as information to another system with the communication processor 18 via a bus 19. This can be, for example, an RS-232 connection or a correct industrial bus such as the CAN bus. Regarding Figure 3:

FIG. 3 shows a detailed illustration of the image reading unit, which has already been presented in FIG. 2. Another concept for parallelization is presented here. The image data can thus be transferred from the image sensor to the image memory more quickly.

The diagram shows an image sensor 31 with the associated vertical 32 and horizontal drivers 33. The image sensor can be reset with the signal 34. This means that the entire image sensor does not have to be read out before the next image can be read in again. This is shown in Figure 4. By reading out only part of the image sensor, the clock rate of the image reading can be increased. Instead of a single read-out register, there are now several read-out registers 35. Each of the readout registers is provided with its own preprocessing unit 36. The preprocessing is an analog signal processing and contains necessary elements such as variable gain called "automatic gain control", "correlated double sampling" CDS and "sample and hold" element. After each signal processing, an AD converter 7 is present which contains the analog pixel values The digital data is passed on to a logic, in order to then store the individual pixels in a frame buffer in parallel as wide data values such as 32-bit or 64-bit. The individual pixels have a width of 8 to 10 bits Reading out the image sensor with all driver elements is controlled by the control element 39. The control element also generates the addresses for storing the digital pixels in a memory and also controls the generation of the wide data words.

By parallelizing the readout registers, the pixels can now be converted in parallel and can therefore be quickly transferred to a memory as wide data words. Regarding Figure 4:

The principle of partial reading of the image window is shown in FIG. At point A it is shown how an object is shown on the far right on the edge of the picture. Now the entire image window is read out. Only after the entire window has been read out is the image sensor reset and a new image is acquired. At point B, the same object is on the right in the edge of the picture. However, only 60% of the image is read out, then the image sensor is reset and a new image can be read out. This is possible because in this case the image sensor is read in the direction where the object is. The readout register of the image sensor is immediately to the right of the object and the image is gradually shifted from one column to the next column with the vertical driver into the readout register. This is only possible if the object is on the side where the readout register is located. If the object were on the left in the image window, you would have to move the entire image to the right to the readout register. This would not achieve a higher clock frequency. In position C you can see how only 30% of the image is being read out. This is enough to read out and digitize the entire object as an image.

Claims

Expectations

1. Image processing system consisting of: - An image reading unit consisting of an electronic image sensor

(1, 31), one or more AD converters (5, 37) in order to convert the analog signals read by the image sensor (1, 31) into digital image data, - a processor unit, - at least two image memories (9, 10) , with each frame buffer from the

Image reading unit and controlled by the processor unit and can be used for reading and / or writing image data, characterized in that it has a configuration which allows the image reading unit to write a read image into a first image memory (9, 10) and a next frame in a next frame store

(10, 9) can write; as soon as all image memories (9, 10) have been written with an image, the newly acquired image is written into the first image memory (9, 10) again; At the same time as the image recording, the processor unit has access to the image memory (9, 10) which was last written by the image reading unit and currently no longer by the

Image reading unit is described.

2. Image processing system according to claim 1, in which the image reading unit is reset after reading out a certain partial window in order to read in a new image.

3. Image processing system according to claim 1, in which the image sensor (1, 31) has one or more readout channels and / or readout registers, which enables the parallel readout of the image data.

4. Image processing system according to one of the preceding claims, characterized in that the image reading unit Has communication element (55) which serves to exchange information between the processor unit and the image reading unit.

5. Image processing system according to one of the preceding claims, characterized in that the image reading unit in the read image

Comparison to a predefined threshold value binarized before the image is stored in one of the image memories.

6. Use of an image processing system according to any one of claims 1 to 5, characterized in that the image reading unit writes a read image in a first image memory (9, 10) and writes a next image in a next image memory (10, 9); as soon as all image memories (9, 10) have been written with an image, the newly acquired image is written into the first image memory (9, 10) again; At the same time as the image recording, the processor unit has access to the image memory (9, 10) which was last written by the image reading unit and is no longer being written to by the image reading unit.