SE523318C2 - Camera based distance and angle gauges - Google Patents

Abstract

A method for determining an angular position of a reflector from a vehicle reference position in relation to an angular reference position including the step of arranging vertically extended reflectors in a working area. From the vehicle reference position vertical image slices of at least one horizontal segment of the working area are obtained, each image slice comrising a plurality of pixels and intensity values of pixels in each vertical image slice are added into a set of column sums, one position in teh set forming an angular reference point. Then the angular position of a reflector is determined as the position of a peak column sum or a subset of adjacent peak column sums in the said set in relation to the angular reference point. The invention also includes a method for determining the distance between a reflector and a vehicle reference position using reflectors with a predetermined vertical extension. The number of consecutive pixels having intensity values exceeding a predetermined intensity value is calculated and based on said number of consecutive pixels the distance is calculated. A device for determining the angular position comprises an image sensing means. An adder is provided for adding intensity values of pixels in vertical image slices into a set of column sums and computing means are provided for determining the angular position of a reflector as the position of a peak column sum or a subset of adjacent peak column sums in the said set in relation to the angular reference point.

Description

25 30 523 318 2 | ø c | no so measures to continuously determine the distances of the reflectors and a reference point on the vehicle.

The use of a laser or other light source to create a beam sweeping over a work area limits the vehicle's design possibilities, as the light source should not be covered in any direction by objects on the vehicle. The laser device used in prior art systems usually requires a complex system of mirrors to wrap around the beam. One problem that exists with the mirror system is that many moving parts are used.

SUMMARY OF THE INVENTION An object of the present invention is to eliminate the problems and disadvantages mentioned above. In accordance with the invention, there has been provided a method and apparatus for determining an angular position of a reflector relative to an angular reference position. The invention can generally be used to determine the position of a vehicle. In particular, the invention can be used in a system for navigating driverless vehicles. In accordance with the invention, an image of a work area is taken. A simple and fast algorithm processes the image to greatly limit the amount of data required for the evaluation of the image. Intensity values of pixels in vertical strips are added to sets of column sums.

A high sum indicates the presence of a reflector in an angular position, which corresponds to a strip or a set of adjacent strips. A very accurate value of the angular position of the reflector can be determined by calculating the center of gravity of the adjacent strips related to a reflector.

By using center of gravity calculation, it is possible to achieve accuracy that exceeds the width of a pixel.

By using reflectors with a predetermined vertical extent, it is also possible to calculate the distance from a reference position to the reflector. A method for calculating the distance includes the measures of calculating the number of consecutive pixels in a column, or vertical strip, with intensity values exceeding a predetermined threshold value. The number of such 10 15 20 25 30 523 318 3 pixels corresponds through a non-linear relationship to the vertical extent of the reflector.

The sums of intensity values from each vertical strip are preferably stored in a first vector. The number of elements in the first vector corresponds to the number of vertical image strips that can be obtained from an image sensor, such as a CCD device. Similarly, the number of pixels with an intensity value exceeding a predetermined value is collected in a second vector. Subsequent calculations for determining the angular position of and the distance to a reflector are then performed on the basis of the contents of said first and second vectors.

In accordance with the invention, it is possible to use a conventional CCD device for capturing an image of the working area in which the reflectors are arranged. Depending on the angular working range of the CCD device, it may be appropriate to provide a CCD device in each corner of a vehicle to cover the entire circle 360 ° around the vehicle.

Preferably, so-called retro-rectifiers. Such reflectors will reflect a large part of the incident light towards the light source. Therefore, the light source is preferably arranged near the CCD device.

Additional features and advantages of the invention will become apparent from the following detailed description and the appended claims and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS A better understanding of the features and advantages of the invention may be obtained by reference to the description below and the accompanying drawings, in which Fig. 1 is a top view of a driverless vehicle in accordance with an embodiment of the invention; Fig. 2 is a side view of the vehicle in Fig. 1, Fig. 3 is a schematic circuit diagram showing an embodiment of a column summing part of the invention, Fig. 4 is a diagram showing the result of the addition in Fig. 3. in a segment of the captured image.

Fig. 5 is a schematic block diagram showing an embodiment of a threshold value and addition part of the invention, Fig. 6 is a diagram showing the result of the addition according to Fig. 5 in a segment of the recorded image, and Fig. 7 is a schematic block diagram showing an embodiment of suitable hardware in accordance with the invention.

DETAILED DESCRIPTION In the embodiment shown in Fig. 1, a driverless vehicle (AGV) 10 comprises a drive unit 11 and a lifting fork 12. The drive unit 11 comprises the electronic and hydraulic means required to operate the lifting fork.

The drive unit 11 is rectangular and an image sensing means 13, such as a CCD camera, is arranged in each corner. The angle of view of the cameras 13 is approximately 90 °, which is indicated by dashed lines. Each of the cameras 13 is connected to image processing means 14, which will be further described with reference to Figs. 3-6. The image processing means 14 is operatively connected to computing means 15, such as a computer.

The driverless vehicle 10 is designed to operate in a work area. On walls 16, which surround the work area, and on other suitable objects inside the work area, a plurality of reflectors 17 are arranged. The reflectors 17 are preferably so-called retro-reflectors, ie they reflect light efficiently in the direction of a light source. The reflectors preferably have a smaller extent horizontally than vertically, as shown in Fig. 2.

The direction of a reflector 17 is defined in relation to a reference direction D on the vehicle. As can be seen from Fig. 1, an angle defin is defined between the reference direction D and the direction of a reflector 17.

As can be seen from Fig. 2, the cameras 13 are mounted in a vertically low position. If the driverless vehicle 10 carries load 18, which is indicated by dotted lines, a low position of the cameras will allow a clear view under the load. It is also possible to arrange the cameras 13 in a different vertical position and closer together, in order to achieve a more complete point of view.

Each reflector 17 has a well-defined vertical height extension, which can be used to determine the distance between the reflector and the driverless vehicle 10, which will be described in more detail below. The cameras 13 generate a video signal, which is an input signal to a circuit shown in Fig. 3 and Fig. 5. The video signal is converted in an analog-to-digital converter 19 to provide a digital signal for further processing. It should be noted that digital cameras can also be used. In such cases, the converter 19 can be omitted. If a PAL video signal is used, the video signal comprises 625 lines and each picture is updated 25 times per second. An NTSC video signal contains 525 lines, which are updated 30 times per second. To obtain vertical strips of the image perceived by the camcorders 13, it is possible to use an image capture to produce an image containing 512x512 pixels. A line at the Video signal lasts 52 us. There is a need for 512 measuring points across each line. Each measuring point across a line corresponds to one pixel.

The analog-to-digital converter 19 is operatively connected to an adder 20, which in turn is operatively connected to a first memory means 21. The intensity value in each position of a matrix comprising 512x512 pixels is added to the corresponding intensity values of other lines in the image.

Accordingly, a vector is provided, each element of the vector containing the sum of the intensity values of a vertical strip in the image. The entire image information is condensed into a one-dimensional vector. The positions in the one-dimensional vector that correspond to a vertical strip of the image of a fl-vector will have a higher column sum value CS than other vertical strips. A high column sum CS consequently indicates the presence of a reflector in a direction corresponding to the position of the element in the vector.

Fig. 4 is a diagram showing the column sum CS of a segment in the image from pixel position (PP) 250 to pixel position 270. In approximately 10 pixel positions, from 253 to 263, there is a peak in the column sum. The top indicates the presence of a reactor in an angular position corresponding to these pixel positions PP. The width of the top illustrates that the width of a 10 15 20 25 30 523 318 6 a a n o | ao reflector exceeds the width of a pixel. This is of course also related to the distance between the reflector and the camera 13. By calculating the center of gravity of the top, shown in Fig. 4, it is possible to determine the angle of the reflector with a resolution that exceeds the pixel resolution. By designing the reflectors in the form of a parallel trapezoid with horizontal bases, as shown in Fig. 2, and thereby reducing the sharpness effect built into discrete pixel cameras, such as CCD-equipped cameras, the resolution can be further increased.

The pixel position, or rather the center of gravity position, of the top according to Fig. 4 corresponds to an angular position of a reflector in relation to the vehicle reference direction D. For cameras with an image angle that does not include the reference direction D, an angular displacement O can be used. The circuit of Fig. 3 will provide a first vector. On the basis of the first vector, the calculating means 15 can calculate with high accuracy the angle between the reference direction D of the vehicle and the reflector.

The circuit of Fig. 5 will also provide a vector, similar to the vector provided by the circuit of Fig. 3. The video signal is converted in the analog-to-digital converter 19, and all subsequent calculations are based on digital values of the picture intensity in each pixel. A threshold value corresponding to an expected intensity value from a reflector is stored in a threshold value memory 22. Each intensity value in digital form for a pixel is compared in a comparator means 25 with the value stored in the threshold value memory 22. If the incoming intensity value exceeds the threshold value , the content of that vector element is increased by one in a second addition circuit 23. The new summed value is stored in a second memory means 24. The result from the circuit in Fig. 5 will process intensity values of a Video signal to produce a second vector to result. Each element in the vector contains the calculated number of pixels in a vertical column having an intensity value exceeding the threshold value stored in the threshold memory 22.

The diagram in F ig. Fig. 6 shows the contents of a segment of the vector produced by the circuit of Fig. 5. The segment of pixel positions PP from about 253 to about 263 indicates the presence of a reactor in counter 523 318 7 see a | a u aa corresponding angular position. The diagram shows that the number of pixels with intensity values exceeding the threshold value is 70. The number of such pixels indicates the distance to the reflector, since the number indicates the height of the reflector, as visible from the camera 13.

The relationship between the number of pixels in the vector and the distance between the reflector and the camera can be expressed by the following equation. ph = k * atanvqí / Z) X where ph = height of the reflector In number of pixels h = geometric height of the reflector in meters x = distance between reflector and camera k = optical constant In one example, the optical constant k was determined to be 1180. the stand was calculated with acceptable accuracy up to about 25 m using a 0.75 m long reflector. It should be possible to measure greater distances by improving the light conditions during the measurement process.

Fig. 7 shows a more detailed diagram of a hardware that can be used in connection with the use of the invention. The video signal from the cameras 13 is used as an input signal to the analog-to-digital converter 19, as described above. The video signal is also routed to a sync signal separator 26 for line and field sync pulse signals. In the separator 26, both the line sync pulses and the field sync pulses are separated from the video signal. The sync signals are used in various elements of the hardware to control the image processing. The video signal is also used to generate a lock signal, which is used by the analog-to-digital converter 19. A synchronizing means 27 is used to synchronize the cameras and other components of the hardware. The synchronizer is operatively connected to the separator 26. The converter 19 converts the analog signal from the camera 13 into an 8 bit long digital word. At the beginning of each line, the black level of the video signal is locked with a lock signal.

The digital signal is transmitted to a comparator and adder 28.

At the comparing and adding means 28, the digital value of the intensity 10 15 20 25 30 523 318 n n a n n eo 8 is compared with a threshold value. If the digital intensity value is greater, ie. if the measured pixel is lighter than the threshold value, an accumulated threshold value sum is calculated with one for that vector position. The digital signal is also fed to an adder 29, which sums all the digital intensity values related to a vertical strip of the image.

The enumerated number of bright pixels is fed to a first FIFO memory 30 through a first latch 31. The F1FO memory (First in First Out) does not require an address bus. Instead, reading and writing take place in memory through a pulse and a clock input. An F IFO memory is a suitable form of memory in time-critical applications, such as in the present case.

Similarly, the result of the addition of intensity values in the adder 29 to a second FIFO memory 32 is passed through a second latch 33. The first and second latches are used to hold the information and thus enable the next unit to read the information.

An output of the first F1FO memory 30 is connected to a third latch 34, and similarly, an output of the second F1FO memory 32 is connected to a fourth latch 35. The third and fourth latches are also used for reset. below the first line of the image, when no accumulated value is to be added.

Several elements of the hardware require a timing signal. In the embodiment shown in Fig. 7, a finite state machine type circuit 36 is used. The time control can be divided into three parts. A first party refers to the line currently being processed, a second party refers to the position on the line and a third party controls the generation of control signals. It should be noted that the video signal also contains redundant image information. The relevant information starts a few lines from the top of the image and the information at each line starts at some distance from the synchronization signal.

Two special time management situations arise. Below the first line of the image, no information should be read from the first FIFO memory 30 and the second FIFO memory 32. In these cases, incoming digital image data should be added to zero and the result of the addition stored in a FIFO pixel pixel memory. A second problem is handling the last line of the image. The complete vector is then inserted into the first and second two-port memory means 21, 24.

The computing means 15 essentially comprises two elements. A first element is a microprocessor 37, which in a practical embodiment is a Motorola 68332 type circuit. The microprocessor 37 is operatively connected to the first memory means 21 and the second memory means 24 for processing the contents of the memories. The microprocessor 37 is also operatively connected to the image processing means 14 for receiving interrupt signals and status information, for example from the finite state machine 36.

The first memory means 21 contains a first vector, each element therein containing the number of pixels in each column which is brighter than a threshold value. The second memory means 24 contains a second vector, each element therein containing the sum of the intensities in each column.

Based on these vectors, the microprocessor 37 can calculate the relevant angular positions and distances to the reflectors.

An external control register 38 is operatively connected to the microprocessor 37. The external control register 38 generates control signals for various components of the image processing means 14. A control signal may comprise the threshold value used in the comparing and adding means 38.

The detection of the retroreflective reflectors 17 is facilitated if a light source is arranged in the vicinity of each of the cameras 13. It is desirable that this does not disturb the environment and therefore the wavelengths of light from the light source may be adjacent to or near the infrared area where the human eye have low sensitivity. The light source can be a strobe lamp or a plurality of series-connected infrared diodes. The camera is usually equipped with an electronic shutter, which is open for a short time interval (eg 1/1000 s). The short open time of the shutter makes it possible to increase the current through the diodes during the short time interval.

The infrared light source is synchronized with the camera by using the camera's frame synchronization pulse. Preferably, a separation signal is received from the separator 26.

Due to the relatively large image angle required, most lenses create a distortion of the captured image. Although the above-described evaluation algorithms still apply, the device must nevertheless be calibrated to compensate for angular errors that occur.

Claims (10)

10 15 20 25 30 523 318 -fi 11 PATENT REQUIREMENTS A method of determining an angular position of a reflector from a vehicle reference position relative to an angular reference position, comprising the step of arranging vertically extended reflectors in a working area, characterized by taking up vertical image strips from the vehicle reference position of at least one working area of the horizon. , each picture strip comprising a number of pixels, summing intensity values for pixels in each vertical picture strip to a set of column sums, a position in the set constituting an angular reference point, and determining the angular position of a reflector as the position of a peak column sum, or a subtotal of adjacent top column sums in said set in relation to the angular reference point. A method according to claim 1, also comprising the step of calculating the center of gravity of a peak formed by a peak column sum or a subtotal of adjacent peak column sums, and determining the angular position of a reflector as the position of the center of gravity. The method of claim 1, wherein the reflectors are made to a predetermined vertical extent, also comprising the steps of calculating the number of consecutive pixels with intensity values exceeding a predetermined intensity value and calculating the distance from the reference position to the reflector depending on the number of consecutive pixels. The method of claim 1, also comprising the steps of occupying the vertical image strips with a discrete pixel camera and making the reflectors as parallel trapezoids to reduce the sharpness that occurs in such cameras. The method of claim 1, also comprising the step of calibrating for such image distortion created by the camera lens. 10 15 20 25 30 523 318 12 The method of claim 1, wherein an image is captured by a camera producing an analog video signal, also comprising the step of extracting line and field image sync signals and intensity values from the video signal and transmitting the line sync signal to image processing means to associate an intensity value with a specific image. vertical strip depending on a time period lasting from the last occurrence of the line sync signal. Method for determining the distance between a reflector and a reference position on a vehicle, comprising the measure of arranging vertically extended reflectors in a working area, characterized by arranging the reflectors with a predetermined vertical extent, recording of vertical image strips from the reference position of the vehicle, which strips amount to at least one horizontal segment of the working area, each picture strip comprising a number of pixels, calculating the number of consecutive pixels with intensity values exceeding a predetermined intensity value, and calculating the distance from the reference position to the number of reflectors picture element. 8. An apparatus for determining an angular position of a vertically extending reactor arranged in a working area from a reference position of a vehicle relative to an angular reference position, characterized by imaging means for receiving from the reference position of the vehicle vertical picture strips comprising at least one horizontal segment of the working area, each picture strip comprising a picture element, an adder for summing the intensity values of picture elements in vertical picture strips into a set of column sums, a position in the set forming an angular reference point, and calculating means for determining of a reactor as the position of a peak column sum, or a set of adjacent peak column sums, in the set relative to the angular reference point. 10 15 523 318 13 nu u »~ u av The device of claim 8, wherein the image capturing means is a CCD camera. Driverless vehicle comprising a device for determining an angular position of a reflector vertically extending in a working area from a reference position of a vehicle in relation to an angular reference position, characterized by image capturing means on the vehicle for capturing vertical from the vehicle reference position picture strips amounting to at least one horizontal segment of the working area, each picture strip comprising a plurality of picture elements, an adder on the vehicle for summing intensity values of picture elements in vertical picture strips into a set of column sums, a position in the set forming an angular reference point, and calculating means on the vehicle for determining the angular position of a reflector as the position of a peak column sum or a set of adjacent peak column sums in the set relative to the angle reference point.