JP2731681B2 - 3D measurement system - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional measuring system for irradiating a target with slit light, receiving the reflected light with a two-dimensional image sensor, and measuring the three-dimensional shape of the target at high speed. About.

[0002]

2. Description of the Related Art In general, three-dimensional recognition ability is required as a visual function required for automatic assembly in a factory, positioning and shape determination in automatic inspection, position identification and route guidance in a mobile robot. I have. Alternatively, three-dimensional measurement is also required for drawing as CAD (computer-aided design) data of existing products, creating real scene data in artificial reality in computer graphics, and the like. In particular, when a three-dimensional measurement system is applied to a mobile robot or the like as a non-contact three-dimensional visual sensor, high-speed measurement is desired as much as possible.

[0003] For three-dimensional measurement, passive measurement such as a stereo method (also referred to as binocular stereovision) with high versatility, and controlled energy are projected onto an object to be measured, and the behavior of the reflected energy is measured. There is active measurement. In active measurement, light is often used as the projected energy, and the process is simple if the reflected light of the projected light is sufficiently large compared to the background light. Can measure distance with high accuracy. In this active type measurement, there are many reports as described below because the measurement method using the principle of triangulation by replacing one of the two cameras in the stereo method with a projector is simple and easy to carry out.

A method of projecting a laser beam onto an object to be measured, and obtaining a three-dimensional position from images taken from cameras placed at different angles from spots formed thereon, is a spot light measurement method (also referred to as a spot light projection method). ). In this measurement method, two-dimensional scanning with a laser beam is performed to measure the entire object, and one image is required for one point measurement, so that much time is required for the measurement. Therefore, if a PSD (Position Sensitive Detector: a position detection element that outputs an analog signal proportional to the position of the incident spot light) is used instead of a camera, measurement per point can be performed at high speed, and distance measurement of the entire target object can be performed. Can be done at high speed. Canada NC
Company R has developed a system that combines spot light scanning and one-dimensional PSD, and has demonstrated that high-speed, high-accuracy range data can be obtained. This configuration example is described in M. Riox's paper (M. Rioux, “Laser Range Finder Based on”
Synchronized Scanners (Laser Range Finder Based on Synchronized Scanner) ”, Applied Optic
s, vol.23, no.21, pp.3837-3844, 1984).

However, since the above-mentioned PSD is easily affected by the background light, it is necessary to limit the measurement environment such as by setting the measurement place in a dark room. On the other hand, when the slit light is projected on the object to be measured instead of the spot light, the three-dimensional position of the slit image appearing along the scanning line by one image input can be obtained at many points, so the efficiency is higher than the spot light. good. An example is Shirai's paper (Y. Shirai, "Recognition of Polyhedra wi
th a Range Finder (Polyhedron Recognition Using Range Finder) ", Pattern Recognition, vol.4, no.2, pp.24
3-250, 1972). An outline of this method is shown in FIG.

In FIG. 1, when the slit light is at an angle θ _i from the reference position, the three-dimensional coordinates (X, Y, Z) of the image Pk (xk, yk) are given by the following equation (1).

X = xk * L / (xk + f * tan θ _i ) Y = f * L / (xk + f * tan θ _i ) Z = yk * L / (xk + f * tan θ _i ) (1) where f is a pinhole camera Focal length (fixed value),
L is a distance (fixed value) between the camera lens and the projector lens of the slot light source. K is a constant corresponding to the interval between the pixels. This method is called a slit light measurement method or a light cutting method (also referred to as a linear light projection method).

However, even in this slit light measurement method, in order to measure the entire object, one-dimensional scanning of the slit light is required, and one image is required for measuring one slit light. Takes time. Therefore, when a plurality of slit lights are projected on the object, it is impossible to identify the projected slit light and the slit image seen from the camera. To solve this problem, an algorithm was developed to solve the identification problem using the same principle as three-eye viewing using another camera.
This method is based on a paper by the present inventors (T. Echigoand M. Ya
chida, `` A Fast Method for Extraction of 3-D Inform
ation using Multiple Stripes and Two Cameras (a rapid method for extracting three-dimensional information using multiple slit lights and two cameras), IJCAI-85, pp.155-159, 1985)
Is disclosed.

There is also an apparatus which gives a color to each of a plurality of slit lights, determines the slit light by determining the color from an image captured by a color camera, and obtains range data at high speed. An example of this configuration is shown in the paper "Rainbow
Range Image Acquisition by Range Finder ”, Jikken Kenho Computer Vision, vol.8
6, no.44-3, 1986.

Further, a method has been developed in which a binary coded pattern in which the light and dark pitches are doubled from a projector is projected in time series, the space is coded, and the same effect as when a large number of slits are projected is obtained. Was done. In this method, since 2 ⁿ slit images are obtained by inputting n images, range data can be obtained in a relatively short time. An example of the configuration is shown in the paper "Liquid crystal range finder-" by Sato and Iguchi.
High Speed Range Image Planning System Using Liquid Crystal Shutter ", IEICE, vol.J71-D, no.7, pp.1249-1257, 1988.

Since the conventional measuring method described above is based on the premise of image input, the video frame time has reached the limit of the measuring time. Therefore, a method has been developed in which each of the image sensors corresponding to pixels has a function of detecting slit light and holding / transferring the projection angle of the slit light, two-dimensionally arranging them, and integrating them on a semiconductor chip. Was. This method fixes the line-of-sight direction for each pixel of the image sensor, detects the peak value within the scanning time of the slit light, and holds the projection angle of the slit light at that time, so that the intersection of the slit light and the line of sight Since a certain distance calculation can be performed, the effect of the background light can be suppressed, and there is an advantage that the environment need not be limited as in a dark room. Further, according to this method, range data can be obtained in the scanning time of the slit light, so that there is high speed.
For example, at Carnegie Mellon University (CMU), photo sensors, amplifiers, and sample and hold circuits
We are developing a chip sensor with 28 pixels. The configuration example is described in A.I. A paper by Mr. Glass et al. (A.Gruss, LRCarle
y, and T. Kanade, "High Speed VSLI Range Sensor", CMU, 1990).

Osaka University has also developed an equivalent system using discrete circuits. An example is “Real-time range image sensor” by Kida, Sato and Iguchi, No. 5.
Times sensing forum. pp. 91-95, 1988.

[0013]

However, in these conventional systems (the system of Carnegie Mellon University and the system of Osaka University), the area of the light receiving section of each pixel is reduced because each pixel has a sample hold circuit. Since the pixel becomes narrow and the pixels become sparse, resolution cannot be sufficiently obtained, and dense range data (depth data in a certain area, that is, three-dimensional distance data) of the object cannot be obtained. Further, in these systems, since all processes including time measurement are performed in analog, it is necessary to compensate for non-linearity of a circuit (particularly a circuit for time measurement) and a change in characteristics due to temperature. In addition, it is generally difficult to compensate for all of them with high accuracy. Also,
In these systems, the circuit occupied by one pixel is complicated, so there is a limit in reducing the chip size,
When the size is reduced, the sensitivity of the light receiving unit becomes a problem, and there is a point that the aperture ratio cannot be increased.

[0014] In addition, using a slit light for scanning a scene (a measurement object, for example, a part of a room) and an array type PSD (position detecting element), the three-dimensional shape of the object is reduced to one.
A non-contact three-dimensional visual sensor system capable of continuous measurement with a period of 30 seconds has been developed in a joint research between Chukyo University and Matsushita Giken, and "High Spead and ContinuousRan by Araki et al.
gefinding System (High-speed, continuous range finding system) "IEICETRANSACTIONS, VOL.E 74, NO.10,
PP.3401-3406, 1991. The feature of this visual sensor system is that a newly developed array type PSD is used as an image sensor. This PSD is
The vertical direction is divided into 128 rows, and the light receiving position can be detected with a resolution of 1/128 in the horizontal direction. As a result, 128 points arranged in the vertical direction in one image pickup are set to 1/40 as a two-dimensional image.
It can be read in 00 seconds. When the sensor information is read 128 times while scanning the slit light using this image sensor, 128 slit images (images of the slit light) projected on the object can be obtained.
It is reported that it can be measured in 32 seconds. Note that the distance to the object is calculated between the image sensor and the slit light source by the triangulation method.

However, in the image pickup device using the PSD as described above, the position of the slit light is read in all the horizontal rows each time the image is read, but at this time, there is a sufficient difference between the background and the light receiving position of the slit light. There must be. PSD
In this method, the position of one point where slit light is received is output for each column, but a sufficient amount of light received at that point and a low amount of light received at other positions must be secured. However, since the reflected light of the background and the slit light differs depending on the target object, measurement conditions such as darkening the surroundings and changing the sensitivity depending on the target object are necessary when acquiring data. Thus, the PSD
In some systems, measurement cannot be performed depending on the brightness of the background and the intensity of the reflected light of the slit light with respect to the background, and the application target is greatly limited.

Accordingly, an object of the present invention is to solve the above-mentioned problems of the prior art, and to obtain a slit image of the surface of a three-dimensional object obtained from an imaging mechanism and a projector of slit light from a camera to an object surface. An object of the present invention is to enable high-speed detection of a range image.

Another object of the present invention is to detect a range image at high speed while obtaining good resolution, sensitivity and accuracy.

Still another object of the present invention is to not particularly limit the measurement environment, that is, to enable measurement regardless of the brightness of the background and the intensity of the reflected light of the slit light with respect to the background.

[0019]

In order to achieve the above object, a three-dimensional measuring system according to the present invention comprises: a slit light projecting means for projecting slit light onto a measurement object while changing its direction; The two-dimensional imaging device having a matrix arrangement that receives reflected light of the slit light and outputs signals in parallel from a plurality of output terminals, and for each minute change in the direction of the slit light, the two-dimensional imaging device A signal conversion means for extracting signals in parallel and converting the signals into digital signals, and the digital signals transferred from the signal conversion means are input in parallel, and each of the two-dimensional imaging devices is scanned during one scan by slit light. It is characterized by comprising a detecting means for detecting the direction of the slit light at which the brightness at the pixel becomes maximum.

Further, according to the present invention, the detection means comprises a plurality of circuits arranged in parallel, and each circuit obtains a digital signal in each pixel of an image read from the two-dimensional imaging device up to the previous time. Comparing means for comparing with the maximum value of the digital signal of the pixel, and storing the larger signal determined by the comparison by the comparing means, and using the stored signal data for the next signal comparison. 1 F
An FIFO buffer; and a second FIFO buffer for storing the direction data of the slit light when the comparing means determines that the current signal is larger than the maximum value of the previous signal. It is characterized in that the maximum output value of each pixel during scanning and the direction of slit light at that time are recorded.

[0021] The present invention preferably further includes a computer graphic device for creating and displaying an image based on the distance data calculated by the calculation means.

[0022]

The two-dimensional image pickup device used in the conventional television camera sequentially reads the data of the two-dimensional array sequentially from one place.
Spending seconds. On the other hand, a line sensor used in a facsimile or a scanner is a product that reads data of a one-dimensional array of 512 pixels in 1/25 milliseconds, and higher-speed elements have been developed.

Therefore, in the present invention, a plurality of one-dimensional imaging devices such as line sensors are arranged in parallel to read data in a two-dimensional array, and reading data in parallel allows reading a two-dimensional image of one device. The reading speed is used to increase the speed. For example, if n line sensors of 512 pixels described above are arranged, a two-dimensional image having a resolution of n * 512 is divided by 1 /.
It can be read in 25 milliseconds.

However, since only one distance data along one slit image can be obtained from one slit light, a scene (measurement target) is densely scanned with the slit light to obtain a dense distance image. I have to do it. According to the imaging device proposed in the present invention, a plurality of slit images can be read out densely even when the slit light is scanned at high speed. However, it is necessary to record the direction of the slit light. Therefore, the present invention has means for reading out each pixel value at a high speed while scanning the slit light, determining whether or not the slit light is applied, and simultaneously recording the direction of the slit light at that time.

As an example, in the embodiment of the present invention described below, the maximum value of the brightness and the direction of the slit light at that time with respect to the time change in each pixel are determined using a FIFO (fast-in fast-out) memory. Recorded.
That is, a plurality of one-dimensional line sensors are arranged in parallel, signals are taken out in parallel from the plurality of output terminals, and further,
On the output side, the read signal is compared with the maximum value of the signal obtained up to the previous time, and the larger signal is put into the FIFO buffer. The data stored in the FIFO buffer is used for comparison with the next signal. Therefore, the brightest value (ie, when the slit light is projected) and the direction of the slit light at each pixel during one scan of the scene with the slit light are recorded. In this way, the maximum output value and the direction of the slit light at that time are recorded among the plurality of images obtained during one scan of the slit light, and the distance to the object is calculated by the triangulation method based on the recorded data. Is done. Therefore, in each pixel, it is sufficient that the difference between when the slit light is applied and when it is not applied can be detected, and it is not particularly necessary to darken the surroundings. Further, since distance information can be obtained for all pixels, dense distance image data can be obtained.

[0026]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 2 shows a main configuration of a three-dimensional measuring system according to one embodiment of the present invention. This three-dimensional measurement system includes a two-dimensional image sensor 1 as an imaging device that outputs analog video signals in parallel from a plurality of output terminals, and a plurality of analog-to-digital (A / D) converters 2 that digitize these video signals. Elements 3 to 7 described later with reference to FIG. 3 are connected to the respective A / D converters 2.

FIG. 3 shows a detailed configuration of a circuit connected to one output terminal of the imaging device. A fan-shaped slit light emitted from a laser slit light source 8 having a cylindrical lens at a laser output port for converting into a slit light irradiates an object 10 to be measured while its direction is controlled by a rotating mirror 9. The reflected light image from the object 10 is condensed on the two-dimensional image sensor 1 by the imaging lens 11.

As shown in FIG. 4, the image sensor 1 is a two-dimensional matrix array image pickup device in which a plurality of line sensors having a plurality of photosensors are arranged in parallel. Can be taken out in parallel. The image sensor 1 of FIG. 4 has 256 × 128 pixels, and the number of lines (n) is 128. The output of one pixel from the image sensor 1 is an analog voltage corresponding to the illuminance (light reception amount) of the pixel, and is sequentially read out by a CCD (charge coupled device) or FET (field effect transistor) switch. The relationship between the output and the amount of received light may be a monotonic function, and may not be proportional. This output is A
A / D conversion is performed by the / D converter 2, and thereafter, all are treated as digital signals. Therefore, if monotonicity can be guaranteed at the output section of the image sensor 1, no error will occur in the subsequent sections. With this configuration, there is no problem of linearity or temperature compensation of the imaging device at all, as compared with a system of a type in which all analog processing is performed in a chip, so that highly accurate measurement can be performed. Data of each pixel can be read in synchronization with the reference clock f _c in the system can read the data of one pixel per clock.

The rotating mirror 9 is rotated and controlled by a pulse from the pulse generating circuit 12 through a pulse motor (not shown), thereby the direction of the slit light can be changed on the step in units of theta _i. This pulse is also counted by the pulse counter 13, and the count value is supplied to the second data selector 6 as data indicating the current direction of the slit light. Furthermore, this pulse is multiplied is changed to the reference clock f _c of the by the shift clock generation circuit 14.

The intensity FIFO 4 is a buffer memory for extracting the maximum luminance value of each pixel, and the orientation FIFO 7 is a buffer memory for storing the direction data of the slit light at the maximum luminance. These FIFO4 and 7 both have a number d _n equal to the number of pixels (photodetector) image sensor 1 arranged in the direction corresponding to the scanning direction of the slit light (see FIG. 2), in synchronization with the reference clock f _c in the system Perform fast-in fast-out write / read operation. The output data from the A / D converter 2 is supplied to the comparator 3 and the first data selector 5, and the count value of the pulse counter 13 is supplied to the second data selector 6. This count value indicates the reflection angle of the slit light, and does not change until all the pixels are transferred.
Comparator 3 has an intensity FIFO for each pixel
4 and the maximum luminance value a and A / D up to the previous time
Brightness value b representing the current amount of received light sent from converter 2
And compares the magnitude relationship information (that is, an indication of which input is greater) with the two data selectors 5,
Send to 6.

Each of the data selectors 5 and 6 selects one of the two inputs a and b according to an instruction from the comparator 3 and outputs it to the output c. The rule of this selection is that when the input a of the comparator 3 is larger than b, the two data selectors 5 and 6 select their own input a (previous value), otherwise (when a ≦ b) Select b (current value) of your input. Input a of data selector 6
Is supplied with the data output from the orientation FIFO 7, and the input b thereof is always supplied with the current direction of the slit light.

The operation of the above-described circuit will be described in more detail along the actual processing flow.

Processing 1. First, all the contents of the intensity FIFO 4 and the orientation FIFO 7 are set to 0 as initialization. Also, the direction of the slit light is brought to the initial position.

Processing 2. An image capture in the image sensor 1, takes out the luminance data pixel by pixel from the image sensor 1 in synchronization with the reference clock f _c, while applying the above selection rule, moving the Intenshiiti FIFO4 and orientation FIFO 7, the image sensor All pixels are transferred. Intensity FI as described above
Since the initial value of FO4 was 0,
After finished transmission of the number of pixels d _n lines, the contents of Intenshiiti FIFO4 has a value of all A / D converter 2. Further, the contents of the orientation FIFO 7 are all initial positions in the direction of the slit light.

Processing 3. One direction of slit light (θ _i )
Proceed, processing 2. Similarly to the image capture in the image sensor 1, taken out reference clock f by one pixel in synchronization with the _c, while applying the above selection rule, moving the Intenshiiti FIFO4 and orientation FIFO 7, the transfer of all the pixels of the image sensor I do. When this is done, the value of the intensity FIFO 4 is compared with the value of the brightness of the current image sensor, and if the value of the brightness of the current image sensor 1 is larger, the intensity FIFO 4
4 and the value of the orientation FIFO 7 are replaced by new values.

Processing 4. 2. Processing until the direction of the slit light reaches the end position repeat.

Processing 5 When the direction of the slit light reaches the end position, the measurement result is taken out from the intensity FIFO 4 and the orientation FIFO 7.

When the above processing 1 to processing 5 are completed, the value of the maximum luminance in each pixel enters into the intensity FIFO 4, and the direction of the slit light at the time when the maximum luminance is obtained enters the orientation FIFO 7. . That is, the value of the orientation FIFO 7 is in the direction in which the slit light impinges on the object 10 corresponding to each pixel.

FIG. 5 shows the relationship between the change in the output of one line sensor in the image sensor 1 and the contents of the intensity FIFO 4 to which this output is transferred. As can be seen from FIG. 3, the change in the direction of the slit light (θ _{1 to} θ _n )
Along with this, the slit light irradiation position of the object 10 moves,
Since the reflected light image on the image sensor 1 also moves, the pixel position of the maximum luminance also moves sequentially as shown in FIG. Therefore, when the scanning of one scene is completed, the value of the maximum luminance in each pixel is stored in the intensity FIFO 4 as shown in FIG. As described above, since only the value of the maximum luminance in each pixel is detected, it is sufficient to consider the difference between when the slit light is shining on the image sensor 1 and when the slit light is not illuminated. Since only light is not detected, there is no need to darken the surroundings. In addition, since the laser light generally has a sufficiently higher light amount than the ordinary ambient light, the above-mentioned consideration is not usually necessary.

Processing 6 By the way, it is necessary to perform correction for removing background noise when reflection of slit light is not obtained because the object 10 corresponding to a certain pixel is at an infinite point. FIG. 6 shows a configuration example of a circuit for this correction. Pixel data that does not exceed a certain threshold value It (intensity output) in the comparator 21 is not affected by slit light, that is, the data is determined to be background noise, and a process of invalidating the data is performed. The calculation for obtaining the distance from the direction of the slit light (orientation output) is performed by the CPU 23 based on the above-described equation (1) of the triangulation method. Note that this calculation can be performed by a look-up table.

That is, the output from the intensity FIFO 4 and the luminance threshold It are compared with the comparator 2.
Compared with 1, if the output from the intensity FIFO 4 is smaller, the value from the orientation FIFO 7 cannot be calculated out-of-range (for example, 0).
Or -1 or ∞). The distance data calculated by the CPU 23 is sent to a computer graphic system (not shown), and is subjected to known interpolation processing, surface creation processing, and the like, and an image is displayed on a screen of a monitor device such as a CRT display (not shown). it can. At that time, the grayscale data of the pixels stored in the intensity FIFO 4 can also be sent to the computer graphic system, and the grayscale state of the surface can be displayed at the same time.

Although a specific example for measuring a one-dimensional distance image has been described with reference to FIGS. 3 and 6, the case where a two-dimensional distance image is obtained by the configuration of FIG. 2 is basically the same as described above. In the processing procedure (processing 1 to processing 6), a distance image can be obtained quickly and accurately by parallel processing.

[0044] Further, the time t _s required to obtain a distance image of one screen in the measurement system of the present invention, the number of movement of the slit light ((slit light in the direction of the end position - the initial position in the direction of the slit light ) / θ _i) a s _n, the number of pixels d _n of the line sensor, when the transfer rate of one pixel to f _c, the _{t s = d n s n /} f c ... (2). For example, f _c = 20 [MHz] ... (3) If _{d n = 256 ... (4)} s n = 256 ... and (5), and t _s ≒ 0.3 seconds ... (6). As described above, according to the present invention, a range image can be obtained at a very high speed while using a solid-state imaging device.

(Other Embodiments) In the above-described embodiment of the present invention, the reference clock f for driving the image sensor 1 and the like is obtained by multiplying the pulse generated from the pulse generation circuit 12 using the shift clock generation circuit 14. is configured so as to produce a _c, the present invention is not limited to this, the shift clock of the shift clock generator is usually provided for example in an image sensor 1 as the reference clock f _c F
The IFO and the like may be driven, and the clock may be frequency-divided to generate a pulse to be supplied to the rotating mirror or the pulse counter. The point is that the timing of rotating the rotating mirror and the driving timing of the image sensor need only be matched, so that separate independent clock generation means can be used to synchronize at the time of initialization of measurement start with a trigger signal. Operation is obtained.

In the above embodiment, two FIFO memories are used in order to obtain the maximum brightness of each pixel within the scanning time of the slit light and the projection angle of the slit light at that time. Instead of using dedicated hardware, for example, as shown in FIG. 7, software can be realized by address control using general-purpose memories (for example, IC-RAM) 28 and 29.

In the system of the embodiment shown in FIG. 7, one of the two memories is used as an intensity memory 28 and the other is used as an orientation memory 29. Each of the line sensors constituting the image sensor 1 has a memory 28,
A series of 29 addresses are secured, and a memory control circuit 30 for accessing the addresses is prepared. Memory 28, 29 assigned to each line sensor
Are the same. An address generation circuit 31 included in the memory control circuit 30 receives a shift clock, which is a synchronization signal for pixel reading, and sequentially and cyclically generates addresses of the memories 28 and 29 assigned to the corresponding line sensors.

The memory control circuit 30 includes an address generator 31
, A comparator 32, a set of selectors 33, 3
4, including a read buffer 35 and a write buffer 36. Comparator 32 and first and second selectors 3
Reference numerals 3 and 34 correspond to the comparator 3 and the selectors 5 and 6 in FIG. The read buffer 35 temporarily stores read data from the memories 28 and 29, and the write buffer 36 temporarily stores write data to the memories 28 and 29. Therefore, the circuit including the address generator 31, the read buffer 35, the write buffer 36, the intensity memory 28, and the orientation memory 29 corresponds to a pair of FIFOs 4 and 7 in FIG.

In the embodiment shown in FIG. 7, each time a pixel is read, the addresses of the memories 28 and 29 are updated using the address generator 31, and the contents of the read buffer 35 read from the intensity memory 28 are used. Two memories 2
The comparator 32 determines whether or not the contents of 8, 29 should be rewritten.
Is selected using a set of selectors 33 and 34, and when it is selected to rewrite, the current input data from the A / D converter 2 and the current input data from the pulse counter 13 are transmitted through the write buffer 36 to the intensities. Memory 2
8. The data is sent to the orientation memory 29 and written.

The rotating mirror for deflecting the slit light is not limited to a flat mirror, but a polygon mirror is also suitable when continuous measurement is required as a non-contact three-dimensional visual sensor. Also, without using such a mirror, a laser slit light source is mounted on a pulse motor driven turntable so that the slit light emission port coincides with the center of rotation, and the direction of the slit light is deflected by rotating the turntable. The same operation can be obtained even if the above operation is performed.

Further, the object to be measured is placed on a turntable and rotated by a predetermined angle corresponding to the deflection range (for example, 60 degrees) of the slit light, or the measuring device itself is rotated around the object. By doing so, three-dimensional measurement of the entire surface of the object can be quickly performed.

The line sensor constituting the image pickup device used in the present invention is not limited to the CCD type, but is
Various existing types of solid-state image sensors such as the OS type can be used. Recent developments in packaging technology have made it easier to provide more than 500 terminals from a single semiconductor chip, so that current technology can easily realize about 256 parallel reads.

As a light source of the slit light, other electromagnetic waves other than the laser light can be used. However, the light does not diffuse, the thickness of the beat does not change depending on the depth, it is safe for humans, Visible laser light that satisfies conditions such as ease is considered most practically preferable.

The three-dimensional information obtained by using the three-dimensional measurement system of the present invention is used for positioning and shape discrimination in automatic assembly and automatic inspection in a factory. This is useful information for identification and return to route planning. To promote automation, safety and reliability are important.Compared to conventional linear motion-only automatic equipment, robots with more degrees of freedom need to know a wide range of external information at high speed and with high accuracy. The three-dimensional system of the present invention is effective for that purpose. Further, there is a demand for input of existing products as CAD data. For example, in the automotive industry, the shape data of other companies' engines are input to CAD in order to compare them with their own products. However, conventionally, contact-type or non-parallel processing three-dimensional measuring devices were used, so data input It took time. The high-speed data extraction of the three-dimensional measurement system of the present invention can meet the demands of such development and design departments. Further, in applications in computer graphics, usefulness as data provision of real scenes in artificial reality is greatly expected. However, most of the currently used apparatuses use a single slit light, and there is a disadvantage that it takes time to extract data of the entire target. For non-rigid objects such as clothing, instantaneous data extraction is indispensable, and high-speed range sensors are expected. The system of the present invention is fast, without limiting the environment,
Since high-accuracy range data can be extracted, it can be expected to be used in a wide range of fields, such as factories and offices, such as provision of actual scene data in the above-described artificial reality.

According to the embodiment of the present invention described above, the following effects can be obtained.

(1) For each minute movement of the slit light, the signal of each column of the two-dimensional imaging device is taken out in parallel, digitally converted, and the direction of the slit light at which the brightness reaches a peak at each pixel when the slit light is fully scanned is determined. Since detection is performed, by combining an imaging mechanism combining an existing imaging element with a mechanism that detects the direction of slit light at a high speed, a relatively simple configuration is inexpensive, and very quickly, and a high-precision distance image ( Range data).

(2) After the brightness value of each pixel is digitized, all are processed in a digital amount, so that they are not affected by the non-linearity of the circuit or noise. Also, the photodetector of each pixel and the transfer path from it (CCD or FET)
The characteristic of the switch may be non-linear as long as it is monotonic, and an error in the analog circuit does not affect the measurement result. In addition, since an existing image pickup device in which only an image pickup function is integrated in the image pickup unit is used, characteristics such as an aperture ratio and a transfer speed of the image pickup device can be used, and the size of the image sensor can be reduced.

(3) The video signal at each pixel of the image read from the two-dimensional imaging device is compared with the maximum value of the video signal of the corresponding pixel obtained up to the previous time, and the larger signal is put into the FIFO. The data stored in the FIFO is used for comparison of the next video signal, and the maximum output of each pixel during scanning of the slit light among a plurality of images obtained during one scanning of the slit light is used. The value (the brightest value) and the direction of the slit light at that time are recorded, and the distance to the object is calculated by triangulation using the recorded data, so the measurement environment is limited to dark rooms. In other words, the operating environment of the image sensor is sufficient, and it is only necessary to detect the difference between when the slit light hits each pixel and when the slit light does not hit it. Therefore, measurement can be performed under general light regardless of the brightness of the background and the intensity of the reflected light of the slit light.

[0059]

As described above, according to the present invention, three-dimensional measurement can be performed at high speed. Further, three-dimensional measurement can be performed at high speed under general light regardless of the brightness of the background and the intensity of slit light with respect to the background. Further, the resolution, sensitivity to laser reflected light, and accuracy of the obtained distance data can be improved.

[Brief description of the drawings]

FIG. 1 is a conceptual diagram showing the principle of performing three-dimensional measurement by triangulation using slit light.

FIG. 2 is a block diagram illustrating a main configuration of a three-dimensional measurement system according to an embodiment of the present invention.

FIG. 3 is a block diagram illustrating a detailed configuration of a circuit connected to one output terminal of the image sensor of FIG. 2;

FIG. 4 is a conceptual diagram illustrating a configuration example of the image sensor of FIG. 2;

FIG. 5 shows a change in output of one line sensor of the image sensor of FIG. 3 and an intensity F to which the output is transferred;
FIG. 7 is a waveform diagram showing a relationship with the contents of an IFO.

FIG. 6 is a block diagram showing a configuration of a correction circuit for removing background noise when slit light reflection is not obtained in one embodiment of the present invention.

FIG. 7 is a block diagram showing a main part circuit configuration in another embodiment of the present invention.

[Explanation of symbols]

 Reference Signs List 1 2D image sensor 2 A / D converter 3 Comparator 4 Intensity FIFO 5, 6 Selector 7 Orientation FIFO 8 Laser slit light source 9 Rotating mirror 10 Object to be measured 11 Imaging lens 12 Pulse generation circuit 13 Pulse counter 14 Shift clock Generator 21 Comparator 22 Selector 23 CPU 28 Intensity memory 29 Orientation memory 31 Address generator 35 Read buffer 36 Write buffer

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Tomio Echigo 5-19 Sanbancho, Chiyoda-ku, Tokyo IBM Japan, Ltd. Tokyo Basic Research Laboratory (56) References JP-A-62-228106 (JP) , A) JP-A-2-223811 (JP, A)

Claims (11)

(57) [Claims] 1. A slit light projecting means for projecting slit light onto a measurement object while changing its direction, receiving reflected light of the slit light from the measurement object, and outputting signals from a plurality of output terminals in parallel. A two-dimensional imaging device having a matrix arrangement to perform, signal conversion means for taking out a signal of each column of the two-dimensional imaging device in parallel for each minute change in the direction of the slit light, and converting the signal into a digital signal; Detecting means for detecting the direction of the slit light at which the brightness at each pixel of the two-dimensional imaging device is maximized during one scan by the slit light by inputting the digital signals transferred in parallel from one another. A three-dimensional measurement system characterized by: 2. The apparatus according to claim 1, further comprising calculating means for calculating distance data of the object to be measured by a triangular metric method based on data on the direction of the slit light detected by the detecting means. The three-dimensional measurement system as described. 3. The image processing apparatus according to claim 2, further comprising a correction unit connected between the detection unit and the calculation unit, the correction unit invalidating direction data of the slit light when the slit light is not received. 3. The three-dimensional measurement system according to 2. 4. The detecting means comprises a plurality of circuits arranged in parallel, each circuit obtaining a digital signal at each pixel of an image read from the two-dimensional imaging device by a corresponding pixel obtained up to the previous time. And a first FIFO buffer for storing the larger signal determined by the comparison by the comparing means and using the data of the stored signal for the next signal comparison. And a second FIFO buffer for storing the direction data of the slit light when the comparison means determines that the current signal is larger than the maximum value of the previous signal. 4. The three-dimensional measurement system according to claim 1, wherein the maximum output value of each pixel and the direction of the slit light at that time are recorded. 5. The two-dimensional imaging device according to claim 1, wherein a plurality of one-dimensional line image sensors including a plurality of photosensors are arranged in parallel, and each of the line image sensors is provided with an output terminal for reading a signal. The three-dimensional measurement system according to any one of claims 1 to 4, wherein: 6. The apparatus according to claim 1, further comprising a driving signal generating unit for generating a driving signal for driving the slit light projecting unit, the two-dimensional imaging device, and the detecting unit in synchronization with each other. The three-dimensional measurement system according to any one of the above. 7. The apparatus according to claim 1, wherein said slit light projecting means comprises a laser slit light source for generating laser slit light, and a rotating mirror for deflecting said slit light. 3D measurement system. 8. The three-dimensional measurement system according to claim 1, further comprising a computer graphic device for creating and displaying an image based on the distance data calculated by said calculation means. 9. A mobile robot, an automatic assembly machine, an automatic inspection machine, and a transfer device, comprising the three-dimensional measurement system according to claim 1 as a non-contact three-dimensional visual sensor. Equipment. 10. A means for minutely moving the position of slit light with respect to the object to be measured, a means for causing a two-dimensional imaging device to receive reflected light of slit light from the object to be measured, and for each minute movement of the slit light. Means for extracting a signal from the two-dimensional imaging device and converting the signal into a digital signal; and inputting the digital signal, and the slit light having a maximum brightness at each pixel during one scan by the slit light. A three-dimensional measurement system, comprising: means for detecting a direction; and means for calculating distance data of the object to be measured by triangulation based on the detected direction of the slit light. 11. A minute movement of a position of the slit light with respect to the object to be measured, a reflected light of the slit light from the object to be measured is received by a two-dimensional imaging device, and the two-dimensional image is received every minute movement of the slit light. The signal of the imaging device is taken out, converted into a digital signal, and the digital signal is input, and the direction of the slit light at which the brightness of each pixel is maximized during one scan by the slit light is detected. A three-dimensional measuring method, wherein distance data of the object to be measured is calculated by a triangulation method based on the direction of the slit light thus obtained.