JP3269222B2 - Distance measuring device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a distance measuring device,
In particular, the present invention relates to a distance measurement device suitable for application to an image processing device that recognizes a three-dimensional shape of a target object such as an industrial part.

[0002]

2. Description of the Related Art Conventionally, various methods and apparatuses for obtaining distance information of an object in an image have been developed for the purpose of recognizing an obstacle of a self-supporting vehicle, the shape of an object in the field of factory automation, and the like. That is, three-dimensional information is to be obtained from image information that is two-dimensional information.

Conventionally, stereo vision has been most commonly used to obtain distance information from an image to an object. In this stereo vision, an image captured by a TV camera located at two separate positions is input, and a position where the same part of the same object is captured is obtained from the two images (corresponding point search).
The distance to the object is obtained from the relationship between the point positions and the positions of the two TV cameras by the principle of triangulation. However, the method of obtaining a distance based on the principle of triangulation involves the following contradictory problems. In other words, while the distance between the two cameras increases, the distance measurement accuracy improves. On the other hand, if the two cameras separate, the deformation of the object in the left and right images, the difference in brightness, and the left image can be seen but not the right image It is very difficult to perform a corresponding point search for finding the same part in the left and right images. If the two camera positions are close to each other, the corresponding point search is relatively easy, but the distance measurement accuracy deteriorates.

[0004] Various studies have been made of a passive stereo system that obtains a depth, that is, a distance image by detecting a correspondence relationship between left and right gray-scale images and detecting parallax from the corresponding left and right same portions.
-Dimensional Object UsingStereo `` IEEE J. Robotics
and Automation, RA3, 4, 1987, pp. 362-373 (Reference 1), "A Computational Theo" by D. Marr and T. Poggio
ry of Human Stereo Vision '' Rpoc.R.Lond., B204, pp.3
01-328 (Reference 2), "Stereo correspondence search by two-step dynamic programming considering the consistency between scanning lines" (Reference 3), "Segment correspondence search processing in stereo images" Various studies have been made, such as "Speeding up" (Ref. 4). The methods used in these methods can be broadly divided into region-based methods and feature-based methods. less than,
Each of these conventional methods will be briefly described. The method based on the area-based method is a method of obtaining a corresponding area by correlating a gray value between certain areas on a screen. In this method, if the area to be correlated is large, the resolution is reduced and the calculation cost is increased. Conversely, if the area is small, a multi-correspondence problem corresponding to a plurality of areas occurs. Further, when the target surface is not parallel to the screen, the patterns in the left and right regions do not exactly match. The feature-based method is a method of extracting features from an image and associating the features with each other. The features to be extracted are roughly classified into edge segments having a certain length and features in point units shorter than the edge segments. In the case of an edge segment, the similarity of the shape of the segment in the left and right images is a major problem in association, but when the target is complicated, the shape of the segment in the left and right images does not always match. Such a problem does not occur because the point-by-point feature is local,
A multi-response problem arises because a plurality of similar features appear. The technique of performing matching using the dynamic programming (see References 3 and 4) is not effective for images in which the order of partial regions is reversed. Further, the method based on the stochastic relaxation method (see Reference Document 1) does not have a clear criterion indicating suitability, so the amount of calculation becomes enormous and the range of the solution can be narrowed, but the solution is not always obtained. There is also a problem that the correspondence finally obtained is not guaranteed to be the optimal solution because it depends on the initial value. The method using the coarse / fine search method (see Reference Document 2)
In order to reduce erroneous correspondence, matching is first performed on a strongly-smoothed low-resolution image, and the parallax is obtained by using the result to restrict the matching range in high-resolution. However, depending on the image, it may be necessary to take correspondence from a high resolution.

[0005]

The prior art described above is
In both cases, the correspondence search between the left and right images is to be solved by various algorithms. However, the inverse problem of restoring the three-dimensional information from the two-dimensional information, that is, the method of completely solving the defect setting problem has been developed up to the present. Not even known. In addition, real-time performance is required in fields such as obstacle recognition of autonomous vehicles and object shape recognition in the field of factory automation. There was also an inconvenience that it was not possible to respond sufficiently.

The present invention has been made under such circumstances, and an object thereof is to obtain distance information in an image without solving the above-described defect setting problem, that is, a problem of correspondence between outline edges of right and left images. It is an object of the present invention to provide a distance measuring device capable of measuring the distance.

[0007]

A distance measuring apparatus according to the present invention captures an image of an object while changing the image capturing position at predetermined intervals and outputs image signals of frames at predetermined intervals corresponding to each image capturing position. Means for sequentially smoothing an image signal of each frame from the imaging means, and calculating and outputting edge strength and gradient angle for each pixel of each frame after the smoothing processing by edge enhancement processing. Means for extracting a peak pixel whose inner edge intensity of a pixel in the region has a maximum value based on the edge intensity of the pixel in the region determined by the gradient angle of the pixel to be subjected to edge detection for each frame. A peak extracting unit, and an offset amount from the pixel center of an edge position for each peak pixel based on the extracted peak pixel and an output of the edge enhancement processing unit. Sub-pixel position detecting means for outputting, for each frame, edge position information at the sub-pixel level comprising the gradient angle and the gradient angle, and for each pixel based on the gradient angle and the pixel center based on the sub-pixel position information for each frame. A line drawing generating means for generating an edge straight line in the area of the peak pixel determined by the offset amount, and calculating a moving distance of the edge straight line between adjacent frames output from the line drawing generating means. Distance calculating means for calculating the distance to the object from the moving distance.

[0008]

According to the present invention, the object is imaged while the image pickup means changes the image pickup position at predetermined intervals, and an image signal for each frame corresponding to each image pickup position is output. The edge enhancement processing means sequentially smoothes the image signal of each frame, and then calculates and outputs the edge intensity and the gradient angle of each pixel of each frame after the smoothing processing by the edge enhancement processing. In the peak extracting means, the edge strength and the gradient angle output from the edge detection processing means are inputted, and the edge strength is determined based on the edge strength of the pixel within the area determined by the gradient angle of the pixel to be subjected to edge detection for each frame. Extracts the peak pixel having the maximum value. In the sub-pixel position detecting means, based on the peak pixel extracted by the peak extracting means and the output of the edge enhancement processing means, the sub-pixel level of the offset amount from the pixel center of the edge position and the gradient angle for each peak pixel. Edge position information is output for each frame. The line drawing generation means generates an edge straight line in a region of a peak pixel determined by a gradient angle and an offset amount from a pixel center for each pixel based on sub-pixel position information for each frame. The distance calculating means calculates a moving distance of an edge straight line between adjacent frames output from the line drawing generating means, and calculates a distance to the object from the moving distance of the edge straight line.

Therefore, according to the present invention, true edge position information having sub-pixel data between pixels can be obtained without solving the problem of the correspondence between the outline edges of the left and right images, and the true edge position information can be obtained. Depth information, that is, distance information can be obtained from the difference (corresponding to parallax in the related art).

[0010]

【Example】

<< First Embodiment >> A first embodiment of the present invention will be described below with reference to FIGS.

FIG. 1 schematically shows the entire configuration of a distance measuring apparatus 10 according to a first embodiment.

The distance measuring device 10 includes a CCD camera 1
2, an optical path selecting means 14, an A / D (analog-digital) converting means 16, a smoothing / differentiating means 18 as an edge detecting means, a peak extracting means 20, a sub-pixel position detecting means 22, a line drawing It has a generation unit 24 and a distance calculation unit 26.

More specifically, the CCD camera 1
2 images an object and outputs an image signal (512 pixels × 512 pixels). The optical path selecting unit 14 sets a small distance (a distance corresponding to 1 to 2 pixels on the image pickup screen) every predetermined time, for example, every 33 ms (also referred to as a video rate, which corresponds to a processing time of one frame of a normal television). ) For selecting the optical paths L1 and L2 with an interval of, for example,
The configuration is as shown in FIG. That is, FIG.
As shown in the figure, the optical path selecting means 14 is arranged on the optical path L2 toward the CCD camera 12 at 45 ° with respect to the optical path L2.
A rotating mirror 90 that rotates on an inclined surface, and a fixed mirror 92 that is disposed substantially parallel to the rotating surface of the rotating mirror 90
It is comprised including. The rotating mirror 90 is shown in FIG.
When viewed from the direction of arrow A in FIG. 20A, as shown in FIG. 20B, the motor 94 includes a circular frame 90A and a substantially semicircular mirror 90B fixed in the circular frame 90A. Thus, the motor is rotated in synchronization with the predetermined time (33 ms). Therefore, in the optical path selecting means 14, for example, when the mirror 90B is at the position shown in FIG. 20A, the optical path L2 is blocked by the mirror 90B, but the optical path L1 is blocked by the fixed mirror 92 and the mirror 90B.
The mirror 90B is guided to the D camera 12, for example, as shown in FIG.
When the optical path is rotated by 180 degrees from the position shown in FIG. 2A, both the optical paths L1 and L2 pass through the space inside the frame 90A, and the optical path L2 is guided to the CCD camera 12. In this way, the CCD camera 12 is moved to the CCD camera 12 every predetermined time (33 m
s), the optical paths L1 and L2 are alternately guided.

In this embodiment, the optical path selecting means 14 and C
An imaging unit 11 that changes the imaging position with the CD camera 12 every 33 ms, and switches and selects the optical paths L1 and L2 is configured here. That is, the combination of the CCD camera 12 and the optical path selecting means 14 causes a minute distance interval (approximately 1 to 2 pixels, for example, several m) corresponding between the scanning lines.
m), it is possible to capture an image of an object equivalent to the case of imaging while switching images every 33 ms from one camera to another, and one frame image of the object is obtained every 33 ms.

The A / D conversion means 16 is a conversion means for converting an image signal S1 which is an analog signal output from the imaging means 12 into a digital image signal S2.

The smoothing / differentiating means 18 receives a digital image signal S2 which is gray-scale image data for each frame, sequentially smoothes the digital image signal S2 to remove noise of the image signal, and spatially differentiates the noise to remove noise. This is a means for calculating the edge strength (which will be described later) and the gradient angle (which will be described later) for each pixel after removal.

The smoothing / differentiating means 18 is, specifically,
As shown in FIG. 2, a smoothing means 28, a window 30 composed of 3 × 3 pixels, an X direction differentiating means 32, a Y direction differentiating means 34, an edge intensity gradient calculating means 36,
Look-up table (hereinafter referred to as “LUT”) 38,
40, and latch circuits 42 and 44. Here, each component of the smoothing / differentiating means 18 will be described in more detail.

The smoothing means 28 receives the image signal S2, which is output data from the A / D conversion means 16, and applies an 8 × 8 convolution obtained by a Gaussian distribution function to S2 to remove noise components. image data _{S3 (f s (x, y} )) of 16 bits and outputs a. Note that the size of the convolution mask size may be larger than 8 × 8.

This image data (data of pixel values such as density and luminance) S3 is input to a pixel at the center of a 3 × 3 window 30.
Data S4 in the direction of both sides (i.e., front, rear, right and left) (f _s (x
+1, y)), the data _{S5 (f s (x-1} , y)), data _{S6 (f s (x, y} + 1)), data S7 (f
_s (x, y-1)), and outputs these data as shown.

In the X-direction differentiating means 32 and the Y-direction differentiating means 34, S4 and S5 and S6 and S7 are respectively differentiated (actually, differences) and expressed by the following equations (1) and (2). Data S8 (g _x (x, y)) and data S9 (g _y
(X, y)).

[0021]

(Equation 1)

When the data S8 and S9 are input to the edge intensity gradient calculating means 36, the edge intensity data S10 (cont (x, y)) and the gradient angle data S11 (orient ( x, y)) according to the following equations (3) and (4).

[0023]

(Equation 2)

Here, the edge intensity data are the outputs of the X-direction differentiating means 32 and the Y-direction differentiating means 34, S8 and S8.
9 is a vector having an x-direction component and a y-direction component, that is, a magnitude cont (x, y) of a spatial first derivative vector (gradient vector) of the image data S3, and the gradient angle data is the gradient vector Direction data or
ient (x, y).

The data S10 and S11 are output as 16-bit and 12-bit data, respectively.

The LUTs 38 and 40 store these data S1
This circuit converts 0 and S11 into 8-bit data S12 and S13, respectively, and an arbitrary function can be set in the LUTs 38 and 40. Thereby, the data S1
It is possible to emphasize a specific range of 0 and S11 and compress data. These LUTs 38 and 40 are effective in preventing the peak extracting means 20 and the sub-pixel position detecting means 22 in the subsequent stage from increasing in scale.

Here, as an example, the LUTs 38, 40
It is assumed that functions defined by the following equations (5) and (6) are set in the following.

[0028]

(Equation 3)

Here, E (x, y) is edge strength data.
The output of cont (x, y) and G (x, y) indicate the output of gradient angle data orient (x, y), respectively.

In the latch circuits 42 and 43, these data S12 and S13 are temporarily latched, and 8-bit edge intensity data E (x, y) and gradient angle data G (x,
y) is output.

In this embodiment, the smoothing / differentiating means 18
For an image signal for each frame input every 33 ms,
A series of processes described above are performed, and 8-bit edge intensity data E (x, y) and gradient angle data G (x, y) are output at a video rate.

The peak extracting means 20 calculates a maximum value of the edge intensity based on the edge intensity of a pixel within a region determined by a gradient angle of a pixel to be subjected to edge detection (hereinafter, referred to as a “target pixel”) for each frame. Is extracted. More specifically, is the peak intensity around the edge intensity of the target pixel (target pixel) compared to the edge intensity of the eight nearest pixels (also referred to as “eight neighboring pixels”)? This is a means for determining whether an edge exists at the target pixel if a peak is present (see FIG. 7).

This peak extracting means 20 is, specifically,
As shown in FIG. 3, it is configured to include a nearest pixel edge strength selecting means 46 and a peak comparison judging means 48. Here, these components will be described in more detail. In the nearest pixel edge intensity selecting means 46, edge intensity data E, which is the final output of the smoothing / differentiating means 18, is output.
The gradient angle data G is input, and as shown in FIG. 7, edge intensities E _{0 to} E ₈ on a 3 × 3 window are output for all pixels. At this time, the gradient angle data G is 0 ° ≦ G <360 ° as shown in FIG.
337.5 ° ≦ G <360 ° and 0 ° ≦ G
0, 22.5 ° G <67.5 ° or 20 in the area (1) where <22.5 ° or 157.5 ° ≦ G <202.5 °
In the region (2) where 2.5 ° ≦ G <247.5 °, 1,67.
5 ° ≦ G <112.5 ° or 247.5 ° ≦ G <29
2, 112.5 ° ≦ G <15 in 2.5 ° region (3)
In the region (4) where 7.5 ° or 292.5 ° ≦ G <337.5 °, 3 is set, and decoding is performed so as to indicate four directions at 45 ° intervals.

[0034] That is, the data S14 in in Figure 3, the edge intensity E ₀ to E _8, a data including a ground stringent angle data g _o the decoded target pixel.

In the peak comparing / deciding means 48, the data S1
G in 4_oGradient from 8 neighboring pixels based on the value of
Two in the direction of the vector and the opposite direction
Identify data. And the eight surrounding pixels
Edge data E of the nearest pixel of₁~ E₈Out of
Edge intensity data of two pixels on both sides orthogonal to the direction
E_i, E_{i + 4}(I = 1,2,3,4), and then
J strength E₀And E_i, E ₀And E_{i + 4}The relationship of each
And determine the peak position. Here, at the same level
When the edge strength is continuous in the direction perpendicular to the edge direction
Also include an equal sign in one of the above comparisons (eg, E_i
≦ E₀<E_{i + 4}), In the direction orthogonal to the edge direction
Only one pixel can be selected as peak
Becomes Therefore, it is possible to prevent the contour line from being interrupted.

As a result of the comparison by the peak comparison judging means 48,
A peak candidate pixel mark signal is output. This signal is output when the target pixel E ₀ is determined to be a peak with respect to an adjacent pixel. In practice, a peak is erroneously extracted due to noise. The peak candidate pixel mark signal P is output when the edge strength E of the target pixel is equal to or higher than a predetermined threshold (Th) in order to exclude data below a certain level from the target of the peak extraction so as not to cause a problem. Is desirable, and this is the same in the present embodiment. Further, in this embodiment, the L _th of the high-level H _th and it than the low level as a peak candidate pixel mark signal P is provided as the threshold, it small for a large peak pixel mark signal "HPEAK" the edge intensity of the edge strength Two signals of a peak pixel mark signal "LPEAK" are obtained. This is because the provision of the two thresholds H _th and L _th enables finer peak detection. Further, the present invention can be applied to peak extension (hysteresis threshold processing) using the peak pixel mark signals “HPEAK” and “LPEAK”.

By executing the above-described operation for all the pixels, it is possible to extract the edge peak, that is, the contour, of the grayscale pixel by "HPEAK" and "LPEAK". “HPEAK” and “LPEAK” are assigned to the 7th and 6th bits of the 8-bit peak pixel data (all other bits are 1). Part) is a peak pixel with a large edge intensity, slightly dark white (low brightness)
The portion is displayed as a peak pixel having a small edge intensity.

The peak extracting means 20 performs the above-described processing for each frame, and outputs a peak candidate pixel mark signal P for each frame at a video rate.

The sub-pixel position detecting means 22 interpolates between the pixels of the output of the smoothing / differentiating means 18 on the basis of the extracted peak pixels, and for each pixel, the offset amount of the edge position from the pixel center and This is a means for outputting edge position information at a sub-pixel level consisting of a gradient angle, that is, a means for obtaining a displacement coordinate vector IP from the output P of the peak extracting means 20 to a true peak position of a contour edge in a peak pixel. The above-described series of edge detection up to the interpolation in the sub-pixel position detection means 22 is performed by “Finding edges and lines” by JFCanny.
in images ", MIT. Artificial Intell. Lob., Camb
ridge, MA, Rep. AI-TR-720, 1983.
It is a hardware version of nny's edge finder.

As shown in FIG. 4, the sub-pixel position detecting means 22 specifically selects four edge intensity data to be interpolated from the edge data E _{1 to} E ₈ of the nearest pixel. the nearest interpolation edge intensity data selection means 50 to take out, from the gradient angle data G in the target pixel, x-direction, interpolation E _a of the edge intensities in the y direction,
Edge intensity interpolation and interpolation means 52 for obtaining E _b , gradient angle data G at the target pixel, and E which is the output of the preceding stage
_a, the fitting function than E _b and pixel E _0, x-direction to detect the true peak position, offset in the y-direction Δ
sub-pixel edge position calculating means 54 for obtaining x and Δy
It is comprised including. In the present embodiment, the sub-pixel position detecting means 22 first performs an interpolation by a linear function in the x or y direction from the edge intensities of the eight nearest pixels around the target pixel. , And Δx and Δy are obtained as data at the subpixel level (for example, a resolution of 1/10 of one pixel of the CCD sensor 12) with the maximum value as the position of the true edge. This process is performed at the video rate for all pixels in each frame.

More specifically, as shown in FIG. 7, the nearest interpolation edge intensity data selecting means 50 calculates the peak direction from the gradient angle data indicating the four directions considering the symmetry at 45 ° intervals. The edge intensity of the four closest pixels in the direction perpendicular to. For example, as shown in FIG. 8, the direction d perpendicular to the edge is (0 ≦ θ ₀ <45).
In the case of ゜), E1, E2, E5, and E6 are selected and output as data S15. This selection is made according to the direction of the gradient vector.

The edge intensity interpolation interpolating means 52 performs internal interpolation using a linear function. That is, as shown in FIG. 9, the interpolation point of the two internal division when edges are perpendicular ₍₀ ≦ θ 0 <45 °) E _a, E _b is E _m by tan (θ _0), since the edge intensity of a point which internally divides the E _n, the following equation (7), represented by (8).

[0043]

(Equation 4)

The output S of the edge strength interpolation interpolation means 52
Reference _numeral 16 denotes Ea and _Eb described above. In the sub-pixel edge position calculating means 54, as shown in FIG.
= Ax ² + bx + c is fitted to E _a , E ₀ , and E _b , the maximum value of the curve is obtained, the offset amount from the target pixel is obtained, and the offset in the x and y directions is obtained from the gradient angle data at the target pixel. Quantity Δ
x and Δy are obtained.

Here, an algorithm for obtaining the offset amounts Δx and Δy will be described. Here, the case of (0 ≦ θ ₀ <45 °) will be described as an example.

The three points (−1 / cos (θ ₀ ), E _b ), (0, E
₀ ), (1 / cos (θ ₀ ), E _a ) is the curve y = ax ² + b
Since they are on x + c, a, b, and c are represented by the following equation (9).

[0047]

(Equation 5)

[0048] In point E _p,

[0049]

(Equation 6)

[0050] a since, the offset Δ of E _p from E _0,

[0051]

(Equation 7)

Is represented by Δ is represented by an offset amount (Δx, Δy) from (x, y), that is, when a component is displayed, Δ
Since x = Δcos (θ ₀ ) and Δy = Δsin (θ ₀ ), the above can be summarized as follows:

[0053]

(Equation 8)

In the same manner, the output S1 is output for other cases.
7 (Δx, Δy) is obtained. The output S17 has a data structure of a 7-bit signed expression. At the position where the pixel mark signal extracted by the peak extracting means 20 exists, a displacement coordinate vector IP (Δx, Δy) from the true peak position is provided.
Is obtained (the same applies to other angles). Although the minimum resolution of the hardware is 1/128, the experimental result shows that 0.03
It was confirmed that the resolution was about 0.04.

The line image generating means 24 generates an edge straight line of a pixel within a region determined by a gradient angle for each pixel based on S17 (sub-pixel edge position information IP) for each frame. Generate edge position data.

Specifically, as shown in FIG. 5, the line drawing generating means 24 includes a double buffer memory 56 for taking in a small area and enlarging it by 16 times and outputting it, and a line drawing storing a line drawing reference map in advance. Reference table memory 58, line drawing pattern generation memory 60, and (x, y) designating circuit 62
And a counter 64.

In FIG. 5, input data S 20 is a displacement coordinate vector IP to the true peak position of the contour edge at the peak pixel output from the sub-pixel position detecting means 22 and a gradient output output from the smoothing / differentiating means 18. This is data composed of one word (16 bits) selected from the corner data G.

As shown in FIG. 11A, this data S
The lower 20 bytes are dx and dy composed of the upper 4 bits of the subpixel edge position data Δx and Δy in the x and y directions obtained by the subpixel position detecting means 22.
Which are represented in two's complement notation. Also,
As shown in FIG. 11B, the upper byte of the data S20 is composed of the edge position pixel marker bits obtained by the peak extracting means 20 and the upper 7 bits of the gradient angle data G by the smoothing / differentiating means 18. .

The double buffer memory 56 has a mechanism of expressing one pixel in a captured 512 × 512 image by 16 pixels. That is, writing to the double buffer memory 56 is performed by a mechanism for designating arbitrary x and y designated by the (x, y) designating circuit 62 and extracting a 32 × 32 area from the input data S20 with the x and y as starting points. (See FIG. 12), and reading from the double buffer memory 56 is performed when one output S21 by the counter 64 is operating. Further, in the present embodiment, the output S22 enlarged 16 times in 33 ms can be obtained by switching the memory to be used every 33 ms by a double buffer mechanism (not shown).

The counter 64 counts 512 in the horizontal (x) direction and 512 in the vertical (y) direction, respectively.
The one output S21 is an output of the upper 5 bits among them, and the other output S23 is an output of the lower 4 bits.

In the line drawing reference table memory 58, since the input data S22 is 16 bits, the number of line drawing templates that can be created with currently available memories is 2 ¹² , so that the line drawing reference memory 58 looks as continuous and smooth as possible. It is a memory that compresses data into simple data.

Here, a specific example of the data compression will be described. When the edge position pixel marker bit is 1, the upper byte takes a value from 128 to 255. Assuming that the value of the upper byte is upbyte, the gradient ig used for the map is reduced to 4 bits by the following equation (12).

[0063]

(Equation 9)

However, when ig> 15, ig = 0. ig means that 360 ° is divided into 16 regions from 0 to 15. dx and dy are each mapped by the lower byte by 4 bits. Therefore, the function mapped from the values of dx and dy is

[0065]

(Equation 10)

Can be defined as The label value indicated by Llabel is output as data S24. Note that a label value Llabel that does not display a line segment in the line drawing pattern generation memory 60 is assigned to a pixel that is not a peak. As a result, a line segment can be expressed with an angular resolution of 11.25 °.

In this regard, in this embodiment, the line drawing display pattern generation memory 60 uses a memory of 1 MB (megabyte). However, if a memory having a larger capacity can be realized, the angle information is compressed by the line drawing reference map. It is also possible to assign without doing. The output data S24 is 12
It becomes bit data.

The line drawing pattern generation memory 60 stores the data S
The line drawing is generated by referring to the line drawing pattern created in advance by the label designated by the number 3. Also, a pattern of a 16 × 16 area is drawn by the lower 4 bits in the x and y directions by the counter 64.

The direction patterns created so as to be assigned in the line drawing reference table memory 58 are 16 patterns as shown in FIG. A pattern in which they are shifted in the dx and dy directions is prepared in advance. For example, i
When g = 4, dx = 2, dx = 2, that is, when Llabel = 1058, it is represented as shown in FIG. FIG. 15 shows a representation obtained by applying this to actual parts. FIG. 3A shows a peak pixel of a connector as an electronic component in a size of 512 × 512. FIG. 2B shows the upper right hole of the connector by the line drawing generating means 24. When the peak pixel is enlarged 16 times, the 16 × 16 area is painted out and is represented as a rattling curve. However, by generating a line drawing from the sub-pixel data at one point, the curves can be connected smoothly. With this method, it is possible to generate an edge line image between pixels, which cannot be obtained by the conventional technology.

The line drawing generating means 24 performs the above series of processing for each frame, and outputs the data S2
6 is output at a video rate (33 ms).

The distance calculating means 26 is a means for obtaining a parallax by obtaining a difference of the input data S26 for each frame (33 ms), and calculating a distance from the parallax based on the principle of triangulation.

As shown in FIG. 6, the distance calculating means 26 includes a one-frame delay means 66, a crossbar switch 68, a sub-pixel peak position extracting means 70, and an image developing means 72. AND operation means 7
4 and a distance calculation memory 76.

Here, each component will be described.
The frame delay unit 66 receives the output data S26 of the line drawing generation unit 24 as input, outputs data S28 delayed by one frame (33 ms corresponding to one image screen time), and uses a memory capable of writing the next frame at the same time. It is configured. Time-phase adjusted data S26 and S
Finally, an image having two subpixel edge line drawings as shown in FIG. 16 is obtained at a video rate (33 ms).

The crossbar switch 68 is provided to keep the left-right relationship between these two sub-pixel edge line drawings constant. The cross bar switch 68 always uses the one on the left side of the image as the reference edge line drawing S30 and refers to the one on the right side. Edge line drawing S3
It is a switch that switches to 2, for example.

The sub-pixel peak position extracting means 70
This is a circuit for extracting a sub-pixel peak position of the reference edge line image S30.

The image developing means 72 comprises a line memory and a shift register. When the width of the sub-pixel edge point is m, the shift register outputs a signal S34 at the position of m. For example, when the peak pixel position is separated by n pixels or more on the captured image, S34 is output from the line memory to a position corresponding to L = 16 × n + m.

The logical product calculating means 74 is a circuit for obtaining the logical product of the output of the sub-pixel peak position extracting means 70 and the output of the image developing means 72, and includes a plurality of AND gates provided corresponding to the line memories. Be composed. In the AND operation means 74, an output is obtained at a position corresponding to the above L.

In the distance calculation memory 76, the following equation (14) for obtaining the distance to L from the output of the AND operation means 74 is stored in advance.

[0079]

[Equation 11]

[0080] and the focal length f of the optical path L1, L2 minute distance interval δ and the imaging means 12 by the principle of triangulation, 1 pixel exact L _p and on the imaging surface, the distance from the imaging surface to the object Z Then, since the camera is actually moved by 1/16 pixel on the imaging plane, Z can be obtained by the above equation.

The distance information S36 thus detected
Is output from the distance calculation memory 72. As shown in FIG. 16, assuming that the width between subpixel edge line drawings is L, the distance Z _L calculated by Expression (14) is output as distance information S36 to the pixels of the reference edge line image.

In this case, the sub-pixel peak position extracting means 70 is provided with three
Judgment is made with a pattern of × 3, and the image developing means 72 also has a configuration in which the line memories and the shift registers are arranged in three rows, so that distance information can be obtained stably. This can be achieved at a video rate (33 ms).

As described above, according to the first embodiment, a video rate line drawing is generated from the displacement coordinate vector output up to the true peak position of the contour edge, and 1 to 33 ms is generated.
By providing the optical path selecting means 14 capable of capturing a grayscale image of an object shifted by a small distance interval corresponding to two pixels,
Distance information can be obtained at a video rate without searching for corresponding points. Therefore, according to the present embodiment, since it is not necessary to search for a corresponding point that requires a great deal of time and calculation cost, it is possible to construct an image processing apparatus that extracts three-dimensional coordinate information of a contour edge of a target object at a video rate. .

<< Second Embodiment >> Next, a second embodiment of the present invention will be described with reference to FIGS. Here, components that are the same as or equivalent to those of the above-described first embodiment are given the same reference numerals, and descriptions thereof are omitted or simplified.

FIG. 17 shows the overall configuration of a distance measuring device 80 according to the second embodiment. This distance measuring device 8
0, the point at which the CCD camera 12 is configured to be movable along the movement guide 82 as shown by the arrow A in FIG. Means 24 and distance calculation means 2
6 in that a storage means 84 is provided between them.

That is, the CCD camera 12 and the image pickup position changing means constitute an image pickup means 81 for picking up an image of an object while changing the image pickup position and outputting an image signal for each frame. Then, the CCD camera 12 is sequentially moved from one side to the other at intervals of a minute distance (a distance corresponding to 1/16 pixel on a captured image), and an image of an object is taken every time the minute distance moves (change of the imaging position). The image signal is output from the CCD camera 12.

The storage means 84 stores the input data S26 every time the imaging means 12 moves by a small distance, stores the data from the movement start position to the movement end position in a superimposed manner, and outputs the data S26 '. The output data 26 ′ of the storage unit 84 is obtained by taking an image at a plurality of times with the movement interval being extremely short, and as shown in FIG.
That is, it becomes binary image data having a certain width w in the trajectory of the edge line drawing. For example, when five pixels are moved from the movement start position to the movement end position, binary image data of a predetermined width is obtained as superposition of data obtained by imaging 5 × 16 = 80 times.

In the distance detecting means 26, the input data S26
First, the parallax is obtained by calculating the distance w from the movement start to the movement end position of the edge position of the line image at the imaging start point (measuring this w in the horizontal direction). Specifically, scanning is performed in the horizontal direction, an address having a pixel value other than zero is stored, a counter is incremented to an address having a pixel value of zero, and the increment value is written to the stored address. This increment value corresponds to w, and the parallax is w / 1.
Since this corresponds to 6, parallax can be obtained without finding a corresponding point.

According to the principle of triangulation, the minute movement amount of the imaging means 12 is ε, the focal length of the imaging means 12 is f, the actual size of one pixel on the imaging surface is L _p , and the distance from the imaging surface to the target object is Assuming that Z, Z can be obtained at the edge line drawing position at the start of movement by the following equation.

[0090]

(Equation 12)

The above equation is stored in advance in a distance calculation memory 72 constituting the distance calculation means 26, and the distance Z is output as data S36 '. This S36 'is as shown in FIG. 19, and when the width is w, the distance Z _w calculated by the equation (15) is obtained.
Is output to the pixel of the edge line image at the start of movement. Width v
In the case of, the distance _Zv is output to the pixel of the edge line image at the start of movement.

Therefore, according to the second embodiment, the moving range of the image pickup means is made up of the binary image data filled with a constant value of black (or white) by superimposing the edge line images at every minute distance interval. Therefore, the parallax (w / 16) of the moving imaging unit is expressed as the width (w) in the moving direction in the binary image data, and the parallax can be directly extracted.

Therefore, two-dimensional data representing the shape of the object can easily have distance information.

[0094]

As described above, according to the present invention,
Even without solving the correspondence problem of the contour edges of the left and right images, true edge position information having subpixel data between pixels can be obtained, and a difference between true edge positions (corresponding to parallax in the related art) can be obtained. There is an unprecedented excellent effect that depth information, that is, distance information can be obtained.

Therefore, according to the present invention, it is possible to construct an image processing apparatus for extracting three-dimensional coordinate information of a contour edge of a target object at a video rate.

[Brief description of the drawings]

FIG. 1 is a block diagram schematically showing an overall configuration of a first embodiment of the present invention.

FIG. 2 is a block diagram showing a specific configuration of a smoothing / differentiating means of FIG.

FIG. 3 is a block diagram illustrating a specific configuration of a peak extracting unit in FIG. 1;

FIG. 4 is a block diagram showing a specific configuration of a sub-pixel position detecting unit of FIG.

FIG. 5 is a block diagram illustrating a specific configuration of a line drawing generation unit in FIG. 1;

FIG. 6 is a block diagram showing a specific configuration of a distance calculation unit in FIG. 1;

FIG. 7 is a diagram showing a direction area when performing peak extraction.

FIG. 8 is a diagram for explaining interpolation.

FIG. 9 is a diagram for explaining the operation of the edge intensity interpolation and interpolation means.

FIG. 10 is a diagram for explaining an operation of a sub-pixel edge position calculating means.

FIG. 11 is a diagram for explaining a format of input data to a line drawing generation unit.

FIG. 12 is a diagram for describing extraction of a small area by a line drawing generation unit.

FIG. 13 is a diagram for explaining the resolution in the direction of a line drawing pattern.

FIG. 14 is a diagram illustrating a display example of a line drawing pattern corresponding to one pixel.

FIG. 15 (a) shows a connector which is an electronic component at 512 ×
FIG. 5B is a diagram showing a peak pixel at a size of 512, and FIG. 5B is a diagram showing a part of the connector by line drawing generating means.

FIG. 16 is a diagram for explaining the operation of the distance calculation means.

FIG. 17 is a block diagram schematically showing an overall configuration of a second embodiment of the present invention.

FIG. 18 is a diagram illustrating an example of output data of a storage unit.

FIG. 19 is a diagram for explaining the operation of the distance calculation means of the second embodiment.

FIG. 20 is a diagram illustrating an example of an optical path selection unit.

[Explanation of symbols]

 REFERENCE SIGNS LIST 10 distance measuring device 11 imaging means 18 smoothing / differentiating means (edge enhancement processing means) 20 peak extracting means 22 subpixel position detecting means 24 line drawing generating means 26 distance calculating means 80 distance measuring means 81 imaging means

Continuation of the front page (56) References JP-A-6-102027 (JP, A) JP-A-5-216534 (JP, A) JP-A-5-141930 (JP, A) JP-A-63-250508 (JP) , A) (58) Fields investigated (Int. Cl. ⁷ , DB name) G01B 11/00-11/30 102 G01C 3/00-3/32 G06T 1/00 G06T 7/00 G06T 7/60

Claims (1)

(57) [Claims] 1. An image pickup means for picking up an image of an object while changing an image pickup position at a predetermined interval and outputting an image signal of a frame at a predetermined interval corresponding to each image pickup position, and an image signal of each frame from the image pickup means After sequentially performing smoothing processing, edge enhancement processing means for calculating and outputting the edge intensity and gradient angle of each pixel of each frame after the smoothing processing for edge enhancement, and an object of edge detection for each frame Peak extracting means for extracting a peak pixel having a maximum value of the inner edge intensity of the pixel in the region based on the edge intensity of the pixel in the region determined by the gradient angle of the pixel, and the extracted peak pixel and Based on the output of the edge enhancement processing means, the sub-pixel level comprising the offset amount of the edge position from the pixel center and the gradient angle for each peak pixel. Sub-pixel position detecting means for outputting edge position information of each pixel for each frame, and the peak pixel determined by the gradient angle and the offset amount from the pixel center for each pixel based on the sub-pixel position information for each frame A line drawing generating means for generating an edge straight line in the area of, and calculating a moving distance of the edge straight line between adjacent frames output from the line drawing generating means, and calculating a distance to the object from the moving distance of the edge straight line. And a distance calculating means for calculating the distance.