JPS6375979A - Visually recognizing device - Google Patents

Abstract

PURPOSE:To recognize a picture in a visual field at a high speed, by recognizing the picture by one-dimensionally processing the output of each one-dimensional line sensor which receives the optical images each different in rotating angle to the optical axis of the visual field. CONSTITUTION:Each one-dimensional line sensor 3a-3c is set in such a relation that they can receive the optical image of each different rotating angle to the optical axis X and Y of a visual field 1a. For example, the sensor 3a is set at 0 deg. (X axis) and the sensor 3b is set at 90 deg. (Y axis), and then, the sensor 3c is set at 135 deg.. By using the image picking-up outputs of the one-dimensional sensors 3a-3c the position, etc., of a two-dimensional picture are recognized. Since the picture in a two-dimensional visual field is received by the three or more one-dimensional line sensors 3a-3c by changing the rotating angle in such a way, locations of an object in three or more directions in a two-dimensional visual field can be detected. Therefore, locations of the object in three or more directions can be detected by the one-dimensional processing of the one-dimensional line sensors and, as a result, inexpensive high-speed recognition is made possible.

Description

[Detailed description of the invention] [Table of contents] Overview Industrial field of application Prior art (Fig. 8) Problems to be solved by the invention Means for solving the problems (Fig. 1) Working examples ( a) Explanation of the configuration of one embodiment (Fig. 2, Fig. 3, Fig. 4) ('b) Explanation of the operation of one embodiment (Fig. 5, Fig. 6, Fig. 7) (C) Others Description of embodiments of the invention Effects of the invention [Summary] In a visual recognition device that recognizes the position or shape of an object from a two-dimensional image within the field of view, an image in which the two-dimensional image is optically divided into at least three images. A creation unit and a one-dimensional line sensor corresponding to each of the two divided images are provided, and the one-dimensional line sensor receives light images with different rotation angles with respect to the optical axis of the field of view. It is used to recognize the position of objects.

[Industrial application field]

The present invention relates to a visual recognition device that recognizes the position or shape of an object from a two-dimensional image captured within the field of view. Regarding equipment.

In recent years, there has been active development of visual recognition devices that mechanically realize functions similar to Iruma's vision, and it has become possible to recognize the position, shape, etc. of objects 2 members within the visual field.

Such visual recognition devices are widely used for inspecting and recognizing objects.2For example, a square pad on a printed circuit board is used to recognize the position of a square pad in an automatic assembly robot equipped with an IC, etc. It is used to recognize ICs and IC chips at 1m when handling square ICs and IC chips.

Such visual recognition devices are required to be capable of high-speed recognition and to be inexpensive.

[Conventional technology]

A conventional visual recognition device uses an imaging means 10 such as a television camera as shown in FIG.

IC chip) 17 is imaged, and the two-dimensional image of the target member 17 is converted into an electrical signal, then converted into a digital value by an A/D (analog/digital) converter 11, and stored in the image memory 12.

Then, the CPU 13 such as a microprocessor reads out the two-dimensional image information from the image memory 12 via the bus 16, performs recognition processing, and as shown in FIG.
, center G, inclination α, etc. were obtained.

The contents of the second image memory 12 are also provided to a video RAM (random access memory) 14 via a bus 16.
The imaged content is displayed on the monitor television 15 for confirmation by the operator.

Such a visual recognition device is attached to an automatic machine such as an assembly robot 2, and is used to self-correct the deviation between the position of the gripping part 2 and the position of the target member 17 when the automatic machine grips the target member 17. 2. Grasp smoothly.

Visual processing for determining the position of this target member is shown in Figure 8 (
B) As shown in the flow diagram, an image within the field of view of the imaging means 10 is captured and converted into an electrical signal, and the A/D converter 11
After A/D conversion, shading correction (background correction) is performed and further binarization is performed. Then, after smoothing processing, the image is captured (developed) into the image memory 12.

Store. Next, the KCPU 13 reads out the image information in the image memory 12, extracts features, and obtains one position information (11°t, , G,
α) is calculated.

In such conventional visual recognition technology, a two-dimensional image of the captured image itself is stored in a memory, and the CPU 13 extracts two-dimensional features to obtain the position.

[Problem that the invention seeks to solve]

However, since such conventional visual recognition technology is based on the idea of using two-dimensional images as they are, they require two-dimensional image memory for the imaging means and two-dimensional image memory, and the CPU also performs two-dimensional processing, so high-speed processing is required. This problem becomes more difficult as the target member becomes more complex in shape, such as a rectangle.

Furthermore, there is also the problem that the device itself becomes expensive because it handles a two-dimensional configuration.

SUMMARY OF THE INVENTION An object of the present invention is to provide a visual recognition device that can perform one-dimensional processing on a two-dimensional image and extract position information of an object at high speed and at low cost.

[Means for solving problems]

FIG. 1 is a diagram explaining the principle of the present invention.

In FIG. 1 (5), numeral 1 is an image introducing section for optically capturing the image within the field of view la.

For example, it is composed of an optical fiber, and 2 is an image forming unit that optically divides a two-dimensional image captured within the field of view la into at least three images.

3 is a one-dimensional line sensor group, and there are at least three one-dimensional line sensors 3a corresponding to each of the images divided into υ2.
, 3b, 3C and a plurality of light-receiving elements for 2, for example, - lines (Charge Couple
ed Device).

As shown in FIG. 1 (2), each one-dimensional line sensor 3a,
3b and 3c are arranged in a relationship such that they each receive light images at different rotation angles with respect to the optical axis (x, y) of the field of view 1a.For example, in the example shown in the figure, the one-dimensional line sensor 3a is
(X axis), one-dimensional line sensor 3b is 90° (Y axis),
The one-dimensional line sensor 3C has an angle of 135°.

To obtain this relationship, one-dimensional line sensor 3a.

3b and 3c may be arranged at an angle as shown in the figure.

Alternatively, the divided images may be rotated by the image creation section 2.

The position of the two-dimensional image, etc. is recognized by the imaging outputs of the one-dimensional line sensors 3a, 3b, and 3c.

[For production]

In the present invention, by changing the rotation angle of an image within a two-dimensional field of view,
Since the light is received by three or more one-dimensional line sensors, the position of the object in three or more directions in the two-dimensional field of view can be detected. Therefore, the position of an object in three or more directions can be detected by one-dimensional processing using a one-dimensional line sensor, and recognition can be performed at high speed and at low cost.

〔Example〕

(a) Description of the configuration of one embodiment FIG. 2 is an overall configuration diagram of an embodiment of the present invention.　　.

In the figure, the same components as those shown in FIG. 1 are indicated by the same symbols, and 4 is an image processing circuit.

It reads out the one-dimensional line sensor group 3 and calculates the position of the target member 17, which will be explained in detail later in FIG. Position information is obtained based on the detected position from the image processing circuit 4.

In this example, the target member 17 is an IC and the image introduction part (
An example is shown in which the tip of an optical fiber (hereinafter referred to as optical fiber) 1 is attached to the hand of a robot, and the control section 5 and a robot control section (not shown) control the position of the arm of the robot. Then, the position of the IC 17 is determined based on the image introduced from the optical fiber 1.

This allows the robot's hand position to self-correct and I
This allows the C17 to be held accurately by the hand.

The optical fiber 1 can bend and transmit a light image.

Since it can be freely attached to the robot's hand and the image creation unit 2 etc. can be installed on the base, the load on the robot can be minimized and it will not interfere with the robot's operation.
In addition, because it is optically transmitted, it is resistant to noise and can be used even in harsh work environments.

FIG. 3 is a block diagram of the image creation section 2 and the imaging section (one-dimensional line sensor group) 3 configured in FIG. 2.

In the figures, the same parts as shown in Figures 1 and 2 are indicated by the same symbols! J), 20 is a beam magnifying lens that magnifies the light image from the optical fiber l, 21 is a collimating lens that returns the light image expanded by the beam magnifying lens 20 to a parallel image, and 22 is a half lens. The prism divides the light image from the collimator lens 21 into two identical
23 is a half prism to obtain two optical images.

24 is an aurastone prism that divides the light image from the half prism 22 to obtain two light images, 24 is an aurastone prism that rotates the incident light image by 190°, and 25° and 26.27 are collimating lenses, respectively. The optical images are guided to one-dimensional line sensors 3a, 3b, and 3C, respectively.

3' is a one-dimensional line sensor, which is a combination of one-dimensional line sensors 3a and 3b, and is composed of a large number of light-receiving elements arranged in a row or in a staggered manner. The light-receiving elements are arranged in a row, and the 1024 areas in the upper half of Figure 2 are rotated by -90'' by the Aurastone prism 24 to the one-dimensional line sensor 3b for receiving the light image. This area is used for the one-dimensional line sensor 3a for receiving the unrotated optical image from the half prism 23.

In addition, it is composed of one-dimensional line sensor 3c4CCD elements,
1024 light-receiving elements are arranged in a row, and are arranged at an angle of 45 degrees with respect to the optical axis of the collimating lens 27 (135 degrees with respect to the optical axis of the field of view).

Therefore, the optical image in the field of view 1a of the optical fiber 1 is divided into three parts by the -7 prisms 22 and 23, and the optical image rotated by 90'' by the Aurastone prism 24 is rotated nine times by the one-dimensional two-in sensor 3' 3b. The optical image that is not detected is incident on 3a of the one-dimensional line sensor 3', and furthermore, the optical image of the half prism 23 is incident on the one-dimensional line sensor 3C arranged at an angle of 45 degrees.

Therefore, the one-dimensional line/sensor 3a captures the optical image of O'' in the X-axis direction of the visual field 1a, and the one-dimensional line sensor 3b captures the optical image of the visual field 1a.
A 90″ optical image in the Y-axis direction is captured by a one-dimensional line sensor 3C.
will receive a light image of 135@ in the field of view 1a.

FIG. 4 is a block diagram of the image processing circuit 4 configured in FIG. 2.

In the figure, the same components as those shown in FIGS. 1 to 3 are indicated by the same symbols, and 40 is a clock generation circuit that generates a read clock for the one-dimensional line sensors 3' and 3C. , 41 is an address counter υ, which counts the clock of the clock generation circuit 40 and calculates the read address of the one-dimensional line sensors 3', 3C.
42b each has an amplifier, and each one-dimensional line sensor 3
', 3C amplify the readout output (serial signal of the light-receiving element), 43a, 43b are each sample-and-hold circuits, A7', 42a, 42b outputs are sampled and held, and noise components are cut. Surumo (Z
), 44a and 44b are respectively binarization circuits that binarize the outputs of the sample and hold circuits 43a and 43b, and 45a and 45b are differentiating circuits; Differentiating the output, 46a
, 46b are each a gate circuit, and each is a differential circuit 45a.
, 45b is synchronized with the clock of the clock generating circuit 40 and outputs to a microprocessor (described later). 47 is a microprocessor (hereinafter referred to as MPU). Read permission for the one-dimensional line sensors 3' and 3 is provided. Signal Ml
, M2 to cause the one-dimensional line sensors 3', 3 to perform a read operation using a clock, and to
1, reads the address a of the address counter 41 using the differential outputs of the gate circuits 46a and 46b, calculates the reference coordinates of the object to be described later from the address a, and provides the result to the control unit 5.

(b) Description of operation of one embodiment FIG. 5 is an illustration of light receiving operation of one embodiment of the present invention, FIG. 6 is an illustration of reference coordinate calculation of one embodiment of the present invention, and FIG. 7 is an illustration of one embodiment of the present invention. FIG. 0 is an explanatory diagram of example position information calculation. First, the member 17 is guided into the field of view la of the optical fiber 1. To do this, since the approximate position of the member 17 is known in advance, it is sufficient to guide the optical 7-eyeper 1 to that position, and specifically, it is sufficient to move the hand of the robot provided with the optical fiber 1. As a result, a fifth prisoner appears within the field of view 1a of the optical fiber 1.

When the member 17 enters, a second image (light image) is captured.

■ The introduced light image is enlarged at y-z 20. Via the collimating lens 21.

It is divided into three images by half prisms 22,23. Therefore, as shown in FIG.
Three optical images are obtained from the half prism 22 toward the Aurastone prism 24.

■ The light image toward the Aurastone prism 24.

The optical axis is rotated by 190'' by the Aurastone prism 24, and the horizontal axis of the field of view 1a is the Y axis as shown in FIG. 5 (Q).

The image is rotated so that the vertical axis becomes the Y axis.

■ These three light images are each collimated lens 25.2
6. 3b and 3a of the one-dimensional line sensor 3' through 27
, enters the one-dimensional line sensor 3C. by this,
3b of the one-dimensional line sensor 3' includes the Y axis of the field of view 1a, 3
In a, there is an X@ in field of view 1a.

The one-dimensional line sensor 3C receives a one-dimensional image on a 135° line in the field of view 1a.

(2) In order to obtain the six absolute coordinates n1 to n6 of this member 17, the image processing circuit shown in FIG. 4 operates.

That is, the MPU 47 first enables the read permission signal M1 in response to a start command from the control unit 5, and resets the address counter 41. As a result, the readout clock of the clock generation circuit 40 causes the one-dimensional line sensor 3'
operates and serially outputs an analog signal corresponding to the amount of light received by each light receiving element to the amplifier 42a. Amplifier 42a
is amplified, sampled and held in a sample and hold circuit 431, and further binarized in a binarization circuit 441 to obtain the binarized output shown in FIG. This binarized output is differentiated by a differentiating circuit 45a to obtain the differentiated output shown in FIG. The position of the differential pulse of the differential output is 2 as shown in Figure 6.
This corresponds to the edge position (coordinate point) of the member 17 Nl-N4.

This differential pulse is synchronized with the clock by the gate circuit 46a and is given to the MPU 47. The address counter 41 counts the clocks of the clock generation circuit 40 as described above,
Since the read address is calculated, the MPU 47 takes in the read address a of the address counter 41 every time the differential pulse of the gate circuit 46H is received. Therefore, MPU4
7 is the edge address a1.7 on the one-dimensional line sensor 3' of the two members 17 as shown in FIG. a
,.

al, obtain a4.

When the address counter 41 counts the number of light receiving elements (2048) of the one-dimensional line sensor 3', the MPU 47 drops the permission signal M1 and stops the reading operation of the one-dimensional line sensor 3'.

Next, M::u47 enables the permission signal M2.

Address counter 41 is reset. by this,
This time, the one-dimensional line sensor 3C is connected to the clock generation circuit 4.
It operates with a read clock of 0.

Outputs an analog signal according to the amount of light received by each light receiving element.

Similarly to the above, the output of the one-dimensional line sensor 3C is amplified by the amplifier 42b, sampled and held by the sample-and-hold circuit 43b, further binarized by the binarization circuit 44b, differentiated by the differentiation circuit 45b, and output by the gate circuit 46b. A differential pulse is given to the MPU 47 via. The MPU 47 takes in the read address a of the address counter 41 using a differential pulse as described above, and obtains the edge address "I#al" on the one-dimensional line sensor 3C of the second member 17.

Furthermore, when the address counter 41 counts the number of light-receiving elements (1024) of the one-dimensional line sensor 3C, the MPU 47 drops the permission signal M2.
The read operation is stopped.

In this way, the MPU 47 obtains the edge addresses a, ~a6 of the member 17 in the three optical images from each one-dimensional line sensor 3', 3C, and then determines the center of the coordinate axis of the field of view 1a of the coordinate points N1~N@. Calculate dimensions nl to n6 from .

For this purpose, the addresses of the coordinate axis centers of the field of view 1a on the one-dimensional line sensors 3' and 3C are set in advance as As and A in FIG.
Since x and As are known, the dimensions n1 to n@ are obtained using the quadratic equation.

The MPU 47 outputs these dimensions n1 to n6 to the control section 5.

■Next, the control unit 5 determines the dimension rl1% from the center of the coordinate axis.
In n4, the center position G, inclination α, and dimension 'ip't of each side of the support member 17 are calculated.

This will be explained using FIG. 7.

The calculation formula below is based on the assumption that the position of one member 17 in the field of view la is n, and n6 is on one side as shown in FIG. 5(a).

n4. The opposite side of the side passing through n@, nl, ri, is the one when it touches the remaining two sides.If the conditions are different, a different calculation formula is used.

The coordinate points NI-N, O reference coordinates (with the center of the visual field 1a as the coordinate axis) are determined by the dimension n1yn from the center.

Here, the reference coordinates of N, %N, are (xl, yl), (
xl.

I2)... (X・pya), it can be found by the following formula.

x,=nden=)'1=0・・・・・・・・・・・・・・・
・(4) xx = nz p Yz = 0
・・・・・・・・・・・・・・・(5)xs
"O#)'s = ns...
・・・・・・・・・・・・(6) I4 = Oe'
Ja = ”4 ・・・・・・・・・・・・
...(7) Next, the linear equation of each side of the two members 17 is expressed as coordinate points N1 to N.

Find the reference coordinates of.

Regarding the straight line A-B of member 17.

)' : rnl・x −1−bl ・
・・・・・・・・・・・・・・・α [However, (xs−
xs) → 0.

ITh=('5h-Ys)/(Xl-Xs).

If (xg-x,)=0, m+:O.

bl = 3'm -mx fist XI straight line C-
Regarding D.

y = ml * x −4−bl−−−−−−1−
-------・(11) However, b4= ya -m,
a Regarding the Xa straight line B-C.

y = ml @X + b3...
・・・・・・・・・・・・(YzHowever, if 2m1←0, 2m5=-1im□.

If m1=0, rn3=Q.

b, = y, -ms @Xt Regarding straight line A-D.

y = mz・X + ba...
・・・・・・・・・・・・(1st floor, however, b4 = 3
'*-m4・X.

In this way, the linear equation for each side of one member 17 is found as 2
The reference coordinates of the corner point A-D of the member 17 are determined.

For example, the reference coordinates (x-a, ya) of corner point A are straight line A
-B and the straight line A-D, the linear cylinder el1 equation and the 13th equation are simultaneous equations.

y, = ml・X@+bx y, = m, e X, + b4 Find yoshi p (xat ym).

Similarly, the reference coordinates of the corner point B-D ("b, yb) p(x
c p )'c ) r (Xd*yd) can also be obtained by solving simultaneous equations of intersecting lines.

Next, the dimension 11, 1 of the inclination α2 side of the member 17. seek.

The slope α is.

α= t=-” (m, ) song,
song,,,,,, by a,a.

The length of the straight line C-D is Kt, and the length of the straight line B-C is t, respectively.

t+=v'(Xe-Xd)"+0'c-yd)"=
--Song...α9i,=y'(Xb-Xc)"+yb-y
c)” +...song...song...(obtained by 1e.

Furthermore, the center coordinates G (x, y) of one member 17 are determined as shown in Fig. 7).

Polar coordinates (r.

θ) is.

γ-v'('I/2)" + (t2/2)"
・Song/Song/Song (17) θ=tal-”(tt/l
l') ・-Four songs-(I'm looking for 18.

Accordingly, the center coordinates G (xg, 7g) in the reference coordinate system are:

K, please.

The unit of this number is one pixel of the one-dimensional line sensor 3'p3c, but the resolution of the field of view 1a and the one-dimensional inner sensor (
The actual size can be obtained by multiplying by the size per pixel determined by the number of pixels).

Therefore, in controlling the hand described above, α of the equation (I4) and the center coordinates of the α-th dominant equation are given to the robot control unit.

By positioning the hand to match the member 17, the hand can grip the member 17.

(C) Description of other embodiments In the embodiment described above, the optical axis rotation of the Y-axis is performed by the Aurastone prism 24 by the two-image creation section 2, but the one-dimensional line sensor 3' is divided into 32 and 3b. , if the one-dimensional line sensor 3b is placed at the Y-axis position, the Aurastone prism 24 becomes unnecessary. In this case, in addition to the readout circuit for the one-dimensional two-in sensor 3b shown in FIG. 4, one more is required.

Similarly, the one-dimensional data of 135' is obtained by arranging the one-dimensional line sensor 3C at that angle with respect to the optical axis;
The image may be optically rotated by 135 degrees.

In this case, the one-dimensional line sensors 3a to 3C may be one one-dimensional line sensor. Furthermore, the one-dimensional line sensor 3' may be replaced by two one-dimensional line sensors 3a and 3b, and any suitable configuration can be adopted depending on the configuration of the optical system and the configuration of the line sensor.

Furthermore, the one-image introducing section 1 is not limited to an optical fiber, but may be of any other well-known type, and the one-dimensional line sensor is not limited to a CCD, but may be another light-receiving element such as a light-receiving diode array.

Furthermore, in the above embodiment, rectangles in limited positions are recognized by obtaining three one-dimensional data on the radiation line. It is also possible to obtain two pieces of data parallel to each other and separated from the center by a certain distance, or three or more pieces of one-dimensional data may be obtained. In this way, which data of the visual field 1a to collect depends on the shape and size of the object to be recognized,
Depending on the location, etc., various methods can be adopted that suit the conditions in order to perform recognition with the minimum amount of data and processing time.

Moreover, it is not only possible to recognize the displacement of a rectangle by one rotation, but also to recognize other shapes with less than two dimensions.

It may be used for selecting and inspecting objects that match the dimensions using 1R, etc., and is not limited to gripping objects.
It can be used for various purposes.

Although the present invention has been described above with reference to embodiments, the present invention can be modified in various ways according to the gist of the present invention.

These are not excluded from the present invention.

〔Effect of the invention〕

As explained above, according to the present invention, two-dimensional image recognition can be performed by one-dimensional processing.

This has the effect of enabling high-speed recognition, and can particularly improve the functionality of automatic machines such as robots.

Further, since complicated two-dimensional processing is not required, it can be constructed at low cost, which greatly contributes to the spread of such visual recognition devices. In particular, by applying it to assembly, etc., it is possible to provide a low-cost automatic machine with a visual function.

[Brief explanation of the drawing]

FIG. 1 is a diagram explaining the principle of the present invention. FIG. 2 is an overall configuration diagram of an embodiment of the present invention. FIG. 3 is a configuration diagram of the image creation section and the imaging section configured in FIG. 2. FIG. 4 is a block diagram of the image processing circuit configured in FIG. 2. FIG. 5 is an explanatory diagram of the light receiving operation according to the configuration shown in FIG. FIG. 6 is an explanatory diagram of the reference coordinate calculation operation according to the configuration shown in FIG. FIG. 7 is an explanatory diagram of position information calculation using the configuration shown in FIG. FIG. 8 is an explanatory diagram of the prior art. In the figure, l...image introduction part. 1a...Visual field. 2... Image creation department. 3, 3', 3a, 3b, 3C-・One-dimensional r
fn*nsa. 4... Image processing circuit. 17...Object.

Claims (1)

[Claims] An image creation unit (2) that optically divides a two-dimensional image taken into the field of view (1a) into at least three images, and an image creation unit (2) that is provided corresponding to each of the divided images. at least three one-dimensional line sensors (3a to 3c), the one-dimensional line sensor (3
a to 3c) have a relationship in which they each receive light images at different rotation angles, and the image within the field of view (1a) is recognized by one-dimensional processing of the output of each of the one-dimensional line sensors (3a to 3c). A visual recognition device characterized by performing.