JPS61198014A - Object information processor - Google Patents

Abstract

PURPOSE:To process an object three-dimensionally based on distance information by forming a mask means obtained by forming a prescribed pattern of a relation of negative and positive, in two divided optical paths in a light guide means, from a stripe-shaped color filter. CONSTITUTION:An object image 60 contains the same part of an image from a lens 41 and an image from a lens 42, and it is constituted so that it is shifted in the direction of a scanning line and forms an image. As a result, on a photosensor 52, images which have passed through both optical paths from the lenses 41, 42 are not overlapped but arranged alternately. In such a state, by aligning the array direction of mosaic masks which are arranged alternately, with the direction of a scanning line on the photosensor 52, the same object images from both the optical paths by the lenses 41, 42 can be projected adjacently, for instance, as shown by B or D in the figure, onto the same scanning line of the photosensor 52. With regard to the adjacent same object images which have been obtained in this way, distance information can be detected, based on a principle for measuring a distance.

Description

DETAILED DESCRIPTION OF THE INVENTION [Technical Public Fund] The present invention relates to an object information processing device that has a simple configuration and is capable of recognizing the environment using distance information.

[Prior art]

Various technologies are being researched to recognize the environment using visual sensors. Many of these systems imitate the human visual recognition function, and are performed by extracting the characteristics of the environment's brightness, darkness, and color distribution from signals obtained by a television camera, and comparing these with stored information. For example, there is a method in which a camera is used to analyze the shape of an object, its edges, two colors, shading, etc., and determine what the object is. However, such a method requires time on the order of minutes even to recognize a simple object such as building blocks, and real-time processing is not possible, making it difficult to put it into practical use. A method of measuring the distance distribution to an object by projecting a slit-shaped light beam has also been proposed (Television Society of Japan Vol. 34, No. 3,
1980 etc.). Furthermore, a method of measuring the distance to an environmental object using an ultrasonic sensor has also been proposed. but,
It is difficult to recognize the environment three-dimensionally using the method based on the distribution of brightness and darkness, and the light cutting method using slit light projection is effective for recognizing three-dimensional shapes, but requires a large amount of data to be processed and requires a relatively large computer. Even if one screen is used, processing time of several tens of seconds to several minutes is required for one screen. Furthermore, since it is necessary to move the slit light, mechanically moving parts are required, which may cause problems with accuracy, and in order to recognize bright environments, a lot of energy must be supplied to the light source of the slit light. There was also the drawback that it had to be done.

Further, in the case of using an ultrasonic sensor, due to its characteristics, it is difficult to make the ultrasonic projection beam narrower, and therefore, there is a problem in that the resolution of the image cannot be increased.

Furthermore, when performing stereoscopic recognition by measuring the distribution of distance to a target object using a two-dimensional optical sensor, there is a method that calculates the distance based only on changes in the brightness of the target object, as described above. Conceivable. In this case, even a difference in color that is clearly visible to the human eye may not cause a change in the output of the optical sensor, and in this case, distance cannot be measured. In Kametsu distance measurement, it is possible to measure the distance r* to a single point on the subject, and adjust the distance measurement position to a contrasting part within the 7-pointer. When recognizing and measuring distance patterns, distances in multiple directions within the field of view are measured, which increases the chances of failure in measurement.

On the other hand, in a single-chip video camera, a color mosaic filter is applied to the surface of the two-dimensional optical sensor to take color pictures.
In the distance measurement method that compares the standard field of view and the reference field of view with an optical sensor with this mosaic filter applied, it is not possible to apply the same color filter to images from the same object point in both fields of view. This is because the image position changes depending on the distance. Therefore, due to the change in color, the distance information becomes inaccurate, resulting in erroneous environment recognition.

〔the purpose〕

In view of the above points, the purpose of the present invention is to provide a simple circuit configuration,
It is an object of the present invention to provide an object information processing device that processes an object three-dimensionally based on distance information. An object of the present invention is to provide an object information processing device capable of processing.

□ Also, the mask pattern for creating the distance pattern is composed of a plurality of striped color filters.

〔Example〕

Next, a visual sensor to which the present invention can be applied will be explained.

FIG. 1 is a principle diagram of one embodiment of a visual sensor. 10.
11 is a lens with sufficiently low distortion, f is the focal length of the lens, d is the distance to the object 12, B is the distance between the lenses, and δ is the deviation of the object image from the optical axis. As is clear from the figure, 1 = - Therefore δ = ``Nie...
...■.

B δ d f' d Similarly to the above, in the visual sensor shown in FIG.
3. The total reflection mirror 14 is used to form an object image on the self-scanning sensor array 15 (light receiving element 15a). Note that an auxiliary light may be provided using a light emitting section.

Note that multidirectional distance measurement is possible outside the side base shown in the figure, and even if an object point is not on the optical axis of the lens on the reference visual field side, the distance from the object point to the plane perpendicular to the optical axis is d== B It is calculated as 2J-. The actual distance is d - sec, where θ is the angle between the object δ direction and the optical axis.

However, for example, when controlling the movement of a robot, etc., there is no problem even if it remains as d. The self-scanning sensor is, for example, a CCD array, and is composed of light-receiving elements 15a in the form of a large number of minute segments with a width of about 10 μm. As is well known, the self-scanning sensor 5 converts electrical signals corresponding to the illuminance of each pixel portion of the object image detected by each of the light receiving elements 15a of the plurality of minute segments into time-series signals in a predetermined order. It has an output function. Note that the measurement accuracy may be made variable by changing the distance B between the lenses, and the lateral viewing range of the lens 11 can be changed by changing the focal length of the lens. Here, the scanning output for the two images 12' is the signal S-1, as shown in FIG. 3(b) (see FIG. 3(a)).
Since it is extracted as a waveform such as S-2, the interval between the signals s-i and s-2 corresponding to the two images must be detected by the electrical signal processing circuit. It is used to perform measurements.

Now, the visual sensor shown here uses a self-scanning sensor (hereinafter abbreviated as sensor array) based on the distance M measurement principle described above to detect the mutual distance between two images. However, in various visual sensors, it is necessary to clarify the object space to be recognized, in other words,
It is necessary to define the measurement visual field Ls by some means, and at the same time define the visual field for capturing the other image (this is referred to as the reference visual field Ln).

In the sensor shown here, the measurement field of view is set near the imaging position of the image 12, and the range in which the image 1f moves in accordance with the distance of the object 12 is set as the reference field of view.

A detailed block diagram of the principle of the visual sensor described above is shown in FIG. 15 is a self-scanning sensor array such as the above-mentioned COD (number of cells equivalent to measurement field of view N1 number of cells equivalent to reference field of view N+
M-1) is driven by a clock control circuit 21, a drive circuit 20, a shift clock, and a transfer lock, and the data on the CCD array is compared with a predetermined threshold value by a quantization circuit 22 (for example, by a comparator) and is Therefore, it is divided into 0 and 1.) I is quantized. Next, the measurement field of view,
The quantized data of each reference field of view is put into the shift registers 26 and 27, and while being shifted by the clock output from the clock control circuit 21, the gate 28 detects whether or not the respective quantized data match, and calculates the number of matches. The comparison circuit 24 calculates the number of matches by the counter 2.
After the completion of N comparisons, the output signal of the counter 24-1 is input to the latch 24-3. Naturally, the latch circuit 24-3 is initially cleared. After comparing N values while shifting them by the shift registers 26 and 27, the shift registers 27
advances one value and then compares the second to N+1st values again with the value of NH in the shift register 26 in order. For the third time, the 3rd to N+2nd values are compared with N values in the shift register 26. Repeat this M times, finally M
The (M+N-1)th values are compared. That is, N comparisons are performed while shifting one by one M times. The matching position is determined by counting the number of times (≦M) of these N comparisons in the matching position detecting circuit 23 and checking the matching number comparing circuit 24.
If the number of times is latched every time the outputs of A) and B of the funparator 24-2 are output, the number of times with the largest number of matches is finally obtained, and this value corresponds to the matching position. Next, the object information signal output circuit 25 uses this number of matches, that is, the value of the matching position, and calculates the distance to the object from the relationship of equation (2). The distance information is output to the CPU, memory, etc.

Next, the scanning method of the visual sensor will be described.

FIG. 5 is a control block diagram of the visual sensor.

30 is a scanner for selectively scanning the visual sensor; 31
is a rounding memory that stores patterns such as distance information, 32
is a rounding CP that controls Senna and performs arithmetic processing, etc.
It is U.

Based on the above explanation, one embodiment of the application of the present invention will be described. FIG. 6 is a block diagram of a visual sensor that is an embodiment of the present invention. 41.42 is an objective lens, 43, 44°, 45 is a total reflection mirror, 46 is a positive mask, 48 is a negative mask, 47.49 is a condenser lens, 40 is a semi-transparent mirror, 51 is an imaging lens, 52 is a one-line image It is a 7-photo sensor such as COD that reads the image in pixel units.

In the above configuration, images of the environment to be recognized are formed in the vicinity of masks 46 and 48, respectively, via objective lenses 41 and 42 provided at predetermined intervals. Mask 46 is the seventh
As shown in FIG. 1, a portion of which is enlarged, an opaque portion 46b is provided in a mosaic pattern on a transparent glass plate, for example. The mask 48 is obtained by inverting the transparent part 46a and the opaque part 46b of the mask 46 as shown in FIG. 7-2, with 48a being the transparent part and 48b being the opaque part.

FIG. 8 is a schematic diagram of the visual sensor shown in FIG. 6. 60 is an environment (object) image.

Note that although the explanation has been made here in a simple two-dimensional manner, it is actually a three-dimensional object. Also, 41.42 is the aforementioned objective lens, 43.44.45 is a total reflection mirror, 46 is a positive mask, 48 is a negative mask, 40 is a semi-transparent mirror, 51 is an imaging lens, and 52 is a one-line image. COD (3 lines in Figure 8), which is a photosensor that reads the
This is an object image formed by the lens 51 on the CD.

Note that the photosensor does not need to be a COD, or may be a planar sensor, and the number of lines is not limited to this. Note that the masks 46 and 48 are configured so that their transparent and opaque parts match. Furthermore, the object image is the same part of the image from lens 41 and the image from lens 42.

minutes (in this example, B and D are displayed on the CCD 52, each 2 minutes).
Individuals exist. ), and is configured to form an image shifted in the scanning line direction. As a result, images from the lenses 41 and 42 passing through both optical paths are arranged alternately on the 7-photo sensor 52 without overlapping. By aligning the direction of the scanning lines on the photo sensor 52 with the arrangement direction of the mosaic masks arranged alternately, the same object image from both optical paths by the lenses 41 and 42 is displayed on the same scanning line of the photosensor, for example as shown in FIG. They can be projected adjacently like B or D. Note that the object image may be projected onto a plurality of scanning lines. Distance information can be detected for adjacent images of the same object obtained in the above manner based on the principle of distance measurement shown in FIGS. 1 to 4.

Next, the CCD, which is the 7-point sensor 52, will be explained. FIG. 9 is a diagram showing a part of the image projected on the CCD, and the blocks 61-1.61-2 and 61-3.61-4 are shown in FIGS. 2 to 4. Each corresponds to the sensor array 15. In FIG. 9, an example is shown in which the object image rBJ is projected onto the CCD 52 across four scanning lines K, but the number depends on the reading accuracy and
The number may be any number depending on the speed etc. In addition, 61-5 is shown in FIGS. 3 and 4 [with the measurement field of view explained].
corresponds to the reference field of view. In this embodiment, the measurement field of view 61-
5 is 32 bits, and reference field 61-6 is 94 bits.

Therefore, the number of divisions is 62 (94-32), but this number of divisions may be configured to be arbitrarily variable depending on the accuracy of distance measurement. Also, the size of the plane CCD 52 is 6.6 in the Y direction.
mmXX direction 8.8 mm, scanning line is X direction 7
It consists of 57 lines and 245 lines in the Y direction. Note that it is not necessary to be limited to this configuration. Also, in Fig. 9, there are 61-1.61-2.61-3.6 in the X direction and Y direction, respectively.
It goes without saying that there are many blocks (not shown) similar to 1-4.62-1...

Next, scanning switching control in the CCD 52 having the plurality of blocks will be explained. As explained in Fig. 4, in order to obtain distance information in one block,
NXM clocks are required. Then, by counting the number of clocks, the clock control circuit 21 in FIG. 4 switches the block to be scanned from 61-1 to 62-IK and then to 61-2 in FIG. 9, for example, in a predetermined area. Switching control is performed to scan for.

Next, FIG. 10 shows a four-pot system diagram using the above visual sensor. Reference numeral 52 denotes the above-mentioned 7 optical sensor (CCD), 70 an A/D converter, and 71 a switching circuit which selects the sensor position of the CCD 52, and the visual sensor sensor data processing circuit 72 obtains the distance information of the object. It is something. The detected distance information is 73 RA
It is written on M as a distance pattern.・32 is CP
In U, 74 is a map memory. The map memory 74 stores past distance patterns or command maps, and compares the distance pattern on the map memory 74 with the current distance pattern to perform optimal arm control, movement control, etc. Note that the CPU 32 may be a non-Neumann type computer that has a plurality of processors and processes various data simultaneously and in parallel with each processor. It may also have a terminal for receiving command signals from the outside. Furthermore, the direction and speed of movement.

Drive control 75 based on data from a distance sensor or arithmetic processing unit and information from the distance pattern. Arm control 76 is performed.

In the above embodiment, an example was shown in which the visual sensor block processing circuits were arranged side by side with Wfi, but this was done in order to match the processing time and CCD scanning time. It is not necessary to arrange multiple . Further, the order of the A/D converter 0 and the switching circuit 71 may be reversed.

Furthermore, since distance information can be obtained, it is possible to accurately avoid or capture high-speed flying objects, and it can also be used as a sensor for avoiding obstacles in automobiles and the like. Other uses need not be limited to this.

In addition, it is possible to measure the distribution of distances to environmental objects within the field of view with higher resolution, especially in the lateral direction, than with ultrasonic measurements, without using mechanically moving parts. Images of the same object passed through are formed adjacent to each other on the 7 otosensors, which solves the problem of difficulty in matching corresponding points with the stereo method (TV Society Journal, Vol. 34, No. 3, 1980, p. 211). do. Furthermore, it is possible to perform arithmetic processing in parallel, it is possible to mount a non-Neumann type computer, and it also becomes possible to perform high-speed processing required for robots. As described above, measuring the distance distribution is effective in recognizing objects three-dimensionally, and is particularly applicable to robot eyes, sensors for walking aids for blind people, and the like.

As described in detail above, it has been shown that it is possible to provide an object information processing device that can three-dimensionally recognize the environment as a distance pattern with a simple circuit configuration.

Next, FIG. 11 briefly shows a part of a control flowchart of a robot capable of mounting the apparatus of the present invention. This program is stored in the ROM of the CPU 32, for example.

In this example, each visual sensor block processing circuit 72 shown in FIG. 10 corresponds to each visual sensor block such as 61-1 on the CCD shown in FIG. Depending on the switching of each block of image data and the processing speed of the visual sensor block processing circuit 72, the visual sensor block processing circuit 72 may be
It's good if you have one. In this example, the number of processing circuits for one line on the CCD 52 is n3 (the visual sensor block is also n1).
). In step 1 of FIG. 11, first n=
1, and in step 2, the distance data from block ■ (there are n8 blocks in one line) in the CCD is supplied to the sensor block processing circuit ■ (there are n1 blocks in one line corresponding to the visual sensor block). do. Next, step 3
Then, the object information signal (2) is outputted from the visual sensor block processing circuit (2) as shown in FIG. 4, and is stored in the distance pattern shown in FIG. 10. This process takes approximately 3 ms, for example.
In step 3, the determination □ may be performed using a timer. Next, in step 4, the distance data from the next block ■ is processed.
In this case, n≦n□ in step 5, and the process proceeds to step 2, where the same process as described above is repeated.

When the processing of one line is completed, the process goes to NO in step 5, and in step 6, Y-Y+1 is set to process the next column, and in step 7, yt (plane CCD 52 is (Assuming there are 11 columns in the direction), it is determined whether a column, that is, one screen has ended.

In step 7, if the distance data processing for one screen has not been completed, the process returns to Step 2 to process the distance data in the next column.

If Y≦y is not established in step 7, that is, COD-
If the distance data for the screen has been processed, the first
In the distance pattern RAM 73 in Figure 0, a distance pattern for the CCD screen has been completed, and in step 8, this CCD
Various driving units 75. The arm section 76 is controlled to determine, for example, a direction in which there is no object nearby from the distance pattern data, and the four-pot body is moved in that direction.

Next, a case will be described in which a change in color is also used as information for calculating distance. In other words, it is possible to measure the distance to an object that has the same brightness but changes in hue or saturation, and by making the mask pattern into stripes, alignment is easier than with a mosaic pattern. Therefore, manufacturing advantages also arise, which will be explained in detail below.

Mask patterns 46, 48 of FIG. 6, ie.

Instead of the mask patterns in Figures 7-1 and 7-2,
In this example, a pattern shown in FIG. 12 is used. 10
1a is a base, 10o is a two-dimensional sensor such as COD, 102
is a color filter.

FIG. 13 shows an enlarged view of a color filter pattern which is a mask pattern. 102a, 102b.

102C has three colors of red, green, and blue, or four colors including a transparent part, repeatedly applied in a striped pattern, and the width of each line is equal to the width of the scanning line on the optical sensor. Then, the images projected on the same scanning line of the optical sensor configured in this way,
Since the distance is calculated by the above-described processing from the positional relationship of object images that have passed through a plurality of optical paths, the information to be compared is all light that has passed through filters of the same color. Also, the distance of the object image passing through different color filters is calculated for scanning lines adjacent to each other in the sub-scanning direction (Fig. 9, °Y direction in Fig. 13).
Within sight! Distances of objects projected onto the same number of IF scan lines can be calculated using a plurality of color-separated images.

Note that the filter (mask pattern) may be attached directly onto the optical sensor by vapor deposition or sputtering. Further, if the optical sensor has sensitivity to light other than visible light, it is also possible to separate information on infrared light and ultraviolet light using a filter.

As described in detail above, even if the change in brightness is small, the distance to an object with a change in chromaticity can be easily measured, and a more accurate distance pattern can be created.

[Effects] As detailed above, according to the present invention, objects can be processed three-dimensionally based on distance information with a simple circuit configuration, and further, changes in not only brightness but also hue, saturation, etc. It has become possible to provide an object information processing device that can accurately process object information even for objects that have a certain shape.

[Brief explanation of the drawing]

FIG. 1 is a diagram showing the principle of a visual sensor. FIG. 2 is a diagram showing the configuration of the visual sensor. FIG. 3 is a diagram showing the distance signal of the visual sensor. FIG. 4 is a diagram showing a detailed block diagram of the visual sensor. FIG. 5 is a diagram showing a scanning method of the visual sensor. FIG. 6 is a block diagram of a visual sensor according to an embodiment of the present invention. FIG. 7-1 is an explanatory diagram of the mask 46. FIG. 7-2 is an explanatory diagram of the mask 48. Figure f$8 is a schematic diagram of the visual sensor. FIG. 9 is an explanatory diagram of the scanning platform on the 7-point sensor 52. FIG. 10 is a system block diagram of the robot. FIG. 11 is a diagram showing a part of a control flowchart of the robot. FIG. 12 is a diagram showing the configuration of an optical sensor and a mask pattern. The ita diagram is an enlarged view of the mask pattern. 102...color filter, 32...-C
P U52...Visual sensor, 31...Song me%! J72...Tune visual sensor block, 73...
...ffi [Pattern RAM Fig. 5 Fig. 6 Fig. 72 Fig. 7B section

Claims (1)

[Scope of Claims] Imaging means for forming an image of optical information from an object point;
a light guiding means which divides the optical path into two optical paths and guides it to the imaging means; in the light guiding means, a predetermined pattern having a negative and positive relationship is formed in the divided first optical path and the second optical path, respectively; An object information processing device characterized in that the formed mask means is composed of a striped color filter made of a plurality of colors.