JP2007025830A - Method and device for recognizing three-dimensional object - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method and device for recognizing a three-dimensional object for acquiring the clear whole image of a three-dimensional object existing in a general space, and for measuring the shape of the object. <P>SOLUTION: This method for recognizing a three-dimensional object is provided to extract and compound one pixel image of an image pickup element corresponding to a selected image pickup position based on luminance information when an image is picked up from a plurality of images picked up at positions where optical axial directions are different, and to acquire the image of the three-dimensional object. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a method and apparatus for recognizing a three-dimensional object from information imaged by an imaging means such as a camera.

  Conventionally, in a case where an image of a three-dimensional object such as an observation object is picked up by a microscope, when the three-dimensional object has a height in the optical axis direction of the microscope, the above-described three-dimensional There is a problem in that objects cannot be captured and displayed in a batch. In order to solve this problem, as shown in Patent Document 1, a magnified image of a three-dimensional object is divided into a plurality of optical paths, and the magnified image of the divided three-dimensional object is placed at different observation positions in the height direction. Each of the imaging means is configured so as to be able to be focused and photographed by a plurality of synchronously coupled imaging means, and the video signal obtained from each imaging means is electrically cut out and combined within an arbitrary range, and the height direction Some obtain a composite video signal for one screen focused on different observation positions.

  Further, Non-Patent Document 1 discloses a method for measuring the surface irregularity shape of a precision processed product under a microscope using interference of white light.

Japanese Patent Laid-Open No. 62-295015 Transactions of the Society of Instrument and Control Engineers (Vol.36, No.1,16 / 25 (2000))

  In the above prior art, when the structure of the three-dimensional object is to be observed in detail, it is necessary to divide the observation position into a plurality of parts in the height direction. If the observation position is divided into a plurality of parts as described above, a camera and a monitor corresponding to the number of divisions are required, and the configuration becomes complicated. In addition, there is a problem that the size of the image captured by each camera varies, and a clear composite image cannot be obtained.

  Further, in the shape measurement of precision processed products under a microscope, the optical system is made parallel and the measurable range depends on the wavelength of light and can only be applied to a very precise measuring range.

  The present invention has been made based on the above-described matters, and provides a 3D object recognition method and apparatus capable of obtaining a clear overall image of a 3D object existing in a general space and measuring the shape of the object. It is intended to provide.

  In order to achieve the above object, the three-dimensional object recognition method of the present invention images an image to be imaged at different positions in the optical axis direction, and addresses this image for each pixel of the image sensor. The distance information to the imaging target is added for each address, and the highest luminance when each image is captured from a plurality of images captured at different positions in the optical axis direction based on the distance information. An image of a three-dimensional object can be obtained by selecting an image pickup position that is high or has a large luminance change, and extracting and synthesizing one pixel image of the image pickup element corresponding to the selected image pickup position.

  The three-dimensional object recognition apparatus of the present invention divides an image picked up by an image pickup means for picking up images at different positions in the optical axis direction and the image picked up by the image pickup means for each pixel of the image pickup device. An image processing means for assigning an address, a distance measuring means for calculating or measuring a distance to the sample reference surface, and a distance for creating distance information to the imaging target for each address based on the distance measured by the distance measuring means The information processing means and the image composition means for synthesizing the images, and based on the plurality of images taken at different positions in the optical axis direction and the distance information, the most luminance when each image is taken An image of a three-dimensional object can be obtained by selecting an image pickup position with a high brightness change or a large luminance change, and extracting and synthesizing one pixel image of the image pickup element corresponding to the selected image pickup position. .

  According to the present invention, the shape of an object existing in a general space is not a perspective-transformed shape captured using a general camera, but an isometric projection image, and distance information to each pixel is obtained. The shape of the three-dimensional object can be accurately recognized.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 shows an example in which imaging means for imaging the shape of a three-dimensional object according to the present invention is provided at the hand of a manipulator. In FIG. 1, the shape measuring apparatus 20 includes a manipulator unit 31 and a control unit 30 that controls the manipulator unit 31. A lens system 1 and an imaging means (CCD camera) 3 are provided at the tip of the manipulator 31. The control unit 30 receives a signal from the imaging means and transmits the signal to the image processing unit 8; the image recording unit 9 that records the image processed image signal; and the lens system 1 from the image processing signal. Distance information processing unit 10 that processes the distance between the object and the three-dimensional object 5 that is the observation object or the object to be measured, and position information, and image synthesis that generates a composite image from signals from the distance information processing unit 10 and the image recording unit And a display unit for displaying the synthesized image. Further, the control unit 30 includes a manipulator driving unit 12 that drives and controls the manipulator unit 31 based on data of the distance information processing unit 10, and a manipulator command input unit 14 that commands driving path control of the manipulator driving unit 12. ing.

  On the other hand, the manipulator unit 31 is composed of an articulated robot for determining an imaging direction with respect to the three-dimensional object 5, and is composed of a plurality of arm portions 4a to 4c and rotatable joint portions 16a to 16d. The arm 4 is provided with a slide shaft 2 at 4a. This manipulator system performs work by a manipulator command input unit 14 and a manipulator drive unit 12.

  Next, the operation of the above-described shape measuring apparatus of the present invention will be described based on the flowchart shown in FIG.

  It is assumed that the space range in which the three-dimensional object 5 and the manipulator unit 31 exist is between the farthest distance point B and the nearest distance point A when represented by the distance from the surface of the imaging means 3. At this time, first, the manipulator unit 31 is driven so that the farthest distance point B of the three-dimensional object 5 is focused on the far distance point side (step 101). At this time, in order to measure the shape dimension of the three-dimensional object 5, it is on a straight line perpendicular to the reference surface of the three-dimensional object (in this embodiment, the table surface on which the three-dimensional object is placed is used as the reference surface). The posture of the manipulator unit 31 is set so that the slide shafts 2 coincide with each other. When imaging is performed in this state, the trouble of coordinate conversion after imaging can be saved. Further, it is desirable that this posture is such that the surface to be measured of the three-dimensional object is not hidden. Next, an optical image is captured by the imaging means 3 (step 102). This optical image is assigned to each pixel in the image processing unit 8. For example, the image number n, the row number x, and the column number y are represented by p (n, x, y). The addressed image is stored in the image storage unit 9. Next, the maximum luminance value at each pixel position of the image captured in the past is compared with the luminance value of the pixel at the same position of the image captured this time (step 103).

  At this time, if the maximum luminance value MAX so far is larger than the luminance value measured this time, the maximum luminance value MAX and the index M of the image having the maximum luminance are left as they are (step 104). On the contrary, if the brightness value captured this time is larger than the maximum brightness value at each pixel position of the image captured in the past, the maximum brightness value MAX is updated with the brightness value captured this time and has the maximum brightness value. The image index is rewritten to the index value N of the image captured this time (step 105).

  Next, the slide shaft 2 of the manipulator is raised by the measurement resolution of the three-dimensional object 5 (determined with the required accuracy by the imaging pitch at the time of measurement) (step 106). The distance from the farthest distance point A of the manipulator slide shaft 2 is moved by the measurement resolution, and the steps 102 to 106 are repeated until the closest distance point B is reached (step 107). For example, when the object to be measured is made of a straight line and a flat surface as shown in FIG. 6 (d) and the shape inspection is performed simply, the measurement object is shown in FIGS. 6 (a) to 6 (d). Such a shape is imaged according to the height of the surface to be measured. In this case, the shape of the triangular pyramid shown in the figure cannot be reproduced, but it is possible to take out information on the cross-sectional outer shape at the height to be measured and perform measurement.

  In addition, when it is necessary to reproduce the entire shape three-dimensionally, the entire shape is displayed in a combination of tomographic photographs of several micrometers by executing imaging every several micrometers. It is possible to measure the dimensions of faults. For example, if you want to measure the dimension in the height direction with an accuracy of 100 micrometers, the accuracy is improved by taking an image every 10 micrometers of 1/10 so that it is not affected by the error of the luminance difference. I can plan. However, since the measurement accuracy is affected by the positioning accuracy of the manipulator and the minimum moving resolution and accuracy of the camera drive system, it is necessary to prepare peripheral devices according to the accuracy. As a result, n images are stored in the image storage unit 9 with the width of the measurement resolution.

  Next, a distance value corresponding to each pixel of the image is selected. This selection method will be described below.

  First, when selecting an optical image of a minute region (X, Y) = (1, 1) of a full-screen focused image, NA images p (1, 1, 1) to p (NA, 1, The relative position of the three-dimensional object is calculated from distance information in which the position of the manipulator and the lens system are in focus when the image with the index M of the image having the highest luminance value in 1) is captured.

  By selecting the index of the image showing the maximum luminance in this way over the entire screen, an image in focus on the object at each pixel is selected. The selected pixel is synthesized with other pixels in the image synthesizing unit 11, and a focused image with a clear outline can be provided on the entire screen.

  Next, a method for extracting a composite image for a triangular pyramid shape as an example of the three-dimensional object 5 by the apparatus of the present invention will be described.

  FIG. 3 shows a three-dimensional pyramid-shaped three-dimensional object 5 as an example of an observation object. The three-dimensional object 5 has a triangular pyramid shape. First, as shown in FIG. 3, an imaging range (in FIG. 3, the imaging range (square area) is set larger than the imaging range imaging object) such that the three-dimensional object 5 is within the camera field of view is determined. . Adjust the aperture of the camera so that the depth of field is narrow, with the aperture open, and set the shutter speed to a high speed to reduce imaging shake due to the vibration of the three-dimensional object or manipulator being imaged. I do.

  If the focal lengths of the lenses attached to the camera are different, the manipulator position for keeping the three-dimensional object within the entire angle of view is changed. This is shown in FIGS. 4 and 5. FIG. FIG. 4 shows a case where a telephoto lens is used, and FIG. 5 shows a case where a wide-angle lens is used. In both cases, the imaging range is set within the frame of −2 <x <2 and −2 <y <2, but in FIG. 5 in which an image is taken close to a three-dimensional object, −2 <x <2, −2 The inner side 22 of the wall in the Z-axis direction at the boundary of <y <2 is seen taking a large area on the screen. That is, it can be seen that the effect of perspective transformation is large, in which a portion close to the camera is shown large and a portion far away is shown small. As described above, when a three-dimensional object is captured from one point, a portion far from the camera is small and a close portion is large, and it is necessary to correct the perspective conversion effect in the measurement for the purpose of dimension measurement. One method that does not require the perspective transformation correction is a method of configuring an optical system in which light from the imaging lens becomes parallel light. However, in this case, the lens aperture and the imaging range become the same size, and it is necessary to prepare a lens with a large aperture in order to measure a large object. There is a limit to the size of the lens, and as long as this method is used, a method of repeatedly moving the field of view and covering a wide range must be taken. Therefore, this apparatus uses a fixed focus method that does not require perspective transformation correction as shown in FIG.

  In the case of this apparatus, first, after focusing on the farthest point A of the three-dimensional object, a method of moving the entire camera away from the three-dimensional object without changing the focal point of the lens is taken. Yes. In this way, by fixing the focal length and gathering only the focused portions from a plurality of images to create one image, the final synthesized image has a size obtained by perspective transformation. An image from which the influence of the change is eliminated can be obtained.

  The imaging process when imaging is performed according to the flowchart of FIG. 2 will be described in detail with reference to FIG. First, FIG. 6A shows a cross-section that can be captured as a focused portion when focusing on the farthest point A of the three-dimensional object. Since the imaging unit 3 forms an optical system so that the depth of focus becomes a very narrow region, the image on the cross section shown in (a) is focused, and even a very small change in luminance is captured on the imaging device. An image is formed. The image of the part away from this cross section is not in focus, and thus has a brightness value that is averaged with surrounding pixels due to blurring.

  Next, the next imaging is performed by moving the slide shaft 2 of the manipulator away from the three-dimensional object by the moving distance corresponding to the predetermined height direction measurement resolution. At this time, the focal length of the lens is not changed, and the in-focus portion is moved by the amount of movement of the manipulator.

  Next, the slide shaft 2 of the manipulator is further raised and set to a position as shown in FIG. The optical image obtained by the image pickup means 3 at this time is an image focused on an object on the cross section shown in (b). By sequentially raising the manipulator, an optical image focused on the vicinity of the apex of the three-dimensional object 5 shown in (c) and (d) is executed. These images are sent from the image storage unit 9 to the image composition unit 11.

  In general, the shape of the three-dimensional object in the coordinate system 33 where the manipulator is installed can be measured using the hand coordinates converted from the posture of the manipulator and the focal length of the lens. However, since the hand coordinates of the manipulator are calculated using the angle sensor 16 attached to the joint, an error may occur in the calculation of the spatial coordinates due to the influence of backlash and the like. In order to eliminate such an error, as shown in FIG. 7, a method of attaching a distance sensor 35 to the hand of the manipulator and acquiring shape data of a three-dimensional object with respect to the distance sensor coordinates is conceivable.

  A coordinate sensor can be calibrated by attaching a distance sensor, for example, a laser distance sensor 35 to the hand of the manipulator, applying a laser beam 34 to a predetermined reference point 32, and performing measurement.

  In the above-described embodiment, a position where an image having the largest luminance value among pixels at the same position with respect to n optical images is picked up as a focused image to obtain an omnifocal image, The shape data of the three-dimensional object was calculated. As another method for extracting the in-focus portion, the following method can be employed.

  The image processing unit 8 detects a difference in brightness between adjacent pixels. Then, the pixel of the image including the point having the largest luminance difference is selected from n images, and the image composition unit 11 synthesizes the image.

  The reason for configuring in this way is that the ridgeline of the three-dimensional object 5 appears clearly at the point of focus, so the difference in luminance from the adjacent pixel becomes large, and the ridgeline of the ridgeline is blurred at the point of focus. The image is blurred. For this reason, by utilizing the fact that the difference in luminance from the adjacent pixels is small, it is possible to obtain a focused image of the entire screen by collecting the focused portions from the n optical images.

  In the present embodiment, the in-focus portion is detected based on the difference in luminance from the adjacent pixel. In addition, the method using the differential value of the change in brightness and the differential value of the second floor, and the brightness, color, and other image information not only with the adjacent pixels but also with the top, bottom, left, right, and surrounding pixels are in focus. It is also possible to use means for detecting the part.

  In the above-described embodiment, when a plurality of optical images of the three-dimensional object 5 are captured, the slide axis of the manipulator is moved in the vertical direction. However, the three-dimensional object is placed on the observation table, and the observation table is moved in the vertical direction. May be. In FIG. 1, the slide axis of the manipulator is set so that the observation axis of the three-dimensional object is in the vertical direction. However, when observing the internal shape as shown in FIG. 8, the slide axis of the manipulator with respect to an arbitrary space axis. By setting, it is possible to perform measurement with a posture without a blind spot even for a three-dimensional object having a complicated shape.

By adopting the above configuration, the focal length of the lens system 1 does not change, so the magnification of the image in the in-focus portion does not change, and an image obtained by combining a plurality of optical images is In addition, it is possible to synthesize without any difference in the junction points.
In the present invention, the synthesized images displayed on the display device 13 from the image synthesizing unit 11 may be displayed collectively, or a plurality of optical images may be sequentially displayed and synthesized. Furthermore, a plurality of optical images can be displayed in color.
With this configuration, shape recognition of the three-dimensional object 5 is facilitated.

It is a figure which shows the structure of one Example of the recognition apparatus of the three-dimensional object of this invention. It is a flowchart explaining the recognition method by the recognition apparatus of the three-dimensional object of this invention. It is a perspective view which shows an example of the three-dimensional object used for this invention. It is a figure which shows an example of the image | video at the time of image | photographing the three-dimensional object used for this invention with a telephoto lens. It is a figure which shows an example of the image | video at the time of image | photographing the three-dimensional object used for this invention with a wide angle lens. It is a figure which shows the cross section to which a focus is set when focusing on the farthest point of the three-dimensional object used for this invention. It is a figure which shows the structure of the calibration by a laser sensor in another Example of this invention. It is a figure which shows the attitude | position in the case of performing the internal recognition of a three-dimensional object in another Example of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Lens system, 2 ... Slide axis, 3 ... Imaging means, 4 ... Manipulator, 5 ... Three-dimensional object, 8 ... Image processing apparatus, 9 ... Image storage apparatus, 10 ... Distance information processing apparatus, 11 ... Image composition apparatus, DESCRIPTION OF SYMBOLS 12 ... Manipulator drive device, 13 ... Display device, 14 ... Manipulator command input device, 16 ... Angle detector, 20 ... Shape measuring device, 22 ... Inner wall, 23 ... In-focus section, 32 ... Reference point, 33 ... Manipulator coordinate origin 34, laser light, 35 ... distance sensor.

Claims (2)

  The image of the imaging target is captured at different positions in the optical axis direction according to the measurement resolution, and the captured image is addressed for each pixel of the image sensor, and distance information to the imaging target is added for each address. Then, from the plurality of images captured at different positions in the optical axis direction, the position having the highest luminance or the largest luminance change when each image is captured is selected and selected based on the distance information. A method for recognizing a three-dimensional object in which an image of a three-dimensional object can be obtained by extracting and synthesizing one pixel image of an image sensor corresponding to the imaging position. An image pickup means for picking up images at different positions in the optical axis direction, an image processing means for dividing the image picked up by the image pickup means for each pixel of the image pickup device and addressing each pixel, and a sample reference plane Distance measuring means for measuring distance, distance information processing means for creating distance information to the imaging target for each address based on the distance measured by the distance measuring means, and image synthesizing means for synthesizing images From the plurality of images captured at different positions in the optical axis direction, based on the distance information, the imaging position having the highest luminance or the largest luminance change when each image is captured is selected and selected. An apparatus for recognizing a three-dimensional object that can obtain an image of a three-dimensional object by extracting and synthesizing one pixel image of an image sensor corresponding to an imaging position.

Images