JP5610579B2 - 3D dimension measuring device - Google Patents

Description

  The present invention relates to a three-dimensional dimension measuring apparatus.

  When obtaining the dimension from the image, the dimension calculation accuracy is improved by acquiring a high-magnification (high-definition) image. However, when light receiving elements having the same size are used, a high-magnification image has a narrow photographing range, so that the size of a measurable measurement target is small. In addition, when the shooting range is changed according to the size of the measurement target, camera parameters such as image distortion change depending on zooming and focusing states, and accurate dimension measurement cannot be performed. Further, when shooting at a high magnification, only a part of the measurement object can be confirmed, so it is difficult to determine which part of the entire measurement object is being shot.

  In response to these problems, in Japanese Patent Application Laid-Open No. 2008-241491 (Patent Document 1), Sasaoka et al., In a three-dimensional measurement apparatus using a stereo camera having a zoom and focus function, has high accuracy even when the zoom and focus change. A three-dimensional measuring apparatus capable of performing good three-dimensional measurement is described. That is, a plurality of image pickup means including one or more image pickup means with variable zoom and focus, a zooming and focusing control means, and a stereo for processing three-dimensional information by processing a picked-up image output from the plurality of image pickup means. In a three-dimensional measuring apparatus having image processing means and camera information storage means for storing camera information necessary for performing stereo image processing, the camera information storage means depends on the zoom state and focus state of each imaging means. As for the camera information that changes, the camera has a plurality of camera information corresponding to the state, the stereo image processing unit acquires the zoom state and the focus state of each imaging unit, and the corresponding camera from the camera information storage unit It describes a method for acquiring information and using it for stereo image processing.

  In Japanese Patent Application Laid-Open No. 2008-70120 (Patent Document 2), Tsujioka et al. In a distance measuring device using stereo imaging with variable zoom, a distance that performs distance measurement by accurate stereo image processing even when the zoom changes. A measuring device is described. In other words, stereo image processing is performed by performing stereo image processing using the first and second imaging means with variable zoom for imaging and the stereo captured images output by the first and second imaging means, and calculating distance information. And a zoom change amount detecting means for performing image processing using the captured images output from the first image pickup means and the second image pickup means and detecting a zoom change amount. When there is a difference between the detected zoom magnifications of the first and second imaging means, the first and second imaging means are controlled so as to correct the difference. Also, the stereo image processing means performs stereo image processing using the zoom change amount output from the zoom change amount detection means, so that accurate stereo image processing can be performed even when the zoom changes, and accurate distance measurement can be performed. States that it will be possible.

  Further, in Japanese Patent Laid-Open No. 8-14861 (Patent Document 3), Kondo et al. Used two imaging units having a zoom lens to accurately measure the three-dimensional shape of the target object, and each captured stereo image. The distance measuring unit obtains subject distance data ZS by a stereo distance measuring method. On the other hand, distance data ZR and ZL are obtained by the zoom distance measuring method in the distance measuring unit while changing the zoom ratio, and the load data is averaged according to the reliability of each data ZS, ZR and ZL by the distance data integrating unit. A method of stably measuring the three-dimensional shape of the subject with high accuracy while obtaining the final subject distance and complementing the measurement results of the two distance measuring methods is described.

  Also, in Japanese Patent Laid-Open No. 2009-284442 (Patent Document 4), Kawano et al. Accurately determined the positional relationship between a plurality of cameras and accurately used a plurality of cameras, which had been difficult to use effectively. Provided are a plurality of camera mounting head configurations that facilitate positioning and camera operation, and are easy to operate while always maintaining a positional relationship between a plurality of cameras, and a plurality of camera system devices that control these. In this device, by controlling and maintaining the positional relationship of multiple cameras accurately, the use of multiple images by multiple cameras, speedy and accurate capture of target objects, grasping of the positional relationship of objects, high-speed zoom focus, It describes a camera system device with more advanced functions using multiple cameras. In other words, in the electronic camera system apparatus including a plurality of cameras, the plurality of cameras are mounted on a pan head that can be controlled in the pan direction and the tilt direction, and the plurality of cameras can be operated in cooperation with each other. An electronic camera system featuring the above is described.

JP 2008-241491 A JP 2008-70120 A Japanese Patent Laid-Open No. 8-14861 JP 2009-284552 A

  However, in the techniques described in Patent Document 1, Patent Document 2, and Patent Document 3, when photographing at a high magnification, only a part of the measurement object can be confirmed, so which part of the entire measurement object is photographed. There is no mention of a solution to the difficulty in determining whether or not you are doing.

  The technique described in Patent Document 4 describes a technique for acquiring a low-magnification image and a high-magnification image at the same time and photographing a desired location in the low-magnification image at a high magnification. There is no description of a method of shooting at a higher magnification that does not fit in a single high-magnification image.

  Accordingly, an object of the present invention is to obtain an image with a simple operation and calculate a size even when acquiring an image at a magnification that is too high to fit in one image in measurement using an image. A device capable of calculating dimensions with higher accuracy than calculating a dimension from an image that can be accommodated in a single image by using an image taken at a magnification that does not fit in a single image. It is to provide.

  The present invention extracts a contour of a workpiece from an image of a wide camera, determines a scanning trajectory of a driving mechanism and imaging magnifications of a plurality of cameras based on the extracted contour information, and scans the determined scanning trajectory during scanning. A plurality of images are acquired from a plurality of cameras, a plurality of images are combined to generate a high-magnification image, and a contour dimension is obtained from the generated high-magnification image with high accuracy. .

  According to the present invention, in measurement using an image, when acquiring an image at a magnification that is too high to fit in one image, the image can be acquired with an easy operation, and the dimensions are calculated. In addition, an apparatus capable of calculating a dimension with higher accuracy than calculating a dimension from an image that can be accommodated in one image by using an image captured at a magnification that does not fit in one image can be provided. be able to.

It is a figure which shows the whole block diagram of the dimension measurement system using the wide camera and the stereo camera with a zoom function in 1st Example of this invention. It is a figure explaining the difference in the position of the X-axis direction of a wide camera and a stereo camera in 1st Example of this invention. It is a figure explaining the difference in the position of the Y-axis direction of a wide camera and a stereo camera in 1st Example of this invention. In the first embodiment of the present invention, an image taken with a wide camera and an image taken with a zoom camera of a specified portion of the image taken with the wide camera are each displayed on separate monitors. It is a figure explaining a mode. Explained in the first embodiment of the present invention is a state in which an image taken with a wide camera and an image taken with a zoom camera of a specified portion of the image taken with the wide camera are displayed on the same monitor. It is a figure to do. In the first embodiment of the present invention, the extracted outline of the attention site, a state in which the outline is divided into a plurality of areas, and an image including the extracted outline of the attention area among the divided areas are extracted. It is a figure which shows a mode that it does. In the first embodiment of the present invention, the outline of the extracted target region, the state of dividing the contour into a plurality of regions, and the center of the image including the extracted contour of the target region out of the divided regions It is a figure which shows a mode that a coordinate is extracted with subpixel precision. It is a figure which shows a mode that CAD information and a measurement result are compared and displayed in 1st Example of this invention. It is a figure which shows the measurement flow in the 1st Example of this invention.

  An embodiment of the present invention will be described with reference to FIGS.

  FIG. 1 shows the overall configuration of a measurement system used in the present invention.

  First, the wide camera 3 is mounted on the pan mechanism 6 and acquires a wide range of images. The lens of the wide camera may be a fixed focus lens that does not require focus adjustment, or a lens with a varifocal function.

  Next, a part of an image photographed by the wide camera 3 is photographed by the zoom camera 1 and the zoom camera 2 having a higher magnification than the wide camera 3. In this embodiment, the zoom camera 1 and the zoom camera 2 constitute a stereo camera in order to obtain a three-dimensional shape to be measured. As the lenses of the zoom camera 1 and the zoom camera 2, a zoom lens capable of adjusting the magnification and focus is used. The zoom camera 1 and the zoom camera 2 are mounted on the pan mechanism 4 and the tilt mechanism 5 and can photograph an arbitrary direction. The zoom camera 1, the zoom camera 2, and the wide camera 3 are mounted on the pan mechanism 4 and the pan mechanism 6, and can be rotated in the horizontal direction. Although not shown, if the zoom camera 1, the zoom camera 2, and the wide camera 3 are mounted on the tilt mechanism, the zoom camera 1, the zoom camera 2, and the wide camera 3 can be integrally directed in any direction. It becomes possible.

  The zoom camera 1, the zoom camera 2, and the wide camera 3 are connected to the computer 15 via camera controllers 9, 10, and 11, respectively, and can set a shutter speed and capture an image.

  The pan mechanisms 4 and 6 and the tilt mechanism 5 are connected to the computer 15 via the pan mechanism controllers 12 and 14 and the tilt mechanism controller 13, respectively, and the zoom cameras 1 and 2 and the wide camera 3 are connected. To control the attitude.

  The captured image is stored in the storage device 18. Further, the images are displayed on the displays 19 and 20.

  Here, since the zoom cameras 1 and 2 and the wide camera 3 are respectively arranged at different positions, the coordinates on the measurement target image in the acquired image are different. In order to measure the dimensions, it is necessary to perform coordinate correction in order to handle the same coordinate system on the image. In this case, the positional relationship may be obtained from an image obtained by using the bundle adjustment method. Further, as shown in FIGS. 2 and 3, the rotation adjustment is performed in advance. The zoom cameras 1 and 2 and the wide camera 3 are shifted by a distance of 30 in the X-axis direction, and the zoom camera 1 and the wide camera 3 are shifted by a distance of 31 in the Y-axis direction. The wide camera 3 is shifted by a distance 32 in the Y-axis direction (that is, the zoom camera 1 and the zoom camera 2 are shifted by a distance 33 = distance 31 + distance 32 in the Y-axis direction. This is the parallax between the camera 1 and the zoom camera 2). By calculating in this way, the position correction of the image becomes easy.

  FIG. 9 shows a flow of shape measurement in this embodiment. First, a measurement object (work) is photographed by the wide camera 3 and displayed on the display 19 as shown in FIG. In the acquired image, distortion caused by lens aberration or the like is corrected by a computer. Next, as shown in FIG. 4, the measurement region 41 is enclosed by using the mouse 17 so as to include a target region 42 that is a portion of the measurement target whose dimension has been calculated. At this time, the measurement area 41 may be designated on the display using a touch panel display.

  Next, the zoom cameras 1 and 2 are moved by panning and tilting so that the designated measurement area appears on the entire screen, and an image is acquired. The acquired image is copied onto the display 20 as shown in FIG. At this time, in FIG. 4, one of the images acquired by the zoom camera 1 or the zoom camera 2 is displayed, but both images may be displayed. FIG. 4 shows an example in which the image of the zoom camera 1 and / or the zoom camera 2 is displayed on a display 20 different from the display 19 that displays the image of the wide camera 3, but as shown in FIG. In addition, all the images may be displayed on the same display 19.

  Next, as shown in FIG. 6, the extracted contour 43 is divided into a plurality of divided regions 44. Among the divided areas, the center coordinates of the area 45 including the extracted outline 43 are obtained with sub-pixel accuracy as shown in FIG. The center coordinates are obtained for all regions including the extracted outline 43 in each of the divided regions. The obtained center coordinates are shooting trajectories with the zoom cameras 1 and 2. That is, the posture of the zoom cameras 1 and 2 is controlled using the pan mechanism 4 and the tilt mechanism 5 so that the center of photographing with the zoom cameras 1 and 2 becomes the center coordinate 47. At this time, the magnifications of the zoom cameras 1 and 2 are automatically set so that the shooting range is slightly larger than the area 45. In addition, after performing focus adjustment thereafter, images with the zoom cameras 1 and 2 are acquired.

  Next, distortion correction of the images acquired by the zoom cameras 1 and 2 is performed. At this time, the zoom magnification and the correction parameter for each focus are prepared in advance so that the image can be corrected at an arbitrary magnification and focus. When this operation is finished for all areas including the extracted outline 43, the obtained images are synthesized. Thereafter, stereo matching is performed on the synthesized image, and a geometric dimension such as a radius for calculating the three-dimensional coordinates of the extracted outline 43 is calculated.

  Finally, the calculated dimension is compared with the three-dimensional design data (CAD data), and the remaining amount, which is the difference between the calculated dimension and the target shape, is calculated. As shown in FIG. The remaining amount 53 is displayed together with the calculated size 52 and the target shape size 51.

  According to the present invention, center coordinates are obtained for all the regions including the extracted outline 43, and the obtained center coordinates become shooting trajectories with the zoom cameras 1 and 2. For this reason, in measurement using an image, even when acquiring an image at a magnification that is too high to fit in one image, the image can be acquired with an easy operation, and when calculating dimensions, one image By using an image taken at a magnification that does not fit in one image, it is possible to provide an apparatus capable of calculating dimensions with higher accuracy than calculating dimensions from an image that fits in one image.

  The present invention relates to a method and apparatus for measuring dimensions from an image.

1, 2 Zoom camera 3 Wide camera 4, 6 Pan mechanism 5 Tilt mechanism 9, 10, 11 Camera controller 12, 14 Pan mechanism controller 13 Tilt mechanism controller 15 Computer 16 Keyboard 17 Mouse 18 Storage device 19, 20 Display 30 Zoom camera 1 The distance 31 in the X-axis direction between the zoom camera 2 and the wide camera 3 The distance 32 in the Y-axis direction between the zoom camera 1 and the wide camera 3 The distance 33 in the Y-axis direction between the zoom camera 2 and the wide camera 3 The zoom camera 1 and the zoom camera 2 Distance in the Y-axis direction (parallax)
40 Measurement object 41 Measurement area 42 Part of interest (part whose dimensions are to be calculated)
43 extracted contour 44 divided region 45 region 46 one pixel 47 of image 47 center coordinates 50 of the divided image in which the region of interest is shown target shape 51 target shape size 52 calculated size 53 calculated size and Difference in target shape dimensions (remaining amount)

Claims (4)

3D having the zoom variable plurality of zoom cameras, and a drive mechanism for operating the pan direction and the tilt direction so as to be integrated with the plurality of cameras, the imaging range than the zoom variable multiple cameras wide wide camera A dimension measuring device,
The contour of the workpiece is extracted from the image of the zoom camera, and based on the extracted contour information, the scanning trajectory of the driving mechanism and the imaging magnification of the plurality of zoom cameras are determined, and the determined scanning trajectory is scanned. During this process, a plurality of images are acquired from the plurality of zoom cameras, the plurality of images are combined to generate a high-magnification image, and the contour dimension is accurately determined from the generated high-magnification image. What is required is a three-dimensional dimension measuring device. The three-dimensional dimension measuring apparatus according to claim 1, wherein the plurality of zoom cameras with variable zoom are configured by stereo cameras.   2. The three-dimensional dimension measuring apparatus according to claim 1, wherein the extracted shape contour and the design shape as the target shape are simultaneously displayed on the screen. 3D dimension measuring apparatus according to claim 1, characterized in that the display on the obtained dimensions and at the same time on the screen design dimensions is the target shape.

Images